Applied DevOps, Chef-gm

Note: This was drafted 2/23/2014 but i’ve decided to publish for historical purposes. DevOps has evolved a lot since this post was written but I believe the goals and purpose are still applicable. – 6/17/2016

As an application developer, I didn’t know much about the emerging philosophy of DevOps but I did know that manually provisioning servers was slow and error prone. My existing development workflow needed an overhaul so I decided to introduce Chef into my team’s workflow to solve the manual server provisioning issue.  To my surprise, Chef has a fairly steep learning curve because the best practices are still evolving and it requires knowledge of a wide range of topics.  You’ve got things like ruby, chef-solo vs. chef-server, knife, and berkshelf to think about.   Here are some thoughts from my first week with Chef to hopefully provide others some perspective and insight into getting started and also show you what is working for me.

My goals with Chef:

  • An environment that is reproducible, scalable and testable.
  • Automate provisioning of production, staging and development so everything matches as close as possible.  No more manual provisioning at the command line or using tools like MAMP.
  • Reduce bugs introduced by developing with mismatched configurations.
  • New developer project setup time reduced to a few commands.
  • Test different server configurations quickly.
  • Faster disaster recovery.
  • Future scaling opportunities.

I was introduced to Chef when learning about the providers in Vagrant.  I initially tried using Vagrant configured with Puphpet and it worked until I tried to share the configuration and it all blew up.  After attempting to debug the issues, I decided to learn to configure the infrastructure directly with Chef.

Infrastructure as code solves many of the growing pains I’ve encountered. I’ve created Chef-gm (general manager) as a workflow and boilerplate for developing web applications. Its core is a combination of concepts built on Chef Solo, Vagrant and Knife for getting past the Chef learning curve. The boilerplate is a set of scripted sysadmin tasks that automate your team’s development and production environments. The workflow gives you some practical use cases and encourages further automation. Fork the Chef-gm repository to get started.

 

Continue Reading

Dig South Hackathon #hackcharleston

Last weekend I participated in the Dig South Hackcharleston: Code for the Cause event with my sister as team Apparctica.  The overnight hackathon was held during a five day Interactive Festival out of Charleston, SC called Dig South and organized in partnership with The Iron Yard.  Ten groups of one to three developers, designers and pitch men assimilated in the TD Arena downtown on Friday evening until Saturday afternoon to compete for the $500 first place cash prize.  Groups where randomly assigned to one of six topics based around the needs of the Lowcountry Open Land Trust and were tasked with building a mobile application.  Food was provided to the participants in the form of snacks, sodas, energy drinks, Verde wraps, and a mix of PBR and Palmetto Brewing Company lagers.  Repositories were managed on Github and are now open-sourced.

The topics that were available:

  1. Historic Landscapes – Upload images and text of significant historic landscapes and identify what landscape is being reviewed based on location.
  2. Soul of the Lowcountry – Relate the four core values of SOL and upload photos for each.
  3. Yelp for the Outdoors – Search outdoor location to identify outdoor activities.
  4. Land StoryCorps – Tell the story of regional histories of protected properties.
  5. Merlin for Waterways (Citizen Science) – Maritime, aquatic animals, and flora.  Linking waterways with the land.
  6. Local Food Sourcing – What is available where? Sources and buyers can post where local and organic produce is being sold.

My team was assigned to #6, Local Food Sourcing.  Our vision for the idea was that the app would work primarily as an educational tool for lowcountry residents to learn where and when items would be available.  The app should list farms, farmers markets, and local groceries with and show which items are season.  It should also allow information to submitted for updates.

The USDA maintains a database of farmers markets that encourages market owners to submit updates. The USDA farmers market API would be the fastest way to a working dataset for our app while allowing people to update the information through the USDA website.

The application we made was developed with an Angular framework called Mobile Angular UI.  It provides a boilerplate of basic yet stylish responsive UI elements made with Angular and a subset of Bootstrap. All of the application logic was client side javascript with no backend.  The application was built with grunt.js and deployed to Heroku with a basic node.js Express server.

I spent the first half of the hackathon figuring out how to deploy the application which I should have sorted out beforehand.  I was focused on this first step because most hackathons won’t qualify your application for competition if it’s not deployable to it’s specifications.  Unfortunately this restriction was lifted at the end and became an unnecessary requirement for competition.

Come crunch time, I didn’t have enough time to sanitize the return data from the farmers market API.  You’ll notice that the dataset contains random HTML, returns the distance to the location in the same parameter as the market name, and that the schedule returns a string of spaces when empty.  All of these API issues lead to some unusual user experiences.

In the end, we didn’t win the prize but we received all the other benefits of hackathons like friends, food, experience and booze.  Until next time, check out what we came up with.  And if you’re curious how to deploy a site built with Grunt to Heroku, check out this tutorial :).

Best viewed on a mobile device.  lolt.herokuapp.com

 

LOLT Home Screen   Menu Options   Seasonal education

Locations from USDA API   LOLT Markets

Continue Reading

New WXRY-FM 99.3 Playlist

Lately I’ve been building large applications and haven’t had a chance to do any small units of work so when I saw this post on Facebook, I decided it’d be a fun thing to try.

ceti-facebook

A local non-profit radio station in Columbia, 99.3 WXRY-FM, was looking for a volunteer to make a searchable playlist for their WordPress website.  Having been involved with music for a long time and a chance to help a non-profit through programming, I volunteered and built them a new playlist.  They had an existing playlist that wasn’t searchable and only showed the last 10 tracks played.  So, I whipped up a quick solution that has logged almost 20,000 tracks so far and lets you search for tracks as far back as the implementation date.  Check it out the new playlist here.

WXRY-Playlist
The new playlist. They play an assortment of alternative rock music.

 

The technical details of the project consist of a WordPress page with an iframe of a jQuery Datatable plugin that loads resources from a remote API hosted on the music server.  The API is a modification of the server-side ajax feature of the Datatable plugin.  It’s an endpoint that sends back JSON to the page to be parsed.  The Shoutcast music server routinely outputs an XML file to the filesystem so I created a cronjob to store the playlist records in MySQL which provides the datasource for the Datatable plugin.  The cronjob runs every minute to load the XML playlist, get the last few tracks from MySQL using the timestamps and if they are not stored, store the new track.

The project took about a week in my spare time and was a satisfying unit of work.  I’m happy I was able to provide something of value for a non-profit using my programming skill set and not general IT knowledge.  In exchange for my volunteer work, the station gave me two tickets to see The Killers.  They also let me promote my local food app, Grubbly, on the playlist.  I encourage you, if you’re in the Columbia metro area, to check out 99.3 WXRY-FM.  Also, try to volunteer every now and then.

Continue Reading

Xdebug breakpoints not working MAMP, PHPStorm

I had an issue with Xdebug not stopping at breakpoints except for the first line of code in only one of my local PHP projects.  Strangely enough xdebug_break() still worked everywhere I put it.  My first thought was that a PHPStorm project setting with the debugger was awry. After searching and trying many different php.ini settings someone suggested checking the xdebug logs for anomalies.

In your php.in file specify a Xdebug log path file and then restart apache.

  • xdebug.remote_log=/[path to where you want the log to live]/xdebug.log

After setting a breakpoint on line 7, running the script, checking the log, and seeing that my breakpoint was being registered, the only inconsistency I saw was the case in the file path.

<- breakpoint_set -i 10 -t line -f file:///Users/na/PhpstormProjects/Sample/httpdocs/index.php -n 7
-> <response xmlns=”urn:debugger_protocol_v1″ xmlns:xdebug=”http://xdebug.org/dbgp/xdebug” command=”breakpoint_set” transaction_id=”10″ id=”429140001″></response>

<- stack_get -i 11
-> <response xmlns=”urn:debugger_protocol_v1″ xmlns:xdebug=”http://xdebug.org/dbgp/xdebug” command=”stack_get” transaction_id=”11″><stack where=”{main}” level=”0″ type=”file” filename=”file:///Users/na/PhpstormProjects/sample/httpdocs/index.php” lineno=”4″></stack></response>

Turns out this particular project folder name was capitalized and others were not.  This was probably a project path setting within PHPStorm somewhere; however, I changed the folder to be all lowercase in the finder and Xdebug began working perfectly.

So if you can’t get a breakpoint to work, check the Xdebug logs and your project path names.  Hopefully this saves you from a few hours of frustration.

Continue Reading

Configure Windows 7 with MAMP virtual hosts and Virtual Box

I’m using MAMP Pro for local web development on OSX Lion and needed to test some javascript compatibility with IE so I booted up a virtual machine. It took me a few minutes to figure how to have the DNS resolve to my local development domains that were set up on my main OS.  I couldn’t find this tip immediately online so here it is.

If you want to be able to access your named virtual hosts that you set up on MAMP on a Windows 7 virtual machine make sure you update your hosts file on the Windows VM.

Here’s what works for me.

  1. Install MAMP Pro
  2. Setup local development host in MAMP
  3. Install Virtual Box
  4. Setup Windows 7
  5. Edit the Windows 7 hosts file
    1. Run Notepad as admin in Windows.
    2. Open C:\Windows\System32\drivers\etc\hosts
    3. Add 10.0.2.2 yourdevdomain
      • Add more if you have want.
    4. Save the file.
    5. Instance restart may be required.
If that doesn’t work, check what port Apache is running on and edit the hosts file accordingly.

Continue Reading