Lessons from Devblocks: Automating incremental upgrades in your PHP apps and their databases

(This post was inspired by this Twitter conversation: http://twitter.com/cakephp_dennis/status/1303663188)

Scenario:

You want to deploy copies of your PHP application that are outside your control (e.g. customers install the software on their own servers).  You want to make it easy for customers to stay current by having your application automatically patch its database schema and perform arbitrary upgrade tasks when it detects the project files have changed. This is especially simple when paired with a public repository (e.g. Subversion) that customers can use to grab the latest stable updates to your app.

The Devblocks Philosophy:

The approach I’m going to explain is based on how I implemented automatic database upgrades in Devblocks.  Devblocks is the PHP5 framework I’ve been building while rewriting Cerb4 from scratch over the past 2 years; and its philosophies are based on the development lessons (a.k.a. “hard knocks”) I’ve picked up from 7 years of working on the same project.  In a nutshell, it focuses on reusability of components (like any good framework) and making “change” as cheap as possible.  Even though Devblocks isn’t something I’m ready to support publicly yet, it’s currently powering thousands of copies of Cerb4.  Most of the Devblocks philosophy applies to building any kind of web application that strives to constantly evolve from ongoing user feedback.

The Nitty-Gritty:

You need a BUILD constant somewhere, which Subversion could even set into a particular file automatically; though it’s useful to set by hand during active development.

Your bootloader needs to compare the BUILD constant to a copy sitting in the cache somewhere.  When your files are swapped out this cached copy will persist with the previous BUILD version.  An update is triggered when the BUILD doesn’t match the cached copy or when the cached copy doesn’t exist.

You should have an /upgrade controller that users are automatically redirected to when the app detects a new version has been swapped in place of the previous files *and* that a database patch is necessary.  This would lock down the application to prevent any requests from being served until the patch is complete.  It should also kill existing sessions to prevent sessions being tied to stale data or objects.  You should have a security mechanism allow administrators to proceed with the upgrade (by IP, etc). The upgrade process also sets a lock file to ensure it’s not run simultaneously by two different administrators.  If the files change but the platform doesn’t have any pending database changes then there’s no reason for the app to lock itself, and users should be able to seamlessly continue with what they were doing.

In the case of Devblocks, everything is a plugin and each plugin has a manifest (in XML) that is independent of its code.  The /upgrade process reads these manifests to learn about new changes.  Devblocks has an “extension point” for “patch containers”, which means plugins can contribute their own scripts to be run by the platform during an upgrade.  A patch container can have multiple patches (e.g. 1.0, 1.1, 1.2, …) which are just PHP files.  The container associates a revision number with each patch.  Patches are run order when their revision number is greater than the last run revision for that patch.  You can reuse a single patch for all the database changes needed during development of a specific version or milestone.  You should always add the latest changes to the end of a patch and resist trying to group changes by table; as a day will come where you make a change, release it, and then change or revert it.  You need to handle the cases where someone (such as QA or other developers) had schema changes that you’re reverting.  Patch changes should always be in chronological order, top to bottom, even if it means you’re adding a change just to drop it a few lines later.  Trust me.

Example:
1.0 (revision 20)
1.1 (revision 105)
1.2 (revision 232)

The revisions for old patches will stop once that version is released and the revision will continue to increment on new files.  New files don’t reset the revision (it’s plugin-wide).  If a customer was upgrading from revision 100 of a hypothetical plugin, according to the example above they’d end up running the 1.1 and 1.2 patches  Each script is designed with the requirement that it needs to be capable of running multiple times, so patches check the schema before making changes (if column “X” doesn’t exist then add it; if column “Y” exists then drop it).  This means a customer can be at any past version and seamlessly upgrade to the latest version in one step.

Since everything in Devblocks is a plugin, the “core” plugin has its patch run first since all plugins are dependent on it.  Though it’s important that plugins only modify tables they created.  The platform ensures all patches are run in order and return successful status codes, and any errors will halt the process to prevent inconsistency.

This process is also really handy in development since you can run the /upgrade controller to make database changes to your local database and know you’re using the same code that customers ultimately will.  You won’t accidentally have tables and columns that you forget to commit.  It’s also useful during development for easily sharing changes with other developers (or QA) since the incremental diffs can be modified by multiple people and merged without incident.  Running /upgrade manually should ignore the version check and always try to run any pending updates.  When making changes in development you could increment the revision number of a patch without needing to change the BUILD of the app.  They have no need to stay in sync; the global BUILD triggers an automatic update, and customers will update far less often than developers.

When setting the app’s BUILD by hand, all you need to do is make sure you’re incrementing.  If you use Subversion you could commit with a build number one higher than the current revision (which your commit will use) or you can have Subversion automatically substitute the latest build number into the app on each commit using svn:keywords.

As an added tip, we also write all our resource URLs (images, scripts, etc) to the browser by appending the latest app BUILD as a query argument (like “script.js?v=BUILD”), which will force most browsers to recache your content every time the files are updated.

If there’s any interest, I can make a minimalistic Devblocks project to demonstrate this in action.  You could also download a free copy of Cerb4 and study the code.

-Jeff

Updating Subversion in OS X 10.5.6

I’ve been using Aptana for my PHP IDE lately, and it comes with a newer version of the Subversion client libraries than OS X 10.5.6 does.  That ended up breaking my ability to use the command line or the IDE interchangeably.

Note: As a precaution, you may want to temporarily move your /usr/lib/libexpat* files out of the way until the compile is completed. They gave me some trouble. When finished you can move them back.

Here are the quick steps I took to compile a new version of Subversion in /usr/local that won’t conflict with future OS X updates:

mkdir ~/svn-src && cd ~/svn-src

curl -O http://subversion.tigris.org/downloads/subversion-1.5.6.tar.bz2

curl -O http://subversion.tigris.org/downloads/subversion-deps-1.5.6.tar.bz2

tar xvjf subversion-1.5.6.tar.bz2

tar xvjf subversion-deps-1.5.6.tar.bz2

cd subversion-1.5.6

./autogen

./configure --prefix=/usr/local --with-ssl

make

sudo make install

After you’ve compiled and installed, you want to move /usr/local/bin to the front of your $PATH.  To do so, edit your ~/.profile (or create it) and add the line:

export PATH=/usr/local/bin:$PATH

Open up a new Terminal session and type:

svn --version

You should see a first line of output like:

svn, version 1.5.6 (r36142)

That’s it!

Automating the search for missing iTunes album art using the OS X terminal

Since switching to OS X (and consequently iTunes) I’ve really become used to looking through my albums by artwork.  I have a bunch of albums that I didn’t get from iTunes which are missing artwork, either because they’re not in the iTunes Store or things are worded slightly differently in my meta tags.  I either buy albums on Amazon MP3 or I download free tracks from last.fm’s site.

I looked at a couple different apps for OS X that scan through the missing artwork and pull hits from Amazon or Google to use as replacements; but the free ones weren’t very automated and the paid ones were asking for about $50.

Maybe you’ve found a quicker way, but I always like a good excuse to do some shell scripting.  In a couple minutes of idle time, I figured out a quick way to automate the search for missing artwork using Terminal.

1. Create a new playlist in iTunes called something like “No Artwork” and drag all your aesthetically unappealing albums to it.
2. On that new playlist, highlight all the albums and copy to the clipboard.
3. Open a text editor (TextMate, vi, whatever) and paste into it.  This is a tab-delimited list of all the tracks for your albums in the format: “Song Name, Length, Artist, Album, Genre #”. Save the file as something like “no_art.txt” and close it.
4. Open Terminal.app and navigate to the directory with your new text file.  Run the following commands (on one line):
   

   cat no_art.txt  | awk -F”\t” ‘{print $3″ - “$4}’ | sort | uniq | sed -e “s/^/open \”http:\/\/images.google.com\/images?q=/” -e “s/$/\”;sleep 2/” | bash

Here’s a quick explanation:

• cat no_art.txt
  Print out the contents of the text file.
• awk -F”\t” ‘{print $3″ - “$4}’
  Split each line into tokens delimited by tabs.  Print out the artist, a dash, and the album name.
• sort
  Fairly self explanatory, make sure the output is sorted by artists first, albums second.
• uniq
  Only print unique lines.
• sed -e “s/^/open \”http:\/\/images.google.com\/images?q=/” -e “s/$/\”;sleep 2/”
  Replace the beginning of each line with an ‘open’ command (which will launch the application associated with URLs).  Append the query info to a Google Images search and enclose the artist and album string in quotes.  End each line with a 2 second sleep (which can be easily changed) to not spam Google or your machine with too many concurrent connections.
• bash
  Run the output through the BASH interpreter as if it was typed at the console.

I have my Firefox3 set up to open new URLs in tabs, so I get a nice browser with a batch of tabs and the best suggested artwork.  I usually look for something around 500×500.  Google is also helpful in pointing out if an artist or album name is mistyped.

With this process you still have to pick a decent match and drag the art into iTunes yourself, but I figured the steps above would be a good starting point for somebody out there who wanted to spend more than the 5 minutes I did.

Enjoy!

-Jeff

Catch(BoringBlogException)

My previous blog just threw a BoringBlogException.  I didn’t need to crawl up the stack trace to figure out that the thing had no readership traction because I positioned myself way outside of my element.  I don’t remember what I was originally thinking, but somewhere in that flawed thought-process I decided my audience was going to be “biz-minded techies”; and that I was qualified to share my opinions with them because I bootstrapped a startup into a comfortably successful company over the past several years.

The problem is, I don’t have a treasure chest of sound business advice.  I built my company by being a relentless, indefatigable workaholic.  I didn’t set out to be any kind of manager or coach, I just wanted the freedom to always be working on projects I’m passionate about.  Enough people continue to find my productive output useful enough to fund my entire life.  That’s a powerful drug.  I’m addicted.

While I’m working on these non-stop projects, I chat and debate for hours with my team and techie friends about usability, architecture, aesthetics, frameworks, cloud computing, and everything else in the world of software.  And, like with anything involving passion, I have epiphanies, jousts, manic ups, and depressive downs — and they’d all make great blog fodder — but conspicuously absent are scenes where I’m dispensing business advice to anybody about anything.  It’s just not in my DNA.

I’m not a business guru.  I’m not a social media expert.  I’m the kind of guy who’s most happy when people in the real-world are enjoying my creative output, happily paying for it, talking about it, and putting it to good use; when the office is engaging the community and paying the bills; and when I’m left free to explore (and hopefully build) the kind of innovative jumps in software that everyone will be talking about a year from now.

So Jeff the business guru is out.  Who’s the real Jeff?  On the best days I lock myself away in a productive isolation, turn the music up, gorge on information, find that undercurrent of creative inspiration, and I ride it until I exhaust myself down to the reserves.  I let that “empty” light blink on my mental dashboard for a while before lay down, and then I end up taking notes by iPhone light until my brain segfaults.  I usually wake up either 3 hours or 12 hours later and wonder for a good 10 minutes whether it’s A.M. or P.M. outside.  Then I eat, sync up with the office, and try to do it all over again.

“Code Hermit” seems to be a much better fit.  Let’s see how I blog in my natural habitat.

-Jeff