Safe from the Losing Fight

Like most developers I suspect, I enjoy having a side project to work on during vacations. Almost always it’s something completely different from the day job; something I’m interested in, or never done before, or allows me to learn and explore something new. That’s the major attraction for me to software development: there’s always something new to learn. It’s also why my attention is span is like that of a labrador retriever’s in a field full of squirrels. My subconscious can never commit to which squirrel to chase for the next five minutes, let alone day.

This Christmas vacation is no different. Although I’m spending it with my immediate family, we’re all introverts (save Dad), so there’s plenty of down time for side projects. During these times, I usually work on a project from a set of domains that I, for whatever reason, find most interesting. And one of those domains happens to be compilers. I’m not sure why I settled on that this time, as opposed to, say, an RSS sync engine. Maybe my subconscious likes alliteration.

I’m not a compiler expert by any stretch of the imagination, but I’ve written a few of small ones before. They’re like a data structures winter wonderland. (Or maybe like a fat, slow moving squirrel, my subconscious helpfully points out.) I’ve read through the LLVM Kaleidoscope tutorial many times before and even used LLVM itself as a backend for a (half finished) Pascal compiler (codenamed “Pasco” for “Pascal & Company” or after the noted guitar amplifier photojournalist). So as I was reading through the tutorial recently, I thought: what can I do with this? It’s actually a great step-by-step tutorial for building a small, but interesting language, regardless of the chosen compiler toolset. But what toolset? Erlang seemed the most interesting, because my conscious self hates alliteration and/or my subconscious.

I taught myself Erlang a couple of years ago, I think. It was an easy dip into the functional languages pool, just to see what that was all about. Erlang has strict evaluation, a simple syntax, and a lack of pretentious academics (also known as monads), which made learning it easier than, say, Haskell. I love the actor model of concurrency and BEAM (the Erlang VM), but always wondered if there were a way to put a more powerful front end on it. (And apparently I’m not the only one who’s wondered this, given the existence of Elixir and other BEAM targeted languages.)

So that’s what I did. I used the Kaleidoscope tutorial to gradually implement the Kaleidoscope language in Erlang, targeting BEAM. Since the language didn’t map perfectly to the new runtime, I renamed the compiler and language to kalerl, which can be found on Github. (kalerl is pronounced “Kal-Earl”, like Superman’s redneck cousin. My subconscious thinks that’s hilarious.) I learned a lot from this exercise, mostly about generating Erlang parse trees, which are called “abstract forms” for some reason. I also had a lot of fun, a) because of learning of new stuff and b) because there’s something strangely satisfying about a tool that takes a programmer’s textual ramblings and spits out an executable binary. There’s magic going on inside of it, and I made that magic.

I’m contemplating writing a series of blog posts about the project (my subconscious likes love and acceptance, and has deluded itself into thinking it is achievable through these means). The difficulty is deciding how to break it up into digestible chunks, and figuring out what is interesting that someone (other than my subconscious) would consider worthwhile. Then I need to find the time and energy to do it, which is severely lacking in my current phase of life. But here’s to optimism, blog posts, and fat, slow moving squirrels.

(P.S. My subconscious also likes parenthetical asides and unedited prose, but I’d wager that’s obvious by now.)

At the risk of becoming part of something trendy (which my nerd contract expressly prohibits), I’ve finally ditched Wordpress in favor of static solution. Unlike everyone else though I’m not ditching it for performance reasons, because honestly, I don’t get enough traffic to bring down London bridge, which, according to a song I once heard is constantly falling down.

No, what prompted this change was my Wordpress install getting hacked for a third time (to inject spam links, of course). It’s not entirely Wordpress’s fault; each time my version was out of date, despite Wordpress making updating to the latest version super easy. Unfortunately I don’t go the Wordpress admin panel every day to check for updates, so I’m routinely out of date.

All of which to say, I needed something other than Wordpress to get around my update delinquency. Specifically here’s what I wanted:

• Little to no code running on the server, with a strong preference for none. No database, either.

• It should generate a static site, from flat text files.

• It should be able to import all my Wordpress entries

• Support the Wordpress site/link structure so that it wouldn’t break existing links into the site.

And a couple of nice to haves:

• Support some version of Wordpress themes, or at least make it easy to port them.

• Work with MarsEdit

In other words, I wanted Wordpress without the Wordpress.

There were a couple of solutions that got close, but none that did all that I wanted. So, being an engineer, I decided to write my own, which I codenamed Postword. I actually started the project back the last time I got hacked, and even used an early prototype for the blog on Hearts & Tactics.

Postword is more or less a collection of Ruby scripts. It has one to take a Wordpress XML file and convert it into a bunch of flat files (one for each post) that can be processed later by the build script. The flat files are nothing special; they have mail like headers for the categories and title, and then body which is just text with some HTML markup allowed. The build script takes these flat files and builds a full site that looks like Wordpress site structure wise. The script is unique in that it supports the idea of themes, whose API just happens to be the exact subset of Wordpress APIs that my custom Wordpress theme happens to use.

And that’s all Postword was up until a couple of days ago. But if was going to use Postword on this site, which I update somewhat frequently, I needed to reduce the friction. The first thing to do was to write a publish script that pushed the local static site to my server via FTP. I wrote my own script in Ruby, which actually turned out to be not that bad, since I could make a lot of assumptions. The important one was that the local site was always the truth. Which meant if the local and remote ever differed, the local overwrites the remote, and if the remote had a file which the local didn’t the remote gets deleted. In effect, the site becomes self-healing.

The last piece of the puzzle was create a local server that MarsEdit could talk to. MarsEdit supports a lot of different APIs, so I had to decide which one to implement. Wordpress was an obvious choice, but it offered a lot of functionality to I didn’t have or want, plus it was more complex than the others. I decided on MetaWeblog, with the Blogger API as a fall back for when MarsEdit tried to delete a post. It was surprisingly easy to implement, since almost all of the functionality was already in the other scripts, and Ruby provides an XMLRPC server out of the box.

The really nice part of all this is I now have the entire source to this site (flat post files + images + build scripts) in one Mercurial repository. I can download that and publish from any machine, as well as have instant back ups.

Not to say Postword isn’t without its flaws. The build script is slow; it takes a little while to generate all those archive pages that Wordpress creates on the fly. And if the build script is slow the publish script is glacial. When posting from MarsEdit, I can’t actually call the publish function after building the site, because MarsEdit will time out on the HTTP connection. That said, the pros easily outweigh the cons for me.

All in all, I’m tickled pink about how Postword turned out.

In order to prove that I exist, I have posted photos — in case you thought the photo to the left was doctored. I can assure you someone that homely does, in fact, exist.

OK, that’s not true, I just posted them so I could play with some new software. Its the thought that counts, but apparently not as much as getting to play with new toys.

In the photo albums you’ll find pictures of myself and the very lovely Elaine, pictures of what my family might have looked like several years ago, and pictures of cats that I do not own. If you find the pictures to harsh to look at, simply put your thumb over my photo. That’s what I do.