The circumstances of my job are a bit unusual.  I work on heavy math computational engines, and am the sole developer on these pieces of code.  Often there isn’t, at first, a clear path to solving the problem, and I will find myself trying out different approaches to see if they achieve the desired results.  It’s not unusual to have to discard code developed over two weeks because it turns out not to work for the problem.

This sort of coding progression can lead to very messy code.  You start down one path, set up your calling contexts and your APIs, then decide that this is the wrong approach and you have to come at the problem completely differently.  Now your APIs are wrong, or at least inefficient for the task.  There might be a pile of dead code intended to handle conditions that no longer arise.  You distort the code you’ve retained until it can be applied to the new approach, but your data structures are wrong, you’re re-doing expensive computations because it’s easier to call the function than to figure out how to cache it effectively, and so on.

The result of this tends to be that, by the time the code is done and working, it really desperately needs a rewrite for maintainability, ease of understanding, and support.

And this is where my employers have given me a very useful and valuable freedom.  Instead of shipping the messy prototype that works, they give me the time to rewrite it.  They use the prototype for demos and customer feedback, but meanwhile, I’m applying the lessons learned from the first time through to rewriting the code.  While this means that the project likely takes longer, it comes out more efficient, more maintainable, and more reliable.

However, programmers are famously lazy.  A rewrite can, all too easily, become a cut-and-paste edit session where the code is only modified in a few places.  To avoid this impulse, I write my prototypes in a language that is syntactically alien to the final version.

The code I write for work is in C++.  I prototype in Common Lisp.  Those languages are so different that I’m not going to be copying code from the prototype to the final C++ version.  I can apply the lessons I learned in redesigning my code paths.  I rewrite the code from the bottom-up.  I now understand what every class will be asked to do, so I can generate the write internal APIs, and then tie them together at the end.

Lisp has some very helpful features for this prototyping.  The ability to pass lambda functions to other functions makes it easy to try out several different options very quickly from the command line, without having to sit through a recompilation, and without needing to put in all the boilerplate that comes with building functions in C++.  This isn’t really a weakness of C++, the two languages are very different, and each has its own strengths.

I’m going to be talking a bit about Lisp in the next little while, bringing up some of the interesting parts of the language, the things that I miss when I’m not programming in Lisp.  I’m not likely to say anything that hasn’t already been said elsewhere, and probably more clearly, but maybe the perspective will be interesting.

I’ve been programming in Lisp for some years now.  A variant of it, muLisp, was one of the first languages that I programmed in, but I was inspired to come back to Lisp by this essay by Paul Graham.