How To Rewrite A Project

(Succesfully)

This is motivated by watching several teams fail to complete a rewrite at my day job, my own mixed experiences with big rewrites, and my love for high quality snark[1].

Big rewrites are a tricky thing. People have written lots of things about how they are terrible and fraught. They are right! Your first choice should be to do something other than rewrite it, like writing integration tests[2]. Integration tests are significantly less fun than redoing everything from scratch, and occasionally you get stuck with a codebase so awful that it doesn’t matter what you do as prep, you’re still going to have to rewrite the thing to keep people sane.

The Pitfalls

The most common failure modes of a rewrite are “too unfinished to live”, and “deprecated by ongoing developments”. The first is where you’ve made progress, but not enough to use, and after some amount of time that makes everyone upset, you all just abandon the thing. “Deprecated by ongoing developments” is when ‘emergency patches’ made to the original codebase, but not necessarily ported back to the hot-new-thing, become significant enough that no one trusts or loves the hot-new-thing enough to keep it going.

You should design your rewrite around making these failure modes as unlikely as possible. It’s going to take some sacrifices though.

Stop Having Fun

The goal, at all times, needs to be shipping. Improvement is how you get stuck. If I had a nickel for every time people wandered off on a yak shaving quest during a rewrite or refactor that got them totally stuck I’d have another cup of coffee.

Ship It Piecemeal

If you’re not rewriting in a new language, or you are, but you’re writing a server backend, an excellent trick is to port over API endpoints piecemeal, until the original piece of software is just a facade that calls the new. Then you can delete the old.

This is great for several political reasons. First, it means you can put the new software into production pretty quickly. All software does trivial things- port those things first. As soon as it’s production code, it’s difficult to delete or abandon. This saves you from bit-rot and enthusiasm drop after you’ve all been pulled away to deal with some unplanned nonsense. It also means you’ll get a chance to sort out the dreaded “unknown-unknowns”, where you discover that the new logging library is is a tire fire, or that you don’t know how to not leak memory in your new language, before it’s an emergency.

If you’re not blessed with the ability to use a shim, you should still sit around and scheme up a way to use the new and old system in parallel, because it’s likely you will. This is a time to break out the clever hacks and the bad puns.

Port It Automatically

If you can, this is the best way. Strongly structured things, like deployment projects, are best for this. You get a system that works at the end. Will it be all-singing, all-dancing, filled with candy and Best Practices?

It will not. It will work though, which means you can continue making it better, instead of going back to whatever was so awful that you decided it was impossible to improve.

Do not write a gnarly script to convert it, and then check in the results. Your first try is going to be wrong. So is your second, and so on. Check in the scripts, check in usage instructions, add a .gitignore thing to ignore the automatic output for now. Or your version control system will be useless, and filled with thousands and thousands of lines of garbage.

You Will Not Get To Finish Uninterrupted

The Iron Law of commercial software development is There Will Be Interruptions. Plan for it. There’s going to be a hotfix that pulls you away.

Keep Expectations Low

Everyone hates $oldthing. We’re all mostly sold on $newthing. There will be flaws. Try not to let it all drown in an orgy of grumpiness whenever you discover that Go’s garbage collector is not quite as sophisticated as Java’s G1.

[1]: there are always two sides

[2]: the great thing about integration tests is that they’re sort of portable, and ideally less brittle and more useful than the tautological unit tests favored by cargo-cult TDD-ists. If you’re committed to rewriting something, maybe do integration tests first.