Dreaming In Code / Amazon’s Amazing Recommendation System

I recently finished the book Dreamingin Code, grabbing acopy after it started appearing in all of the “customers whobought this book also bought“* sections for other books I wascontemplating.

Wonderful book.

It documents the progress (or lack thereof) and missteps thatoccurred during the lengthy birthing of Chandler, an open sourceOutlook/Exchange “Killer”.

Started with a lengthy, largely abstract wish list, and anability to use whatever technologies and techniques that theywanted — financed by the deep pockets of the undemandingMitch Kapor— Chandler was the sort of project that developers dreamilycontemplate while analyzing the patterns on their cube wall,imagining how much better it would be than the tight confines anddemanding deliverables of their current reality.

Chandler was the sort of project where the only apparentlimitation seemed to be the collective imaginations of thecontributors. The sky was the limit!

The result of such free reign and lack of demands can onlybe classified as a disaster: After a long period ofdevelopment by a fairly large and expensive team, the releases thusfar have been a rather dubious calendar product, in a veryalpha state, and a server product that backtracks on many of theinitial goals (for instance the “P2P solves everything” silverbullet).

It’s difficult to categorize this book: It really isn’t a how-tobook at all. Nor is it really a book for a layman, as it frequentlygoes into a level of detail that could be daunting (sometimes thelevel of technical detail seems a bit like filler, not reallyplaying a part in the telling of the story. Unnecessary detail forthe story, but too little detail to be of interest for a technicalanalysis).

Really it’s a how-not-to book: A reporting of many of theclassic mistakes of software development happening in a real,visible project, and how they impacted the progress. TheOSAF (Open Source Applications Foundation) team certainly isn’t thefirst to make these mistakes, and they certainly won’t be the last,but they had an uncommon honesty and openness to allow a reporterto document their every step.

I would imagine Mitch figured this would all turn out quite abit differently, with Chandler rocketing to Firefox-like success,and this book documenting the new age approach to softwarecreation.

This book came to mind yesterday when I hit publish on the”MostDestructive Metric In Software Development” entry. In thatentry I mentioned that developers abhor using other people’s codewhen they have the option of spinning something “original”themselves, and this materializes several times during the Chandlerproject. After eliminating the Zope ZODB (a Python object database)from consideration, for instance, and going off and going throughseveral iterations of their own database, the individual behind thedecision declared that they “never really looked atZODB“.


* – Around four years ago we bought the great book”TheBaby Name Wizard“, and found it to be a helpful in choosingnames for our first child. We’re now a month away from welcomingour third child, and have begun the name contemplation game, and Ijust received a “People who bought the Baby Name Wizard alsolike…” email from Amazon. Their system has more inputs thanI imagined.