Build, Buy, or Uncertainty – The Great Dilemma of Software Development

Project Completion

You just finished a gruelling six-month projectto create the Greatest Software Widget Ever Made.Your blood, sweat and tears have been invested into what youbelieve is a first-rate solution, whether solving a corporateneed, or building a better mousetrap to bring untold riches toyour ISV.

You email news of your accomplishment to peers and/or clients,proudly sitting back to await the certain accolades.

Amidst the expected congratulations, one reply makes you uneasy.”So it basically does what Project X does?” it asks.

Surely they must be mistaken, you think.

You fumble through the feature sheets for Project X to discoverthat, sure enough, on paper it does everythingthat your solution does, and in some ways it is arguablysuperior: Maybe it supports a richer set of features, is crossplatform where yours isn’t, or it’s a component of a more-completeintegrated solution where the whole is greater than the parts.

On paper, at least, it is a better solution.

The Damage

If you’re an ISV, you’ve just discovered that you have morecompetition than you thought you did. This could impact your marketin a wide range of ways.

  • Perhaps it’s a very expensive product from a company like BEA.You can easily underprice them, and it represents little threat toyour product (it might actually help you make sales, giving you amuch more expensive product to contrast with in sales pitches).Great stuff!
  • Perhaps it’s the complex potential use of an existing systemthat requires such a high level of expensive modules, customizingand programming that it turns out to be much more expensive andrisky to actually use.
  • Maybe it’s an esoteric open-source project with a highlyconvoluted code-base, few developers behind it, and it requiressignificant knowledge to configure and use. It represents only atheoretical risk.
  • In the worst-case scenario, the alternative is a free,comprehensive, elegant and usable alternative that is quicklybecoming widely known. Your market has been devastated, outside ofselling in hopes that your customers are blissfully unaware (whichis rightfully morally difficult for most people).

If you’re a corporate developer, it could simply be ademotivating signal of time wasted (developers are heavilymotivated by seeing their solutions doing “good” in the world.Dumping code they invested time and passion into isheart-wrenchingly demotivating. “What have you been workingon?” “Well, I spent the last year working on a projectthat was dumped for Microsoft BizPoint 2009 before delivery“);it could just be ammo for the office Machiavellian (I’ve wascriticized in the past by just such a coworker for not leveraging a”possible solution” that wasn’t even invented until twoyears after my project was deployed, so such Machiavellian Jr’saren’t even bound by the constraints of time); or it could bea career killer.

In the corporate space things can get even uglier, as productsfrom the big names like Microsoft will be preferred, howeverill-equipped for the task they actually are: If theymarginally seem to apply to the problem, there will bethose who push for it. Sometimes this strategically makes sense(e.g. if you adopt a new product early, it’ll likely evolve quicklyif it’s developed by a company with the resources that Microsofthas, for instance), but often it’s just corporate hubris. Someorganizations have a love-hate (mostly hate) relationship withtheir IT department, so any opportunity to undermine and reduce thedependency on them, even if it involves a more expensive,less-flexible, less-empowering solution, will be pursued. I workedat one shop where a large team of developers worked for yearsperfecting and expanding in-house solutions, while at the same timethe organization continually and openly quested to replaceeverything they did and were doing with externally sourcedsolutions. The effect on motivation was profoundly negative, but itwas the result of a Better Not Be Invented Herementality, coupled with a negativity opinion of IT.

The Smaller Scale

Of course such an overlap of functionality or use isn’t limitedto whole products. It applies to even the smallest component of ourprojects.

Developers face this sort of issue regularly: From cryptographylibraries to database abstraction tools to MVC frameworks to simplecoding standards – For virtually every single need, there aresolutions that purport to solve all ills and fulfill all needs.Somewhere amidst the 100,000+ version 0.1 projects onsourceforge, thehuge catalog of software and feature lists provided byorganizations like Microsoft and Oracle, or the millions ofprojects on the web, and even amongst the libraries and frameworksalready included with our platform, there are possiblecompetitors.

From a more optimistic perspective, they are possible allies -code and existing research and knowledge that you can leverage tomake your solution better. Why waste time building a dubiousnon-core cryptography function just to enable security, if youcan just use the System.Security.Cryptography collection ofclasses Why re-invent the wheel when a lot of standard solutionsare crystallized in industry patterns?

The problem is that you have to know about the knowledge,competitors and/or allies that you can leverage.

The Problem

Several months back I wrote that internal code reuse can be dangerous, the premisebeing that most teams don’t spend enough time investigating what isalready available (often available in a superior, more widely-knownfashion) – be it coding standards, frameworks, or whole projects, alot of the code base in many organizations is completely redundant,inferior duplications of what already exists, created purely out oflack of awareness. Even when resources are within grasp they’reoften ignored – A huge number of in-house .NET “utilitylibraries” duplicate what exists in the .NET Framework. All becauseno one bothered looking through the admittingly-massive MSDNLibrary before undertaking a task themselves.

While sometimes thisoccurs because of the Not InventedHere syndrome, more frequently it’s simply ignorance -Either intentional ignorance (such as not bothering doing propercompetitor/options research before getting started), orunintentional ignorance (there is such a huge number ofprojects out there, often with convoluted explanations of what theydo, and difficult and time consuming setups to test it outyourself, that it can be impossible to know what already exists.For some small components and projects it can literally be fasterto just do it than it is to investigate the alternatives).

Huge code repositories grow larger with second-rate solutions,products are developed that enter the market dead-on-arrival, anddevelopers have the passion beaten out of them.

The Paralysis

As bad as all of this sounds – making solutions to discover thatthey’re second rate or obsolete, or huge internal code repositoriesfilled with duplications of better alternatives – the problemexists even where these problems aren’t evident (in fact asecondary-effect occurs because of the prevention of these issues):Many software developers are simplydecision-paralyzed. They’re unable to proceed with anyproject because the number of possible standards, components, andframeworks that they need to consider is unending.

This is made worse by the fact that there are tens of thousandsof software products and components making claims that they can’tback up (especially in realm of nebulous, all-empassingapplications. These high level applications often requiretremendous configuration and programming to be usable, but theappearance is that they’re a drop-in-place solution). Sourceforge,for instance, is full of tens of thousands of version 0.1applications, where the purported feature list could be bedescribed as a wish list. Other products entail tremendous expense,or bring along undesired dependencies or complexity.

Researching, and then understanding every caveat and issue withevery potential solution is a huge undertaking.

The Solution

If only it were so easy to simply rattle off a solution.

Nonetheless, a huge step in the right direction would besimpy allocating adequate time upfront for “competitor research”before committing to a project, or even before undertaking acomponent. This is a tremendous undertaking these days,yet in timelines it’s often forgotten or considered satisfied by aquick 5 minute Google search.

If you’re developing a web application to store documents, couldit be satisfied by SharepointCommunityServer Zope A derivativeof MediaWiki If you’ve committed to one of those projects, but youjust need to extend some of the functionality (say you want todisplay the EXIF data of photos), have you committed the time toinvestigate the alternatives?

For virtually every need, there are a slew of theoreticalsolutions. The truth, however, is that many of those solutionsaren’t a good fit – many of them are simply terrible- but you need to be informed and prepared whenpeers, coworkers and clients ask about them in comparison.

Even if you don’t find an appropriate solution alreadyavailable (in most cases you won’t), document all of yourfindings: If Biztalk looks superficially like it’s the rightsolution, but for some reason it isn’t appropriate, be prepared forthe inevitable grilling you’ll get every single time someone whoknows about Biztalk and then hears about your project gets withinearshot.

Be prepared!

Tagged: [], [],[]