After hanging my shingle in the independent consulting business — trying to pay the bills while exploring entrepreneurial efforts – one of my original specialties was high performance database systems.
I pitched my expertise in large-scale database systems, providing solutions that would often dramatically improve operational performance and relieve pain points. Just as a facet of my network of contacts, this was usually in the financial industry.
It made for relatively easy work: Many orgs don’t understand how to properly leverage their database products, their data needs, or their query profile, so it was incredibly easy to lazily promise at least a 50x performance improvement in some particular pain point, on existing hardware, and achieve that and more with ease.
I started backing away from the database world, however, because many database products have dramatically improved and reduced the ability to shoot one’s own feet, but more significantly hardware has exponentially improved.
From massive memory, caching the entirety of all but the most enormous databases, to flash storage (and now Optane storage) that makes the most inefficient of access patterns still effectively instant. From a big RAID array that could barely manage 1000 IOPS, on a platform with maybe 32GB of RAM, to monster servers with many millions of IOPS and sometimes TBs of very high speed memory.
It has gotten hard to build a poorly performance database system, at least at the scale that many firms operate at. Expert implementations can still yield significant speedups, but to many potential customers a 100ms query is effectively identical to a 1ms query, with low enough loading that hardware saturation isn’t a concern.
Egregious inefficiency is operable. Which is a great thing in many cases, and is similar to our move to less efficient runtimes and platforms.
The possible market for database optimization just isn’t as large or as lucrative. And when things are awry on that million IOPS system with 2TB of RAM and 96 cores, it’s usually so catastrophically broken that it isn’t really a fun pursuit, usually necessitating a significant rebuild.
For the past half year I’ve moved primarily to cryptocurrencies and their related technologies. Not chasing a “whale”, or hoping to ride a bubble to great riches, but rather becoming completely knowledgeable of every emergent standard, and of the currently dominant platforms and their source code, and offering that knowledge to individuals and firms looking to integrate, to leverage, or to bet.
All the while I continue with the entrepreneurial efforts. My latest is with the iOS ARKit, which has been a fun experience. I should announce something on that soon.
Otherwise I continue with fun projects. I still have a contour mapping application on the go. It started as a real estate sales project, but that went so quickly to great success that it became just an interesting project to explore Kotlin, undertow, targeting iOS and Android and then learning about pressure propagation, etc. Then WebGL mapping, millisecond time synchronization (I use four current smartphones in concert), etc. I have a mostly complete posting on that, long promised, that I’ll finish soon.
 – I hit some financial roadblocks, leading to a crisis, when I put aside all revenue production that try to focus entirely on entrepreneurial efforts that seemed, for so long, on the absolute cusp of success. Lessons learned.