Sleepless Nights for Software Developers

A recent Ask HN raised the question “What do you use Machine Learning for?“, and the top answer, by ashark, is golden-

I use it as something to worry about not knowing how to use, and how that might make me unemployable in a few years, while also having no obvious need for it at all, and therefore no easy avenue towards learning it, especially since it requires math skills which have completely rusted over because I also never need those, so I’d have to start like 2 levels down the ladder to work my way back up to it.

I’ve found it very effective in the role of “source of general, constant, low-level anxiety”.

This is an accurate assessment for most of us. We anxiously watch emerging technologies, patterns and practices, trying to decide where to focus some of our limited time. Worried about missing something and finding ourselves lost in the past with a set of obsolete skills. So we endlessly watch the horizon, trying to separate the mirages from the actual, deciding what to dive into.

I recently started casually diving into TensorFlow / machine learning. TensorFlow is the machine learning toolset released by Google, and represents the edge of the current hype field (ousting the industry trying to fit all problems into blockchain-shaped solutions, which itself relegated nosql to the trough of disillusionment). It’s just layers and layers of Things I Don’t Know.

GRPC. Bazel. SciPy. Even my Python skills are somewhat rusty. Most of the maths I’m still fairly adept at, and have a handle on CUDA and the Intel MKL, but getting TensorFlow to build on Windows, itself a remarkably painful process (at least in my experience, where having VS2017 and VS2015 on the same machine is a recipe for troubles, and while you can just install the binaries I am a fan of working with a build to allow me to dive into specific operations), yields such an enormous base of dependencies that it gives the feeling of operating some enormously complex piece of machinery. It was much easier to build on Ubuntu, but still represents layers and layers of enormously complex code and systems.

It’s intimidating. It’s the constant low-level stress that we all endure.

And the truth is that the overwhelming majority of tasks in the field will never need to directly use something like Tensorflow. They might use a wrapped, pre-trained and engineered model for a specific task, but most of us will never yield payoff for an in-depth knowledge of such a tool, beyond satisfying intellectual curiosity.

But we just don’t know. So we stay awake at night browsing through the tech sites trying to figure out the things we don’t currently know.

EDIT: I should add the disclaimer that I’m not actually losing sleep over this. I’m actually calm as a Hindu cow regarding technology, and probably am a little too enthused about new things to learn. But it still presents an enormous quandary when my son asks, for instance, what he should build a service layer for a game he’s building. “Well….”