Filler Post

All has been quiet on here lately during a busy professional and personal time. Just a quick update that everything is great, I certainly haven’t abandoned this, and hope to have some interesting updates very soon.

Technology and Population Density Trends

A bit of a rambling, conversational piece today.

Amara’s Law states-

We tend to overestimate the effect of a technology in the short run, and underestimate the effect in the long run.

We declare that a technology changes everything, realize that entrenched patterns/behaviors and small hangups limit broad adoption. We discount it as over-hyped and not as significant as expected. It quietly takes over and changes the very foundations of society and our lives.

Recently I was pondering what electric cars and self-driving cars would do to population density. The former — using mechanically simpler vehicles with a much less expensive energy source — will significantly reduce the cost of driving as it achieves the economy of scale, while the latter will reduce the inconvenience of driving, commuting in particular.

Self-driving cars will not only allow us to do other things during the ride, it will significantly increase the capacity of our roads to handle traffic by reducing human error and inefficiencies.

Intuitively, at least to me, these changes will propagate lower density living. That home that previously would have been an expensive, three-hour commute becomes a relaxing period to watch a Netflix series or catch up on emails.

Considering the probable social change of self-driving EVs led me to consider the changes over the past several decades. In Canada, as an example, lower density areas of the country — the Atlantic provinces, rural areas, small towns and villages — are hollowing out. The high density areas, such as the Golden Horseshoe around Toronto, is a magnetic draw for all of Canada and continues growing at a blistering pace.

Even if a home in the Toronto area costs 5x the price for a given set of amenities, and even if a hypothetical person might prefer lower density, many forces still draw them in.

Which is strange, in a way. I grew up in a small city and seemed to be completely isolated from the larger world. Calling a relative 20 minutes down the road necessitated long distance. My town had no computer store, a mediocre BBS, few channels on television, no radio station, etc. There were few resources for learning.

I was wide eyed at the options available in the big city.

Yet today we live in a world where that same small town has inexpensive 100Mbps internet, and can communicate with anyone over the globe in an instant. Where you can order just about anything and have it the next day, or even the same day. Every form of entertainment is available. Every resource and learning tool is a couple of clicks away (aside — education is one area that has yet to see the coming change from the new world). Few of the benefits of the density are missing.

But those same changes led to centralization, and a hollowing out of most of the better jobs, entailing the workforce having to follow.

We centralized government and administration, pulling the school boards and government offices, banks, etc, out of those small towns in the quest for efficiency, moving up the density ladder. Those five small villages amalgamated to a single board, that then got pulled into a larger board in the city an hour up the road, etc. Connectivity means that management for the few remaining auspices of structure can be at a far flung location.

Every medical specialty was moved to larger centers as the ownership of cars became prevalent, and long drives were accepted. Seeing a pediatrician 200km away becomes a simple norm. Service and even retailing gets centralized to some unknown place elsewhere on the globe.

Everything centralizes. Because it can.

Most decent jobs require a move to density. The same forces that gave the convenience of the city in far flung locations also relegated it to being essentially a retirement home.

Reconsidering the probable change of EVs and self-driving cars will likely accelerate that migration.

Updates: Pay Apps / Date-Changing Posts / Random Projects

Recently a couple of readers noticed that some posts seemed to be reposted with contemporary dates. The explanation might be broadly interesting, so here goes.

I host this site on Amazon’s AWS, as that’s where I’ve done a lot of professional work, I trust the platform, etc. It’s just a personal blog so I actually host it on spot instances — instances that are bid upon and can be terminated at any moment — and there was a dramatic spike late in the week on the pricing of c3 instances, exceeding my bid maximum. My instance was terminated with extreme prejudice. I still had the EBS volume, and could easily have replicated the data on the new instance for zero data loss (just a small period of unavailability), however I was just heading out so I just ramped up an AMI image that I’d previously saved, posted a couple of the lost posts from Google cache text, and let it be. Apologies.

Revisiting Gallus

Readers know I worked for a while on a speculative app called Gallus — a gyroscope-stabilized video solution with a long litany of additional features. Gallus came close to being sold as a complete technology twice, and was the source of an unending amount of stress.

Anyways, recently wanted a challenge of frame-v-frame image stabilization and achieved some fantastic results, motivated by my Galaxy S8 that features OIS (which it provides no developer accessible metrics upon), but given the short range of in-camera OIS it can yield an imperfect result. The idea with be a combination of EIS and OIS, and the result of that development benefits everything. I rolled it into Gallus to augment the existing gyroscope feature, coupling both for fantastic results (it gets rid of the odd gyro mistiming issue, but still has the benefit that it fully stabilizes with highly dynamic and complex scenes). Previously I pursued purely a big pop outcome — I only wanted a tech purchase, coming perilously close — but this time it’s much more sedate in my life and my hope is relaxed. Nonetheless it will return as a pay app, with a dramatically simplified and optimized API. I am considering restricting it only to devices I directly test on first hand. If there are zero or a dozen installs that’s fine, as it’s a much different approach and expectation.

Project with my Son

Another project approaching release is novelty app with my son, primarily to acclimate him to “team” working with git. Again expectations are amazingly low and it’s just for fun, but might make for the source of some content.

Social Anxiety and the Software Developer

A brief little diversionary piece that I hope will prove useful for someone out there, either in identifying their own situation, or in understanding it in others. This is a very selfish piece — me me me — but I hope the intent can be seen in a good light. I suspect that the software development field draws in a lot suffering social anxiety.

This piece is in the spirit of talking openly and honestly about mental health, which is something that we as a community and a society don’t do enough.

A couple of months ago I endured (and caused others to endure) a high stress event. I certainly haven’t tried to strike it from memory (the internet never forgets), and in many ways a lot of positives have come from it and it has been a profound period of personal growth since.

One positive is that I finally faced a lifelong burden of social anxiety, both pharmacologically and behaviorally, a big part being simply realizing that it was a significant problem. I know from emails to my previous mention of enduring this that it struck some readers as perplexing: I’ve worked in executive, lead, and senior positions at a number of organizations. I have a domain under my own name and put myself out there all the time1. I’m seemingly very self-confident, if not approaching arrogance at times.

That isn’t just a facade: I am very confident in my ability to face an intellectual or technical challenge and defeat it. In the right situation I am forceful with my perspective (not because it’s an opinion strongly held, but because I think it’s right, but will effortlessly abandon it when convinced otherwise).

Confidence isn’t a solution to social anxiety, however. It’s possible if not probable for them to live in excess alongside each other. In many ways I think an bloated ego is a prerequisite.

Many choices — as trivial as walking the dog — were made under the umbrella of avoiding interactions. Jobs were avoided if they had a multi-step recruitment process. Investments were shunned if they weren’t a singular solution to everything, and even then I would avoid the interactions necessary to get to a resolution.

I succeeded in career and personally entirely in spite of these handicaps, purely on the back of lucking into a skillset at a perfect time in history. I am utterly convinced that at any other time in history this would have been devastating to any success. Be good at something and people overlook a lot.

And it was normalized. One of the things about this reflective period is that suddenly many of the people who I know and love realized “Hey, that was pretty strange…” It seemed like a quirk or like being shy (which we often treat as a desirable trait), but in reality it was debilitating, and had been from my formative years.

There are treatments for it. I’m two months into this new perspective and I can say that the results are overwhelming. I will never be a gregarious extrovert, but life is so much less stressful just living without dreading encountering a neighbour, or getting a phone call, etc.

1 – The online existence is almost abstract to me, and I’ve always kept it that way. I have always dreaded people who I know in “real life” visiting this blog (sometimes family or coworkers have mentioned a piece and it has made me go silent for months, hoping to lose their interest), reading any article I’ve written or anything written about me, etc. That is too real, and was deeply uncomfortable to me. Nonetheless there have been times I’ve realized I said something in error and a cold sweat overcomes me, changing all plans to get to a workstation and fix the error.

CMOS and Rolling Shutter Artifacts / Double Precision Compute

EDIT (2017-03-01) – It’s been a bit quiet on here, but that will change soon as some hopefully compelling posts are finished. I’ve taken a couple of weeks to get into a daily fitness and relaxation routine (I would call it a meditation if that term wasn’t loaded with so much baggage), organize life better, etc. Then it’s back to 100% again with these new habits and behaviors.

While I finish up a more interesting CS-style post (SIMD in financial calculations across a variety of programming languages), just a couple of interesting news items I thought worth sharing.

In a prior entry on optical versus electronic image stabilization I noted rolling shutter artifacts –an image distortion where moving subjects or the entire frame during motion can be skewed and distorted — and their negative impact on electronic stabilization.

During video capture, especially under less than ideal conditions, it is a significant cause of distortion that often goes unnoticed until you stabilize the frames.

Sony announced a CMOS sensor with a data buffering layer that allows it to have something approximating a global shutter (Canon previously announced something similar). While their press release focuses on moving subjects in stabilized type situations, the same benefit dramatically reduces the rolling shutter skew during motion of video capture. It also offers some high speed capture options which is enticing.

Sony sensors are used by almost every mobile device now, so it’ll likely see very rapid adoption across many vendors.

EDIT: Sony is already showing off a device with a memory-layer equipped CMOS, so it’s going to become prevalent quickly.

Another bit of compelling news for the week is the upcoming release of the GP100, Pascal-based workstation GPU/compute device by nvidia (they previously released the P100 server based devices).

Double-precision calculations of upwards of 5 TeraFLOPS (for comparison a 72-core/AVX-512 Knights Landing Xeon Phi 7290 offers about 3.4 TeraFLOPS of DP performance, while any traditional processor will be some two magnitudes lower even when leveraging full SIMD such as AVX2). Traditionally these workstation cards massively compromised double precision calculations, so this update brings it into much greater utility for a wide array of uses (notably the financial and scientific world where the significant digits of single precision made it unworkable).