Lies, Damn Lies, and surveys

Survey Results are Meaningless minus the Methodology

If you’re in the survey (or “Marketing Intelligence”) business,an easy way to gain lots of free publicity is to publish someresults that feed into an ongoing skirmish: For little cost a pressrelease will percolate across the web, bubbling up everywhere,spreading your brand. In all likelihood a newswire writer will havethe same motivation, and carefully pick and choose factlets withthe goal of making a story more inflammatory.

Given that smartphones are big business now, and the battle(RIM, Nokia, Apple and Google’s hordes) is a very public one,smartphones and their operating systems are often the focus.

We’ve seen this play out quite a few times already: Some surveyresults come out, usually with a very pro-Apple slant (the Appleenthusiast ecosystem is far more receptive to such material, and ismore likely to run with it), and is quoted everywhere.

As always, methodology and details actually come second. It’sthen that critical, mind-blowing flaws in the survey or itspresentation/interpretation are revealed. Of course by then themessage was already delivered and has become truth through repeatedassertion.

So for all you market research startups out there, I encourageyou to post a self-selective survey on the website of an alreadybiased group where they can profess themselves as wealthy, virile,mega-sexed super-humans. Release these extraordinary findings viapress release and watch as lazy bloggers and tech columnists runwith it.

Remember thisridiculous survey? It was the harbinger of doom for Android.Only it wasn’t. It is only one of a long line of terrible surveys,and only one example where the tech press and wire writersmisrepresented the already poor survey, or at a minimum didn’tbother asking the questions that matter.

So we’ll wait for GfK’s methodology (never expect techjournalists to care to look this up. As an industry it is rife withincredible ignorance and laziness), and the inevitable correctionwill be a whimper to the roar of the original release. At a glanceit is obvious that all is not as it appears, and it isn’t simplythat the numbers don’t confirm my bias: They just don’t add up, andhave little correlation with other surveys on the same topic.

RIM Isn’t Dead

Many years back I applied to RIM. They never contacted me.Clearly that’s the source of all of their travails today.

Really, though, I await great things from RIM: The company hadsome golden handcuffs to their existing platform — the lucrativeenterprise messenger system undermined more innovative initiativesin the same way that Windows and Office hobbles Microsoft — but nowthat they’re seeing some serious attacks that can’t be ignored, thebattle is underway again. The PlayBook looks like it may be awinner, and I’m willing to sandpaper my fingers to use that 7″tablet if it delivers what it promises. QNX is simply a gloriousoperating system (it is elegant and close to perfect), sothat alone has me interested.