The Brutal Winter of 2013-2014. Data and discovery.

Spring is a welcome respite after a brutal winter here in Southern Ontario.

IMG_20131222_092844

The frigid cold came early, hitting us with -23C (-10F) temperatures before the season had officially begun. It blasted us with a destructive ice storm that wreaked havoc on power grids, causing multi-day power outages and ruined holiday seasons.

It caused massive tree damage that is still being cleared: I’ve become a bit of an expert with the chainsaw, and have barely dented the workload before me.

The snow kept coming, the base layer of ice making every trek dangerous. A normal Southern Ontario winter sees intermittent periods of warmth that clears the slate for the next snow fall, but that never happened this year.

It just kept accumulating.

Our cars were stuck multiple times, each yielding frustrating multi-hour recovery efforts. Do you know where the recovery hooks are on your vehicle, or if it even has them? I didn’t know the answer to these questions before this winter, but I do now.

Of course complaining about the weather is something that people are prone to do, but as someone who historically loves the winter season, this outing soured me on the whole concept of a cold season. Even bitter cold is wonderful on occasion, but it is intolerable when it lasts for weeks.

It is the first time I’ve questioned the geography of where I live.

In any case, in putting together a solution for a client recently I needed to make some simple demonstrations with easily available data (a similar motivation yielded the oft emulated Interesting Facts about Domain Names entry years ago), and investigating the season’s weather was an interesting target.

I was curious if it really was that abnormal: In most prior years I’ve held -20C as a magical mark of “extraordinarily cold”, but this year that mark seemed to be hit with regularity.

So I went searching on Environment Canada. They do indeed have data available, even if they make it a little harder to use in bulk than necessary, but what is disheartening is the lack of quality of the data.

Stations go in and out of service frequently, and it’s less than ideal to compare different stations given natural variations. Even sticking with singular stations, however, there are significant gaps of data from all of them. Weeks of data simply vanished. In some cases only some of the data is available.

Surely this is a funding issue, and I suppose with the aggregate of hundreds of stations such gaps can be smoothed out, but for someone trying to use the data it makes it more of a pain than it should be.

In any case, I wanted data that was representative of Southern Ontario, and after evaluating many stations across the region, I settled on Roseville (4816). To fill in missing data, I used a synthesized set of data from two nearby stations in the affected periods, Elora (41983) and Waterloo Wellington A (4832).

Vertica and Pentaho Kettle made quick work of getting the data into a workable form, doing any data clean-up and discovery, and extracting data.

The data still is far from perfect — in many cases the high/low range falls significantly outside of the preceding day’s high/low range, which seems impossible — but it demonstrate trends and comparative results well.

D3 and jQuery made it a trivial task to visualize the data.

In any case, for those who want to visualize exactly what the daily temperature ranges looked like in Southern Ontario this winter (where winter is broadly specified as December 1st to March 31st of the following year), and those for decades before, the following tool makes the task simple. Note that for it to function your browser needs to support SVG, as all modern browsers do.

Note: The interactive temperature chart is also available at http://dennisforbes.ca/wp-content/uploads/weather/index.html. It will likely be non-functional if you are reading this with an RSS reader.





And on the magically cold value of -20C, 2013 had 25 days (between December 1st, 2013 to March 31st, 2014) below -20C. 1993 had 21 such days, while the closest recent year was 2004 with 12 days hitting that magically cold mark.

2012 had 5 days lower than -20C. 2011 had 0. The relative mildness of recent winters — versus the similar brutality of winters such as 1993-1994 — made it feel that much worse.