glory in data

What I talk about when I talk about data

LONDON, ENGLAND, 1-Aug 1996. A young man tried to get a job in a factory today. Human Resources took one look at his nascent skill set and pointed him towards the sales floor. Had anyone from the department been available for comment they would have said something like: “we need someone to work The Computer.”

That’s how I got my first job in data analysis. There really was only one PC and when I had to go and get raw data I obtained it from a black and green behemoth un-ironically known as The Other Computer. The pile of floppy disks on my desk was generally referred to as The Data as in: “What’s that, Pete?” “The Data, Sir”. Given they’d employed me to look at The Data it is somewhat surprising that’s as far as most conversations got.

With 20 years hindsight, these incurious luddites look ridiculous now, relics of a less enlightened time. Only one computer? Backward! Bliss maybe, but backward nonetheless. Calling a pile of floppy disks a database? How quaint. As for the fact that they had to get a 19-year-old in to talk to The Computer? Ah, bless ‘em. We’re just so much more advanced now.

Or are we?

MANCHESTER, ENGLAND, 1-Aug 2016. A not-so-young man tried to do his job today. His MD collared him first thing about speed of The Database. A spokesman for the company was quoted as saying: “Pete just needs to fix the f***ing thing.”

It is tempting to think of today’s technology workers as the cats who got the cream, the geeks who inherited the earth. Whilst I don’t dispute this statement, it covers up a deeper truth. Replace that green and black behemoth with The Cloud (or, as likely at time of writing, The Server) and those floppy disks with any of The System, The KPIs or even plain old The Database. For The Data, read The Data. Now ask yourself: just how much more advanced are we now?

The reason the sales floor wanted me back then was because they had very specific questions that they suspected they could answer with data. Note the chronology here – they already had the questions – inspired by real life, and set in stone. The notion that data could maybe solve them was very much an afterthought. Nowadays the opposite is more often true: we have the data but we don’t know what questions to ask of it. This is a deep and fundamental change in the way businesses – all businesses – are run.

Back in 1996, what happened next is telling: I looked at the data and painted some future scenarios. The questions from the sales floor went from “tell me this?” to “what if we changed that bit there?”. The data became part of the conversation. Part of the conversation. The four vastly experienced sales guys were the other vital parts.

When I talk about data I always hark back to those other vital components. The best databases in a business are often stored in brains, not bytes. The Data is the whole darn shootin’ match: the storage, the system, the sympatico with those who know and care the most. And the whole darn shootin’ match is what I talk about when I talk about data.

Follow me

Featured post

From student to professional in precisely one step

I originally intended this post for data-practitioners, their bosses and those who are both. Based on the small but noisy response it got, I now know I was wrong: this applies to any erstwhile student who has ever managed or been managed. That’s quite a lot more people. Enjoy.

(Original title: A-Level business data: a contradiction in terms)

Continue reading “From student to professional in precisely one step”

The glory years of Excel (and how to recreate them)

This is a blog post for those who want to “wow” people with computers.  The star of this blog is the Excel Macro.  These two statements are not contradictory.

London was a good place to be in the summer of 1995.  The weather was hot, for ages, a specific kind of heat faceted by not only temperature but music, too.  The heatwave actually sounded like Blur, Oasis and, erm, The Boo Radley’s.  Cool Britannia was not only where we lived but how we lived it.

Best of all, though, computers were still scary.

At this remove, this needs some illustration. Believe it or not there was a time before the ubiquity of the PC was an inevitability.  The computer was some twisted entity designed by mad scientists to (supposedly!) make you more efficient.  Untrusted. When the technologists of Generation X entered the working world, this was what we found.  A world where changing a font was considered novel, the SUM functionality miraculous; as for PowerPoint, heads verily boom-squelched.  A member of the Excel cognoscenti could, back then, look way better than they actually were.  This was brilliant.  This was way cooler than Britpop. This was the nineties: the place where you could put in the least effort for most reward.

Time, alas, moved on.  Computers became less scary; experience normalised, skills democratised.

For shame!

At some point between 1995 and 2015, everyone got good.  Excel in particular became, ugh, common.  That which gave me my place amongst the cognoscenti became the norm.  There is no such thing as a normal cognoscenti, you’re just normal.  The very thing that had happened to computers had happened to…me!

For double shame!

Thank God, then, for the Excel Macro.  Impervious to the vicissitudes of progress, democratisation and education, it has just lurked in the background, unused, an ignored menu item.  Untrusted. Even those that love Excel, stay clear of the dreaded Macro.  Somehow, the Macro stayed in the nineties.  Which is brilliant.  Better than Britpop.

My advice then, is simple.  If you want to find a place where you can wow people with computers.  Open Excel (it doesn’t matter which version).  Stick Definitely Maybe on your headphones.  Learn how to code macros properly and party like it’s 1995.  This is the Excel Macro: the place where you can put in least effort for most reward.

Every Generation X-ers dream.


PS Learning to code macros is relatively easy and there are loads of great online resources to help you, be warned though, supporting an Excel Macro can be a job for life, even harder than supporting Excel formulae, and this is not only hard, this is hardcore.


(What’s the story) macro glory

This is a blog for Excel-loving, technical historians who secretly wish it was 1995 – and almost no-one else.

The Excel Macro.  On a literal level, it is a relic.  Microsoft stopped supporting an ancient (but great!) programming language called “Visual Basic 6” (VB6) sometime in the mid-noughties, usurping it with the far classier, future-busting, “Visual Basic Dot Net” (VB.Net).  Nowadays it is nigh impossible to buy the VB6 package without embarking on something of a treasure hunt (I got mine off eBay).

Standing in the face of this neomania is a noble language called “Visual Basic for Applications” (VBA), a sort of little brother to VB6.  Like all little brothers, though, VBA bided its time, waiting in the wings for its shot at the champion.  Like some little brothers, it got its way in the end.  Where VB6 might now be considered Palaeolithic, VBA lives on.  With each new release of Excel or Word, with each degree of modernisation, there it sits in the background, an ancient parasite in perfect symbiosis with its host.

When we write an Excel Macro, we use VBA.  The future never needed to intrude.  To my knowledge there is almost no clamour for this to change.  Based on simple population heuristics* there are (probably) more individual pieces of code stored in Excel macros than in any other language.  This means that the most common coding platform in the world was made in the nineties.

It changed by not changing at all.  The Excel Macro kept it real.



*Excel is used by most of the world’s office denizens, this leads to loads of spreadsheets being created, the proportion of them containing macros represents a minority but a significant one.  That is, there are simply loads of macros out there.

Excel is hardcore

To the casual onlooker all I have done is help with a formula in Excel.  In-house things are different.  We know: this is life changing.  Spreadsheets are faster.  Reports are smarter.  Truth is nearer.

This is part of who I am: the Excel Guru. Every office has one because every office needs one. This is inevitable.  Here’s why:

Every office uses Excel, every Excel User writes formulae.  These formulae are often designed for repeat interrogation.  The Excel User might be blissfully unaware of it, but this is programming.  The Excel User is (for them mercifully) not a programmer therefore they inevitably reach the point of their programming incompetence.  At this point the Excel Guru gets called.  This has to happen.

Before I go on, I should state that it is in no way my intention to denigrate the Excel User, the quality of a spreadsheet is in the vision, not the formulae.  Indeed, most serious Excel Users are good enough to create all but the most taxing of formulae.  This results in two curious side-effects:

  1. The call for Excel Guru is usually not planned (because the Excel User can create nearly all of the formulae they need)
  2. The Excel Guru is an entirely ephemeral being: they exist from the moment they are called to the moment they return to their desk.

To my desk is where I go next.  The finance team chink their tea-cups to the sound of my name.  I re-join the development team to rapturous applause.  I garner envious glances from the other desks.  They want to be me.  I suppress a grin, try to look humble.  This is tough to do for – at this exact moment in time – I am glory.

This glory fades fast.  Yes, I helped, yes, it made me feel good about myself, and, yes, I got the job done.  I always get the job done.  For this reason the euphoria dissipates to be replaced by a distinct nausea.  I have been here before.  For all the plaudits I know what I have truly done.  Nausea itself dissipates and all I feel is a sense of deep foreboding, of impending doom.  This is not far-fetched, I develop software and this means I know what I’ve done, or, more to the point, what I haven’t.

The first thing I haven’t done is planned for the project (this is rendered impossible by 1 above).   Instead I have jumped straight into the live code.  At the Excel User’s behest I have made changes there and then.  This is my second problem: if I were planning the project I would have to consider all of the potential users, not just this one, but the nature of desk side support negates this.  My third omission is at the point of release.  I code in haste say something like “that okay?” and beat a hasty retreat.  No alpha- let alone beta-testing. As for showcasing the “product”, well, without planning we have no definition, without definition the idea of a product is, erm, nebulous (indeed it is nebulous by definition!).

These three things, however, are trifling in comparison to the final omission: my code is live and editable.  My lovingly-but-hastily-prepared code is out there, live, today.  Not only that, it is infinitely changeable.   I have not locked down my code, not been allowed to.  The Excel User can just change it, just like that.  Imagine!  Something worse yet lies in wait: when this spreadsheet is finished it will be sent to others, in the same free-for-all format.  They can do anything with it.  “It” being my code!

To recap: no planning, no user story, no testing, the code raw and exposed.  Foreboding.  Doom.

Lured by the euphoria of using one’s knowledge to help others, the Excel Guru quickly and repeatedly descends into this pit of despair.  The crack cocaine of development jobs.  This is why Excel remains the most hardcore language for any programmer: we do it for the buzz not for the future.

PS If you really want to support your spreadsheets for life, introduce a wonderful macro.

Create a free website or blog at

Up ↑