A scratch pad is not a system

The spreadsheet is a brilliant scratch pad, a dangerous system, and worth a great deal less the day its author leaves.

Every office has a Pete the Sheet. Pete built the sheet. Nobody remembers when, and Pete, if asked, will say "ages ago" and change the subject. It plans the job, or prices it, or tracks it, and it has done so for longer than anyone in the room has worked there. It has tabs called OLD, OLD (2) and DO NOT USE, the last of which is used every day. It has a column that turns red for reasons that were explained once, in a meeting, in 2011. And somewhere near the bottom right there is a cell that everyone agrees is the answer. Nobody agrees why.

The file is called Plan_FINAL_v7_PETE_revised_USE_THIS_ONE.xlsx. There is also a v8.

Pete is not the problem. Pete is a legend. He asked the right question with the only tool he had. The problem is that forty years later we are still asking it with the same tool, and Pete has taken the redundancy package twice. Both times he was hired straight back as a consultant, mainly to explain the spreadsheet. Paid to go, paid to come back, and all of it thanks to one file. It is the best investment the company ever made, just not for the company. Last anyone heard, he was running a beach bar in Mexico, and taking calls about column F.

What it was built for

The spreadsheet began as a ledger. VisiCalc, in 1979, put an accountant's paper grid on a screen and let the totals redo themselves when a number changed. It was a marvel. People bought a computer just to run it, which tells you everything about the state of arithmetic on paper. Multiplan and then Excel followed from Microsoft, and the grid conquered the world.

It deserved to. For adding up, for trying a number and seeing what happens, for thinking on a Tuesday afternoon, nothing beats it. It is a superb scratch pad: the back of a cigarette packet, for professionals.

A brief word about the man behind it, because he is more interesting than the spreadsheet. Charles Simonyi is not the most famous name at Microsoft, but he may be one of the most important. He left Hungary as a teenager, helped build one of the first what-you-see-is-what-you-get word processors at Xerox PARC, then ran the group at Microsoft that made Multiplan, Word and Excel. A naming style for computer code is still called Hungarian notation after him. He went on to spend some of the proceeds visiting the International Space Station. Twice. He is still the only space tourist to have gone back. For years he also owned Skat, a 71-metre yacht painted battleship grey with a number on the hull like a warship's. The number was only the designer's project reference. It still looked as though it had come to invade somewhere. None of this matters to the argument. It is just nice to know that the spreadsheet's family tree includes a man who has been to space, which is more than can be said for most of the spreadsheets.

But look at why it won. It asks nothing of you. A database wants to know, before you start, what a thing actually is: a date, a depth, a node, a name. A program wants to be told the rules. The spreadsheet wants nothing at all. Type anywhere. Type anything. Merge a few cells, colour one yellow, and call it a system. It is the only thing in the office that has never once said no.

That is its genius, and that is the trouble. The spreadsheet is the path of least resistance, which is an excellent route for water and a poor one for evidence. It lets you skip the dull, difficult bit, deciding what your data actually is, and it lets you skip it forever. Forty-odd years of spreadsheet history is mostly the story of busy people reaching for the nearest tool, which is about as human as it gets. We have all done it. Some of us are doing it right now, in another window.

But it was never built to be software. Nobody designed it to hold the official record of a hundred-million-pound operation, and nobody designed it to be trusted by people who did not build it. Out of the box it has no bounds: any cell can hold anything, and nothing stops it. It guesses what you meant: a gene name becomes a date, a code becomes a number, a leading zero disappears. Nobody tests it. It keeps no useful record of who changed what, and none at all of why. It is a free-form table maker, and a very good one. It is not even very good at graphs, which for a tool made of numbers is a bit like a chef who can't do toast.

The world runs on it anyway

Somewhere offshore right now, a supertanker the length of three football pitches is pulling alongside to load a couple of million barrels of oil, and the schedule it is keeping is a spreadsheet, kept by hand. Plans, reports, daily logs, official returns: spreadsheets. Each company has its own Pete the Sheet, each with his own sheet, and none of the sheets talk to each other. Every new one is another hat thrown into a ring that already contains every hat ever made.

And often it is not even a new hat. For thirty years the same handful of spreadsheets have been passed around the industry as canon, copied from job to job and company to company: the production estimator, the battery calculator, the downtime tracker, each one somebody's Pete. In that time the vessels have changed, the nodes have changed, the positioning has changed and the computers have changed beyond recognition. The sheets have not. There are sheets in daily use today with columns for laser positioning kit that nobody has fitted in years, and nobody deletes them, in case column M turns out to matter. They have not evolved; they have accumulated, like barnacles, or management.

This would be harmless if spreadsheets were reliable enough. They are not, and the record is public.

1. In 2010 two eminent economists published a finding that growth stalls once public debt passes 90% of GDP. It shaped arguments for austerity. In 2013 a graduate student asked for the spreadsheet and found that rows had been left out of an average. With them put back, the cliff disappeared.
2. In 2012 a bank lost billions on a trading position its risk model had told it was safe. Part of the model was a spreadsheet, and a formula in it divided by a sum where it should have divided by an average, halving the risk on paper.
3. In 2020 England's test-and-trace system dropped nearly sixteen thousand positive cases from its daily figures for about a week, because the results passed through a file format that could only hold so many rows. The rest simply fell off the bottom.
4. The same year, the committee that names human genes renamed about two dozen of them, because the spreadsheet kept turning names such as MARCH1 into dates. Studies had found the damage in roughly one published gene list in five.

None of these people were fools. That is the point. Carelessness can be trained out of a person; faith in a grid, it seems, survives even a professorship. The tool fails quietly and politely, with no error message, and it fails the careful as happily as the careless. A sum that leaves out five rows looks exactly like a sum that doesn't.

Code earns trust. Spreadsheets borrow it.

Real software is slow to become reliable. It is written, read by someone else, tested against cases that should break it, fixed, versioned, and tested again. The people who do this are unglamorous and they are heroes: they are why a bridge's sums or an aircraft's autopilot can be trusted by people who will never meet them.

Most spreadsheets go through none of it. They are rarely finished and almost never checked. Their formulas are hidden in cells, their logic spread across tabs, their macros understood by one person, if that. They are trusted because they are old, or because they are Pete's, or because they have a nice border round them, not because anyone has shown that they are right. They borrow a trust they never earned, and they never pay it back.

Filling in a number is not quality control. Typing today's figure into the cell that turns green checks nothing; it just makes the cell green. Streamer separations get reported for every line, every day, as text: pages of it, faithfully typed. You cannot see what any of it means until you look at the coverage, and nobody has ever drawn a coverage map from a paragraph. And rolling up on a job with a heavily macro'd spreadsheet, feeling rather pleased with yourself, is not peak engineering. We have all met that person. Some of us have been that person.

And in marine operations nothing stands alone. The weather drives the vessel, the vessel drives the lay, the lay drives the batteries, the batteries drive the schedule, and the schedule drives the bill. In a spreadsheet, that whole web of cause and effect comes down to cell DE4 quietly depending on a pattern in column D of a hidden sheet called Sheet3 (2). Nobody can see the web. Change one number and something three tabs away changes too, without a word. Or, worse, it doesn't.

And then it sits there

Here is the part we do not say out loud. Once the job is done, the spreadsheet is rarely opened again. Why would anyone open it? It has no context. Without the person who built it, it is a tessellation of numbers: rows that once meant something to someone who has since moved on.

Even Pete, handed it back a year later, would struggle. The meaning is smeared across columns I and J, a lookup on another tab and a note in a cell comment, and nothing says which number belongs with which. What was column A again? Nobody knows. Possibly nobody ever did.

The client still gets the seismic data, which is what they asked for. What they don't get is the operational story of how it was acquired: the downtime, the decisions, the near misses, what cost what and why. That stays behind in a grid nobody can read, and the next job starts from scratch. The knowledge did not stay with the company. It left in the same taxi as the people who had it.

And on the rare day somebody does need it, a dispute, a claim, the next job in the same water, Pete has already left. He always has. (See: Mexico.) In the whole history of the industry, nobody has ever found Pete at his desk on the day the question came up.

Be honest about what the day looks like, because we have all done it. Read a number off a PDF. Type it into a cell. Next number. After the first hour you start to wonder whether you are paying attention at all, which is awkward, because paying attention is the job. Monkey see, monkey do, at a professional day rate, and at the end of it a grid with some bold in it. A day spent thinking costs the same and leaves an asset behind. This one leaves a file. You have spent your money on nothing, and filed it carefully.

The questions we cannot answer

Ask any operation three questions. Are we learning? Is this getting cheaper? Are we improving?

With the evidence scattered across a thousand sheets, nobody can say. So we do what people always do with too little data: we fit the story to the points we have, while knowing there are many more to be had. We tell ourselves we are improving. We may be treading water and guessing, with conviction.

The answer needs every job, every reading, every change, kept in one place with its meaning attached, and comparable with the last job and the one before. That is not a spreadsheet. It is a record. What a proper record looks like is another essay.

No person, however good, reads hundreds of data streams at once and checks every hour of every one. Software can, and a person can then put their name to what it finds. That is not a threat to anyone's judgement. It is the first time their judgement has had the whole picture to work with.

Better tools do not replace the consultant; they set them free. Nobody spent thirty years offshore to log the nominal readings. The value was always in the other part: spotting the thing that looks wrong, asking why, digging into the anomaly with everything you know. Hand the logging to something that never gets bored, and the consultant finally gets to spend the day rate on the part of the job that needed a person.

This is not a small step. It is the first real change in how the work gets done in a generation, and it arrived while we were formatting cells. The talent was always there; the industry is full of people who can read a survey like a book. It has just been like watching a world-class cyclist spend twenty years racing on the shop's delivery bike, basket and all, on a haematocrit so low the team doctor would send him home. Give him a proper bike, and some iron, and see what happens.

This was fixable years ago

For a long time the choice was honest: hire an expensive programmer for a year, or open Excel. Excel won by default, and fair enough. The programmer wanted a desk, a specification and a beanbag.

That choice has not existed for years. Not five minutes: years. Building reliable, bounded, tested software for a specific job has become quick and cheap, and the industry has been slow to notice. We are not early to this. We are late.

Which leaves an uncomfortable question. How far could we have got in the last twenty years if this had been the attitude all along? Twenty years of every job teaching the next one, every reading kept, every lesson findable. Instead we have had twenty years of Pete, and some very nice tabs.

The teams that move first, in exploration and reservoir management above all, where every decision rests on the data, will spend less and know more. They will also look good to the people they report to, which has never yet harmed a career.

Forgive us

So forgive us if, when you hand over a spreadsheet, a small part of us dies. It is not because we are lazy, or fussy, or keen to sell you something. It is because we know better, we can do better, and on work that matters, we must.

Keep the scratch pad. Use it every day; we do. Just stop asking it to be the system. What the system should be instead is the next essay.

Come on.