Eloi vs Morlocks - does AI prove the spreadsheet rebels were right all along?

Early in my career I found myself sitting in a meeting about data. Specifically a meeting to agree what ‘a customer’ really was.

The idea was that a central data team was going to build a canonical data model for what it meant to be a customer — one shining model which would rule them all. A model which would define, once and for all, what a customer really was.

As a recent graduate, however, I had questions.

I had already spent some time with different product teams and business functions as part of my rotation, and beyond some basic information, the customer definition quickly became slippery — a ‘customer’ looked very different in the context of sales, finance, and product.

Needless to say my questions ensured that my rotation with that team didn’t end so well — but to be fair neither did the project.

Because the team quickly found that — outside of reporting — ‘good data’ did not mean neat, perfectly modeled data in the abstract.

And throughout my subsequent career I have found this to be a repeating pattern — organizations hoping that the next data technology might unlock the perfect data model. A paean to information that will infuse every actor in the organization with the flaming sword of absolute certainty.

But almost thirty years later the world is littered with data definition projects that flamed out spectacularly as they encountered the messy complexity of real-world relationships — and the hard lesson that truth and meaning are always highly contextual to the specific decision being made.

A lesson that is either being forgotten — or politely ignored — in much of the content being churned out about improving the success of AI implementations through similarly exhaustive data foundations.

Data hygiene as avoidance

For me it’s telling that organizations — whether product providers, consultancies, or end user companies — are increasingly talking about the creation of ‘better data models’ as the best route to successful AI implementation.

Clean up your enterprise data models to make the organization legible to AI, the argument goes, and then AI will ‘work right’. No more hallucinations. No more mis-alignment. No more bull-shittery.

Perfect data. Perfect machine. Perfect decisions.

But this argument rests on two dangerous fallacies — compounded by a fundamental mis-reading of what an enterprise actually is.

The first fallacy is that enterprises have ever had clean data that neatly fits the constraints of data models. Most meaningful work in an enterprise rests on information asymmetries and imperfect data. We only know our customers imperfectly. Solving problems means navigating imperfect information. Building new products is basically swimming in imperfect signals across multiple dimensions.

So when we fall for the idea that data has to be perfectly modelled before we can act, we are falling for a fallacy about how our most meaningful decisions are actually made.

The second fallacy is that it is even possible to fully model an enterprise. Enterprises are messy, organic entities defined by a complex web of evolving relationships. By definition most action in the world revolves around continuous sensemaking and decisions-in-the-moment, with imperfect data. The records, contracts, agreements etc. that end up in conceptual data models and systems of record are just the things we use to keep a highly compressed audit trail of what happened in the past.

So when we fall for the idea that data can ever be perfectly modeled, we are falling for a fallacy about the completeness, accuracy, and timeliness it is possible to achieve via the creation of conceptual data models.

Together these fallacies leave organizations — and the people who advise them — basing their AI plans on a fundamental mis-reading of enterprise reality, of what having ‘good data’ for AI actually means. And this mis-reading can actually become a comforting excuse for deferring real action — a way to postpone further implementation until we have a fully specified data model that delivers full ‘data hygiene’.

But hygiene is a word we use to be polite about things we don’t really want to look at or talk about. A word to distance ourselves from unpleasantness. To avoid talking about mess, ordure, and disease. It frames our shuddering horror at the unclean as a simple cleanliness problem — something to be scrubbed away so we can act without dirtying our hands with unpleasant truths.

And in this context, ‘data hygiene’ is often code for avoidance — for averting our eyes from the dirty, messy and unconstrained reality of real-world decision-making data and pretending it be tamed with clean, orderly data models that make everything look simple, repeatable, and controlled.

But let’s be honest — most enterprise decisions will never be made in hygienic conditions with perfect information. They're always going to be made deep in the dirt.

Where critical data really lives

The desire for perfect data is understandable. It’s how we’ve traditionally built technology systems. And it grows naturally from a parallel desire for AI to act in a more deterministic and repeatable way.

But this thinking fundamentally misleads organizations when they are thinking about where to apply AI — drawing their eyes towards the deterministic processes already covered by existing technologies and away from the much larger scope of work that sits in the messy whitespace and so is irreducible to structured models.

Because enterprises have always had a dirty underbelly where the real work gets done — one in which capacity is constrained by the scale of sensemaking rather than the scale of systemization.

Systems of record provide a transactional backbone for the latter, an audit trail, and a backward-looking record of what happened. And in enterprise technology discussions they suck all of the air out of the room. They are considered to be infallible sources of enterprise truth. The poster child of data hygiene.

But the messy reality that lies beneath this veneer of data hygiene — the thing that people don’t want to look at — is that while IT departments promote the virtues of common data models and infrastructure, businesses work out of highly contextual spreadsheets, databases, and documents. Edge data systems that are continually cut, pasted, and discarded to support the evolving contextual needs of teams. Ad-hoc data aggregations that fill the seams and enable judgement in-the-moment under the demands of real-world uncertainty. Systems without which every business would fall apart.

But until now organizations were able to ignore this reality because it just kind of worked. IT teams could pretend their data systems were a representation of reality, of how work gets done. CEOs could pretend they believed that the CIO had it all under control. And everyone else could get on with their work using whatever information they could assemble to make the best possible decisions within the bounds of their responsibility.

Because all of this work happened in the shadows — a vast network of organic teams absorbing and dealing with uncertainty through a blend of expertise, judgement and localized contextual data that they built for themselves. Teams whose activities had to be tolerated despite the fact they were using unhygienic data.

And this, right here, is where AI needs to live.

AI exists to scale in uncertainty, not to eliminate it

AI is actually something very different from traditional IT. Rather than simply executing tasks that can be fully specified in advance using data whose structure and relevance are known upfront, AI works best in situations where neither the task nor the shape of the data can be exhaustively defined beforehand.

This places AI firmly beyond the capabilities and scope of corporate data infrastructures and into the messy world of local data, messy spreadsheets, and highly contextual decision making. The kind of work long undertaken by the teams who absorb uncertainty and provide a buffer between the chaotic outside world and the prim conventions of enterprise data systems.

Which means that while ‘data quality’ is a recurring issue raised by CIOs as a driver of disappointing outcomes — including in diginomica’s own research — it actually means something quite different in the context of AI. Because traditional measures of data quality were optimized for downstream audit, reporting, and control — not for upstream judgment and action under conditions of uncertainty.

In practice, ‘good data’ for AI is therefore not a comprehensive data model that attempts to make the whole system legible in advance — with complete schemas, stable representations, and long-term relevance — but the much smaller subset of highly contextual data necessary to make a specific decision under conditions of uncertainty in the here-and-now. The kind of highly contextual data that has always been assembled by humans — and without which the organization couldn’t survive.

Which is also what makes it so uncomfortable.

Because AI is effectively pushing the unacknowledged but critical role of ‘dirty’ data into the faces of people who have previously pretended that it either didn’t exist or was just something unpleasant at the periphery. It is asking organizations to really acknowledge how work gets done.

In short it is exposing the fantasy that the vast majority of people in a business have been using clean, hygienic data to make clean, hygienic decisions in ways that are clearly assigned, auditable — and deterministically repeatable.

Which means that there’s an uncomfortable implication here.

For years, organizations have treated the people working in spreadsheets, notes, and informal tools as a problem to be fixed — a sign that data discipline hasn’t yet been imposed. Meanwhile, the architects of clean models and canonical truths have been treated as the custodians of how the organization really works.

Whatever the merits of that argument in a world of transactional systems, AI flips it by showing that the people working in the mess were not undermining the organization but compensating for its inability to perfectly model their operating reality. They were assembling context, exercising judgment, and absorbing uncertainty so that everyone else could pretend to have clean hands.

But AI forces organizations to confront the moral, social, and political risks that humans have been quietly absorbing when making considered judgements with imperfect context — even though many might resist the resulting need to disrupt the status quo by making agency and responsibility explicit so that AI can function.

And this is the quiet misunderstanding at the heart of most enterprise AI debates — because the hardest data problem in enterprise AI is not how to structure data for retention and legibility, but how to assemble the right — often ephemeral — context, at the right time, for the right decision. Because most activities don’t require a perfect view of the whole organization, they only require a relevant view for the decision at hand.

The killer ‘data quality’ question for AI, therefore, is not how to build a better enterprise data model, but how to design systems of agency that reflect the implied boundaries and real-time context necessary for enterprise decision-making.

Seen this way, the most valuable starting point for enterprise AI data is the messy operational context people have already assembled to make decisions in the real world — data that already encodes decades of hard-won field knowledge about constraints, trade-offs, and decision dynamics.

Which puts me in mind of the Time Machine by HG Wells.

In Wells’s imagined future, humanity splits into two classes — the naïve Eloi who live comfortably above ground, and the subterranean Morlocks who keep the machinery of the world running below.

And in the same way, if you want to understand how an organization actually works — and build AI that delivers value inside it — you don’t start with the pristine models floating over the top. You go looking in the filthy engine room.

Which means the future of enterprise AI won’t be built by Eloi with beautiful models.

It will be built by Morlocks standing neck deep in ugly spreadsheets.