Data readiness: the invisible foundation of every AI project

There is a sentence we hear in almost every project that goes off course: "we didn't think data would be the problem". It usually arrives in the third month, when the technical team, having proved feasibility in a few weeks, discovers that scaling stalls not on the model but on what it is fed.

Data readiness is the invisible foundation of every AI project. Invisible because it appears in no demo: a prototype builds perfectly well on a clean, hand-prepared extract of a few thousand rows. A foundation because the gap between that prototype and a production system rests almost entirely on it.

Readiness is not reducible to data quality, contrary to common belief. It rests on four distinct dimensions, and an organisation can be excellent on three and blocked by the fourth.

The first is accessibility. Do the data exist in a form a program can reach, or are they locked inside systems that expose no interface? Perfect data available only through a manual monthly export will never support a real-time system.

The second is freshness. How often are the data updated, and is that rhythm compatible with the intended use? A fraud detection model fed by data consolidated overnight will detect nothing in real time, however sophisticated it is.

The third is semantics — the most neglected. Two systems can hold a "customer" field that does not mean the same thing: one counts legal entities, the other delivery sites. Nobody documented it because each team knows its own definition. The day the two sources are joined, the figures stop reconciling, and it takes weeks to understand why.

The fourth is lineage. Do we know where a data point comes from, who produced it, what transformations it went through? Without that lineage, correcting an anomaly is guesswork, and answering an auditor who asks on what basis an automated decision was made is impossible.

The sequence below aggregates several comparable experiences; it does not recount a single project. A retailer wants a demand forecasting model. Sales data exist, clean, with five years of history. The prototype performs excellently. Production reveals two difficulties. First, the history includes a supply disruption period during which sales reflect availability rather than demand — the model has learned to predict stockouts. Second, product reference systems changed twice in five years, and the mapping between old and new references was never formalised. Rebuilding a coherent history takes more work than building the model itself.

This kind of discovery is the norm, not the exception. Which is precisely why we treat the data readiness audit as a prerequisite to framing, not as a phase of the project. At KAIROS Impulse that audit does not score the organisation on an abstract scale: it answers an operational question, use case by use case — does this data, in this state, support this project, and if not, what is the shortest path? A use case may be set aside not for lack of interest, but because its foundation would require eighteen months of preparatory work, while another, less ambitious, is feasible in three months.

A frequent misreading needs heading off here. The conclusion is not that the entire data estate must be cleaned up before anything starts. Large data governance programmes launched without a specific use case almost always drift: they run for years, cost a great deal and produce reference systems nobody uses, because nothing ever forced the hard trade-offs to be settled. The right sequence is the reverse: pick a use case with real value, and clean up only the data perimeter it requires. The use case gives the data work a constraint, a deadline and a judge.

One last dimension deserves attention: data readiness is as much organisational as technical. Behind every dataset there is a team that produces it and has other priorities. Asking that team to document its definitions, tighten its data entry or maintain a reference mapping is asking for extra work whose benefit accrues to someone else. Without an explicit executive arbitration, that request stays at the bottom of the pile — and the AI project waits.

Three questions to close. For the use case you have in mind, do you know precisely which data it consumes, where they sit and how often they are refreshed? Is there a shared, written definition of the key entities — customer, product, transaction — or does each system have its own? And if an anomaly appeared in a result, could you trace it back to its source? Those three answers are worth any maturity score.