There is no AI without data readiness

When an AI project fails to deliver what it promised, the conversation usually revolves around the model. In most cases I see, the model was fine. What was missing was data in a usable condition, and that almost never appears in the original proposal.

The problem starts before the first experiment

Data readiness is not a synonym for volume. A company with ten years of history can be less ready than one with two, if over those ten years the meaning of its fields changed three times without anyone recording it.

A Microsoft case study on software engineering for machine learning describes data management as a stage that runs through the whole lifecycle, not as an initial step you complete. Collection, cleaning, labelling and versioning come back on every iteration, and teams that treat this as preparation work underestimate the effort systematically.

Data cascades

The research on data cascades, published by Google, describes an uncomfortable pattern: data quality problems are barely visible early on and produce compounding negative effects that only surface much later, often already in production.

What makes the cascade dangerous is the asymmetry. Fixing at the source is cheap and the benefit is invisible, so nobody prioritises it. Fixing after the model is in production is expensive, and the problem arrives disguised as "the model is getting things wrong".

The practical implication is direct: data work done early looks like waste and is the only thing that prevents the large rework.

Four questions that reveal actual readiness

Before any AI project, four questions separate expectation from reality.

Does the data exist in an accessible form? Existing in a legacy system with no API, or inside scanned PDFs, is very different from existing in a queryable table.

Is the meaning stable? If the "status" field has had three sets of values in five years, the history is only usable with an explicit translation.

Is the quality known? Not perfect, known. Knowing that 12% of records have an empty address is workable. Not knowing is not.

Is the use permitted? The source basis, consent and stated purpose have to support the intended use, especially where personal data is involved.

If any answer is "I do not know", that investigation is the project's first deliverable, not a footnote.

Data validation is a test, not a report

The work on data validation for machine learning, presented by researchers at Google, makes a point that holds for any AI application: data errors should be detected automatically, at the pipeline entrance, against an expected schema.

In practice that means declaring expectations and failing when they break: type, value range, required fields, cardinality, approximate distribution. It is the equivalent of a unit test, except for the input rather than the code.

The gain is not academic. Without validation, a source system that silently changes format degrades results for weeks before anyone notices. With validation, the pipeline stops on the first strange batch and someone is alerted the same day.

Governance without turning into bureaucracy

DAMA's data management body of knowledge organises the subject into domains such as quality, architecture, metadata and governance. It is extensive, and reading the whole guide before starting is the most reliable way never to start.

The minimum cut that works for an AI project is three things:

• A defined owner per dataset, a person with a name, not a department.
• A living dictionary saying what each field means and when the meaning changed.
• Recorded lineage, meaning where it came from, what transformations it went through and where it went.

With those three, most audit and investigation questions already have an answer.

A knowledge base has its own requirements

When the project is retrieval augmented, readiness changes in nature. What matters becomes whether the document is current, whether conflicting versions are circulating, whether the text survives extraction and whether the document's permissions can be honoured at search time.

Two versions of the same policy, one revoked and one in force, indexed together, produce a system that answers confidently using the wrong rule. No model tuning fixes that.

What this means for your team

Before approving the next AI project, run a two-week readiness assessment on the datasets it will use. Access, stability of meaning, measured quality and permitted use, with findings written down.

If the assessment finds a serious problem, that is the real project, and it delivers value even without AI: reliable reporting, feasible integration, cheaper audits.

And put automated validation at the pipeline entrance from day one. It is the cheapest piece to build at the start and the most expensive to add once the system is already in production.