Measuring AI ROI: beyond the hype

The question of return on investment in AI puts most organisations in an uncomfortable position. Not because the gains are absent, but because measurement methods inherited from conventional IT projects fit badly, and the alternative — giving up on measurement in the name of the investment being "strategic" — does not survive contact with an audit committee.

The first obstacle is that the value of an AI project rarely shows up where you expect it.

A writing assistant rolled out to save sales teams time often produces its effect elsewhere: not in drafting time, which is marginal, but in the proposals that were never sent for lack of time and now are. The gain is not productivity, it is volume of activity. Measuring drafting time will show nothing.

The second obstacle is attribution. When a business indicator improves after an AI deployment, the share owed to the model is rarely isolable. Other things changed: teams were trained, processes revised, executive attention turned to the subject. That difficulty is not unique to AI, but it is acute here because AI projects almost always come with a transformation of the work itself.

The third obstacle, and the costliest, is reasoning project by project when most of the value is cumulative. An organisation's first use case carries the full weight of setup: the platform, the skills, the governance rules, the teams' learning curve. Its standalone ROI is almost always poor. The third use case, reusing that foundation, posts flattering figures having paid for none of it. Assessing each project separately therefore leads to two symmetrical errors: stopping a programme after a first use case judged disappointing, or overstating a portfolio's returns based on its latest additions.

The example that follows is a composition: it brings together observations from different contexts without reproducing a real case. A department deploys a case-handling aid and calculates a twenty per cent reduction in processing time, the equivalent of several full-time roles. The figure is accurate. It nonetheless produces no saving, for a simple reason: the time freed is spread across forty people, a few dozen minutes each per day. That time is neither measurable, nor transferable, nor removable. It has dissolved. The same organisation, however, obtains a perfectly tangible gain on another use case, more modest in percentage terms but concentrated on a three-person team handling a bottleneck: there, the time freed absorbs a rise in activity without hiring.

The lesson is structural: time saved only becomes value if it is concentrated somewhere. That is the principle we apply from the framing stage at KAIROS Impulse. Before estimating a use case's gains, we ask an unpleasant but necessary question: if this project succeeds perfectly, what concretely changes in the accounts or in the organisation's capacity? If the answer is "teams will be more comfortable", the project may still make sense, but it should not be promised a return it will not deliver.

Three families of value deserve separating, because they are not measured the same way. Efficiency value — doing the same with less — is the easiest to calculate and the most often illusory, because of the dilution effect described earlier. Capacity value — doing what could not be done, handling an unreachable volume, responding within an impossible deadline — is harder to quantify and far more real. Risk value — avoiding an error, detecting an anomaly, documenting a decision — can only be measured in incidents avoided, which requires knowing the prior frequency.

One recurring objection deserves an answer: some executives hold that AI is a strategic investment and escapes return calculations by nature. That is a dangerous position. Not because it is always wrong, but because it makes the programme indefensible at the first difficult budget round. An initiative whose output nobody can describe will be cut, whatever its real worth. Measuring imperfectly beats not measuring.

A final point, often forgotten: cost. The cost of an AI system is not its development. It includes inference, paid per use and rising with adoption — success costs more than failure — plus monitoring, periodic retraining, and human supervision time. Organisations that budget the project without budgeting the three years of operation that follow discover the true cost at the worst moment.

Three questions to close. For each use case underway, can we name the line in the accounts or the capacity indicator it is meant to move? Is the expected gain concentrated on one team or diluted across many people? And have we costed three years of operation, or only the build?