Contents
The first version of our claims outcome card worked, by every technical measure we had. It scored a claim against the firm's own resolved cases and put three probabilities on the screen. The numbers were calibrated. The engineering was sound.
Assessors looked at it and carried on doing exactly what they'd been doing before.
We'd built a product failure and spent a while calling it a modelling problem. Here's what we found, and what we changed.
A percentage is not an instruction
Here's roughly what the old card said:
Supplement risk: 41% Hidden damage: 0.58 Total loss: 12%
Every one of those numbers is defensible. Together they told an assessor almost nothing they could act on, for three reasons.
A single-event probability has no reference class. "41% chance of a supplement" invites the obvious question: 41% of what? People aren't bad at probability because they're stupid. They're bad at it because a bare percentage strips out the population the number came from, and the population is where the meaning lives.
Two numbers on different scales invite arithmetic nobody asked for. One score was a percentage, one a decimal index, and assessors started trying to combine them. They shouldn't have had to. If your reader is doing sums to use your interface, you've offloaded your work onto the person with the least time to do it.
None of them says what to do next. This was the real problem. An assessor has one decision in front of them: authorise it, look harder at it, or send it down the total-loss route. Getting from three probabilities to those three actions takes an interpretive step, and we'd left that step entirely to the reader.
Natural frequencies
The fix came from risk-communication research that's thirty years old and still routinely ignored by people building software. Gerd Gigerenzer and colleagues have shown repeatedly that people reason far more accurately about risk stated as a natural frequency, "about 6 in 10 similar jobs", than as a probability, "58%", even though the two are mathematically identical.
The classic demonstrations are medical. Doctors given a screening result as a percentage get the interpretation badly wrong at alarming rates. Give them the same information as counts out of a hundred patients and most of them get it right. The format is doing the work.
Our card now states every score as a natural frequency, and pairs it with the outcome it refers to:
About 6 in 10 similar jobs found hidden damage once stripped down.
About 4 in 10 similar jobs later needed a supplement, which is a cost revision.
Repairable. Only 8 in 100 similar cars were written off.
The mapping is deliberately coarse. Under 1% becomes "fewer than 1 in 100". Anything under 20% is stated out of a hundred, because "about 1 in 10" throws away too much at the low end. From 20% up we round to the nearest tenth and say "about 6 in 10". Above 95% we stop counting and say "almost all".
That coarseness is the point, and it was the hardest part to sell internally. Rounding 58% to "about 6 in 10" looks like throwing away information. What it's actually doing is refusing to imply precision the model doesn't have. Print a figure to the percentage point and you invite your reader to treat the gap between 58% and 61% as meaningful. Both of them mean "more likely than not, worth a look".
One verdict, one action
Natural frequencies fixed how the numbers read. The missing instruction was still missing. So the card now opens with one sentence of plain English and follows it with a recommendation:
Expect this estimate to grow once the car is stripped down. Scrutinise the strip-down and budget for a supplement before authorising.
or
This estimate looks solid. Similar jobs rarely grew once work started. Proceed as estimated.
The individual frequencies sit underneath, because an assessor who wants to check the working should be able to. But the first thing on the card is now a claim about this job and a suggestion about what to do with it, which is where the assessor was going to end up anyway.
The gauge, and the zone that overrides everything
Above the verdict sits a gauge with four zones. Under 30 is On track. Under 50 is Worth a look. Above that is Likely to grow. The fourth zone is Total loss likely, and it behaves differently from the others.
A written-off vehicle never goes to a garage. It doesn't get stripped down, so it can't turn up hidden damage, and it almost never supplements. Calling the repair-stage risks low for a total loss misses the point. They're meaningless for it.
The first version showed them anyway, because the model produced them and the template rendered whatever the model produced. That's how a card ends up telling an assessor a write-off has a 40% chance of a strip-down surprise.
Now, when the headline outcome is a total loss, the repair-stage signals disappear entirely. The gauge marker moves into the total-loss zone and the recommendation switches to routing: send this down the total-loss path, don't authorise a repair.
There's a small piece of interface arithmetic underneath this worth doing properly. Repairable claims are held out of the top of the gauge however high their risk score goes, so the total-loss zone belongs exclusively to total losses. A repairable job scoring 80 and a write-off must never land in the same place on the dial. The zones mean different kinds of thing, and the geometry has to respect that.
Authored copy, generic fallback
One design decision took some arguing. We author the wording per outcome by hand. "Hidden damage" gets a sentence about strip-down. "Total loss" gets a sentence about being written off. "Supplement" gets a parenthetical explaining that a supplement is a cost revision, because not everyone reading the card is an assessor.
The obvious objection: it doesn't scale. Define a new outcome and there's no copy for it.
So there's a generic fallback: an unknown outcome renders as "about 6 in 10 similar jobs were classed X". It reads acceptably. Not well, acceptably, and that's the right trade. A new outcome works on day one, and improving its wording becomes a small piece of authoring instead of a blocker.
The alternative, generating the copy from a language model at render time, would have been more elegant and considerably worse. This card goes in front of someone making a financial decision about a claim. The sentence has to say the same thing every time, and the person who owns the process has to be able to review it.
What we are measuring
None of this is worth anything if it doesn't change behaviour, so the card records whether it was shown, and whether the assessor accepted or dismissed what it recommended. Every write is best effort. Instrumentation must never be the reason a task page fails to load.
Acceptance rate doesn't measure accuracy. A card can be right and ignored, and a card that's always accepted might just be telling people what they already thought. But it's the only number that speaks to whether the redesign did what we claimed. The model's own metrics can't.
It's also why the whole thing is running as a pilot with one customer, behind a feature flag that's off by default and a role granted to named users. We think the redesign is right. We'd rather find out than assert it.
The general lesson
The modelling was the easy half of this project. The hard half was getting a number onto a screen in a form that changes what a busy person does next, and that's the half most machine learning work in operations software skips entirely.
If your model outputs a probability and your interface displays that probability, you haven't finished. You've handed the interpretation problem to the person with the least time to solve it and the most at stake when they get it wrong.
Further reading:
- Claims outcome prediction: what Project X-Ray does and what it deliberately does not
- Why our AI never agrees a total-loss value: the human-in-the-loop design this sits inside
- Everyone wants AI to decide the claim: why we let AI run the operation and keep a human on the money
- One platform, both sides of the claim: the argument we took to the ILC ClaimsTech final