Contents
Open any case management system in a UK claims or legal operation and watch what it does. It holds the case. It records the status. It stores the documents, timestamps the actions, and produces a report at the end of the month.
Now watch what the people around it do. They chase the garage for an update. They read the engineer's report and work out what it means. They notice that a case has sat at the same status for nine days and nobody has asked why. They decide who should pick it up.
The software recorded every one of those things after a human did them. It caused none of them.
That's the whole category, and it's about to change.
The gap between the statuses
A workflow diagram makes an operation look like a machine. Case arrives, moves to triage, moves to assessment, moves to settlement. Clean arrows, clean boxes.
The arrows are where the work is. Getting a case from triage to assessment means somebody read an email, found the missing detail, phoned someone, updated three fields and decided the case was ready. The system recorded the status change. The forty minutes of judgement that produced it left no trace except a timestamp.
Every operations director I talk to knows this. It's why headcount scales with volume in a business that bought software specifically so it wouldn't. You automated the recording and left the doing alone, so the doing still costs what it always did.
Why everyone is suddenly promising more
Language models changed what's possible in the arrows. A model can read the engineer's report. It can notice that a case has stalled. It can draft the chaser, work out which of six workflows a new email belongs to, and pull the three facts a handler would have gone looking for.
So the category has started promising operations that run themselves. Touchless claims. Autonomous handling. Agentic everything. Every core vendor has a roadmap slide with an agent on it, and a good number of them will ship something that genuinely does the work.
Most of it will fail in regulated operations, and not for the reason people expect. It won't fail because the models aren't good enough. It'll fail because being right most of the time is a different product from being accountable all of the time, and only one of those can be deployed on a book of business.
Bounded autonomy
The version that works has a boring name and three properties.
A calibrated "I'm not sure" beats a confident wrong answer, every time. This is the one the industry keeps getting backwards. A model that handles 80% of cases and correctly flags the other 20% is worth enormously more than one that handles 95% and is silently wrong about the rest, because the second one has no failure mode you can staff around. You cannot review what you were never told to look at.
That means confidence has to be a first-class output, scored and thresholded, rather than something implied by the model's tone. Our email agent scores eleven factors into one number and sorts every message into act, review or hold. The number is visible, the reasoning is attached, and the thresholds are yours to set.
Some things are never automatic, at any confidence. Scores handle ordinary variation. They're the wrong instrument entirely for a category of work where the cost of being wrong isn't proportional to how wrong you were.
So the email agent has four conditions that force human review no matter what it scored: anything classified as needing review, anything that's a complaint or an escalation, anything with negative sentiment, anything critically urgent. A perfectly understood complaint still goes to a person. Under DISP a complaint starts a clock the moment it arrives, and an agent that answers it beautifully while failing to recognise what it is has started that clock late.
The same principle runs through everything we've built. Switchboard never agrees a total-loss value: a named engineer sets it, the AI presents it and captures the response. Project X-Ray says where a claim is heading and recommends a route, and an assessor takes it or leaves it.
Everything it did has to be answerable later. Not a log of API calls. A record a compliance officer can read: what the system did, what it was looking at, what it recommended, who accepted it. When the FOS asks for the file, "the AI handled it" is the answer that loses.
The uncomfortable trade
Bounded autonomy demos badly.
The impressive demo is the one where the AI takes a call, settles the claim and books the payment while everyone watches. Ours stops and asks a person. In a room full of buyers, the autonomous version wins the applause and the cautious version wins the second meeting, because somebody in that room has to sign their name to how the book is handled.
It's also more expensive to build. Confidence scoring, hard overrides, an approval queue that handlers will actually use, an audit trail that survives contact with a regulator: none of that is the fun part, and all of it is most of the work. Anyone can wire a model to a case management API in a fortnight. The fortnight isn't the project.
And it caps the headline number. A system where a human touches every complaint and every payment cannot claim to be touchless, so the automation percentage on the slide is lower. Fine. The number that matters is how much of your team's day went back to them, and you get most of that from the ordinary work rather than from the cases you'd never have automated anyway.
Where this actually goes
My read on the next three years, offered as a view rather than a forecast.
The systems that win regulated operations won't be the most capable ones. They'll be the ones whose behaviour a compliance officer can describe in a sentence and evidence in a file. Capability is becoming a commodity, because the models improve for everyone at once and nobody's moat is a prompt. What doesn't commoditise is the accountability structure around the model, because that has to be designed against a specific regulator, a specific process and a specific set of people who carry personal liability.
I also think the industry will discover the failure mode the hard way, publicly, within eighteen months. Somebody's autonomous claims agent will do something indefensible at scale, it will end up at the FOS or in front of the FCA, and the resulting caution will be indiscriminate. Firms building on bounded autonomy will be able to show their working. The rest will be explaining why their vendor's model did what it did, which is not a position anyone wants to occupy.
Meanwhile the filing cabinet keeps recording. Every one of those forty-minute gaps between statuses is still being filled by a person doing work the system watched and didn't help with.
That gap is the entire opportunity. It's worth taking seriously enough to build properly, which mostly means being willing to ship something less impressive than the thing your competitor is demoing, and being right about it for longer.
Further reading:
- The email the AI answers on its own: confidence tiering and the four hard overrides
- Why our AI never agrees a total-loss value: the same argument applied to money
- Everyone wants AI to decide the claim: why AI runs the operation and a human owns the decision
- Claims outcome prediction: recommending a route without taking the decision
- Human in the loop: the approval design across every channel