Voice reliability: the settings you change after go-live

Contents

We've written before about how a voice agent tells a human from an answering machine and about the acknowledgement pattern that stops silence feeling like a dropped call. Both are about getting an agent good enough to put in front of customers.

This one is about the fortnight after that, when the agent is live and something about it is annoying people. In our experience the fix is almost never the model. It's four settings, and they're worth understanding before you need them.

Barge-in, and why one setting doesn't fit every line

Barge-in is the agent stopping when the caller starts talking over it. Detecting that is a solved problem: watch the inbound audio energy, and when it stays above a threshold for a few consecutive frames, stop speaking.

The trouble is that the right threshold depends entirely on who's calling and from where.

A claims line taking calls from the roadside gets traffic noise, wind, hands-free audio and people standing next to a running engine. Set the detector too eagerly and the agent stops mid-sentence every time a lorry goes past, which reads as the agent being broken. A settlement line where people ring from an office needs the opposite: those callers expect to interrupt and get irritated when they have to say something twice.

So barge-in is configured per agent, with three presets and an escape hatch.

Five numbers, and it's worth knowing what each one does because the preset names only get you so far.

Grace is how long after the agent starts speaking before it will accept an interruption at all. Without it, the tail of the caller's last word cuts off the agent's first word, and the conversation collapses into two people apologising.

Confirmation frames is how many consecutive loud frames are needed before the detector believes it. This is your main defence against a cough, a door, a car horn.

Debounce is the minimum gap between successive interrupts, which stops a noisy line producing a machine-gun of stop-start.

Cooldown is how long the energy detector sits down after firing, so the agent gets a moment to actually respond.

Energy threshold in dBFS is the loudness bar, and the direction confuses people: closer to zero means less sensitive. Patient sits at -18, so it takes a genuinely loud interruption.

Set sensitivity: 'custom' and you can supply any of the five individually. Most deployments run balanced and never touch it. The ones that do are almost always moving a roadside line to patient.

Waiting for the human to say hello

Outbound is a different problem, and the mistake is easy to make.

A person answers the phone and says "hello?". If your agent starts its welcome message the instant the line connects, it talks over that hello, and the caller has now missed the first three seconds and started the conversation confused.

So outbound agents wait for the greeting before playing their opening line. It's on by default, with a timeout of eight seconds, after which the agent speaks anyway.

The timeout matters as much as the wait. Some people answer silently and wait for you to speak first, and an agent waiting forever for a greeting that isn't coming has produced a silent call, which is worse than talking over someone.

Every call ends, and you need to know how

A voice call has a surprising number of endings, and the ones we handle separately are worth listing because they're all things that happened to us in production:

• The caller hangs up
• The agent finishes the job and ends the conversation itself
• A tool decides the call is done
• The agent hears a goodbye and infers the end
• Voicemail was detected and the agent hung up without leaving a message
• Voicemail was detected, the agent waited for the beep and left one
• Voicemail was detected but the beep never came, so the wait timed out
• The call hit its maximum duration

Each of those funnels through one handler with a named reason attached. That naming is not tidiness for its own sake. When someone asks why 30% of yesterday's outbound calls ended in under fifteen seconds, voicemail_beep_timeout answers it and "call ended" does not.

The one that earns its keep most often is voicemail carrying three outcomes instead of one. Detecting an answering machine tells you nothing about whether your message got left. A campaign where every call reports success while nobody hears from you is a campaign nobody debugs until someone asks why the phone stopped ringing.

The duration limit, and a bug worth describing

Agents can be given a maximum call duration, with a message spoken before the line drops. Useful when a caller is looping, or when an agent has got itself stuck and would otherwise sit there running up a bill on telephony, transcription and a model whose input grows every turn.

The implementation contains a mistake we made and had to correct, and it generalises.

The obvious way to speak the closing message is to hand it to the same queue everything else uses. That queue is asynchronous: it accepts the text, returns immediately, and synthesises in the background. Which means the code that queued the message carried straight on to hanging up the call, and the caller heard nothing at all before the line went dead.

The fix is to generate that audio directly and wait for it to finish playing before ending the call. It's an obvious bug once you've seen it and an easy one to write, because everywhere else in the system, fire-and-forget is the correct choice. The last thing you say before hanging up is the one place it isn't.

The warning also gets written into the conversation history, so anyone reading the transcript later sees why the call ended rather than finding it stop mid-flow.

What to actually do with this

If you're running voice agents, three suggestions.

Set your barge-in preset per line. Roadside and office callers want different behaviour from the same agent, and the setting exists for exactly that.

Look at your call endings by reason at least weekly. It's the cheapest operational signal you have, and a shift in the distribution tells you something broke long before anyone complains.

And put a duration limit on every agent, even ones you're confident about. It costs nothing when it never fires, and the call it eventually catches is the one you'd have found on an invoice.

---

Further reading:

• Detecting humans vs machines in voice AI: AMD, VAD and how detection works
• The acknowledgement pattern: why silence feels broken
• What an AI conversation actually costs: why call duration dominates the bill
• AI resilience: the provider-agnostic architecture underneath