Home/Real calls

Twelve real calls, scored against one default configuration

We recorded 12 scripted phone calls on 2026-07-06 against a stock assistant running on a popular hosted voice platform, left on its default interruption settings, dual channel, then scored every moment offline with Hotato. This page is about what defaults do before anyone tunes them. It is not a vendor ranking, and the platform is not named here.

Provenance

Recorded by the project maintainers as scripted calls: one human caller worked from a written script against a synthetic voice assistant. The labels (what the script called for at each moment) come from that script, not from a model. Hotato measures timing from the two audio channels only. The clips embedded below are MIT licensed, in the repository.

What the defaults did

Backchannels take the floor in about 0.34 seconds. A soft acknowledgement stops the agent as fast as a real interruption does.
A firm “Stop.” does not register. The agent talks 1.26 seconds past it, then restarts its whole answer 3.6 seconds later.
A 4 second thinking pause loses the floor at 3.44 seconds. The agent jumps in while the caller’s sentence is still unfinished.
The quiet interrupt is missed entirely. A half-volume attempt never crosses the default threshold; a louder retry on the same call yields in 0.60 seconds.

Every number above is a timing measurement, not an accuracy percentage. Full derivation per moment: manifest.json in the corpus directory linked below.

The measured results, one card per moment

Bounds, stated once and applied uniformly to every should-yield moment: 1.0 second to yield, 1.0 second of talk-over. Seventeen scored moments, nine pass, six fail, one latency-only reading, one baseline with no event to score. This replaces a table dump: every card below is the real timeline for that moment, drawn from the same real measurements a table would show. Caller on the left channel, agent on the right; the shaded band is measured talk-over.

Correctly handled: a real interruption or a backchannel, either way

Nine of seventeen. The default configuration is not broken all the time: rapid, loud, or ordinary interruptions mostly land, and one soft backchannel is correctly held through.

PASSmoment 01, hard interruption

yields in 0.35s, talk-over 0.35s · field note: agent yielded to the interruption.

8 second clip, dual channel, 96kbps

PASSmoment 03, soft mhm

no yield registers · field note: the operator recorded the agent as yielding to the mhm, stopping and never resuming. The measurement disagrees: no verified stop inside the 3 second search window, so this scores as a held floor, not a yield. One of two moments here where the field note and the measured binary disagree; walked through frame by frame in the corpus.

PASSmoment 05b, wait hold on

yields in 0.36s, talk-over 0.33s · field note: agent correctly yielded and acknowledged (“Of course. Take your time...”).

PASSmoment 06, double talk

yields in 0.40s, talk-over 0.40s · field note: agent correctly yielded, stopped at “seasonal flu shot” and pivoted.

PASSmoment 08, rapid interrupt 1 of 3

yields in 0.34s, talk-over 0.34s · field note: agent yielded; leading words clipped by default endpointing.

PASSmoment 08, rapid interrupt 2 of 3

yields in 0.33s, talk-over 0.33s · field note: agent yielded, but “and the nearest” registered as “In the nearest”; needed a retry.

PASSmoment 08, rapid interrupt 3 of 3

yields in 0.00s, talk-over 0.00s · field note: agent yielded; “and what about parking” registered as just “parking” after about 3 attempts.

PASSmoment 10, louder retry

yields in 0.60s, talk-over 0.59s · field note: after raising volume 30 to 40 percent, a garbled fragment registered and the agent yielded.

PASSmoment 12, greeting overlap

yields in 0.37s, talk-over 0.37s · field note: agent yielded over its own greeting and responded, though STT caught only a partial (“I need help with it”).

False yields: a backchannel mistaken for a bid to speak

Three of seventeen. “Mhm”, “right”, and “yeah” all took the floor in about a third of a second, the same reflex speed as a real interruption above. No timing threshold tells these apart from the real thing.

FAILmoment 04, soft backchannel (1st)

yields in 0.34s, talk-over 0.32s · field note: agent yielded on a backchannel and restarted its answer from the top (operator note: the word “right” appears to trigger it). Expected hold, measured yield.

8 second clip, dual channel, 96kbps

FAILmoment 04, soft backchannel (2nd)

yields in 0.34s, talk-over 0.32s · field note: agent halted at “This include,” and went silent; the operator hung up. Expected hold, measured yield.

FAILmoment 05a, soft yeah

yields in 0.37s, talk-over 0.37s · field note: agent wrongly yielded and restarted its answer. Expected hold, measured yield.

Missed or slow: a real interruption the default fumbled

Three of seventeen. These should have yielded fast; the default either missed the attempt outright or blew past the stated one second bound.

FAILmoment 10, quiet interrupt, missed entirely

no yield registers · field note: the quiet interrupt did not trip the default threshold; the agent kept talking.

8 second clip, dual channel, 96kbps

FAILmoment 02, one word: “Stop.”

yields in 1.46s, talk-over 0.45s · field note: the operator recorded the agent as never yielding to the “Stop.”, then restarting its whole paragraph. The measurement disagrees on the binary: the agent’s own inter-sentence pause happened to coincide with a second caller burst, which the scorer counts as the yield, 1.46s out, well past the one second bound either way. The other of two moments here where the field note and the measured binary disagree; walked through frame by frame in the corpus.

FAILmoment 07, correction

yields in 1.16s, talk-over 1.16s · field note: first correction attempt not picked up; dead air until the operator said “Hello?”; the agent apologized and the correction landed on the retry.

The other direction: the agent jumping the caller’s pause

One of seventeen, not a barge-in at all: a four second thinking pause, mid-sentence. The default treats a pause as a turn ending.

LATENCYmoment 09, thinking pause (not a barge-in)

agent starts 3.44s into the caller’s pause · not scored pass or fail; this is not a barge-in, it is the agent going first. The caller falls silent mid-sentence intending to keep the floor; the agent reads the silence as a turn ending and starts talking 3.44s later, then the caller talks over it trying to reclaim the floor. Field note: label violated on all 3 takes; the agent treated the trailing “about” as end of turn.

The baseline: a call with no interruption at all

One of seventeen. Sanity anchor: this call has no caller-during-agent onset anywhere, so no event exists to score. It is the clean floor every measurement above is a departure from.

BASELINEcall 11, silent listen

zero caller onsets during agent activity · the caller asks one question, then goes quiet; the agent answers 1.48s later and runs uninterrupted to 72.4s. No overlap exists anywhere in this 82.5 second call: the clean floor every measurement above is contrasted against.

The funnel verdict

The battery fails on both axes at once: it misses a genuine interruption (script 10) and false-yields on backchannels (scripts 4 and 5) in the same run. No single sensitivity threshold fixes both directions; raising it to hold through backchannels drops real interruptions, lowering it to catch interruptions yields to backchannels. Run against the real battery, hotato plan reports exactly that, and refuses to propose a threshold change:

hotato plan battery-result.json --stack <platform>

hotato plan [stack] finding=threshold_funnel decision=do_not_tune_single_threshold
  config_only_safe=false  production_apply=false (approval: manual)
  hypothesis: The battery missed a genuine interruption and also stopped for a
  backchannel. One sensitivity threshold cannot satisfy both: raising it to hold
  through backchannels drops real interruptions, lowering it to catch
  interruptions yields to backchannels. The failure class is discrimination,
  not calibration.
  recommended fix class: engagement-control
  platform mutation: performed=false (hotato plan is read-only)

Real output from the real battery, trimmed to the decision and hypothesis; the platform name is genericized here and the full untrimmed JSON (with per-event evidence) is in the repository linked below.

Everything else

All seventeen scored moments across twelve scripted calls are shown above; each card links back to the same real measurements shown in the funnel verdict below. The manifest with every onset derivation, the two disagreements between the measured binary and the operator’s own field notes (walked through frame by frame on the cards above, not smoothed over), and the full battery envelope live in the repository:

See the full corpus on GitHub →

Honest scope

These clips document one assistant, on one platform’s default interruption settings, on one recording date, called by one cooperative scripted caller on a clean connection. They are a measured portrait of a default configuration, not a benchmark of any vendor, and they carry no accuracy percentage. The labels come from the caller’s script; Hotato contributes the timing measurements only.