Turn-taking, timed to the millisecond.
Three signals, re-derivable by hand from two audio channels, with no model in the path.
Each channel already belongs to one party, so timing needs no words. Hotato reads exactly three numbers from two lanes of energy, and you can check every one with a ruler on the waveform.
- did_yield
- bool: did the agent stop once the caller started?
- seconds_to_yield
- how long from the caller’s onset to the agent’s stop.
- talk_over_sec
- how long both channels carried energy at once.
$ hotato run --stereo call.wav --onset 12.4 --expect yieldEnergy is not intent.
The scorer measures when each side spoke and for how long. It flags a candidate moment; you decide whether it was an interruption, a backchannel, or a clean handoff.
Timing is read from energy over time, one lane per party: which channel was live, and when. It names no speakers and reads no words, so every number stays re-derivable from the waveform alone.
A flagged moment starts as a candidate you label, then becomes a card: both channels, the overlap window in milliseconds, and numbers that come back the same on a re-run. What every verdict rests on →
Audio, or the transcript you already have.
A timestamped transcript from your call logs or a chat agent is a full input: hotato scores the turn timing in it with no audio, then checks the words.
Four checks read the words, each one a regex, a checksum, a lookup, or a clock: phrase (a pattern present, absent, first or last, for one role), pii (four detectors in must_not_leak mode, which report the hit and never the value), entity_accuracy (captured entities matched against your reference), and latency (a turn or tool answered inside its millisecond budget).
$ hotato assert run --transcript call.transcript.json --assertions assertions.yamlHave audio and want the words too? Produce the transcript locally with --transcribe (faster-whisper, on your machine). Privacy posture →
# transcript artifact: the matched value is never stored turn 6 caller: my card is [redacted] turn 7 agent : thanks, I have [redacted] on file pii_hits: - detector: card_luhn turn: 6 role: caller value: [redacted] # never echoed
Read the scorecard, not one number.
A run yields one score per dimension, never a blended number, because each dimension fails for a different reason and wants a different fix.
| Dimension | What it answers |
|---|---|
| Outcome | Whether the task succeeded. |
| Policy | Whether the required disclosures and PII rules held. |
| Conversation | The turn-taking timing. |
| Speech | Latency, phrasing, and delivery. |
| Reliability | How often the agent holds up across repeated runs. |
Why it is not one figureA red Conversation with a green Outcome tells you the agent did the right thing while stepping on the caller, which is a floor-control fix, not a prompt rewrite. One figure would hide the signal you came for.
How do I test voice agent interruptions?
Score a recorded two-channel call and read three numbers: did_yield, seconds_to_yield, and talk_over_sec. Hotato measures them from audio energy alone, with no model in the path.
One command runs the check on a recording: hotato run --stereo call.wav --onset 12.4 --expect yield. Each number is re-derivable by hand from the waveform.
The agent talks over the caller
talk_over_sec measures it: how long both channels carried energy at once. The overlap shows up as a window on the dual-channel timeline, in milliseconds.
Each channel already belongs to one party, so the measurement needs no words and no transcript. The scorer flags the moment as a candidate; you label whether it was an interruption, a backchannel, or a clean handoff.
In the atlas: the agent starts speaking while the caller still holds the floor: public reports, who filed each one, and the stacks it was reported on.
The agent interrupts customers mid-sentence
did_yield answers it: once the caller started, did the agent stop? seconds_to_yield times the stop, from the caller’s onset to the agent’s stop.
Run the recording with --expect yield and a call where the agent kept talking comes back as a fail, with the onset marker, the overlap window, and the yield line drawn on the timeline.
The agent ignores real barge-ins
Mark the caller’s onset and check did_yield. When the caller takes the floor and the agent keeps going, did_yield reads false and talk_over_sec carries the length of the collision.
In the atlas: interrupt handling that fails to tear down the in-progress turn, reported across stacks.
Energy is not intent: the scorer flags the moment, you label what it was, and the card shows both channels with the overlap window in milliseconds.
Measure the stop, not the intent.
autopsy reads one recording and lists its incidents, timed to the millisecond.
$ hotato autopsy ./call.wav
One recording is enough to start. Get started → · external proof →