Find every bad call, then close the loop

hotato analyze and hotato connect / pull / sweep rank every turn-taking failure across a folder or your whole call history at once. Label the ones that matter, and patch, verify, and loop carry the fix from proposal to a paste-ready artifact to proof, with the two irreversible decisions kept in your hands.

1. Find every bad call in a folder

hotato analyze is zero-config: no scenarios, no labels, no onset, no flags required. Point it at a folder of dual-channel recordings and it walks every .wav with the same whole-call scanner as hotato scan, then ranks the candidate moments across every call by salience so the worst ones float to the top.

# one self-contained, offline HTML dashboard, ranked by salience
uvx hotato analyze ./recordings

# a bare folder as the first argument routes to the same command
uvx hotato ./recordings

# ranked candidates for an agent instead of a browser
uvx hotato analyze ./recordings --format json

The dashboard embeds the real audio around the top moments (base64, nothing uploaded, zero external requests) with a playhead that sweeps the timeline in lockstep with playback, so you press play and hear the talk-over or the dead-air gap land exactly where the chart marks it. Files that cannot be scored (mono mixes, unreadable audio) are reported cleanly in a Skipped section with their reason; a bad file never crashes the run.

2. Or find them across your whole call history

Already have API access to your stack? Connect once, then pull and score every recent call in one shot, no per-call ids. Credentials are stored locally at ~/.hotato/connections.json (file mode 0600) and sent only to the vendor’s own API, never to Hotato.

uvx hotato connect vapi --api-key <key>
# --stack and the key become optional once exactly one stack is connected
uvx hotato sweep --since 7d

sweep is pull (list recent recordings, download each with the same fetch hotato capture uses) then the same zero-config analyze over the pulled folder, into one dashboard. pull and sweep both work standalone too.

LiveKit and Pipecat are capture-in-your-infra: there is no vendor recording to pull, so they are not connectable. See LiveKit / Pipecat capture. Mono/mixed stacks cannot attribute overlap to caller or agent, so their calls surface in the dashboard’s Skipped section, honestly, not faked. Full per-stack endpoint provenance: ADAPTER-STATUS.md in the repo.

3. Label the moments that matter

Hotato never infers intent, in a folder scan or a single call. You listen to a candidate, decide the expected behavior, and freeze it as a permanent fixture:

uvx hotato fixture create --stereo call.wav --onset 42.18 \
    --expect yield --id refund-cutoff-001 --out tests/hotato

The full walk from one clip to a CI gate, including --expect hold for a backchannel: The regression loop.

4. loop: one command, with memory

hotato loop drives the parts Hotato can drive and remembers where it left off in a small local state file (.hotato/loop-state.json by default). It never runs ahead of you: it advances exactly one stage per call, and the next run picks up where the last one stopped.

# run 1: discovery (analyze -> scan -> rank) -> awaiting_label
uvx hotato loop ./recordings

# you label the candidates that matter
uvx hotato fixture create --stereo rec.wav --onset 12.4 \
    --expect yield --id refund-001 --out tests/hotato

# run 2: plans a guarded fix from the labeled fixtures -> awaiting_verify
uvx hotato loop ./recordings --fixtures tests/hotato

5. patch: the paste-ready artifact

Reads a fix plan (from hotato plan, schema hotato.fixplan.v1) and renders its abstract {field, from, to} recommendation into a literal artifact for the target stack. For Vapi and Retell, whose config lives behind a REST API, that is a JSON merge-patch body plus a ready curl against the platform’s own config-update endpoint, using the exact field names from fixmap’s verified knob catalogue.

uvx hotato plan result.json --stack vapi --assistant-id <id> --out fixplan.json
uvx hotato patch fixplan.json

hotato patch [vapi] finding=missed_real_interruption decision=propose_one_step
  config_patchable=true  applies_change=false
  change: stopSpeakingPlan.numWords  4 -> 2  (decrease, bounds [0, 10])
  apply method: rest-merge-patch
    curl (you run this; hotato does not):
      curl -X PATCH https://api.vapi.ai/assistant/<assistant-id> \
        -H "Authorization: Bearer $VAPI_API_KEY" \
        -H "Content-Type: application/json" \
        -d '{"stopSpeakingPlan": {"numWords": 2}}'
  next: review the artifact above and apply it yourself; hotato never applies it
  next: re-capture the failing moment through your stack after applying it
  next: prove the movement across the battery: hotato verify --before before/ --after after/

For LiveKit and Pipecat, whose config lives in your agent source, there is no config-update REST call to hit, so patch emits the exact source edit instead, for example InterruptionOptions(min_words=1), never a fabricated endpoint. For an unknown stack it names the knob family and asks for a concrete target; it emits no literal body it cannot stand behind.

hotato patch [vapi] finding=missed_real_interruption decision=do_not_tune_single_threshold
  config_patchable=false  applies_change=false
  no patch: The plan refused single-threshold tuning: the battery misses a real
    interruption AND false-stops on a backchannel, so no one config value fixes
    both. No config patch is produced.
  recommended fix class: engagement-control
    This is a discrimination problem, not a threshold problem: telling a genuine
    bid for the floor apart from a backchannel or speech not addressed to the
    agent. No single timing threshold separates them.
  next: This is the both-axes case: no single config threshold fixes it, so no
    config patch is emitted. See the engagement-control pointer.
  next: Verify any change with a battery, not one clip: hotato verify --before
    before/ --after after/

Honesty, always: patch makes no network call and pins applies_change to false. It produces the change; it never applies it.

6. verify: the before/after proof

After you apply the change, in your own stack, and re-capture the same failing fixtures, verify scores the old and new run envelopes against each other. Fixtures pair by event_id, then scenario_id; each side is a single hotato run envelope or a directory of them.

# the failing take, before the change
uvx hotato run --scenarios tests/hotato/scenarios --audio tests/hotato/audio \
    --format json > before.json

# ... apply the patch, re-capture the fixtures ...
uvx hotato run --scenarios tests/hotato/scenarios --audio tests/hotato/audio-new \
    --format json > after.json

uvx hotato verify --before before.json --after after.json

hotato verify: 5 fixtures paired (before -> after)
  3 of 3 that used to fail now pass; 2 of 2 hold fixtures still pass
  results: fixed=3, still_pass=2
  CLAIM: 3 of 3 fixtures that used to fail now pass, and 2 of 2 hold fixtures
    still pass. This improvement COINCIDES with your change; hotato measures
    timing and does not attribute cause.
  talk-over before: p95 2.10s (n=5)
  talk-over after: p95 0.35s (n=5)
  hotato reports coincidence, not causation.

It reuses the hotato compare taxonomy per fixture (fixed, regressed, improved, worse, unchanged, still_pass, not_scorable). The rollup is the two axes that matter: the regression axis, how many previously-failing fixtures now pass, and the hold axis, how many hold-labeled guard fixtures still pass.

• Coincidence, never causation. verify says the improvement coincides with your change, never that it caused it. Hotato measures timing; it does not run a controlled experiment.
• Refuses low-n claims. Below --min-n (default 3) the per-fixture facts still print, but the battery-scale headline is withheld and said so.
• Never invents a verdict. An unjudgeable side is not_scorable; a fixture on only one side is reported unpaired, never silently folded into the rollup.

By default verify measures and exits 0; --fail-on-regression exits 1 if any fixture regressed or got worse, for gating a rollout on the proof itself.