One MIT CLI, seven surfaces, and the evidence each one leaves behind.
Hotato is 72 top-level commands (174 counting subcommands) that turn a call recording into a timestamped incident, an incident into a portable check, and a check into a CI exit code. Every page below names the commands it describes.
- MIT
- 0 runtime deps
- exit 0 pass · 1 regression · 2 refuse
- 1.20.0
The surfaces
Start where your problem is.
Each surface stands on its own; together they are one loop from a recording to a gate.
Find the moment, with a timestamp on it.
Barge-in, talk-over, dead air and latency incidents with measured magnitudes, and an input-health gate that refuses audio it cannot score.
pin · contract · provePin one bad moment; CI re-scores it every push.
A portable .hotato bundle, JUnit XML for the dashboard, and a proof whose headline claim is capped by the lanes you actually ran.
Grade the claim, not the wording.
Five deterministic assertion kinds read the trace and your system of record. The model judge is counted separately and never merged into a score.
connect · capture · setupYour stack’s calls, with the key on your disk.
Eight vendor-hosted stacks pull by API key stored at file mode 0600; LiveKit and Pipecat capture in your own infrastructure.
describe · hotato-mcp · pytestA coding agent can read the whole CLI in one call.
A manifest generated from the parser itself, an MCP server with fifteen tools over stdio, and a pytest fixture that asserts on measurements.
serve · consoleReview calls as a team, on loopback.
A stdlib-only web app bound to 127.0.0.1, bearer-token auth on every request, an append-only audit, and a ?format=json mirror on every view.
Every verdict travels with the bytes it was measured from.
Audio, frame-level evidence, the input-health report, the policy, provenance, and a sha256 manifest checked on the way out of the archive.
Five commands take a recording to a red build.
This is the sequence hotato --help prints as its own start-here block, in order.
- Check one call.
The deterministic whole-call scanner lists incidents with timestamps and writes a self-contained HTML report under
hotato-output/. - Scan the folder.
The same engine over a directory, reporting the measured share of calls with no critical incidents — a count, never a blended quality score.
- Pull from your stack.
The health report over recent calls from a connected vendor.
- Pin the incident.
One incident id becomes a portable failure contract carrying its own audio and policy.
- Prove it in CI.
The gate re-scores the stored evidence and exits non-zero when a lane fails or comes back inconclusive.
$ hotato check ./call.wav $ hotato scan ./calls $ hotato vapi health $ hotato pin apx-<id> $ hotato prove --contracts contracts
--demo reads the two-channel call bundled in the install.What the instrument declines to tell you.
Each of these is a scope boundary the CLI states in its own help text, and each one is why a verdict here means something.
| The boundary | What it means in practice | Where |
|---|---|---|
| No accuracy percentage | Results are reproducible timing measurements with every threshold exposed and every frame inspectable. No single number stands in for them. | autopsy, scan, run |
| Timing, never intent | A candidate moment is an overlap onset or a response gap. The tool cannot know whether a caller sound was “mhm” or “stop”; you decide and label it. | scan --stereo |
| Bad input is refused, not scored | Identical channels, a silent required channel, or a mono file on the two-channel path exit 2 with the reason and the next step. | trust, autopsy |
| Inconclusive is never green | A lane that refused or could not tell exits 2 even when nothing failed, so CI cannot read missing evidence as a pass. | prove |
| A model judge stays on its own lane | Deterministic pass/fail/inconclusive counts and the judge count are reported separately, and the two are never merged into one score. | assert run, test run |
| A contract pass re-measures stored evidence | A green contract verify says the evidence, policy and scorer still agree with the human label. Testing the current agent is a separate capture. |
contract verify |