Run the head-to-head on your own calls.
A scorecard hands you the evidence and the command to regenerate every result yourself.
Five dimensions, two lanes, each scored on its own.
Outcome, policy, conversation, speech, and reliability each pass or fail on their own evidence. The advisory model judge sits on a separate lane and never decides the verdict for you.
Every deterministic result re-derives from the same audio under one pinned policy. You recompute it yourself and get the same verdict.
$ hotato test run conversation-test.yaml --agent <id>$ hotato simulate scenario.yaml --out runs/- deterministic lane
- advisory judge, opt-in
Run it on your own recordings.
No account, no vendor number. Point it at a captured call or a simulated scenario: same scorecard either way.
Simulated artifacts are deterministic: seeds derive from the scenario, no model or network, every artifact stamped origin=simulated.
Three questions about one call.
Different tools answer different questions. Match the question to the tool for it; the deterministic fixture layer owns the last one.
| The question | The tool for it | When |
|---|---|---|
| Did the whole call succeed? Task success, transcript rubric, compliance. | Your broad QA suite | before it ships |
| Should the agent yield right now? The live turn-taking decision, mid-call. | Your runtime voice layer | during the call |
| Did the confirmed timing bug come back? Re-scored from the same audio, deterministically. | Hotato, the deterministic fixture | after a confirmed bug |