One bad moment becomes a check your build cannot skip.
A failure contract is a self-contained .hotato bundle: the clipped audio, the frame-level evidence, the input-health report, the policy that decides pass, and the exact command that replays it. Commit it, and CI re-scores that moment on every push.
# the bare ref pins the call's top critical incident $ hotato pin apx-cc33f46fad58 # #N pins one specific incident from the same call $ hotato pin apx-cc33f46fad58#1 --expect hold --id mhm-was-fine-001 # the CI gate on the result $ hotato prove --contracts contracts
apx-<12hex> is the id an autopsy prints on its own pin: line. Resolution is offline, from the stored autopsy envelope.Pinning refuses more often than it succeeds, and that is the feature.
Every refusal exits 2 with the reason and leaves no artifact behind, so a half-written contract never reaches your repository.
- The recording changed.
The source file’s current bytes must still hash to the pinned id. A changed file refuses rather than quietly pinning a different call.
- The incident came from mono.
Contracts require the two-channel deterministic path, so a mono-derived incident is refused outright.
- There is no decision to pin.
An incident kind that carries no yield/hold decision, a rank out of range, or a moment that is not scorable.
Barge-in and talk-over incidents pin. The default is --expect yield, meaning the caller held the floor; pass --expect hold to record that the agent was right to keep talking. The reviewer’s name is bound into the contract identity.
# or build one straight from a moment you already found $ hotato contract create --stereo bad-call.wav --onset 42.18 \ --expect yield --id refund-cutoff-001 --out contracts \ --max-talk-over 0.6 --max-time-to-yield 1.0
The gate
Three exit codes, and one of them is “could not tell”.
The same three codes across the CLI: 0 pass, 1 regression, 2 refuse.
| Code | contract verify | prove |
|---|---|---|
| 0 | Every contract’s re-scored timing passes its own recorded policy, and every embedded assertion is PASS or INCONCLUSIVE. | Every lane you activated passed. |
| 1 | At least one contract regressed — its re-scored timing no longer meets its policy — or is no longer scorable, or an embedded assertion deterministically failed. | At least one lane failed or regressed. |
| 2 | Usage error, an empty directory, or a corrupt contract.json. |
Usage error, or inconclusive: a lane refused or came back inconclusive and none failed. Missing evidence is never green. |
Zero activated lanes is a usage error in prove, not an empty pass: a proof of nothing is refused. The verdict is fail-closed — overall pass only when every activated lane passed.
$ hotato prove --contracts contracts/ --gauntlet $ hotato prove --contracts contracts/ \ --suite ci.suite.yaml --agent support-v3 $ hotato prove --before before/ --after after/ \ --min-n 5 --name support-v3-rc2
prove adds no scoring engine of its own: every number in it is one a contract verify, suite run or gauntlet already measured.- proof.json · hotato.proof.v1
- proof.md
- content_id over the canonical JSON
The headline is capped by the evidence, not by the release you wanted.
A proof names the claim scope its lanes actually support, and it never reaches for a higher one.
- Captured Evidence.
Contracts alone. Re-measuring stored evidence says the evidence, policy and scorer still agree with the human label — it does not test the current agent.
- Test Suite.
A suite or gauntlet run.
- Candidate Revision.
A before/after lane, and only when you also bind the candidate’s identity with
--candidate-config-hashand--provider. - Deployed Revision.
Adds
--deployment-idon top of a bound candidate. Alone, that flag is recorded as evidence and elevates nothing.
The before/after lane needs a minimum number of previously-failing fixtures to support its claim (--min-n, default 3). Below it, the lane is inconclusive rather than a pass.
The proof carries verdicts, counts, relative input names and sha256 digests only: no transcript text, no audio bytes, no absolute paths.
JUnit XML, a step summary, and a scaffold for four CI systems.
The gate reports the way your dashboard already reads.
contract verify --junit writes one testcase per contract; prove --junit writes one testsuite per evidence lane, where a refused or inconclusive lane is an <error> rather than a silent pass.
--step-summary appends a Markdown verdict block made for GitHub’s $GITHUB_STEP_SUMMARY; --pr-comment appends a share-safe block with the verdict line, the caught moment, the one measured number, and an ASCII caller/agent barge-in timeline. Both fail open: a render or write problem reports on stderr and leaves the verify exit code alone.
# the canonical gate config your CI system reads $ hotato init ci --system gitlab # or the whole-repo starter kit: gate, hotato.yaml, fixtures/, contracts/, reports/ $ hotato init starter --stack vapi --out . # the gate itself $ hotato contract verify contracts/ --junit contracts-junit.xml
--system takes gitlab, jenkins, azure or circleci, and writes the file that system already reads. Each gate is guarded, so an empty contracts/ or fixtures/ directory is a normal starting state rather than a red pipeline. Generated files are namespaced — HOTATO.md, not README.md — so a first run does not collide with files your repo already has.A green contract run means the evidence still holds.
Re-scoring a bundle catches what changes underneath you: an engine upgrade, a threshold change, a re-captured audio file. Testing the current agent against that stimulus is a separate capture, and the CLI keeps the two claims apart.