hotato

Local call forensics and regression guards for AI agents, built in the open.

The transcript passed. The call failed. hotato scores turn timing and say-do (what the agent said against what the backend did), then gates CI. Self-hosted, MIT, maintained by Attention Labs.

  • MITlicense
  • 0runtime deps
  • 0egress by default

Read the code that decides.

One repository, MIT, holding every check that decides whether a call passes.

Check a folder of recordings and fail the build when a failure comes back. Deterministic checks decide the verdict; the model judge is advisory unless you opt in. It measures turn timing and say-do, not intent, and reports NOT SCORABLE when the evidence can't support a call.

Full security posture: read the security page →

Maintainer
Attention Labs
Repository
github.com/attenlabs/hotato
Home
hotato.dev
License
MIT
Runs on
Python 3.9-3.13, stdlib-only core

Everything that matters is public.

Deep docs and the numbers behind every claim, readable by a person or a coding agent.

Source of truth

GitHub-first

Every capability ships in the repository first.

github.com/attenlabs/hotato →
Readable by humans and agents

Public deep docs

Deep docs, llms.txt, AGENTS.md, and an MCP server, so a teammate or a coding agent can drive the whole loop themselves.

On the record

Public changelog

Every dated release, each entry tracing to a tagged release and the commit log behind it.

See the changelog →

The best pull request is a labelled recording.

Bug fixes and synthetic scenarios are welcome. One consented, labelled recording makes the test suite credible, not just runnable.

  1. Record and label it.

    Dual-channel audio and one JSON: the label (stop for the caller, or keep speaking), timing bounds, an attestation.

  2. Validate locally.

    Run the checker before you submit. Exit 0 means the label and audio conform.

  3. Submit, get credited.

    Open the corpus-submission issue, or a PR under corpus/, stating provenance and consent. Contributors are named when their clip lands.

corpus · validate
$ python3 corpus/validate.py your_label.json
  schema         pass
  timing bounds  pass
  attestation    pass
exit 0

Full recording and PII rules: CONTRIBUTING · docs/SUBMITTING.

Read the code. Run the suite.

Find what broke in your agent calls. MIT-licensed. One command starts it.

$ hotato autopsy ./call.wav

Read the docs, or browse the source. Get started → · GitHub →