hotato
GitHub
Home / About

Built in the open, by Attention Labs.

Attention Labs, Inc. builds hotato as open-source, self-hosted conversation QA for voice agents.

  • MITlicense
  • 2,761tests in CI
  • 0runtime deps
  • 5dimensions
  • 0egress by default
The maintainer

One company stands behind it.

One repository, one license, one place to read every line that decides a verdict.

Run the whole loop: sweep, score, contract, CI gate, self-hosted on your own infrastructure, forever, with no seat cap or capability paywall.

the facts
Maintainer
Attention Labs, Inc.
Repository
github.com/attenlabs/hotato
Home
hotato.dev
License
MIT
Runs on
Python 3.9-3.13, stdlib-only core
non-negotiable
No blended scoreA run reports a per-dimension scorecard across five dimensions. There is no single number to average away the one thing that broke.
Deterministic by defaultThe deterministic checks decide. The model judge is advisory, on its own lane, and only gates when you opt in.
Your audio stays yoursOffline by default, zero egress on the default path. Audio leaves only when a command's job is to fetch it from your own stack.
What it stands for

The invariants that bind it.

Conversation QA is only worth running if you can trust the result. Three rules bind the code and the copy; a change that breaks one gets sent back, however good the rest is.

When the inputs cannot support a verdict, hotato reports NOT SCORABLE.

Want the security posture in full? Audit the code or run it in your own environment →

The open-source path

Everything that matters is public.

The roadmap, the deep docs, and the numbers behind every claim are all in the open, readable by a person or a coding agent.

Source of truth

GitHub-first

Every capability ships in the repository first.

github.com/attenlabs/hotato →
Readable by humans and agents

Public deep docs

Deep docs, llms.txt, AGENTS.md, and an MCP server, so a teammate or a coding agent can drive the loop without a sales call.

On the record

Public changelog

Every dated release, each entry tracing to a tagged release and the commit log behind it.

See the changelog →
What's next, in the open

Public roadmap

What's shipped, what's gated, and what's still roadmap, all stated plainly.

Answered where you can see it

Questions in public threads

The founder answers in the open, in issues and discussions, so the reasoning is on the record for the next person who asks.

Contribute

The highest-value PR is a labelled call fixture.

Bug fixes, scorer tuning, docs, and synthetic scenarios are welcome. One consented, human-labelled recording is worth more than all of them: synthetic fixtures make the suite runnable, a labelled recording makes it credible.

  1. Record dual-channel.

    Caller on one channel, agent on the other, separated at capture. That physical split makes an overlap a fact of the recording, exact to the sample.

  2. Label it.

    One JSON next to the WAV: the yield-or-hold expectation, the timing bounds, and an attestation. The scorer measures energy over time; your label supplies the meaning.

  3. Validate locally.

    Run the checker before you submit. Exit 0 means the label and audio conform: fields well-typed, bounds consistent, attestation affirmed.

  4. Submit.

    Open the corpus-submission issue form, or a PR under corpus/. State whether the audio is synthetic, role-played, or captured, and affirm consent and PII removal.

  5. Credited in the changelog.

    Contributors are named when their clip lands.

Validate a label before you submit
$ python3 corpus/validate.py your_label.json

Full recording and PII rules: CONTRIBUTING · docs/SUBMITTING.