Contributing
Hotato measures one narrow thing well: the audio timing of turn-taking. All contributions welcome, but the highest-value one is the hardest to fake.
Contribute real, labeled call fixtures. Synthetic fixtures make the eval runnable. Real recordings make it credible.
Ground rules (bind code and copy)
- No accuracy percentages. Report timing in milliseconds and a yield/no-yield confusion matrix against human labels, never a headline “accuracy %” for the scorer.
- No speaker-ID, diarization, transcription, or emotion claims. The scorer sees energy over time. Energy is not intent, identity, or sentiment. Don’t describe it as any, in code, tests, or docs.
- The open core stays MIT, forever. Contributions are accepted under MIT; the core is never relicensed.
- The tool’s output stays vendor-neutral. A both-axes failure points toward a learned engagement-control / addressee-detection layer, not a knob: hotato names the kind of fix, never a product or a number.
- Attention Labs licenses that layer, but that’s marketing, not the scorer’s output: no internals, no invented numbers, no product names in what it prints.
The fixture model: two channels, one truth
Record dual-channel when you can: caller on one channel, agent on the other, separated at capture.
Then overlap is a fact you can point at, not an inference, which lets time-to-yield and talk-over be scored honestly. Mono is accepted but degraded, and must carry a human caller_onset_sec label.
A fixture is a scenario JSON (id, title, category of should_yield / should_not_yield, expected bounds, reference render timings) plus its audio. Copy a bundled scenario’s shape and register new ones in the manifest.
Consent and PII (read before recording anyone)
- Get explicit, documented consent from every party to redistribute the audio in an MIT-licensed public corpus.
- Strip PII: names, numbers, addresses, account identifiers. Prefer synthetic or role-played content over real customer calls. No PHI, ever.
docs/CORPUS-GOVERNANCE.md governs the real corpus: consent template, PII policy, and how validity is reported (milliseconds and a confusion matrix, never an aggregated accuracy percentage). Don’t merge real audio without it.
Running the tests
The core needs no third-party dependencies; tests use pytest.
python -m pip install -e ".[dev]" # pytest + jsonschema python -m pytest
Full guide: CONTRIBUTING.md. We review for correctness, and always for honesty.