hotatodocs
Home Docs GitHub
Home/Docs/Pytest plugin

The pytest plugin

Install hotato and the plugin registers itself. One fixture for asserting on real measurements inside any test, and one opt-in flag that gates the whole session on turn-taking.

bash
pip install hotato   # the plugin auto-registers (pytest11 entry point)

The hotato_score fixture

Same inputs as the CLI, returns the same envelope. Assert on whichever measurement your test cares about.

test_turn_taking.py
def test_call_yields(hotato_score):
    env = hotato_score(stereo="call.wav", expect="yield")
    assert env["exit_code"] == 0
    assert all(e["verdict"]["passed"] for e in env["events"])

def test_selftest_battery(hotato_score):
    env = hotato_score(suite="barge-in")
    assert env["exit_code"] == 0

The --hotato-suite session gate

Opt in on the command line and the battery runs after your tests, printing a summary and failing the session (exit 1) on a regression.

terminal
$ pytest --hotato-suite -q
.                                                     [100%]
================== hotato suite: barge-in ==================
8 of 8 events pass (failed=0)
1 passed in 0.54s

On a regression the gate is explicit:

terminal
FAIL svd-st-missed: fix[config] expected the agent to yield but it kept talking
hotato: regression detected; failing the session.

Gate on your own labelled sets

The bundled battery is a self-test of the harness. Point the same flag at your own scenario and audio directories, including the tiered suites:

bash
pytest --hotato-suite \
  --hotato-suite-scenarios corpus/suites/gold/scenarios \
  --hotato-suite-audio corpus/suites/gold/audio
One flag, both gates

If CI already runs pytest, this flag is the whole integration. Prefer a PR comment with the results table? The GitHub workflow posts one and gates on the same envelope. Use either, or both.