Bring the calls you already have.
Point Hotato at a call from a stack you already run. It pulls a two-channel recording, or takes a timestamped transcript, and scores it offline: turn timing, plus say-do, whether the agent did what it said. Below is every stack we read the API docs for, including the ones whose recording arrives as a single combined channel and the ones that hand back no recording at all.
- 5separated scoring
- 5combined channel
- 6named limits
- 16read, no adapter
One scorer, five ways in.
Every stack ends at the same two-channel WAV or timestamped transcript, read by the same scorer. Only how that file arrives changes.
Vapi
Downloads the call's stereo recording by call id, caller and assistant already on separate channels, and scores it offline.
- capture
- drive · gated
Twilio
Fetches both channels of a recorded call by Recording SID, once each side was recorded on its own channel.
- capture
- drive · gated
Retell
Downloads a call's per-party recording by explicit call id, preferring the copy scrubbed of personal data.
- capture
- capture-only
LiveKit
Your deployment writes one WAV per party; Hotato scores them locally. The audio stays in your environment.
- capture
- capture-only
Pipecat
A drop-in 2-channel recorder writes the session; Hotato scores that file on your own machine.
- capture
- capture-only
Five stacks keep the parties apart.
Each of these hands back a recording with one party per channel, so an overlap names who caused it and no opt-in is needed. Placing a live call is Vapi and Twilio only, and only behind credentials and an explicit opt-in.
| Stack | Capture a call | How Hotato gets the audio | Drive a live call |
|---|---|---|---|
| Vapi | Yes | Auto-pull with your API key, by call id | Gated |
| Twilio | Yes | Auto-pull with your API key, by Recording SID | Gated |
| Retell | Yes | Auto-pull with your API key, explicit call id only | Capture-only |
| LiveKit | Yes | Capture in your infra, score the local WAV | Capture-only |
| Pipecat | Yes | Capture in your infra, score the local WAV | Capture-only |
Vapi · auto-pull by call id
Vapi stores each call with the customer and the assistant on separate channels. Hotato fetches that two-channel WAV (Vapi’s stereoUrl) by call id and scores it on your machine. Older payloads carry the recording at artifact.stereoRecordingUrl or call.stereoRecordingUrl instead, and Hotato reads both.
$ hotato connect vapi$ hotato capture --stack vapi --call-id <id>$ hotato sweep --stack vapi --since 7dsweep pulls up to 50 calls from a window and ranks the worst turn-taking moments across all of them in one HTML dashboard. No key yet? hotato capture --stack vapi --demo runs the whole loop on a bundled call.
Drive-a-call originates a live call from a staging clone of your assistant, waits for it to end, and feeds the recording into the same capture → score path. Every driven run costs a billed phone call, so Hotato dials only when both your credentials and HOTATO_DRIVE_OPT_IN=1 are present. The manifest’s origin records the caller as assistant-originated, never as a human.
Twilio · auto-pull by Recording SID
Twilio stores two-party recordings dual-channel by default, so ?RequestedChannels=2 returns per-leg audio: caller on channel 0, agent on channel 1. Pin it at creation with <Dial record="record-from-answer-dual"> or REST RecordingChannels=dual, since omitting the download parameter follows whatever was requested then. A 400 stops the pull cleanly, or falls back to RequestedChannels=1 under --allow-mono.
$ hotato connect twilio$ hotato capture --stack twilio --recording-sid RE...$ hotato sweep --stack twilio --since 7dTwo limits worth knowing before you trust a number. Accounts on Encryption with Public Key or External Storage receive mono regardless of the parameter. And a conference recording is not per-party: channel 0 is the first participant who joined with recording enabled and channel 1 mixes everyone else, so it carries two channels without carrying two parties — swap or re-map with --caller-channel / --agent-channel. connect stores TWILIO_ACCOUNT_SID and TWILIO_AUTH_TOKEN.
Drive-a-call dials your agent, speaks a caller script at fixed offsets, records both sides, and scores them on the same capture → score path. The scripted caller says its lines at the offsets you set and does not listen back, so every run replays the same turn sequence rather than barging in the moment the agent starts talking. The manifest’s origin records the caller as scripted-twiml, never as a human.
Retell · auto-pull by explicit call id
Once a Retell call ends, its recording is stored with each party on a separate channel. GET /v2/get-call/<id> over Bearer auth returns the call; Hotato prefers scrubbed_recording_multi_channel_url, then recording_multi_channel_url.
$ hotato connect retell$ hotato capture --stack retell --call-id <id> --expect yieldRetell documents a POST /v3/list-calls lister; Hotato takes an explicit --call-id until that lister is wired (issue #58). Retell is the only server it talks to: one request for the call, one for its audio. Everything after runs offline. A single mixed channel cannot say who talked over whom, so Hotato scores one only behind --allow-mono. connect stores RETELL_API_KEY.
LiveKit · capture in your infra
hotato setup --stack livekit prints an Egress scaffold: one audio-only Track egress per participant, so the caller and the agent land in separate files. A RoomComposite egress mixes them into one channel.
$ hotato setup --stack livekit$ hotato capture --stack livekit --caller caller.wav --agent agent.wavYour deployment writes the two WAVs to its own disk and Hotato scores those files in place. Install the live-capture helpers with pip install 'hotato[livekit]'. Pass --onset <sec> if you already know the second the caller started talking.
Pipecat · a recorder you drop in
hotato setup --stack pipecat prints a drop-in AudioBufferProcessor(num_channels=2) recorder that writes the caller and the agent to separate channels of one WAV.
$ hotato setup --stack pipecat$ hotato capture --stack pipecat --stereo captured.wavThe recorder runs in your own process and writes the caller on channel 0 and the agent on channel 1, so every overlap names who caused it. Install the live-capture helpers with pip install 'hotato[pipecat]'.
Every row above ends at the same place: hotato run --stereo captured.wav --expect yield scores any dual-channel WAV you already have. --expect yield reads a caller who starts talking as an interruption the agent should yield to; --expect hold reads it as a backchannel.
Two gatesDrive-a-call places a test call and scores it. Because each run bills a phone call, it needs both your provider credentials and HOTATO_DRIVE_OPT_IN=1. Retell, LiveKit and Pipecat are capture-only: point Hotato at recordings you already have.
Five more pull, on one channel.
Each of these has a list endpoint and a fetch endpoint Hotato calls with your key, and each returns a single mixed recording. One channel cannot say who talked over whom, so Hotato scores it behind --allow-mono and labels the result indicative only. Read off each vendor’s own API documentation on 2026-07-07.
| Stack | What Hotato fetches | What the vendor’s API leaves out | Flag it needs |
|---|---|---|---|
| Bland AI | recording_url off /v1/calls/{id} | No channel field in the recording or call-details schema | --allow-mono |
| ElevenLabs Conversational AI | Combined MP3 off /v1/convai/conversations/{id}/audio | Docs state the audio carries no separate caller/agent channels, only one combined conversation MP3 | --allow-mono |
| Synthflow | recording_url off /v2/calls/{id}, a Twilio Recordings URL | No dual-channel option documented anywhere in the call API | --allow-mono --model-id |
| Millis AI | recording.recording_url off /call-logs/{session_id} | Schema exposes one recording_url; call settings carry only a boolean enable_recording | --allow-mono --base-url for EU |
| Cartesia (Line) | audio/wav off /agents/calls/{id}/audio | Channel count is unconfirmed in the docs, so Hotato treats the file as mixed until a live channel-count check says otherwise | --allow-mono --agent-id |
What indicative only meansHotato validates that a dual-channel file carries one party per channel before it produces separated turn-taking numbers. A combined channel yields degraded scores that read the conversation as a whole, which is why the opt-in is explicit: --allow-mono on the command, or HOTATO_ALLOW_MONO=1 in the environment. Single-call capture or pull by explicit id needs none of the per-stack ids above.
Six stacks route somewhere else.
These have no Hotato adapter of their own, and the reason is a property of the vendor’s API rather than a judgement about the vendor. Where the recording still reaches you, the row names the route it travels.
| Stack | What the API gives back | The route that works |
|---|---|---|
| Deepgram Voice Agent API | A real-time WebSocket only: no list-calls, no fetch-recording, no recording-ready webhook, and the call is never stored vendor-side | Record it in your own infrastructure and score that file, the LiveKit and Pipecat pattern above |
| Regal.ai | No list-calls and no REST fetch-recording endpoint; the audio arrives only on the call.recording.available webhook | That webhook’s properties.recording_link is a Twilio Recordings URL — feed it through Hotato’s Twilio path |
| Sindarin | No Sindarin-hosted recording endpoint; record is submitted with your own Twilio credentials | The recording lands in your own Twilio account, so pull it with the Twilio path |
| Thoughtly | Its OpenAPI types every response as an untyped GenericResponse.data blob, so no recording-URL field path could be confirmed against a live call | Point Hotato at a dual-channel WAV you hold: hotato run --stereo |
| xAI Grok Voice Agent Builder | OpenAI-Realtime-compatible surface with no documented list-calls or fetch-recording endpoint; in-product recording appears in marketing without a confirmable API contract | Point Hotato at a dual-channel WAV you hold: hotato run --stereo |
| PlayAI (formerly Play.ht) | The conversational-agent product is retired: play.ai DNS does not resolve and docs.play.ai returns DEPLOYMENT_NOT_FOUND | The one live domain, docs.play.ht, is text-to-speech with no calls or recordings API |
Three open checks on the stacks above
Where a channel layout stands on inference rather than a line in a spec, it is listed here rather than folded into a claim.
- Cartesia channel count. The
/agents/calls/{id}/audioWAV’s channel count was not confirmable in the docs. It is scored as mixed until a live check reads the header. - Twilio dual-channel edge behaviour. The 400-on-unavailable response and the conference channel ordering stand on earlier notes, carried forward rather than re-read from the spec this pass.
- The Synthflow and Regal Twilio-URL trick. Because both hand back a Twilio Recordings URL,
?RequestedChannels=2might yield stereo — neither vendor documents or guarantees it, and it stands untested against a live account.
Read, no adapterSixteen further platforms were read to the same standard and carry no Hotato adapter, because their documentation sits behind a login or renders as a client-side app, or their API returns a transcript rather than audio: Daily · Ultravox · Hume EVI · Telnyx · Infobip · OpenAI Realtime · Amazon Connect · Parloa · Sierra · Decagon · PolyAI · Voiceflow · Cognigy · Dialogflow CX · Genesys Cloud · NICE CXone. Each keeps its per-platform facts and gaps in the integration spec.
$ hotato sweep --demo$ hotato connect vapisweep --demo scores the two calls that ship with Hotato and writes one HTML dashboard, on your machine alone. Swap vapi for twilio or retell to connect a hosted stack.
See the dashboard before you wire anything.
The demo runs the whole connect → capture → score path on the calls that ship with Hotato, so you can read the scorecard before you wire up a single credential.
hotato connect <stack> stores that stack's key once at file mode 0600 in ~/.hotato/connections.json. It is sent only to the vendor's own API on a capture, never to Hotato. Scoring itself stays offline.
Timing, from separated channels.
Every stack has one job here: keep the two parties on separate channels all the way to the file, so an overlap names who caused it.
- did_yield
- seconds_to_yield
- talk_over_sec
ScopeHotato reads turn-taking timing from speech energy: did_yield (agent stopped), seconds_to_yield (how long), talk_over_sec (overlap seconds). A mixed channel cannot say who talked over whom, so Hotato scores one only behind --allow-mono and labels it indicative only.