nsudhanva/eot-detector

End-of-turn detector for voice agents

This bundle is the serving release v4 of an end-of-turn (EoT) detector. A voice agent uses it to decide when the caller has stopped speaking. The detector scores each pause in the caller audio. When the score is above the threshold, the agent can reply.

The bundle has two models:

• prod/: a stack of a text model and an audio model. The API answers requests with it.
• shadow/: a silence rule that fires after 700 ms of silence. The API scores it and logs it only.

Model parts

Part	Base model	What it does
prod/text/	HuggingFaceTB/SmolLM2-360M, fine-tuned (LoRA, merged)	Scores the transcript prefix: is the sentence complete?
prod/audio/turn-1-mini.step.onnx	p99lab/turn-1-mini, unchanged	Scores the last seconds of audio in streaming steps.
prod/weights.json	Logistic stack, trained in this project	Combines the text score, the audio score, prosody and the pause length.
prod/reference.json	Dev set histograms	Drift reference for the Prometheus PSI metrics.
prod/serving.json	Manifest	Thresholds per audio condition and operating point.

manifest.json lists every file with its size and sha256.

Inputs and outputs

Inputs:

• Audio: 16 kHz mono PCM, the last 8 s before the pause.
• Text: the transcript prefix of the current turn, and the previous agent turn (optional).
• Offset: the pause length in ms, from 0 to 2000.

Outputs:

• p_eot: the probability that the turn is complete, from 0 to 1.
• fire: true when p_eot is above the threshold for the condition and the operating point.

The default operating point is p5 (5% false cut-off budget).

How to serve

The detector runs with Docker on a Mac or Linux machine. Get the code and run:

./start.sh

The script downloads this bundle, starts the API, Prometheus and Grafana, and prints the URLs. It needs no data lake and no MLflow.

To use the bundle directly, set these variables and start the API:

EOT_SERVING_BUNDLE=/path/to/prod
EOT_SERVING_SHADOW_BUNDLE=/path/to/shadow
EOT_SERVING_TRACKING_URI=
EOT_LAKE_ENABLED=false
uvicorn eot.app:app

Endpoints: POST /v1/eot (one pause), WS /v1/eot/stream (live audio), GET /v1/models, GET /readyz.

Results

Test split of release v0.1, clean audio, operating point p5:

Metric	Value
Latency at the 5% false cut-off (FCO) target	842 ms
Achieved FCO	6.6%
AUC	0.872
Real turn ends that never fire within 2 s	19.8%

Telephone audio (tel8k): 874 ms latency, 6.8% FCO.

Out of domain (v0.3-bench), the frozen threshold is too cautious. The latency is about 1.9 s until you calibrate the threshold on your own calls.

Limits

• English only.
• The thresholds come from the dev split. Calibrate them on your own traffic before production use.
• One in five real turn ends does not fire within 2 s. The 700 ms silence rule is the fallback.
• The text model needs a transcript. Without text, only the audio model and the pause length count.
• The audio model does not see the agent audio. Use a separate channel for the caller.

Licenses

• Fine-tuned text weights (prod/text/): Apache-2.0.
• Base text model: HuggingFaceTB/SmolLM2-360M, Apache-2.0.
• Audio model: p99lab/turn-1-mini, Apache-2.0, included unchanged.
• Stack weights, reference and manifests: Apache-2.0.