turn-detector

A streaming turn-detection model for full-duplex voice agents. It reads a two-party conversation, one channel per speaker, and at each decision step answers a single question: has this speaker finished their turn, or are they only pausing?

The model is causal. Every timestamp it emits is the time by which all audio the decision depended on has been heard, so any lookahead is paid as latency rather than hidden inside the stamp.

Components

Encoder. nvidia/parakeet_realtime_eou_120m-v1 — a 120 M-parameter streaming FastConformer built for voice-agent pipelines, under the NVIDIA Open Model License. Only its mel front end and encoder stack are used as a streaming feature extractor; the recognition decoder and its <EOU> token are not.

Pretraining objective. Voice Activity Projection (VAP). The encoder is continued on a self-supervised task that predicts who will be speaking over the next few windows from the current state, instead of on a turn-end label. See Ekstedt & Skantze, Voice Activity Projection: Self-supervised Learning of Turn-taking Events (Interspeech 2022), and ErikEkstedt/VoiceActivityProjection.

Decision head. A small MLP reads the encoder state for the speaker being scored together with a joint two-channel state, and emits a per-step probability that the current pause is a turn end.

Arming gate. The head is asked only while the channel is quiet, and quiet is decided by an averaged ensemble of public streaming voice-activity detectors — Silero VAD, FSMN-VAD and earshot. Only detectors that are themselves causal are included, so the gate adds no lookahead.

Commit policy. Decisions are taken on a fixed 160 ms grid, subject to a minimum quiet duration, with an optional delayed re-commit while the channel stays silent. This is a serving-time policy and is configurable independently of the model.

Audio

16 kHz, one channel per speaker. Higher sample rates are converted with a causal FIR resampler, so resampling introduces no lookahead.

Evaluation

Developed against public streaming turn-taking benchmarks, which score events on a running conversation rather than accuracy on pre-cut clips:

• TurnBench (Sesame) — end-of-turn and interruption detection under a latency deadline and a false-positive budget.
• LiveKit eot-bench — false cutoffs at a fixed response-delay budget.

Status

This repository currently carries the model description only. Weights are not published here.