Smart Turn Hinglish (whisper-tiny)

Audio-only turn detection for voice agents: has the speaker finished their turn, or are they just pausing? Built for English + Hindi/Hinglish from the pipecat smart-turn-v3.2 data. ~7.8M params. The ONNX graph takes a raw 16k waveform and returns P(turn complete), with the log-mel frontend baked in, so inference needs only onnxruntime + numpy.

Code, training, ablations and the full write-up: github.com/CodeWithMoin/smart-turn-hinglish

Files

file	window	notes
`smart_turn_8s.onnx`	8s	shipped model. accuracy pick, ~38 ms/clip CPU
`smart_turn_4s.onnx`	4s	fast variant, ~17 ms/clip CPU

Results (official frozen `smart-turn-v3.2-test`, accuracy @ 0.5)

	this model (8s)	Smart Turn v3.2
Hindi	93.9%	92.8%
English	93.7%	94.7%
Human / real	94.4%	95.5%

Beats the reference on Hindi and sits within a point on English, at a fraction of the size. Overall test AUC 0.983.

Usage

import numpy as np, onnxruntime as ort

sess = ort.InferenceSession("smart_turn_8s.onnx", providers=["CPUExecutionProvider"])

def p_turn_complete(wav_16k):                 # float32 mono @ 16 kHz
    x = wav_16k[-128000:]                     # last 8s
    buf = np.zeros((1, 128000), np.float32)
    buf[0, 128000 - len(x):] = x              # right-align (left-pad with silence)
    return float(sess.run(None, {"waveform": buf})[0][0])

In a live agent, run a cheap VAD (e.g. Silero) continuously; on a ~200 ms pause, call this on the last 8s. Threshold P(complete) — higher = more patient (fewer interruptions).

Limitations

Trained mostly on synthetic TTS. Real casual conversational speech on a laptop/phone mic is out of distribution and where it's weakest. See the report for the honest failure analysis and the plan (real Indian-mic conversational data) to close it.

Provenance

Fine-tuned from openai/whisper-tiny on the openly published pipecat smart-turn-v3.2 corpus. For commercial use, check the upstream terms.