aufklarer/Smart-Turn-v3.2-CoreML

Smart Turn v3.2 — CoreML

End-of-turn detection for voice agents on Apple platforms. Given the last 8 seconds of the user's speech, the model returns the probability that the user has finished their turn, so an agent can reply promptly after a real endpoint and keep listening through a mid-sentence pause. It works on the raw audio (prosody, pace, intonation), not a transcript, and covers 23 languages.

The Whisper log-mel front-end is inside the model, including the zero-mean / unit-variance waveform normalisation the upstream model was trained with. Feed 16 kHz PCM and read one probability.

Model

Property	Value
Parameters	8.0 M (Whisper-Tiny encoder + attention pooling + MLP head)
Precision	float16 encoder and head, float32 audio front-end
Input	audio float32 [1, 128000] — 16 kHz mono, most recent audio last, zeros at the front
Output	probability float32 [1, 1] — turn complete if > 0.5
Window	8 s (longer turns: keep the last 8 s)
Size	16.8 MB
Deployment target	iOS 17 / macOS 14
Languages	Arabic, Bengali, Chinese, Danish, Dutch, English, Finnish, French, German, Hindi, Indonesian, Italian, Japanese, Korean, Marathi, Norwegian, Polish, Portuguese, Russian, Spanish, Turkish, Ukrainian, Vietnamese

Files

File	Size	Description
smart_turn.mlmodelc/	16.8 MB	Compiled CoreML program (float16 weights, float32 front-end)
config.json	—	I/O contract, window size, upstream revision
LICENSE	—	BSD-2-Clause notice for the upstream weights

Performance

Accuracy on 1,000 clips from the upstream pipecat-ai/smart-turn-data-v3.2-test set (shard train-00000-of-00010.parquet, threshold 0.5). Latency is one 8-second window on Apple M5 Pro (macOS 26.5.2) (ONNX Runtime CPU with 2 intra-op threads, CoreML on CPU + Neural Engine). Higher accuracy / F1 is better; FPR is the share of unfinished turns wrongly cut off, FNR the share of finished turns the model kept waiting on.

Model	Accuracy	Precision	Recall	F1	FPR	FNR	Latency (mean)	Latency (p95)
Upstream smart-turn-v3.2-gpu.onnx (mel input)	92.90%	0.912	0.944	0.927	8.48%	5.61%	—	—
smart-turn-v3.2.onnx (our ONNX export)	92.90%	0.912	0.944	0.927	8.48%	5.61%	36.3 ms	51.4 ms
smart_turn.mlmodelc	92.90%	0.912	0.944	0.927	8.48%	5.61%	3.5 ms	5.4 ms

Usage

import CoreML

let config = MLModelConfiguration()
config.computeUnits = .cpuAndNeuralEngine
let url = Bundle.main.url(forResource: "smart_turn", withExtension: "mlmodelc")!
let model = try MLModel(contentsOf: url, configuration: config)

// `turn` holds the user's current turn at 16 kHz; keep the last 128000 samples.
let window = try MLMultiArray(shape: [1, 128000], dataType: .float32)
let tail = turn.suffix(128000)
let offset = 128000 - tail.count
for (i, sample) in tail.enumerated() { window[offset + i] = NSNumber(value: sample) }
let output = try model.prediction(from: MLDictionaryFeatureProvider(dictionary: ["audio": window]))
let probability = output.featureValue(for: "probability")!.multiArrayValue![0].floatValue

# speech-swift CLI
speech turn --audio utterance.wav

Run it after Silero VAD reports a pause, on the whole current turn (up to 8 s). If the user resumes before the agent answers, run it again on the full turn.

Source

Converted from pipecat-ai/smart-turn-v3 (revision f766f81d3cfd, smart-turn-v3.2-gpu.onnx), the open Smart Turn model from the Pipecat project, BSD-2-Clause. Training data and evaluation sets are published by Pipecat as pipecat-ai/smart-turn-data-v3.2-*.

Links

• speech-swift — Apple SDK
• Docs — install and CLI docs
• soniqo.audio — website
• blog — blog