Clearline-VAD

Streaming voice activity detection for LiveKit Agents, a drop-in for silero.VAD. 16 kHz, one probability every 16 ms, CPU only, for wideband and telephone audio. Two models: the release (55k parameters, the plugin default, chosen for turns) and a distilled variant (127k parameters with per-band level trackers, the better frame classifier, taught by a Whisper-encoder teacher what rooms full of other voices sound like).

Work in progress. Clearline is still being trained, and this repository is updated as better models pass our development-set rules; every upload is its own revision, so pin one (STT-API's plugin does). In progress now: the distilled variant on far-field meetings, where it fires more in the pauses than the release (AliMeeting), and heavy noise, where Silero v6 still leads by a little (FLEURS under DEMAND at 0 and -5 dB). The numbers below are the models in this revision.

	Silero v6	Clearline-VAD	Clearline-VAD, distilled
model CPU per 32 ms, one core	103 µs	60 µs	83 µs
plugin CPU per stream, 16 kHz, 8 streams (% of a core)	1.19%	0.47%	not measured
plugin CPU per stream, 48 kHz, 8 streams	1.57%	1.03%	not measured
mean frame ROC-AUC, 13 test sets	0.912	0.921 (higher on 8)	0.921 (higher on 9)
LiveKit: turns detected / median onset	0.801 / 247 ms	0.857 / 239 ms	0.813 / 289 ms
LiveKit: turn ends on time	0.718	0.775	0.737
LiveKit: false triggers per hour, 10 sets	29.6	38.0	24.6
LiveKit: false triggers per hour, without the private telephone sets	28.1	27.5	15.7
telephone, frame ROC-AUC: FLEURS through 5 codecs	0.936	0.923	0.936
telephone, frame ROC-AUC: the dev sets on the phone	0.963	0.960	0.966
telephone, LiveKit on AMI dev: onsets / ends / false per hour	0.877 / 0.750 / 17.4	0.947 / 0.857 / 32.4	0.907 / 0.767 / 20.8

LiveKit rows: every turn replayed through LiveKit's own VAD state machine on 10 test sets, each model at the point it ships with, chosen on the four development sets (AMI dev and VoxConverse dev, wideband and through the phone); Silero v6 at its own best point on the same four (threshold 0.70).

Languages

FLEURS test speech in 40 languages, 60 s streams from 30 to -5 dB SNR and through a phone path, under two noise families: DEMAND (homes, offices, public places, transport) and TUT urban scenes (airports, metro, streets, malls). Frame ROC-AUC per language, conditions pooled; "training h" is that language's hours in our simulated training pool.

language	training h	DEMAND: Silero v6	Clearline-VAD	distilled	TUT: Silero v6	Clearline-VAD	distilled
German	421	0.961	0.974	0.972	0.960	0.979	0.978
English	404	0.943	0.932	0.935	0.951	0.939	0.942
French	214	0.987	0.992	0.991	0.989	0.990	0.988
Spanish	200	0.956	0.955	0.961	0.958	0.949	0.960
Dutch	91	0.889	0.897	0.886	0.893	0.897	0.899
Hausa	88	0.947	0.937	0.939	0.949	0.923	0.949
Italian	71	0.969	0.965	0.967	0.967	0.957	0.961
Mandarin	54	0.946	0.950	0.952	0.932	0.924	0.935
Malay	49	0.929	0.897	0.938	0.919	0.939	0.942
Portuguese	40	0.923	0.922	0.928	0.904	0.906	0.903
Swahili	32	0.903	0.877	0.892	0.961	0.966	0.969
Polish	29	0.981	0.982	0.987	0.977	0.984	0.987
Yoruba	27	0.927	0.905	0.929	0.985	0.981	0.974
Bengali	27	0.960	0.952	0.961	0.958	0.957	0.966
Georgian	21	0.972	0.978	0.981	0.970	0.976	0.978
Japanese	21	0.857	0.867	0.859	0.878	0.886	0.884
Thai	14	0.869	0.849	0.893	0.975	0.973	0.979
Tamil	8	0.946	0.946	0.951	0.949	0.950	0.950
Cantonese	7	0.950	0.962	0.962	0.946	0.943	0.947
Russian	7	0.905	0.901	0.925	0.987	0.984	0.985
Korean	6	0.923	0.899	0.911	0.912	0.892	0.897
Ukrainian	4	0.860	0.791	0.870	0.939	0.940	0.951
Javanese	4	0.944	0.914	0.931	0.885	0.878	0.913
Telugu	4	0.974	0.969	0.976	0.973	0.970	0.980
Burmese	4	0.947	0.941	0.946	0.913	0.926	0.935
Turkish	4	0.873	0.837	0.867	0.913	0.926	0.937
Khmer	3	0.870	0.857	0.877	0.878	0.894	0.897
Urdu	3	0.921	0.915	0.942	0.986	0.989	0.990
Mongolian	3	0.906	0.876	0.894	0.937	0.940	0.940
Persian	2	0.941	0.927	0.930	0.937	0.926	0.928
Arabic	2	0.977	0.975	0.980	0.968	0.970	0.976
Greek	1	0.952	0.971	0.973	0.951	0.971	0.973
Hebrew	<1	0.936	0.952	0.959	0.917	0.927	0.940
Indonesian	<1	0.975	0.958	0.966	0.975	0.968	0.968
Hindi	<1	0.921	0.914	0.919	0.917	0.915	0.922
Amharic	<1	0.990	0.992	0.993	0.974	0.978	0.984
Vietnamese	<1	0.869	0.848	0.858	0.948	0.928	0.943
Lao	<1	0.966	0.963	0.971	0.954	0.940	0.958
Zulu	<1	0.957	0.958	0.958	0.962	0.965	0.963
Filipino	0	0.734	0.695	0.742	0.972	0.972	0.960
all 40, pooled		0.926	0.920	0.932	0.940	0.941	0.946

ahead of Silero v6 in (of 40)	noise	30 dB	10 dB	5 dB	0 dB	-5 dB	phone	pooled
Clearline-VAD	DEMAND	33	15	14	11	13	10	12
Clearline-VAD	TUT	35	21	16	18	24	15	21
distilled	DEMAND	38	27	23	14	15	25	26
distilled	TUT	34	26	22	21	31	20	29

In quiet Clearline-VAD leads in most languages. In noise it depends on the noise: level with Silero under TUT's urban scenes, behind it under DEMAND from 10 dB down. The distilled variant is the stronger of the two under both: ahead of Silero in most languages, and under DEMAND ahead from 5 dB up and a little behind at 0 and -5 dB.

Rooms full of other voices

DEMAND's cafeteria, restaurant and meeting room are where Silero kept its lead, so the distilled variant was chosen on a second unseen noise set with such rooms: FLEURS development sentences under CochlScene (13 scenes recorded on smartphones). Per scene, at 10, 5 and 0 dB, against Silero v6:

It fires more than the release in the pauses of a far-field meeting (AliMeeting), the price of learning that other people's voices in a room are background.

Telephone

The same FLEURS streams at 10 dB through five codecs at 8 kHz (* never in training), and the two development sets through the training codecs.

frame ROC-AUC	Silero v6	TEN VAD	Clearline-VAD	Clearline-VAD, distilled
G.711	0.940	0.913	0.927	0.940
GSM	0.935	0.900	0.929	0.939
Opus	0.939	0.913	0.924	0.938
AMR-NB*	0.931	0.909	0.919	0.930
G.723.1*	0.934	0.912	0.920	0.936
FLEURS, all five	0.936	0.903	0.923	0.936
dev sets on the phone	0.963	0.944	0.960	0.966

On FLEURS through the five codecs the distilled variant is level with Silero and the release a little behind; Clearline-VAD catches more turns on the line, at more false triggers (first table).

Use

import sys
from huggingface_hub import snapshot_download

sys.path.insert(0, snapshot_download("Scicom-intl/Clearline-VAD"))
from clearline.livekit import VAD

def prewarm(proc):
    proc.userdata["vad"] = VAD.load()      # threshold 0.825 (0.85 with window=256), filter 0.35 by default

session = AgentSession(vad=ctx.proc.userdata["vad"], stt=..., llm=..., tts=...)

Raw ONNX: inputs x [1, 512] float32 at 16 kHz, ctx [1, 256], mean [1, 40] (start at the mean0 metadata value), h0, h1 [1, 1, 64]; outputs p [1, 2] and the new states. The distilled graphs add a floor [1, 40] (start at zero) and carry the peak tracker in mean [1, 80], with h0, h1 [1, 1, 96]; the code below reads the shapes from the graph, so it runs either.

import numpy as np, onnxruntime as ort

sess = ort.InferenceSession("clearline_vad.onnx", providers=["CPUExecutionProvider"])
names = [i.name for i in sess.get_inputs()][1:]
state = {i.name: np.zeros(i.shape, np.float32) for i in sess.get_inputs()[1:]}
state["mean"][:] = float(sess.get_modelmeta().custom_metadata_map["mean0"])

def step(chunk):                           # 512 float32 samples at 16 kHz
    global state
    p, *new = sess.run(None, {"x": chunk[None].astype(np.float32), **state})
    state = dict(zip(names, new))
    return p[0]                            # two probabilities, one per 16 ms

Files

file	what
clearline_vad.onnx, clearline_vad_256.onnx	release model, 512- or 256-sample chunks
clearline_vad_distilled.onnx, clearline_vad_distilled_256.onnx	distilled variant (96 units, level trackers)
pytorch/*.pt	PyTorch weights for clearline/vad_model.py
clearline/	the LiveKit plugin, release model bundled

Notes

• Default threshold 0.825, not 0.5 (0.85 with window=256), chosen on development data; smoothing_alpha is LiveKit's ExpFilter alpha (weight of the previous value, 0 = off).
• The distilled graphs have their own points: activation_threshold=0.85 with smoothing_alpha=0.2 at 512-sample windows, 0.825 with 0.35 at 256.
• Feed it 16 kHz audio (AudioInputOptions(sample_rate=16000)): at 48 kHz LiveKit's resampler costs more CPU than the model.
• Weak spot: line silence on telephone legs (more false triggers than Silero); raise activation_threshold there.
• The distilled variant ranks frames best and is the one for noisy rooms and the phone; use it where frames matter more than turns, such as offline segmentation. It costs about 1.4 times the release per 32 ms (still under Silero), its 16 ms graph more than Silero, and it fires more in far-field meetings' pauses.
• Training: simulated streams from 2,728 h of speech in 137+ languages, a telephone line with real codecs, and 792 h of labelled real conversation (podcasts, a private telephone dataset, NOTSOFAR-1). No test corpus is in training; checkpoints chosen on AMI dev and VoxConverse dev, wideband and through the phone.
• License not set yet: some training data is CC BY-NC (Expresso, EARS, WHAM!).

Husein Zolkepli. Clearline: voice activity detection and noise cancellation for voice agents, on one CPU core. Scicom (MSC) Berhad, 2026.