nassimaODL/streaming-vad-crnn

Streaming Neural VAD — Causal CRNN (47,873 params)

A small-footprint, causal, streaming-oriented Voice Activity Detector built for an on-device far-field target. Four causal depthwise-separable Conv1D residual blocks (kernel 5, dilations {1,2,4,8}) followed by a unidirectional GRU (hidden 64) and a per-frame linear head, over 64-bin log-mel features at a 10 ms frame rate.

• Parameters: 47,873 (~0.19 MB FP32)
• Look-ahead: 0 ms in the network; 12.5 ms from the centred 25 ms analysis window
• CPU end-to-end RTF: 0.0035 on 60 s inputs (single thread, Xeon Gold 6248)
• Framework: PyTorch 2.8

Intended use

Frame-level speech/non-speech decisions upstream of wake-word / ASR gating. Trained on clean read English (LibriSpeech-style) with additive MUSAN noise; it is a research prototype, not validated on real far-field, multi-speaker, or reverberant audio (see Limitations).

Results

Held-out, speaker-disjoint clean test partition (93 utterances, 4 speakers). All operating thresholds are selected on validation only, then frozen.

System	Precision	Recall	F1	FA	Miss	Collared DER
Energy (calibrated)	0.945	0.941	0.943	0.294	0.059	7.07%
Silero (calibrated)	0.976	0.976	0.976	0.126	0.024	—
CRNN raw	0.986	0.966	0.976	0.073	0.034	1.15%
CRNN + post	0.987	0.970	0.978	0.069	0.030	0.64%

Continuous-score: CRNN AUROC 0.984 / AUPRC 0.997; Silero 0.982 / 0.996. A paired 10k-utterance bootstrap gives a 95% CI of [0.0009, 0.0035] for the clean F1 difference (CRNN − calibrated Silero).

Noise robustness (frame F1, held-out MUSAN, global-RMS SNR):

SNR (dB)	−5	0	5	10	15	20
CRNN + post	0.932	0.952	0.961	0.969	0.972	0.974
Silero	0.913	0.950	0.966	0.969	0.972	0.974
Energy	0.916	0.917	0.917	0.918	0.919	0.924

Full analysis, curves, and caveats are in the accompanying technical report.

Usage

from inference import VAD
vad = VAD("vad_crnn_export")            # dir with model.safetensors + config.json
probs, mask, times = vad("clip.wav")    # frame probs, binary mask, frame centres (s)

vad_model.py must contain your real architecture — see its docstring.

Files

• vad_crnn_export/model.safetensors, vad_crnn_export/config.json — weights + config
• inference.py — standalone loader / predictor
• vad_model.py — architecture factory (you paste your crnn.py here)
• export_checkpoint.py — regenerates the export from the training checkpoint

Limitations

Clean read English only; additive-noise robustness under a global-RMS SNR definition; reverberation untested in the released model; four test speakers; single training seed; server-CPU (not target-device) latency; no stateful chunked-streaming benchmark. Do not treat in-domain F1 as deployed performance.

Training data & attribution

Provided challenge corpus (LibriSpeech-derived) + MUSAN music/noise for augmentation. The corpus and any derived mixtures are not redistributed here. LibriSpeech (Panayotov et al., 2015), MUSAN (Snyder et al., 2015).