vrc-we-voice-eo

Esperanto keyword-spotting model for vrc-WiredExpressions: wav2vec2 CTC over letters, 20 ms frames, 16 transformer layers, int8 ONNX (254 MB).

Files

File	Content
model.onnx	Input input_values: float32 [batch, samples], 16 kHz mono, zero mean, unit variance. Output logits: [batch, frames, 211]. int8 dynamic quantization of the MatMuls.
vocab.json	Token ids; <pad> (0) is the CTC blank, `
config.json	Wav2Vec2Config of the pruned network.
voice.json	Language, units, firing threshold (-0.6).

Training

Fine-tuned from facebook/wav2vec2-xlsr-53-espeak-cv-ft (Apache 2.0), pruned to its lower 16 of 24 layers, with CTC on letters.

Dataset	Licence	Utterances
Common Voice 19.0	CC0 1.0	161,628
Multilingual Spoken Words Corpus	CC BY 4.0	105,369

Its first stages also trained on the Common Voice, MSWC and FLEURS data of English, German, French, Spanish and Russian.

Noise: MUSAN (CC BY 4.0).

Evaluation

Local keyword-spotting tests, not a standard benchmark. Phrases are registered as text and spotted in continuous audio. Base: facebook/wav2vec2-xlsr-53-espeak-cv-ft. Both models run int8 at their own threshold (this model -0.6, the base -0.75).

Hits % / false fires per hour	wav2vec2-xlsr-53-espeak-cv-ft (base)	vrc-we-voice-eo
Words	68.0 / 0.48	82.0 / 0.24
Words, noisy	56.0 / 1.68	61.3 / 0.00
Crowded	68.0 / 0.48	82.0 / 0.24
Non-speech	— / 0.25	— / 0.07

• Sentences: one phrase per FLEURS test sentence; noisy adds noise at 10 dB SNR.
• Words: isolated Multilingual Spoken Words test clips, from speakers held out of training.
• Crowded: every sentence and word phrase registered at once.
• Non-speech: music (MUSAN) and vocal sounds (VocalSound), every phrase registered; the worst of five operating points.

Addendum: training log

Shared settings: batches of 40 s of audio, 2 accumulated per step; AdamW, 10% warm-up, linear decay; all layers and the head trained; utterances 0.5 to 15 s; MUSAN noise in half the utterances at 5 to 20 dB SNR, and 5% noise-only samples with an empty target.

A continuation was kept when its hit rate, averaged over five false-fire budgets (0.05 to 1 per phrase-hour) and the speech tests, rose by at least one point in total, no test fell by more than one, and non-speech false fires stayed at or under 0.1 per phrase-hour.

Stage	Outcome
Pruned to 16 layers, new letter head; six languages, 30,000 steps	kept
Six languages, English drawn 3×, 15,000 steps, 3e-5	kept
Esperanto only, 8,000 steps, 3e-5	kept
Continuation, 6,000 steps, 1.5e-5	kept: net +4.5
Continuation, 6,000 steps, 1.5e-5	kept on review: net +8.7, noisy words +11.1, clean words −1.2
Continuation, 6,000 steps, 1.5e-5	kept: net +3.2
Continuation, 6,000 steps, 1.5e-5	kept: this model (net +3.5)
Continuation, 6,000 steps, 1.5e-5	rejected: net −3.2 (noisy words −5.6)

Licence

Apache 2.0, as the base models. Dataset licences are listed above.