mrmuminov/uzbek-fastconformer-ctc-uz

Uzbek character-level CTC (FastConformer, cased with punctuation)

A character-level CTC speech recognition model for Uzbek (Latin script). It outputs cased text with light punctuation (. , ? ! : - ") and the Uzbek letters oʻ, gʻ and the tutuq belgisi ’. It uses no language model: decoding is plain greedy CTC.

It is fine-tuned from the encoder of nvidia/stt_uz_fastconformer_hybrid_large_pc (CC-BY-4.0, NVIDIA). The original BPE/RNN-T heads are dropped and replaced with a new character head.

Architecture

• Mel front-end and 17-layer FastConformer encoder from the NVIDIA checkpoint (12.5 frames/s).
• New head: ConvTranspose1d(512, 512, k=2, s=2) -> ReLU -> Conv1d(512, vocab_size, 1). The 2x upsampling gives 25 frames/s so that fast speech and long words fit in CTC.
• Vocabulary: 62 characters sorted by code point (vocab.json), the CTC blank is the last id (62). About 109.4M parameters.

Text normalisation

Training transcripts were normalised (code/asr_uz/text.py), so the model writes in this style:

• ‘ ʻ are unified to ʻ; ' ´ ’ ʼaftero/gbecomeʻ (o’zbek->oʻzbek), otherwise ’`.
• Quotes become ", dashes become -, ; becomes ,, spaces before punctuation are removed.
• Clips containing digits, Cyrillic or other symbols were dropped. The model does not write numbers as digits.

Training data

mrmuminov/uzbek_voice: 860,565 clips (1019 h) of read sentences from 5,558 speakers. After normalisation 855,533 clips remained.

• Split by speaker and by sentence: 60 test speakers never appear in train, held-out sentences never appear in train.
• Evaluation sets use only verified clips (upvotes >= 2 and no downvotes, or is_valid == 1).
• Training used 693,692 clips (826.9 h, 0.5 to 16 s, 5 to 22 characters/s), including clips that nobody reviewed (about 85% of the training audio has no upvotes). 18,050 steps of about 300 s of audio (about 1.6 epochs), bf16, AdamW, EMA weights, SpecAugment, speed/gain/noise augmentation. The released weights are the EMA weights at step 16,500, the best validation WER of the run, which was stopped early because validation had flattened.

Results

Greedy decoding on the original audio. "Relaxed" lowercases and removes punctuation from both sides, so it measures only what was pronounced; "strict" also counts case and punctuation.

Set	Clips	Hours	WER relaxed	CER relaxed	WER strict	CER strict
val	3,000	3.4	2.08%	0.39%	3.41%	0.57%
test_seen_text	3,378	3.9	1.84%	0.32%	3.38%	0.53%
test_unseen_text	485	0.5	1.44%	0.22%	2.50%	0.35%

• val: training speakers reading held-out sentences.
• test_seen_text: 60 held-out speakers, sentences that also occur in training.
• test_unseen_text: 60 held-out speakers, held-out sentences (the strictest set, but only 485 clips).

WER relaxed by accent region (regions with at least 50 clips):

Region	val	test_seen_text
Andijon	2.1%	1.6%
Buxoro	1.8%	0.8%
Farg'ona	1.8%	2.1%
Jizzax	2.7%	2.5%
Namangan	2.6%	1.9%
Navoiy	1.8%	1.8%
Qashqadaryo	1.9%	2.9%
Qoraqalpog'iston	3.1%	0.3%
Samarqand	1.4%	2.4%
Sirdaryo	-	2.6%
Surxondaryo	1.3%	1.7%
Toshkent shahri	3.4%	1.9%
Toshkent viloyati	2.3%	1.7%
Xorazm	1.1%	1.4%

Read these numbers with care

• The evaluation data comes from the same corpus as the training data: read news-style sentences, mostly clean recordings, and only clips that other people verified. This is not conversational or noisy speech.
• The base model lists UzbekVoice among its training data, so it may have seen these sentences or speakers before fine-tuning.
• Length: training clips were at most 16 s, and the model degrades on longer audio. Rough test on same-speaker clips concatenated to a target length (relaxed WER, about a dozen samples each, so only a trend): ~5 s 2.0%, ~16 s 7.1%, ~24 s 11.3%, ~30 s 18.4%, ~45 s 25.3%. Cut long recordings into segments of 15 s or less.
• No evaluation on an independent set (Common Voice, FLEURS) has been done yet. Expect noticeably higher error there.

ONNX

onnx/ holds the encoder and head (mel features in, log-probs out; the mel front-end is not in the graph, use fb.bin and the config for it). On 200 test_unseen_text clips the fp32 graph matches PyTorch exactly and the int8 graph (MatMul only) is 2.6x smaller with a small loss:

Model	WER strict	WER relaxed	Size
PyTorch	2.48%	1.23%	438 MB
ONNX fp32	2.48%	1.23%	459 MB
ONNX int8	2.98%	1.64%	174 MB

Usage

pip install torch torchaudio soundfile "nemo-toolkit[asr]" safetensors pyyaml
git clone <this repo> && cd <this repo>
PYTHONPATH=code python -m asr_uz.infer audio.wav --model .

Any audio format soundfile reads works; it is resampled to 16 kHz mono. The training code is in code/asr_uz/.