SayedShaun/stt_bn_fastconformer_hybrid_large_v1

Bengali FastConformer Hybrid Transducer-CTC Large

Bengali (বাংলা) speech recognition. A hybrid RNNT + CTC FastConformer fine-tuned from NVIDIA NeMo's stt_en_fastconformer_hybrid_large_pc using the conformer-training-pipeline.

One shared FastConformer encoder feeds two decoders trained jointly (auxiliary CTC loss weight 0.3):

• RNNT — the primary path, and the more accurate one.
• CTC — frame-synchronous, for simpler or lower-latency decoding.

Trained on ~300 h of Bengali read speech (262k utterances).

Held-out test set: 14.47% WER / 3.96% CER (RNNT, greedy, no LM).

Usage

pip install nemo_toolkit['asr']

from nemo.collections.asr.models import EncDecHybridRNNTCTCBPEModel

model = EncDecHybridRNNTCTCBPEModel.from_pretrained("SayedShaun/stt_bn_fastconformer_hybrid_large_v1")
output = model.transcribe(["sample_bn.wav"])
print(output[0].text)

Switch to the CTC head:

model.change_decoding_strategy(decoder_type="ctc")   # default is "rnnt"

Input: 16 kHz mono audio (.wav/.flac); resample anything else first. Output: Bengali text in Unicode Bengali script. Output does contain punctuation — sentence-final । and commas, inherited from the source corpora, which were not stripped before training. No capitalization (Bengali script is caseless). Strip punctuation yourself if your downstream scoring expects none.

Model details

Property	Value
Base checkpoint	stt_en_fastconformer_hybrid_large_pc
Model class	EncDecHybridRNNTCTCBPEModel
Framework	NVIDIA NeMo
Tokenizer	Bengali SentencePiece BPE, vocab 1,024
Sample rate	16 kHz
Primary / auxiliary decoder	RNNT / CTC (loss weight 0.3)

Training data

Corpus	Role	License
Common Voice Scripted Speech 26.0 — Bengali (via Mozilla Data Collective)	train / dev / test	CC0‑1.0
OpenSLR‑53 — Large Bengali ASR training data set	train / dev / test	CC BY‑SA 4.0
google/fleurs bn_in	train only	CC BY 4.0

All three are crowdsourced or studio-recorded read speech, predominantly Bangladeshi Bengali (Common Voice, OpenSLR‑53) with some Indian Bengali (FLEURS). Audio is converted to 16 kHz mono.

Statistics

Split	Utterances	Hours	Duration (s): min / mean / median / max	Chars/utt	Words/utt
train	262,415	299.77	0.10 / 4.11 / 3.60 / 38.52	28.3	4.5
dev	1,794	2.03	1.60 / 4.08 / 3.50 / 21.90	27.7	4.4
test	3,561	3.98	1.50 / 4.02 / 3.50 / 28.56	27.1	4.3
combined	267,770	305.78	0.10 / 4.11 / 3.60 / 38.52	28.3	4.5

The corpus is short-form: a median utterance is 3.6 s and ~4.5 words, so the model has effectively never seen long-form audio. Training filters to 0.1–20 s at the dataloader, so the small tail above 20 s present in the manifests is unused. The train split spans 166 unique characters against 106–113 in dev/test — the surplus is punctuation, Latin script and stray symbols that survived normalization.

FLEURS is folded entirely into train — its own dev/test splits included — because the held-out dev/test sets are drawn only from Common Voice and OpenSLR‑53, so there is no contamination risk. The test manifest is the Common Voice bn test split plus 500 OpenSLR‑53 utterances held out at a fixed seed.

Speaker counts, gender and dialect distribution, recording devices and noise conditions are not documented by the upstream corpora at a level that supports a breakdown here.

Training procedure

Property	Value
Hardware	1 × NVIDIA A40 48GB
Total global steps	777,000
Train / val batch size	32 / 4
Optimizer	AdamW, betas (0.9, 0.98), weight decay 0.001
Base learning rate	5e‑4, cosine annealing, 1,000 warm-up steps
Observed LR this segment	~1.36e‑5 → ~3.18e‑6
SpecAugment	2 freq masks (max width 27), 10 time masks (width 0.05)
Checkpoint selection	lowest val_wer (RNNT), top‑5 kept

The encoder is frozen for the first 500 steps of a fresh run so the reinitialized Bengali decoder can settle before gradients reach the pretrained acoustic encoder. The English base checkpoint is adapted via NeMo's change_vocabulary(), which swaps in the Bengali SentencePiece tokenizer. The effective learning rate over the recorded segment sits far below the configured base rate because the scheduler was already deep into cosine decay on resume.

Evaluation

Test set

3,561 held-out utterances (Common Voice bn test + 500 OpenSLR‑53), RNNT greedy decoding, no external language model:

Metric	Value
WER	14.47%
CER	3.96%

The CTC head was not separately evaluated on the test set; the validation gap below is the best available estimate of its cost.

Validation (dev manifest, greedy, no LM)

Selection objective	Epoch	Global step	RNNT WER	CTC WER
Best RNNT (released)	90	745,379	13.61%	21.08%
Best CTC	78	647,089	13.79%	20.80%
RNNT/CTC compromise	85	704,427	13.68%	20.91%
Last complete validation	93	769,953	13.73%	20.82%

Over the recorded segment RNNT improved 15.09% → 13.61% WER (1.48 points, ~9.8% relative) and CTC 22.38% → 20.80% (1.58 points, ~7.1% relative). RNNT stays ~7 points ahead of CTC throughout, as expected — RNNT conditions on previously emitted tokens, CTC assumes conditional independence across frames.

Released weights are the epoch‑90 checkpoint, selected on RNNT validation WER. Do not graft the epoch‑90 RNNT head onto the epoch‑78 CTC head: the two checkpoints carry different shared-encoder weights.

Intended uses

Bengali transcription; ASR research and benchmarking; a starting point for domain-specific Bengali fine-tuning; human-reviewed transcription workflows. Use RNNT by default; reach for CTC when decoding simplicity, alignment behavior, or latency matters more than the ~7-point WER gap.

Out-of-scope uses

Not validated for: languages other than Bengali; speaker identification or verification; medical, legal, or emergency transcription without human review; surveillance or biometric profiling; noisy far-field audio; code-switched Bengali-English speech; streaming production latency; and dialects or demographics absent from the training data.

Bias, risks and recommendations

ASR accuracy varies substantially across dialects, speaker demographics, disability, recording conditions and socioeconomic groups, and nothing in this model's training or evaluation data measures that variation. Before deploying: evaluate on data representative of your own speakers; report WER by dialect, speaker group, audio duration and noise condition; test proper nouns, numerals, dates and code-switched speech separately; document your Bengali normalization and punctuation policy; and keep a human in the loop for consequential transcripts.

License

Released under CC BY 4.0 — free for commercial use, with attribution — the same license NVIDIA applies to the base checkpoint.

These weights are a fine-tune of NVIDIA's stt_en_fastconformer_hybrid_large_pc (CC BY 4.0), trained on Common Voice Scripted Speech 26.0 Bengali (CC0‑1.0), FLEURS (CC BY 4.0) and OpenSLR‑53 (CC BY‑SA 4.0). None of those licenses restrict commercial use. If you redistribute these weights or a derivative, credit NVIDIA's base checkpoint and the three corpora above.

OpenSLR‑53's share-alike term is treated here as not propagating to trained model weights. That reading is common practice but not settled law; if your use requires certainty, seek your own legal advice.

Pipeline code is MIT.

Citation

@misc{shaun_bengali_fastconformer_2026,
  author = {Sayed Shaun},
  title  = {Bengali FastConformer Hybrid Transducer-CTC Large},
  year   = {2026},
  url    = {https://github.com/sayedshaun/conformer-training-pipeline}
}

Architecture references:

@article{rekesh2023fastconformer,
  title   = {Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognition},
  author  = {Rekesh, Dima and Koluguri, Nithin Rao and Kriman, Samuel and others},
  journal = {arXiv preprint arXiv:2305.05084},
  year    = {2023}
}

@article{gulati2020conformer,
  title   = {Conformer: Convolution-augmented Transformer for Speech Recognition},
  author  = {Gulati, Anmol and Qin, James and Chiu, Chung-Cheng and others},
  journal = {arXiv preprint arXiv:2005.08100},
  year    = {2020}
}