SayedShaun/stt_bn_fastconformer_hybrid_large_v2

Bengali FastConformer Hybrid Transducer-CTC Large (v2)

Bengali (বাংলা) speech recognition. A hybrid RNNT + CTC FastConformer, fine-tuned from NVIDIA's English stt_en_fastconformer_hybrid_large_pc on 1,048 hours of Bengali speech from eight corpora — read, spontaneous and broadcast news, both Bangladeshi and Indian.

	WER ↓	CER ↓
RNNT (default)	17.87%	6.14%
CTC	23.38%	7.68%

_{8,924 utterances · greedy decoding · no language model}

---

Quick start

pip install nemo_toolkit['asr']

from nemo.collections.asr.models import EncDecHybridRNNTCTCBPEModel

model = EncDecHybridRNNTCTCBPEModel.from_pretrained("SayedShaun/stt_bn_fastconformer_hybrid_large_v2")
print(model.transcribe(["sample_bn.wav"])[0].text)

Two decoders share one encoder. RNNT is the default and the more accurate; CTC is frame-synchronous, for simpler or lower-latency decoding:

model.change_decoding_strategy(decoder_type="ctc")   # default is "rnnt"

Input / output contract

Accepts	16 kHz mono .wav/.flac. Resample anything else first.
Segment length	0.5–20 s. Segment longer audio yourself — the model has never seen long-form input.
Returns	Bengali script, unpunctuated, no Latin characters. Bengali numerals (০-৯) are emitted; ASCII digits are not.

That output contract is a hard constraint, not a style choice: training transcripts were stripped of punctuation, and any utterance containing a Latin letter or an ASCII digit was dropped outright, so the model was never taught to emit either. Bengali numerals survived that filter and are in the tokenizer, so numbers may come back either as digits (১৮০) or as spoken Bengali words.

⚠️ Normalize your references before scoring. Training text is NFC-normalized, which decomposes the nukta letters (ড় ঢ় য়) to base + nukta — U+09A1 U+09BC, not U+09DC. They are Unicode composition exclusions, so NFC goes this direction, not the reverse. Skip this and the two spellings of one grapheme count as errors:

import unicodedata
reference = unicodedata.normalize("NFC", reference)

---

Results

Test — 8,924 utterances (14.26 h), greedy, no LM, no punctuation restoration:

Decoder	WER ↓	CER ↓
RNNT (default)	17.87%	6.14%
CTC	23.38%	7.68%

Validation — 4,494 utterances, best logged during training:

Decoder	WER ↓
RNNT	17.67%
CTC	23.23%

RNNT stays ~5.5 points ahead of CTC, as expected: RNNT conditions each token on the tokens already emitted, while CTC assumes conditional independence across frames.

The wide CER/WER gap (6.14% vs 17.87%) is characteristic of Bengali — most errors are a single morpheme or inflectional ending inside an otherwise correct word, rather than a wholly wrong word.

---

Model

Base checkpoint	stt_en_fastconformer_hybrid_large_pc (English)
Model class	EncDecHybridRNNTCTCBPEModel
Framework	NVIDIA NeMo
Parameters	~115 M
Tokenizer	Bengali SentencePiece BPE, vocab 1,024
Features	80-dim log-mel, 25 ms window / 10 ms stride
Objective	L = 0.7·L_RNNT + 0.3·L_CTC

---

Training procedure

Property	Value
Hardware	1 × NVIDIA A40 48 GB
Precision	bf16-mixed
Train / val batch size	64 / 4
Gradient accumulation	1
Gradient clipping	1.0
Optimizer	AdamW, betas (0.9, 0.98), weight decay 0.001
Learning rate	3e‑5, cosine annealing, 1,000 warm-up steps, min 3e‑6
Final logged effective LR	~4.25e‑6
Checkpoint monitor	val_wer (RNNT), top‑5 kept
Training steps	~257,950 global steps

The English base checkpoint is adapted with NeMo's change_vocabulary(), which swaps in the Bengali tokenizer and reinitializes both output heads.

The learning rate is deliberately low — 3e‑5, an order of magnitude below the 5e‑4 a from-scratch FastConformer run would use — because this is a fine-tune of an already converged acoustic model.

Built with the conformer-training-pipeline.

---

Training data

Eight corpora, pooled; all audio converted to 16 kHz mono.

Corpus	Register	License
Common Voice Scripted Speech 26.0 — Bengali (via MDC)	Crowdsourced read	CC0‑1.0
OpenSLR‑53	Crowdsourced read	CC BY‑SA 4.0
OpenSLR‑37 (bn_bd, bn_in)	Studio read, multi-speaker	CC BY‑SA 4.0
google/fleurs bn_in	Read	CC BY 4.0
bengaliAI/Ben-10	Spontaneous, 10 regional dialects	CC0
ai4bharat/Shrutilipi	All India Radio broadcast news	CC BY 4.0
ai4bharat/Kathbath	Read, 200+ Indian districts	CC0
ai4bharat/IndicVoices	Spontaneous / extempore	CC BY 4.0

Both major varieties are well represented — Bangladeshi Bengali (Common Voice, OpenSLR‑53, OpenSLR‑37 bn_bd, Ben-10) and Indian Bengali (FLEURS, OpenSLR‑37 bn_in, Shrutilipi, Kathbath, IndicVoices) — and a substantial share of the hours are spontaneous rather than read. No source targets rural or far-field speech specifically.

Split	Utterances	Hours	Duration (s) min / mean / median / max	Chars/utt	Words/utt	Chars
train	657,250	1,048.39	0.10 / 5.74 / 4.57 / 40.60	61.5	10.5	80
dev	4,494	7.15	0.21 / 5.72 / 4.60 / 33.04	60.8	10.4	72
test	8,924	14.26	0.22 / 5.75 / 4.60 / 25.79	61.9	10.7	73
combined	670,668	1,069.79	0.10 / 5.74 / 4.57 / 40.60	61.5	10.5	80

A median utterance is 4.6 s and ~10.5 words. Training filters to 0.5–20 s at the dataloader — the lower bound drops alignment failures too short to contain their transcript, the upper bound caps peak activation memory — so the tails outside that range go unused.

Speaker counts, gender balance, recording devices and noise conditions are not documented by the upstream corpora at a level that supports a breakdown here.

Text normalization

Applied to every manifest before the tokenizer is built:

1. NFC — unifies the two spellings of the nukta letters (ড় ঢ় য়) onto base + nukta; the eight sources don't agree on which they use.
2. Strip unpronounced characters — punctuation, Devanagari danda (।॥), zero-width joiners, bidi marks. Assamese ra/wa (ৰ, ৱ) are mapped to Bengali equivalents.
3. Drop any utterance still containing a Latin letter or ASCII digit — stripping them in place would leave a transcript that no longer matches its audio. Bengali numerals (০-৯) are deliberately not filtered and remain in the tokenizer.

---

Intended use

Bengali transcription · ASR research and benchmarking · a base for domain-specific fine-tuning · human-reviewed transcription workflows.

Use RNNT by default. Reach for CTC when decoding simplicity, alignment behavior or latency matters more than the ~5.5-point WER gap.

Limitations

The training data makes these impossible, not merely weak:

• Punctuation of any kind
• Latin script and ASCII digits, and therefore code-switched Bengali-English speech — utterances containing either were removed from training outright

---

License

CC BY 4.0 — free for commercial use, with attribution. The same license NVIDIA applies to the base checkpoint.

These weights are a fine-tune of NVIDIA's stt_en_fastconformer_hybrid_large_pc (CC BY 4.0), trained on the eight corpora above: CC0 (Common Voice Scripted Speech 26.0, Ben-10, Kathbath), CC BY 4.0 (FLEURS, Shrutilipi, IndicVoices) and CC BY‑SA 4.0 (OpenSLR‑53, OpenSLR‑37). None restrict commercial use. If you redistribute these weights or a derivative, credit NVIDIA's base checkpoint and all eight corpora.

Several source corpora are gated and carry their own terms of use. Those terms bind your use of the data, not of these weights — but if you rebuild the training set, you must accept them yourself.

The share-alike term on OpenSLR‑53 and OpenSLR‑37 is treated here as not propagating to trained model weights. That reading is common practice but not settled law; if your use requires certainty, seek your own legal advice.

Pipeline code is MIT.

Citation

@misc{shaun_bengali_fastconformer_v2_2026,
  author = {Sayed Shaun},
  title  = {Bengali FastConformer Hybrid Transducer-CTC Large (v2)},
  year   = {2026},
  url    = {https://github.com/sayedshaun/conformer-training-pipeline}
}

Architecture references:

@article{rekesh2023fastconformer,
  title   = {Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognition},
  author  = {Rekesh, Dima and Koluguri, Nithin Rao and Kriman, Samuel and others},
  journal = {arXiv preprint arXiv:2305.05084},
  year    = {2023}
}

@article{gulati2020conformer,
  title   = {Conformer: Convolution-augmented Transformer for Speech Recognition},
  author  = {Gulati, Anmol and Qin, James and Chiu, Chung-Cheng and others},
  journal = {arXiv preprint arXiv:2005.08100},
  year    = {2020}
}