mrmuminov/quran-fastconformer-ctc-ar

Quranic character-level CTC (FastConformer, Uthmani with diacritics)

A character-level CTC acoustic model for Qur'an recitation. It outputs fully diacritized Uthmani text (62 characters, kept verbatim: no Unicode normalisation) and uses no language model: decoding is plain greedy CTC.

It is fine-tuned from the encoder of nvidia/stt_ar_fastconformer_hybrid_large_pcd_v1.0 (CC-BY-4.0, NVIDIA). The original BPE/RNN-T heads are dropped and replaced with a new head.

What this is, and what it is not

This is a transcription model, not a mispronunciation or tajweed detector. It was trained only on professional recitations and has never seen deliberately incorrect recitation. Do not use its output as a verdict on whether a student recited correctly. Treat it as an acoustic baseline for building such a system.

Architecture

• Mel front-end and 17-layer FastConformer encoder from the NVIDIA checkpoint (108.8M parameters, 12.5 frames/s).
• New head: ConvTranspose1d(512, 512, k=2, s=2) -> ReLU -> Conv1d(512, 63, 1). The 2x upsampling gives 25 frames/s, because fast reciters exceed 12.5 characters/s and CTC needs at least one frame per character.
• Vocabulary: 62 characters sorted by code point, the CTC blank is the last id (62).
• About 109.3M parameters in total.

Training data

A 37-reciter recitation dataset (226,129 clips, 6,236 ayat per reciter, all 114 suras, text identical across reciters).

• Excluded from this model: 3 Warsh reciters (Warsh audio with Hafs labels, 18.7k clips).
• Training used 28 reciters, 104 suras, 150,994 clips, 740.6 h (clips of 0.5 to 100 s, at most 22 characters/s).
• Known problem in the training data: two of the 28 reciters, Abdullaah_3awwaad_Al-Juhaynee (62% CER even on validation) and khaleefa (partial, 1,733 clips), have audio that does not match its text. They were still in this model's training set. See "Experiments" below for a retrain without them.

Results

Corpus-level CER (total edits / total reference characters), greedy decoding, on the original 16 kHz audio. "No diacritics" removes combining marks, tatweel and the small waw/yeh signs from both sides.

Set	Clips	Hours	CER	CER (no diacritics)	Sub	Del	Ins
val: 28 training reciters, 10 held-out suras	18,252	86.1	0.0355	0.0344	0.008	0.026	0.002
test: 6 held-out reciters, suras seen in training	33,378	192.1	0.0310	0.0304	0.002	0.022	0.007
test: 6 held-out reciters, 10 held-out suras	4,038	21.4	0.0337	0.0330	0.003	0.026	0.005

Held-out reciters: As-Sudais, Alafasy, Maher Al-Muaiqly and all three Minshawy recordings (one person is never split across train and test). Held-out suras: 4, 23, 26, 31, 45, 58, 73, 96, 100, 104.

Per held-out reciter (held-out suras):

Reciter	CER
Minshawy_Murattal	0.0115
Abdurrahmaan_As-Sudais	0.0120
Alafasy	0.0128
Maher_AlMuaiqly	0.0164
Minshawy_Mujawwad	0.0336
Minshawy_Teacher	0.1157

The aggregate is dominated by reference problems, not only model errors. Most reciters are near 1%. For Minshawy_Teacher the audio of long clips (over 60 s) contains neighbouring ayat that the reference text lacks, and for Minshawy_Mujawwad clips over 100 s repeat phrases that the reference does not contain. In both cases the model transcribes what is audible, so the reference is wrong rather than the model; CER still counts it as error.

Experiments

Dropping the two mismatched reciters did not help on the held-out test. A second run with the same recipe, without Abdullaah_3awwaad_Al-Juhaynee and khaleefa (26 reciters, 143,827 clips, 713.7 h), reached a better validation CER (1.48%, but on a different validation set because those reciters were removed from it) and a worse CER on the same held-out test set: 5.13% against 3.37% on the unseen suras, worse for all six test reciters (for example As-Sudais 1.43% against 1.20%, Minshawy_Teacher 20.8% against 11.6%). The cause was not isolated: it may be run-to-run variance (a single seed per run) or the acoustic variety of the removed reciters. This published model is the one that scored better on the test set, so it was kept.

Limitations

• Professional reciters only. No amateur or student speech and no noisy microphone conditions were evaluated.
• "Held-out reciter" is weaker than it sounds. The reciter names match EveryAyah folders, and the NVIDIA base encoder was trained on EveryAyah, so the encoder may have heard these reciters (and possibly the exact clips). The numbers are not evidence of generalisation to unknown speakers.
• Unseen suras are not fully unseen text: the Qur'an repeats phrases (the basmala, refrains), so memorised text can still help. A text prior in the encoder or head can also auto-complete what was not pronounced; this has not been probed yet (splice and non-Qur'an tests are not done).
• Training clips were limited to 100 s; longer clips are handled worse (see the Mujawwad note above).
• Remaining errors are mostly deletions (about 2.5% of characters).

Usage

Requires nemo-toolkit[asr], torch, soundfile, torchaudio, safetensors, pyyaml (tested with Python 3.12, nemo-toolkit 3.0.0, torch 2.14).

cd code
python -m quranic.infer --model .. --confidence recitation.wav

from quranic.model import CharCtc
from quranic.vocab import Vocab
from quranic.infer import transcribe

model = CharCtc.from_release("path/to/this/repo", "cuda")
vocab = Vocab.load("path/to/this/repo/vocab.json")
print(transcribe(model, vocab, ["recitation.wav"], "cuda"))

The output includes the per-character posterior (char_confidence), so low-confidence regions can be flagged rather than silently accepted.

Training

Code is in code/quranic/ (make_split.py, train_ctc.py, evaluate.py).

• Optimiser: AdamW (betas 0.9/0.98), encoder LR 5e-5, head LR 1e-3, weight decay 1e-3 on encoder matrices, encoder frozen for the first 400 steps, linear warm-up, cosine decay to 1%, gradient clip 1.0, EMA 0.999.
• 20,000 steps of about 390 s of audio each (about 2.55 epochs), bf16 autocast, one RTX 4060 laptop GPU (8 GB), about 2.9 h.
• Augmentation: SpecAugment (2 freq masks of 15, 4 time masks of 12), speed 0.9/1.1, gain +-6 dB, noise 20-40 dB SNR. Speed perturbation changes madd duration; do not reuse it for duration-sensitive downstream tasks.
• Checkpoint: EMA weights with the best validation CER.

ONNX and in-browser use

onnx/model_int8.onnx (174 MB) takes log-mel features [batch, 80, frames] and lengths, and returns log-probabilities [batch, 2 * frames / 8, 63]; the CTC blank is the last index. MatMuls are quantized to int8 (convolutions stay fp32, since the browser WASM runtime has no integer convolution). onnx/fb.bin is the mel filterbank (80 x 257 float32). On 200 held-out clips the int8 model scored CER 1.30% against 1.32% for fp32 (Python, ONNX Runtime); a JavaScript re-implementation of the mel front-end plus onnxruntime-web scored 1.81% on a different 40-clip set where Python fp32 scored 1.95%. code/quranic/export_onnx.py reproduces the export.

A static, in-browser demo lives in the Space mrmuminov/quran-recitation-asr.

Demo app

space/ contains a Gradio demo (transcript, per-character confidence colours, optional comparison with an expected text). Run it locally:

cd space
pip install -r requirements.txt gradio
python app.py

It downloads this model from the Hub on start. To host it, create a Gradio Space and upload the contents of space/ (app.py, requirements.txt, README.md); note that Hugging Face currently requires a PRO subscription for Gradio Spaces on free CPU hardware.

License and attribution

CC-BY-4.0. This model is derived from NVIDIA's stt_ar_fastconformer_hybrid_large_pcd_v1.0 (CC-BY-4.0); please credit NVIDIA and link the base model when using it.