Nemo-Arabic-STT-Diacritized

An Arabic speech-to-text pipeline that produces fully diacritized (tashkeel) transcripts. It combines two independently developed models in sequence; it is a packaged inference pipeline, not a single jointly trained architecture.

Pipeline

1. Speech recognition: NVIDIA NeMo FastConformer Hybrid (stt_ar_fastconformer_hybrid_large_pcd_v1.0) transcribes Arabic speech to plain, undiacritized text.
2. Diacritization: CATT (Character-based Arabic Tashkeel Transformer) adds tashkeel to the transcript.

Audio in, diacritized Arabic text out. Neither checkpoint was retrained or fine-tuned for this repository.

Files

File	Description	Source
stt_ar_fastconformer_hybrid_large_pcd_v1.0.nemo	ASR checkpoint, unmodified	NVIDIA, CC-BY-4.0
best_ed_mlm_ns_epoch_178.pt	Diacritizer checkpoint, unmodified	abjadai/CATT, Apache-2.0
diacritize.py, catt/	Diacritizer inference code (vendored from CATT)	abjadai/CATT, Apache-2.0
pipeline.py	DiacritizedASR — loads both models once, audio in / diacritized text out in a single call	This repository
server.py	Reference FastAPI server implementing the full pipeline	This repository

Usage

from pipeline import DiacritizedASR

model = DiacritizedASR(
    nemo_path="stt_ar_fastconformer_hybrid_large_pcd_v1.0.nemo",
    catt_ckpt="best_ed_mlm_ns_epoch_178.pt",
)
diacritized = model.transcribe("audio.wav")

Both models load once at construction; each .transcribe() call runs ASR followed immediately by diacritization in the same process. For the two steps individually:

import nemo.collections.asr as nemo_asr
from diacritize import Diacritizer

asr_model = nemo_asr.models.EncDecHybridRNNTCTCBPEModel.restore_from(
    "stt_ar_fastconformer_hybrid_large_pcd_v1.0.nemo"
)
diacritizer = Diacritizer(ckpt="best_ed_mlm_ns_epoch_178.pt")

text = asr_model.transcribe(["audio.wav"])[0].text
diacritized = diacritizer.diacritize_texts([text])[0]

Or run server.py directly for an HTTP API (POST /transcribe, GET /health).

Example

Input audio (Arabic speech) transcribed and diacritized:

plain:       السلام عليكم ورحمة الله وبركاته كيف يمكنني مساعدتك اليوم؟
diacritized: السَّلَامُ عَلَيْكُمْ وَرَحْمَةُ اللَّهِ وَبَرَكَاتُهُ كَيْفَ يُمْكِنُنِي مُسَاعَدَتُكَ الْيَوْمَ؟

Limitations

• Diacritization quality depends on ASR transcript quality; transcription errors propagate to diacritization.
• CATT diacritizes using full-sentence context; very short or ambiguous transcripts may diacritize imperfectly.
• Developed and tested on general Modern Standard Arabic conversational speech, not evaluated on Quranic recitation.

Attribution and License

• ASR model: NVIDIA, stt_ar_fastconformer_hybrid_large_pcd_v1.0, CC-BY-4.0.
• Diacritizer: abjadai/CATT, Apache-2.0.
• This repository (packaging and inference code) is released under CC-BY-4.0, consistent with the ASR model's license and compatible with CATT's Apache-2.0 terms.

Built for the Muslim Arabic voice AI companion project.