VoxaPlus: Moroccan Darija speech-to-text (60M)

VoxaPlus is a 60M-parameter speech recognizer for Moroccan Darija that also handles Modern Standard Arabic. It is a fine-tune of oddadmix/Nawah-ASR-60M-Max-Arabic, a FastConformer encoder with a TDT (token-and-duration transducer) decoder and an auxiliary CTC head. It is small enough to transcribe in real time on a laptop CPU.

The current voxaplus.pt is phase 4, the best model so far (see the training history below).

Usage

import soundfile as sf
import parakeet_model as P                      # shipped in this repo, with fastconformer.py and parakeet_data.py
from tokenizers import Tokenizer

model = P.load("voxaplus.pt")                   # add device="cuda" for GPU
tok = Tokenizer.from_file("tokenizer_bpe1024.json")
wav, sr = sf.read("audio.wav", dtype="float32") # mono, 16 kHz, values in [-1, 1]
print(P.transcribe(model, wav, tok))

• Input is 16 kHz mono audio. Resample other rates first.
• Clips up to about 20 s give the best results, since training used 1 to 20 s segments. For long recordings, split on silences or use a sliding window, as the authors' real-time microphone demo (FastAPI + WebSocket) does.
• Output is Arabic script with no diacritics and no punctuation, matching how the model was trained.

Results

All numbers use the same scorer: corpus-level WER and CER after the Open Universal Arabic ASR Leaderboard normalisation. That normalisation removes punctuation and diacritics, unifies alef, hamza, پ/ڤ and digit variants, and attaches the conjunction و.

Model	Params	Darija: MoulSot test, 1,962 clips (WER / CER)	MSA: MASC test, 400 clips (WER / CER)
Nawah-ASR-60M (base)	60M	51.94 / 19.14	20.66 / 8.47
VoxaPlus phase 1	60M	39.84 / 15.14	30.78 / 10.45
VoxaPlus phase 3	60M	38.30 / 14.55	31.66 / 10.86
VoxaPlus phase 4 (this model)	60M	36.87 / 14.08	20.99 / 8.04
atlasia/moulsot.v0.3 (teacher)	1.7B	34.35 / 12.62	not measured

• Compared with the base model, Darija WER drops from 51.9 to 36.9 (a 29% relative reduction).
• The MSA replay data in phase 4 removed the MSA loss that the Darija-only phases had caused. MSA WER is back at the base model's level, and MSA CER is slightly better (8.04 vs 8.47).
• VoxaPlus comes within about 2.5 WER points of its 1.7B teacher at about 28× fewer parameters.
• The MoulSot v0.3 card reports 38.99 / 12.58 on MoulSot. That figure comes from a different normalisation, so it can't be compared directly with this table. Measured with the same scorer, the teacher scores 34.35 / 12.62.
• The MSA set is 400 clips from the first shard of the MASC test split (1 to 20 s). It contains no training data.

Approach

Base model

Nawah-ASR-60M-Max-Arabic has the following setup:

• Encoder: FastConformer, 8× subsampling, 128-bin log-mel input.
• Output: TDT decoder plus an auxiliary CTC head, with a 1,024-token BPE vocabulary.
• Training code: the upstream repo ships no training code. We wrote a pure-PyTorch TDT loss and checked it against brute-force path enumeration.
• Objective: TDT + 0.2 × CTC, with SpecAugment, AdamW, a cosine schedule with warmup, fp16 autocast, and batches packed by total audio duration.

Training history

Phase	Data	Outcome
1	MoulSot 100-gt-2.5: about 102 h of transcribed Darija	Darija 39.84 WER
2 / 2b	Phase 1 data plus DVoice, DODa audio and anassdabaghi/tts_darija, with and without quality filtering	Worse than phase 1, likely from spelling and domain mismatch. Dropped.
3	Phase 1 data plus about 300 h of pseudo-labels from half the unlabelled MoulSot pool	Darija 38.30 WER
4	Real labels ×2, plus 541 h of pseudo-labels, plus extra Darija and MSA replay: 1,021 h per epoch, 2 epochs, LR 5e-5, started from phase 3	Darija 36.87, MSA 20.99

Pseudo-labelling (phases 3–4)

1. Teacher selection: atlasia/moulsot.v0.3 was chosen after a bake-off against Voxtral-Mini-4B-Realtime-Arabic. The two were about equally accurate on Darija, but the moulsot teacher was much faster: about 600× real time on an A100 with vLLM.
2. Labelling: the teacher labelled the clean unlabelled MoulSot pool (extra-gt-2.5, PESQ ≥ 2.5): 242 shards, about 1,200 h, 915k clips.
3. Second opinion: VoxaPlus's own CTC head transcribed every clip as well. A clip is kept only if the two transcripts disagree on at most 40% of words and the teacher output isn't degenerate (repetition loops, implausible length for the duration). On the test set this filter cut teacher-label WER from 34.3 to 23.6, at a 35% keep rate.
4. Result: 416k clips and 541 h kept.

Extra data (phase 4)

• Darija: oddadmix Moroccan collections (noone-stories, ameed, wak3i). These have AI-assisted transcripts, so they were kept only where VoxaPlus's CTC output disagrees on at most 60% of words. Non-verbal tags were removed.
• MSA replay: about 100 h of MASC (44 of 400 train files, spread evenly), plus MediaSpeech (ar) and ClArTTS. This prevents the forgetting of MSA seen in the earlier phases.
• Total: the extra sources add about 277 h, roughly 60 h of noone-stories, 94 h of MASC, 11 h of ClArTTS, and the rest from ameed, wak3i and MediaSpeech.

Leakage control

The MoulSot test set consists of 6 YouTube channels that are absent from the training split. However, the unlabelled extra-gt-2.5 pool contains 1,850 of the test clips verbatim. All clips from those 6 channels were therefore excluded from pseudo-labelling and training:

@BASSOU, @Boughatate, @EddiePacerOfficer, @Tobiss_bla_fran, @lballonbdarija, @walo_maydi3_family

Datasets and credits

Dataset	Use	Licence (per its card)
atlasia/MoulSot-Full by AtlasIA	Labelled Darija (100-gt-2.5), unlabelled pool for pseudo-labels, test set	not stated: check the card
atlasia/moulsot.v0.3 by AtlasIA	Teacher for pseudo-labels	Apache-2.0
oddadmix/Nawah-ASR-60M-Max-Arabic	Base model, architecture code, tokenizer	not stated: check the card
oddadmix/arabic-audio-collection-moroccan-noone-stories, -ameed, -wak3i	Extra Darija	other (rights kept by the original channels)
MohamedRashad/MASC-Arabic	MSA replay + MSA test set	CC-BY-4.0
ymoslem/MediaSpeech (ar)	MSA replay	CC-BY-4.0
MBZUAI/ClArTTS	Classical Arabic replay	CC-BY-4.0

Thanks to AtlasIA for MoulSot and the moulsot teacher, to oddadmix for Nawah and the Moroccan collections, and to the MASC, MediaSpeech and ClArTTS authors. Tools: PyTorch, Hugging Face datasets / huggingface_hub, qwen-asr with vLLM (for the teacher), and Google Colab (A100).

Limitations and gaps

• Still behind the teacher on Darija. It's about 2.5 WER and 1.5 CER points behind moulsot.v0.3. Because the pseudo-labels come from that teacher, imitation alone is unlikely to beat it.
• WER is high in absolute terms. Darija spelling isn't standardised (اللي/لي, هاد/هذا, attached vs separate words), so part of the 36.9 WER is spelling disagreement rather than mishearing. CER (14.1) is the more reliable signal.
• French and code-switching. French-origin words are sometimes written in Latin script or phonetically misspelled (e.g. الطوبيس). French and English segments aren't a target: Arabic-script output only.
• Domain. Training data is mostly YouTube vlogs, stories and podcasts. Phone calls, heavy noise, children's speech and regional varieties (Amazigh-influenced, northern, Hassaniya) are under-represented and untested.
• No diacritics, punctuation or casing, and no timestamps exposed by the API.
• Single test set for Darija. Results come from the MoulSot test set, which covers 6 channels. No other Darija benchmark has been run yet.
• Licences. The oddadmix Moroccan collections are "other" (rights kept by the channels), and MoulSot and Nawah declare no licence on their cards. Commercial use requires checking those sources first.

Next steps to improve

1. More real labels, which is the biggest lever. Add human-verified Darija, especially hard cases (French code-switching, noise, phone audio). Even 20–50 h of carefully transcribed audio could help more than further pseudo-labels.
2. Use the noisy pool. Pseudo-label MoulSot extra-lt-2.5 (about 1,500 h, PESQ < 2.5) with the same two-model agreement filter, to gain noise robustness.
3. Better pseudo-labels. Combine two teachers (moulsot.v0.3 + Voxtral-Arabic: when they agree within 20%, label WER was 17.2 vs 29.3). Iterate self-training rounds, and weight samples by confidence instead of a hard threshold.
4. Longer and wider training. Run a third epoch at low LR, average the last checkpoints (SWA/EMA), and try a larger encoder if latency allows.
5. Spelling normalisation. Normalise Darija spelling consistently on both the training targets and the references (a Darija orthography map) to separate real errors from spelling variation.
6. Language model. Add shallow fusion or rescoring with a Darija n-gram or small LM trained on Darija text, such as social media or Darija Wikipedia.
7. Broader evaluation. Score on UBC-NLP/Casablanca (Morocco split), atlasia/Darija_ASR_Bench and a held-out code-switching set, and report latency and real-time factor on CPU.
8. Deployment. Export to ONNX or TorchScript with int8 quantization for mobile and edge use, and add streaming (chunked) inference with timestamps.