sailu4/lmaana-v2

Lmaana v2: Controlled Mixed-Replay Adaptation for Moroccan Darija ASR

Lmaana v2 is a native fairseq2/OmniASR CTC checkpoint adapted for Moroccan Darija automatic speech recognition. It continues the Lmaana v1 line with controlled mixed replay over Dataset13 and MoulSot.

Status: research checkpoint. It is useful for studying the trade-off between Dataset13 retention and MoulSot adaptation; it is not a production ASR release.

TL;DR

• Architecture: fairseq2 wav2vec2_asr, 1b_v2 model family.
• Tokenizer: official omniASR_tokenizer_written_v2.
• Adaptation: controlled mixed replay of Dataset13 and MoulSot.
• Selected variant: v2-B, peak learning rate 2e-7, step 250.
• Validation: MoulSot WER improves from 40.5473% for Lmaana v1 to 40.4653%; Dataset13 changes from 43.2483% to 43.2973%.

Model Details

Property	Value
Model family	wav2vec2_asr
Architecture	1b_v2
Training objective	Connectionist Temporal Classification (CTC)
Tokenizer	omniASR_tokenizer_written_v2
Training continuation	From the retained Lmaana v1 checkpoint with a fresh experiment output directory
Selected step	250
Peak learning rate	2e-7

Data

The mixed dataset contains Moroccan Darija speech from two corpora:

Corpus	Validation examples	Test examples
Dataset13	6,893	6,576
MoulSot	5,198	1,960

The physical Parquet records use text, audio_bytes, and audio_size. Dataset and split identity are supplied through the partition layout. The evaluation identities were verified against the raw audio bytes; eight validation rows correspond to duplicated audio content.

Training and Selection

Two learning-rate variants were trained for 250 local continuation steps: v2-A (1e-7) and v2-B (2e-7). Checkpoint selection was based on validation WER using a retention/adaptation gate: Dataset13 regression no greater than 0.10 absolute WER points and MoulSot improvement of at least 0.03 absolute WER points. Both candidates passed; v2-B was selected because it had the larger MoulSot improvement.

The held-out test set was not used for selecting the checkpoint.

Results: Validation

Split	Dataset13 WER	MoulSot WER
Validation	43.2973	40.4653

Corpus	CTC loss	UER	WER	Examples
Dataset13	176.3170	17.3914	43.2973	6893
MoulSot	74.9353	13.2148	40.4653	5198

Results: Held-Out Test

The final test evaluation used the selected v2-B checkpoint after validation-based selection.

Corpus	CTC loss	UER	WER	Examples
Dataset13	184.1440	17.7805	44.2674	6576
MoulSot	74.8520	14.6398	45.2140	1960

Compared with the Lmaana v1 test reference, v2-B improves MoulSot WER by 0.0249 absolute points and increases Dataset13 WER by 0.0446 absolute points. Therefore, v2-B is the preferred MoulSot-adaptation candidate, not an unqualified improvement on every domain.

Robustness Audit

• dataset13: WER 43.30% over 6893 examples.
• moulsot: WER 40.47% over 5198 examples.

Figures

Files

• checkpoint/: complete native resumable fairseq2 checkpoint.
• assets/: generated model-card figures.
• metadata/: evaluation summaries and robustness metrics.

Evaluation Protocol

WER, UER, and CTC loss were computed separately on the Dataset13 and MoulSot validation/test partitions with greedy CTC decoding. The same checkpoint, tokenizer, split definitions, and evaluation procedure were used for both corpora. The robustness audit additionally reports duration buckets, a reference-script proxy, and substitution/deletion/insertion counts.

Reproducibility

The complete native checkpoint is stored under checkpoint/. The training configuration, evaluation summaries, audit metrics, plotting scripts, and notebook are maintained in the companion GitHub repository. Large private datasets and raw transcriptions are not included in this public model card.

Limitations

This checkpoint is a research artifact, not a production ASR system. The robustness audit uses verified audio identities; its CER is diagnostic and is not a replacement for tokenizer-level UER.