yecsASR-omni-llm

⚠️ Experimental research model — released as a downstream demonstration of the YECS corpus. Not production-hardened; see Limitations.

An automatic speech recognition model for Yoruba–English intra-sentential code-switching, finetuned from Meta's Omnilingual ASR LLM model (omniASR_LLM_300M, a ~1.63B-param wav2vec2 encoder + LLaMA decoder) on the Yoruba-English Code-Switching (YECS) Corpus by LyngualLabs. It transcribes naturally code-switched Yoruba/English speech, preserving Yoruba tone diacritics.

Results — held-out YECS test set (9,905 utterances)

Scored with a single normalization pipeline (lowercase, punctuation stripped, tones kept).

Model	WER	CER
yecsASR-omni-llm (this model, ~1.63B, 5 epochs)	16.15%	6.30%
whisper-small-yoruba (reported)	20.76%	—
mms-300m-yoruba-english (reported)	29.11%	—

Tone-insensitive: 13.53% WER / 5.24% CER (the ~2.6-pt gap is the tone-marking cost). Best dev WER 13.93 at step 5,000 (checkpoint kept).

Fairness notes: trained 5 epochs, matching mms-300m-yoruba-english. Re-scoring mms through this exact pipeline reproduced ~0.298 (≈ its reported 0.291), confirming the normalization is consistent across models.

Training

• Base: omniASR_LLM_300M (Meta Omnilingual ASR, Apache-2.0) — ~1.63B params.
• Data: YECS train split, ~95.6h, 16kHz mono. Targets cleaned to the tokenizer's encodable charset (lowercased, punctuation/symbols removed, tone marks preserved) — verified zero out-of-vocab characters.
• Regime: 5 epochs (~6,500 steps), lr 1e-5, bf16, single A100 80GB, keep-best.
• Tokenizer: omniASR_tokenizer_v1 (character tokenizer).

⚠️ Limitations

• Experimental, single-corpus finetune — errors on harder clips, some English content words, and proper names.
• Output is lowercase and unpunctuated (normal for ASR; the tokenizer models only spoken characters + tone marks).
• Trained/evaluated only on YECS-domain audio; may not generalize to other accents, conditions, or domains.
• Not a transformers model — load with the omnilingual_asr library (below).

Usage

git clone https://github.com/facebookresearch/omnilingual-asr && cd omnilingual-asr
pip install -e . && pip install huggingface_hub

import torch
from huggingface_hub import hf_hub_download
from fairseq2.models.hub import load_model
from fairseq2.data.tokenizers.hub import load_tokenizer
from omnilingual_asr.models.inference.pipeline import ASRInferencePipeline

model = load_model("omniASR_LLM_300M", dtype=torch.bfloat16)
sd = torch.load(hf_hub_download("LyngualLabs/yecsASR-omni-llm", "model.pt"), map_location="cpu")
model.load_state_dict(sd, strict=False)        # if this errors, try sd["model"]
tok = load_tokenizer("omniASR_tokenizer_v1")
pipe = ASRInferencePipeline(None, model=model, tokenizer=tok)
print(pipe.transcribe(["sample.wav"], lang=["yor_Latn"]))   # 16kHz mono

License & data

• Model weights derive from omniASR_LLM_300M (Apache-2.0, Meta Omnilingual ASR).
• Training data: YECS Corpus (LyngualLabs), licensed NOODL-1.0 — review its terms before redistribution or commercial use.

Acknowledgements

LyngualLabs (YECS corpus); Meta AI (Omnilingual ASR base model + recipe).