kmbrly123/mupun-eng2mupun

Mupun Machine Translation (English → Mupun)

The first machine translation model for Mupun (ISO 639-3: mcu), a West Chadic (Afroasiatic) language of Plateau State, Nigeria. Mupun is a "Left-Behind" language, absent from NLLB, FLORES, MAFAND, and every major NLP benchmark.

Mupun is a distinct language, not a dialect of Hausa or Mwaghavul.

⚠️ Honest status: early research baseline

The translations are not good yet. In native-speaker evaluation by a fluent Mupun speaker, this model averaged about 1.5 out of 5 for adequacy, performing worst on simple everyday sentences (for which almost no written Mupun training data exists) and somewhat better on complex, scripture-like sentences. This model is a proof that Mupun MT can exist at all, an honest baseline to improve upon — and not suitable for real-world translation. The core limitation is data: there is almost no everyday written Mupun to train on.

Model details

• Base model: facebook/nllb-200-distilled-600M
• Adaptation: LoRA fine-tuning on a multi-domain Mupun–English corpus (New Testament + dictionary + grammar examples), merged into the base for release.
• Direction: English → Mupun.
• Language tag: Mupun has no tag in NLLB, so this model borrows Hausa's tag (hau_Latn) as its Mupun target ("Mode A"). You must force the target token hau_Latn to produce Mupun. A known side-effect is occasional Hausa word intrusion in the output.

Usage

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("kmbrly123/mupun-eng2mupun")
model = AutoModelForSeq2SeqLM.from_pretrained("kmbrly123/mupun-eng2mupun").eval()

tok.src_lang = "eng_Latn"
enc = tok("Good morning, how are you?", return_tensors="pt")
out = model.generate(
    **enc,
    forced_bos_token_id=tok.convert_tokens_to_ids("hau_Latn"),  # targets Mupun
    max_length=128, num_beams=5, no_repeat_ngram_size=3, repetition_penalty=1.5,
)
print(tok.batch_decode(out, skip_special_tokens=True)[0])

Runs on CPU (no GPU required) and offline once downloaded.

Evaluation

On a held-out scripture test set (379 sentence pairs):

Direction	chrF	BLEU
Mupun→English, zero-shot NLLB	16.75	0.93
Mupun→English, Qwen2.5-7B (5-shot)	20.61	1.15
Mupun→English, fine-tuned	38.81	17.60
English→Mupun, zero-shot NLLB	12.31	0.05
English→Mupun, fine-tuned (this model)	22.56	6.89

Automatic scores are measured on scripture; native-speaker validation on everyday sentences gives a much lower mean adequacy (~1.5/5), reflecting the register gap. See the paper for full analysis.

Training data

Openly released and citable:

Dawap, K. T. (2026). Mupun Language Resources: parallel corpus, lexicon, and grammatical examples for a Left-Behind West Chadic language [Data set]. Kaggle. https://doi.org/10.34740/KAGGLE/DS/10984681

Credits

• Manji Yiljep — translator of the Mupun New Testament (the foundation of the training data and the largest body of written Mupun in existence) and native-speaker validator.
• Zygmunt Frajzyngier — A Dictionary of Mupun (1991) and A Grammar of Mupun (1993), source of the lexical and grammatical data.
• The Mupun people, to whom the language belongs.

License

This model is a derivative of NLLB-200 (CC-BY-NC 4.0, non-commercial) and is trained on data including the CC-BY-SA Mupun New Testament. It is released for non-commercial research use only; any use must respect the upstream NLLB and data licenses.

Limitations

An early baseline for a language that previously had no MT at all. Output is often incorrect, uses a scriptural register far from everyday speech, and may contain Hausa intrusions from the borrowed language tag. It should not be relied upon for any consequential translation. It is released in the belief that an honest, imperfect, openly available starting point serves Mupun better than none.