SaintsStudios/Tumbuka_NLLB-200_QT

---

language: - tum - eng tags: - translation - seq2seq - nllb - nllb-200 - instruction-tuning - malawi license: mit datasets: - mwanzau/tumbuka-language-malawi metrics: - bleu model-index: - name: tumbuka_nllb-200 results: []

tumbuka_nllb-200 (Version 2.0)

This model is a highly optimized, fine-tuned checkpoint of Meta's facebook/nllb-200-distilled-600M, specifically adapted for high-fidelity translation and instruction-following tasks in the Tumbuka (Chitumbuka) language, spoken primarily in Northern Malawi and Eastern Zambia.

While Version 1.0 laid the groundwork for basic translation, Version 2.0 fixes deep-seated regional language leakage (such as Nyanja/Chichewa loan-words) by undergoing 9+ hours of advanced multi-stage training. It combines premium English-Tumbuka translation pairs with dedicated Tumbuka instruction-tuning datasets.

Key Improvements in V2.0:

• Linguistic Purity: Eradicated common baseline model bleed (e.g., automatically corrects baseline defaults like Takulandilani into pure Tumbuka Mwapokeleleka).
• Contextual Precision: Significantly improves technical and situational vocabulary mapping (e.g., shifting the translation of "reliable irrigation" from rigid variations to natural maji ghakugomezgeka ghakuthira).

---

Model Description

• Developed by: SaintsStudios
• Model type: Encoder-Decoder (Seq2Seq)
• Language(s) (NLP): English (eng_Latn) to Tumbuka (tum_Latn), and Tumbuka Monolingual Instruction Tasking
• License: MIT
• Finetuned from model: facebook/nllb-200-distilled-600M

---

Performance Examples

English Input	Baseline NLLB / V1.0 Output	V2.0 Pure Tumbuka Output
Welcome to our home.	Takulandilani ku nyumba yithu (Nyanja leak)	Mwapokeleleka ku nyumba yithu
We need reliable irrigation for our farms.	Tikukhumba maji ghakukhora...	Tikukhumba maji ghakugomezgeka ghakuthira minda yithu
Good morning, how are you today?	Mulenji uwemi...	Mulenji uwemi, muli wuli mwahuno?
Who are you?	Kasi ndiwe njani?	Kasi ndiwe njani?

---

How to Use the Model

You can use this model directly with the Hugging Face transformers library for inference.

Python Inference Code

import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

model_id = "SaintsStudios/tumbuka_nllb-200"

print("Loading model and tokenizer...")
tokenizer = AutoTokenizer.from_pretrained("facebook/nllb-200-distilled-600M")
model = AutoModelForSeq2SeqLM.from_pretrained(model_id)

device = "cuda" if torch.cuda.is_available() else "cpu"
model.to(device)

def translate_to_tumbuka(text: str):
    # Set the language configs
    tokenizer.src_lang = "eng_Latn"
    inputs = tokenizer(text, return_tensors="pt").to(device)
    
    # Force target generation to start with the Tumbuka language token
    forced_bos_token_id = tokenizer.convert_tokens_to_ids("tum_Latn")
    
    translated_tokens = model.generate(
        **inputs,
        forced_bos_token_id=forced_bos_token_id,
        max_length=128,
        num_beams=4,
        early_stopping=True
    )
    
    return tokenizer.decode(translated_tokens[0], skip_special_tokens=True)

# Test run
sample_text = "Let us learn together."
output = translate_to_tumbuka(sample_text)
print(f"English: {sample_text}")
print(f"Tumbuka: {output}") 
# Output: Tiyeni tisambire pamoza


Training Hyperparameters (V2.0)
Dataset Blend: 50% Translation Pairs / 50% Monolingual Alpaca Instruction Datasets

- Epochs: 3
- Learning Rate: 2e-5
- Optimizer: Adafactor
- Batch Size: 2 (Gradient Accumulation Steps: 8)
- Precision: FP16 Mixed Precision
- Hardware: Kaggle Cloud Compute (GPU T4)

Acknowledgements
Special thanks to the open-source contributors maintaining datasets for Malawian localized languages, making high-quality regional adaptation pipelines possible.
Commit directly to the
main
branch
Open as a pull request to the
main
branch
Commit changes
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Rephrase