AmplifiedAccess/Swahili-nllb-1.3b

Swahili NLLB 1.3B

A LoRA fine-tune of facebook/nllb-200-distilled-1.3B for English → Swahili translation, using NLLB's purpose-built translation architecture (encoder-decoder, SentencePiece) rather than a general-purpose chat LLM.

Highlights

• BLEU 38.74 on English→Swahili FLORES devtest
• chrF++ 63.37
• Beats the CraneAILabs Swahili-Gemma-1B reference (27.6 BLEU / 56.8 chrF++) on both metrics
• LoRA adapter only 23.6M trainable parameters (~1.7% of the 1.37B base model)

Results

FLORES devtest, English→Swahili:

Model	BLEU	chrF++
CraneAILabs Swahili-Gemma-1B (reference)	27.6	56.8
This model	38.74	63.37

Usage

This repo holds a LoRA adapter, not merged weights — load the base NLLB model first, then apply the adapter.

import torch
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
from peft import PeftModel

base_model_id = "facebook/nllb-200-distilled-1.3B"
adapter_id = "AmplifiedAccess/Swahili-nllb-1.3b"

tokenizer = AutoTokenizer.from_pretrained(base_model_id, src_lang="eng_Latn")
model = AutoModelForSeq2SeqLM.from_pretrained(base_model_id)
model = PeftModel.from_pretrained(model, adapter_id)
model.eval()

text = "A community officer can help residents complete forms, check eligibility, and even process applications offline."
inputs = tokenizer(text, return_tensors="pt")
target_lang_id = tokenizer.convert_tokens_to_ids("swh_Latn")

with torch.no_grad():
    outputs = model.generate(**inputs, forced_bos_token_id=target_lang_id, max_new_tokens=128)

print(tokenizer.batch_decode(outputs, skip_special_tokens=True)[0])

Training details

Data

Curated parallel corpora, capped as noted:

Source	Examples	Notes
Rogendo/English-Swahili-Sentence-Pairs	59,353	Capped at 60k
MAFAND-MT (en-swa)	32,414	Human-translated news domain (Masakhane)
Sunbird SALT (text-all)	23,947	Full/uncapped
Svngoku/Inkuba-Swahili-MMT	29,993	Capped at 30k
Total	145,707

Configuration

Parameter	Value
Base model	facebook/nllb-200-distilled-1.3B
Method	LoRA (no quantization — full-precision base)
LoRA rank / alpha	16 / 32
Language pair	eng_Latn → swh_Latn (one direction)
Epochs	3
Effective batch size	32 (4 per device × 8 gradient accumulation)
Learning rate	3e-5 (cosine schedule, 3% warmup)
Max sequence length	128
Optimizer	AdamW
Training steps	13,600 / 13,662
Final training loss	10.02

Trained on a single Kaggle T4 GPU. Evaluated on FLORES devtest via sacrebleu after training (BLEU/chrF++ above); the reported training loss reflects raw seq2seq cross-entropy and is not directly comparable to the causal-LM losses reported for our Gemma-based models — treat the FLORES BLEU/chrF++ scores above as the quality signal for this model.

Limitations

• Trained and evaluated only for English→Swahili; the reverse direction is untested and not claimed.
• Evaluated on FLORES devtest only; performance outside that domain (news, casual text, dialectal Swahili) is unverified.
• LoRA adapter requires the base NLLB-1.3B model at inference time (~5.5 GB combined).

Intended use

• English → Swahili machine translation
• Research on low-resource African language NLP with NLLB-family models

Acknowledgments

• Masakhane for the MAFAND-MT parallel corpus
• Sunbird AI for the SALT dataset
• Meta AI for the NLLB-200 model family

Framework versions

• PEFT 0.19.1
• Transformers 5.0.0
• PyTorch 2.10.0