Fallovski/french-serer-nllb-lora-srr-token

French–Serer NLLB LoRA with Native srr_Latn Token

Exploratory follow-up experiment, added during the revision process. Not part of the six configurations (A–F) benchmarked in the original manuscript.

Motivation

Configurations B–D in this project decode Serer under the wol_Latn (Wolof) NLLB-200 language tag, since srr_Latn is not natively supported. This model instead adds a genuine srr_Latn token to NLLB-200's vocabulary — its embedding initialized by copying wol_Latn's embedding (following the proxy-initialization approach validated for unseen low-resource languages, e.g. Limbum–English, arXiv:2608.07629) — plus 5 Serer implosive consonant characters missing from the base vocabulary (ƥ, ƈ, Ƥ, Ƭ, Ƈ), then fine-tunes with LoRA using the exact same hyperparameters as configuration C.

Model summary

• Experiment ID: G (post-submission addition)
• Base model: facebook/nllb-200-distilled-600M + native srr_Latn token
• Best checkpoint: french_serer_nllb_lora_srrtoken-epoch=09-val_bleu=20.0926.ckpt
• LoRA: rank 16, alpha 32, dropout 0.1
• Learning rate: 0.0003, 10 epochs, warmup 500 steps
• Random seed: 42

Evaluation

Metric	Value
BLEU (test, beam=5)	19.5865
chrF	41.2796
ROUGE-1	0.4546
ROUGE-L	0.4107
BERTScore-F1	0.8816
Test loss	1.859

Evaluated on the held-out test split (2890 sentence pairs, SHA-256: 01d14d982a3c0cce172b5099e2db064d05bdef89677fe72a45f56715d6364ee2), identical protocol to configuration C for direct comparability.

Comparison with configuration C (wol_Latn proxy, same LoRA hyperparameters)

Metric	G (native srr_Latn)	C (wol_Latn proxy)
BLEU	19.5865	17.9291
chrF	41.2796	38.7191
ROUGE-L	0.4107	0.391
BERTScore-F1	0.8816	0.8764

G outperforms C on every metric under an identical training protocol, suggesting that adding a dedicated target-language token — rather than reusing a related language's tag as a decoding proxy — is worth the modest extra setup cost when adapting NLLB-200 to an unsupported language with a well-resourced phylogenetic neighbor.

Intended use

Research use only. Private repository, not validated for production deployment. Requires the custom tokenizer bundled with this repo (includes the srr_Latn token and the 5 added Serer characters) — do not swap in a stock NLLB-200 tokenizer.

Usage

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

repo_id = "Fallovski/french-serer-nllb-lora-srr-token"
tokenizer = AutoTokenizer.from_pretrained(repo_id, src_lang="fra_Latn")
model = AutoModelForSeq2SeqLM.from_pretrained(repo_id)

text = "Bonjour, comment allez-vous ?"
inputs = tokenizer(text, return_tensors="pt")
target_id = tokenizer.convert_tokens_to_ids("srr_Latn")
output = model.generate(**inputs, forced_bos_token_id=target_id, num_beams=5, max_new_tokens=128)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Training data

Same French–Serer parallel corpus as configurations B–D (23113 train / 2889 val / 2890 test pairs, ~90% religious register, ~10% educational glossaries, predominantly Siin dialect). Full provenance is kept in a private dataset card, available on request.