asturian-nllb-lora-bleu

LoRA adapter for Spanish→Asturian (spa_Latn → ast_Latn) machine translation, fine-tuned on top of NLLB-200-distilled-600M. Model checkpoint selected by highest BLEU score on the FLORES+ dev set.

Training Data

• Train: ES-AST Parallel Corpus (AINA) — 704,378 sentence pairs
• Dev/Test: FLORES+

Training Details

• Base model: facebook/nllb-200-distilled-600M
• Method: LoRA (r=16, alpha=32, target modules: q_proj, k_proj, v_proj, out_proj)
• Batch size: 32 (16 per device × 2 gradient accumulation steps)
• Learning rate: 2e-4 (cosine schedule, 5% warmup)
• Epochs: 1.5
• Validation: BLEU on FLORES+ dev set
• Precision: bfloat16

Results on FLORES+ devtest

BLEU	chrF++	COMET	BLEURT
16.32	46.09	68.27	44.98

Usage

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
from peft import PeftModel

model_name = "facebook/nllb-200-distilled-600M"
adapter_name = "ikergf/asturian-nllb-lora-bleu"

tokenizer = AutoTokenizer.from_pretrained(adapter_name)
tokenizer.src_lang = "spa_Latn"

base_model = AutoModelForSeq2SeqLM.from_pretrained(model_name)
model = PeftModel.from_pretrained(base_model, adapter_name)
model.eval()

forced_bos_token_id = tokenizer.convert_tokens_to_ids("ast_Latn")

inputs = tokenizer("Hola, ¿cómo estás?", return_tensors="pt")
outputs = model.generate(
    **inputs,
    forced_bos_token_id=forced_bos_token_id,
    max_new_tokens=128,
    num_beams=5
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))