Muhammed-sheheen/NLLB_FINETUNIG

NLLB English-to-Malayalam Fine-tuned Model

This is a fine-tuned and merged version of facebook/nllb-200-distilled-600M for translation from English to Malayalam.

Developed at ICFOSS (International Centre for Free and Open Source Software).

The LoRA adapter has already been merged into the base model, so this repository can be loaded directly with Transformers without installing PEFT or downloading the base model separately.

Usage

import torch
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

model_id = "Muhammed-sheheen/NLLB_FINETUNIG"
source_language = "eng_Latn"
target_language = "mal_Mlym"

tokenizer = AutoTokenizer.from_pretrained(model_id, src_lang=source_language)
model = AutoModelForSeq2SeqLM.from_pretrained(model_id)

text = "Artificial intelligence can improve language technology."
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=256)

with torch.inference_mode():
    generated = model.generate(
        **inputs,
        forced_bos_token_id=tokenizer.convert_tokens_to_ids(target_language),
        num_beams=4,
        max_new_tokens=128,
    )

translation = tokenizer.batch_decode(generated, skip_special_tokens=True)[0]
print(translation)

Model details

• Developed by: ICFOSS (International Centre for Free and Open Source Software)
• Task: English-to-Malayalam machine translation
• Source language code: eng_Latn
• Target language code: mal_Mlym
• Base model: facebook/nllb-200-distilled-600M
• Architecture: M2M100/NLLB sequence-to-sequence Transformer
• Fine-tuning method: LoRA, merged into the model weights
• LoRA rank: 16
• LoRA alpha: 32
• LoRA dropout: 0.05
• Target modules: q_proj, k_proj, v_proj, and out_proj
• Training epochs: 20

Evaluation

The saved training output reports a held-out test loss of 1.772. Translation quality metrics such as BLEU, chrF++, or COMET were not included in the saved evaluation results. Users should evaluate the model on data representative of their intended domain before deployment.

Limitations and responsible use

This model may produce inaccurate, incomplete, or biased translations, especially for specialized terminology, uncommon names, dialectal Malayalam, long inputs, and text unlike the fine-tuning data. Human review is recommended for medical, legal, safety-critical, or other high-impact content.

The fine-tuning dataset and its preprocessing details are not documented in the available training artifacts. This limits reproducibility and makes it difficult to fully characterize domain coverage and potential dataset biases.

License

The base model is distributed under the CC BY-NC 4.0 license. This fine-tuned model is published under the same license and is restricted to non-commercial use.

See the NLLB-200 model card for additional information about the base model, its intended uses, and its limitations.