mbart-fa-religious-final

Model description

This model is a fine-tuned version of facebook/mbart-large-50-many-to-many-mmt for English-to-Persian (Farsi) translation of religious and doctrinal text.

Off-the-shelf multilingual MT models handle general-purpose text reasonably well, but often struggle with the specific register, vocabulary, and phrasing conventions of religious/doctrinal writing. This model adapts mBART-50 to that domain using a purpose-built parallel corpus of religious text.

• Source language: English (en_XX)
• Target language: Persian / Farsi (fa_IR)
• Base model: facebook/mbart-large-50-many-to-many-mmt (610M parameters)

Intended uses & limitations

Intended for translating English religious/doctrinal text (e.g. talks, sermons, devotional writing) into Persian. Performance on general-domain or conversational text is expected to be weaker than the base multilingual model, since fine-tuning specialized the model toward this domain's vocabulary and register.

This is a personal/portfolio project, not a production translation system. Translations should be reviewed by a fluent Persian speaker before use in any setting where accuracy matters (e.g. official communication, publication).

Usage:

from transformers import MBartForConditionalGeneration, MBart50TokenizerFast

tokenizer = MBart50TokenizerFast.from_pretrained("your-username/mbart-fa-religious-final", src_lang="en_XX") model = MBartForConditionalGeneration.from_pretrained("your-username/mbart-fa-religious-final") model.eval()

def translate(text, num_beams=4): encoded = tokenizer(text, return_tensors="pt") generated_tokens = model.generate( **encoded, forced_bos_token_id=tokenizer.lang_code_to_id["fa_IR"], num_beams=num_beams, ) return tokenizer.decode(generated_tokens[0], skip_special_tokens=True)

Training and evaluation data

Trained on ~49,000 English-Persian sentence pairs scraped from published religious texts (General Conference talks 2014-2025, Liahona messages, and several doctrinal manuals), split 90/10 into train/test.

Sentence pairs were strictly aligned: a document was discarded entirely if its English and Persian paragraph counts didn't match, and individual paragraphs were discarded unless their sentence counts matched exactly on both sides. This trades dataset size for alignment quality.

Evaluated on the full held-out test split (~4,900 pairs):

• Validation loss: 1.171
• BLEU: 26.27 (precisions: 58.1% / 32.7% / 19.9% / 12.6%, brevity penalty: 1.0)

Full training code, data collection methodology, and development notes are available at: https://github.com/tuckj11/mbart-fa-religious

Training data was derived from copyrighted materials published by The Church of Jesus Christ of Latter-day Saints; this model is a personal/educational project and is not affiliated with or endorsed by the Church.

Training procedure

The model was fine-tuned from facebook/mbart-large-50-many-to-many-mmt using Hugging Face's Seq2SeqTrainer, with source/target languages set to en_XX/fa_IR.

A few adjustments were made from initial defaults during development, documented in the repo's development notes:

• Label smoothing was disabled (set to 0), since mBART-50's large (~250k-token) vocabulary causes label smoothing's loss term to dominate and misrepresent training loss.
• decoder_start_token_id was explicitly set to the eos token (not the target language tag) to match how the decoder was actually trained via label-shifting.
• Evaluation and checkpointing ran once per epoch, so the final saved checkpoint reliably reflects the true end of training.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• warmup_ratio: 0.1
• weight_decay: 0.01
• train_batch_size: 4
• eval_batch_size: 4
• gradient_accumulation_steps: 8
• total_train_batch_size: 32
• optimizer: AdamW (betas=(0.9,0.999), epsilon=1e-08)
• lr_scheduler_type: linear
• num_epochs: 3
• mixed_precision_training: Native AMP (fp16)
• generation_num_beams: 4
• label_smoothing_factor: 0

Training Results

Training Loss	Epoch	Step	Validation Loss
9.4660	3.0	4122	1.1712

Note: reported training loss is averaged across the entire run, including early, less-stable steps before learning rate warmup completed -- it is not directly comparable to validation loss, which reflects only the final, best checkpoint.

Final evaluation (on the full ~4,900-example held-out test set):

• BLEU: 26.27
• Precisions (1-4 gram): 58.1% / 32.7% / 19.9% / 12.6%
• Brevity penalty: 1.0
• Length ratio: 1.046

Framework versions

• Transformers 5.14.1
• Pytorch 2.11.0+cu128
• Datasets 5.0.1
• Tokenizers 0.22.2