yasmineee/Fine-tuned_NLLB-600M_on_opus_dataset_for_Arabic_to_English_MT_using_QLoRA

NLLB_LoRA

This model is a fine-tuned version of facebook/nllb-200-distilled-600M on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.3193
• Bleu: 32.3755
• Rouge: 0.5949
• Gen Len: 17.4185

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 7

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Rouge	Gen Len
No log	0.9989	437	1.6309	31.0574	0.5662	18.7485
2.5273	2.0	875	1.3828	31.329	0.5835	17.4015
1.5129	2.9989	1312	1.3496	32.067	0.5903	17.4015
1.4432	4.0	1750	1.3340	32.1914	0.5917	17.3865
1.432	4.9989	2187	1.3252	32.2798	0.5939	17.38
1.413	6.0	2625	1.3213	32.3615	0.5948	17.404
1.3961	6.992	3059	1.3193	32.3755	0.5949	17.4185

Framework versions

• PEFT 0.12.0
• Transformers 4.42.3
• Pytorch 2.1.2
• Datasets 2.20.0
• Tokenizers 0.19.1