nllb-sango-finetuned-600m

This model is a fine-tuned version of facebook/nllb-200-distilled-600M on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.2383

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 200
• num_epochs: 1
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
16.3649	0.1146	1000	3.7578
15.5959	0.2291	2000	3.5684
15.1041	0.3437	3000	3.4531
14.7656	0.4583	4000	3.3770
14.5981	0.5728	5000	3.3145
14.3408	0.6874	6000	3.2676
14.2934	0.8019	7000	3.2383

Framework versions

• PEFT 0.18.1
• Transformers 5.0.0
• Pytorch 2.10.0+cu128
• Datasets 4.8.3
• Tokenizers 0.22.2