nllb-id-en-ccmatrix

This model was trained from scratch on the ccmatrix dataset. It achieves the following results on the evaluation set:

• Loss: 0.4863
• Bleu: 60.1097

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 4000
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu
0.5523	1.0	28125	0.5070	58.684
0.4789	2.0	56250	0.4965	59.1724
0.4592	3.0	84375	0.4926	59.4186
0.4451	4.0	112500	0.4898	59.7352
0.4338	5.0	140625	0.4876	59.814
0.425	6.0	168750	0.4864	59.9321
0.4179	7.0	196875	0.4860	59.9852
0.4124	8.0	225000	0.4861	60.0556
0.4084	9.0	253125	0.4857	60.0791
0.4056	10.0	281250	0.4863	60.1097

Framework versions

• Transformers 4.26.1
• Pytorch 2.0.0
• Datasets 2.10.1
• Tokenizers 0.11.0