opus-mt-en-id-ccmatrix-lr-4

This model was trained from scratch on the ccmatrix dataset. It achieves the following results on the evaluation set:

• Loss: 0.9115
• Bleu: 65.4544

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 4000
• num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu
0.7279	1.0	28125	0.7185	61.1164
0.6185	2.0	56250	0.6849	62.0536
0.5598	3.0	84375	0.6759	62.3915
0.5163	4.0	112500	0.6646	62.9303
0.4795	5.0	140625	0.6665	63.3461
0.4471	6.0	168750	0.6692	63.5319
0.4173	7.0	196875	0.6690	63.7436
0.3897	8.0	225000	0.6739	63.8343
0.3633	9.0	253125	0.6832	63.867
0.3382	10.0	281250	0.6928	64.0481
0.314	11.0	309375	0.7015	64.0177
0.2909	12.0	337500	0.7151	64.3563
0.2687	13.0	365625	0.7265	64.2445
0.2474	14.0	393750	0.7384	64.5093
0.227	15.0	421875	0.7560	64.3729
0.2072	16.0	450000	0.7712	64.6396
0.1888	17.0	478125	0.7876	64.805
0.1713	18.0	506250	0.8052	64.7883
0.1546	19.0	534375	0.8258	64.9535
0.1394	20.0	562500	0.8421	64.9885
0.1251	21.0	590625	0.8593	65.1229
0.112	22.0	618750	0.8757	65.2565
0.1006	23.0	646875	0.8923	65.288
0.0907	24.0	675000	0.9033	65.3973
0.0828	25.0	703125	0.9115	65.4544

Framework versions

• Transformers 4.26.1
• Pytorch 2.0.0
• Datasets 2.10.1
• Tokenizers 0.11.0