Sinhala_Bert_Finetune_BS8_LR2e-5

This model is a fine-tuned version of on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.1821
• Bleu: 0.4279

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu
6.9549	1.0	225	3.1586	2.0557
4.9197	2.0	450	2.5200	2.5634
4.1109	3.0	675	2.1008	3.8007
3.2783	4.0	900	1.7975	13.5566
2.4474	5.0	1125	1.5934	22.6077
1.8907	6.0	1350	1.4460	29.8262
1.6821	7.0	1575	1.3480	32.6221
1.2704	8.0	1800	1.2910	31.4855
1.1577	9.0	2025	1.2595	42.7246
0.9222	10.0	2250	1.2180	40.0441
0.7346	11.0	2475	1.1746	44.1229
0.6097	12.0	2700	1.1689	47.3475
0.6070	13.0	2925	1.1767	42.2465
0.4839	14.0	3150	1.1435	48.0279
0.4263	15.0	3375	1.1408	49.9589
0.3260	16.0	3600	1.1362	53.3992
0.2929	17.0	3825	1.1412	51.4126
0.2679	18.0	4050	1.1183	57.0391
0.2498	19.0	4275	1.1279	56.6146
0.1969	20.0	4500	1.1325	56.9288

Framework versions

• Transformers 5.0.0
• Pytorch 2.10.0+cu128
• Datasets 4.8.5
• Tokenizers 0.22.2