Sinhala_Bert_Finetune_BS16_LR5e-5

This model is a fine-tuned version of on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.1692
• Bleu: 0.4222

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu
6.6045	1.0	113	2.8613	3.7431
4.1461	2.0	226	2.0201	7.3713
2.8864	3.0	339	1.6122	21.2711
2.0655	4.0	452	1.3587	20.4770
1.5190	5.0	565	1.3010	42.1320
1.2361	6.0	678	1.2267	35.9553
0.9272	7.0	791	1.1439	45.5780
0.6205	8.0	904	1.1426	48.1018
0.5086	9.0	1017	1.1114	52.1876
0.3975	10.0	1130	1.1004	55.4769
0.3096	11.0	1243	1.0964	52.4873
0.2712	12.0	1356	1.1202	57.0564
0.2214	13.0	1469	1.1359	51.5924
0.1870	14.0	1582	1.1071	57.5690
0.1617	15.0	1695	1.1132	54.6654
0.1323	16.0	1808	1.0945	62.5243
0.1168	17.0	1921	1.1290	57.0977
0.0973	18.0	2034	1.1397	54.5007

Framework versions

• Transformers 5.0.0
• Pytorch 2.10.0+cu128
• Datasets 4.8.5
• Tokenizers 0.22.2