md_mt5_0109_v2

This model is a fine-tuned version of Buseak/md_mt5_0109 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2368
• Bleu: 0.5272
• Gen Len: 18.9267

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
0.8133	1.0	975	0.4559	0.4502	18.9192
0.7386	2.0	1950	0.4087	0.4689	18.9203
0.6826	3.0	2925	0.3751	0.4854	18.9244
0.6321	4.0	3900	0.3450	0.4959	18.9221
0.6062	5.0	4875	0.3235	0.4977	18.9187
0.5634	6.0	5850	0.3027	0.5123	18.9172
0.5361	7.0	6825	0.2874	0.5146	18.9259
0.5152	8.0	7800	0.2719	0.5211	18.9295
0.503	9.0	8775	0.2661	0.5222	18.9292
0.4868	10.0	9750	0.2550	0.5258	18.9277
0.4691	11.0	10725	0.2485	0.5263	18.9251
0.4672	12.0	11700	0.2432	0.526	18.9238
0.4607	13.0	12675	0.2394	0.5268	18.9259
0.4536	14.0	13650	0.2369	0.5272	18.9267
0.4499	15.0	14625	0.2368	0.5272	18.9267

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.0