mt5-small-finetuned-28jan-2

This model is a fine-tuned version of google/mt5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.5078
• Rouge1: 18.7485
• Rouge2: 5.8034
• Rougel: 18.5163
• Rougelsum: 18.4817

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 9
• eval_batch_size: 9
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
5.9652	1.0	242	2.8048	14.036	4.037	13.7766	13.8254
3.474	2.0	484	2.7485	16.821	4.8051	16.6168	16.5782
3.2101	3.0	726	2.6444	17.2659	5.1077	16.9501	16.8998
3.0555	4.0	968	2.6408	17.3002	4.8657	17.0414	16.9794
2.9515	5.0	1210	2.5860	17.6468	5.3816	17.3755	17.3434
2.8694	6.0	1452	2.5586	18.3932	5.3896	18.2521	18.0748
2.7898	7.0	1694	2.5325	18.4954	5.5609	18.2994	18.2112
2.7436	8.0	1936	2.5431	18.8172	5.9338	18.4693	18.4324
2.6955	9.0	2178	2.5588	18.7895	6.1003	18.3593	18.3268
2.6571	10.0	2420	2.5079	19.2525	5.8268	19.0279	18.9846
2.629	11.0	2662	2.5118	18.9191	5.9877	18.6505	18.6
2.5998	12.0	2904	2.5070	18.7181	5.9061	18.4432	18.3931
2.5692	13.0	3146	2.5014	18.4412	6.1983	18.2394	18.1618
2.5751	14.0	3388	2.5125	18.7014	5.9729	18.4366	18.406
2.55	15.0	3630	2.5078	18.7485	5.8034	18.5163	18.4817

Framework versions

• Transformers 4.26.0
• Pytorch 1.13.1+cu116
• Datasets 2.9.0
• Tokenizers 0.13.2