mt5-small-finetuned-14feb-1

This model is a fine-tuned version of google/mt5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.4516
• Rouge1: 20.33
• Rouge2: 6.2
• Rougel: 19.9

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.000275
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel
4.0401	1.0	388	2.5481	16.31	4.7	16.1
2.9776	2.0	776	2.4442	17.25	4.93	16.93
2.7362	3.0	1164	2.4181	19.73	5.74	19.21
2.5767	4.0	1552	2.4071	19.37	5.62	18.89
2.4466	5.0	1940	2.3560	18.98	5.94	18.55
2.3402	6.0	2328	2.3923	20.45	5.5	20.03
2.2385	7.0	2716	2.3639	20.03	5.96	19.76
2.1663	8.0	3104	2.3431	19.17	5.34	18.84
2.0849	9.0	3492	2.4008	19.97	5.58	19.67
2.0203	10.0	3880	2.3948	19.67	5.75	19.26
1.9653	11.0	4268	2.3915	20.06	6.07	19.61
1.9067	12.0	4656	2.4025	20.83	6.46	20.41
1.8592	13.0	5044	2.4194	19.97	6.4	19.69
1.8158	14.0	5432	2.4156	19.87	6.16	19.38
1.7679	15.0	5820	2.4053	19.9	5.99	19.52
1.748	16.0	6208	2.4156	19.68	5.81	19.28
1.7198	17.0	6596	2.4306	20.0	6.26	19.63
1.6959	18.0	6984	2.4499	19.1	6.19	18.82
1.6769	19.0	7372	2.4536	20.62	6.3	20.15
1.6682	20.0	7760	2.4516	20.33	6.2	19.9

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1+cu116
• Datasets 2.9.0
• Tokenizers 0.13.2