mt5-small-finetuned-amazon-en-es

This model is a fine-tuned version of google/mt5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.0294
• Rouge1: 16.4846
• Rouge2: 7.8849
• Rougel: 16.2726
• Rougelsum: 16.2173

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
6.5928	1.0	1209	3.3005	14.7927	6.4642	14.3032	14.2436
3.9024	2.0	2418	3.1399	16.8294	8.5707	16.1439	16.133
3.5806	3.0	3627	3.0869	18.1262	9.1916	17.7121	17.7184
3.4201	4.0	4836	3.0590	17.6134	8.915	17.209	17.1519
3.3202	5.0	6045	3.0598	17.6322	8.5392	17.4438	17.3057
3.2436	6.0	7254	3.0409	16.7409	8.107	16.5385	16.496
3.2079	7.0	8463	3.0332	16.722	8.0183	16.4492	16.4297
3.1801	8.0	9672	3.0294	16.4846	7.8849	16.2726	16.2173

Framework versions

• Transformers 4.23.1
• Pytorch 1.12.1+cu113
• Datasets 2.6.1
• Tokenizers 0.13.1