mt5-small-finetuned-amazon-en-es

This model is a fine-tuned version of google/mt5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.0294
• Rouge1: 16.5993
• Rouge2: 8.0138
• Rougel: 16.1315
• Rougelsum: 16.2931

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
6.5928	1.0	1209	3.3005	14.7775	6.4604	14.2574	14.3422
3.9024	2.0	2418	3.1399	16.8632	8.6474	16.065	16.2114
3.5806	3.0	3627	3.0869	18.2422	9.2647	17.6227	17.7649
3.4201	4.0	4836	3.0590	17.7826	8.9742	16.9951	17.1804
3.3202	5.0	6045	3.0598	17.7808	8.6038	17.2243	17.4125
3.2436	6.0	7254	3.0409	16.8469	8.2339	16.3935	16.5818
3.2079	7.0	8463	3.0332	16.8148	8.2115	16.3166	16.4832
3.1801	8.0	9672	3.0294	16.5993	8.0138	16.1315	16.2931

Framework versions

• Transformers 4.24.0
• Pytorch 1.12.1+cu113
• Datasets 2.6.1
• Tokenizers 0.13.2