mt5-small-finetuned-amazon-en-es

This model is a fine-tuned version of google/mt5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.0280
• Rouge1: 17.3563
• Rouge2: 8.6193
• Rougel: 17.081
• Rougelsum: 17.1297

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
7.0507	1.0	1209	3.3225	12.6324	4.7979	12.3957	12.4312
3.9068	2.0	2418	3.1852	16.432	8.2165	15.7321	15.789
3.5973	3.0	3627	3.0834	16.912	8.2736	16.3027	16.3174
3.4111	4.0	4836	3.0560	16.8768	8.0417	16.209	16.2473
3.318	5.0	6045	3.0464	17.5367	8.364	16.9286	16.9249
3.2435	6.0	7254	3.0371	17.3217	8.398	16.9066	17.0021
3.202	7.0	8463	3.0347	17.1712	8.0887	16.7378	16.748
3.1799	8.0	9672	3.0280	17.3563	8.6193	17.081	17.1297

Framework versions

• Transformers 4.33.2
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3