mt5-small-finetuned-amazon-en-es

This model is a fine-tuned version of google/mt5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.0255
• Rouge1: 17.469
• Rouge2: 8.5134
• Rougel: 17.1167
• Rougelsum: 17.2481

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
8.094	1.0	1209	3.2933	12.7976	5.1617	12.4199	12.5113
3.9263	2.0	2418	3.1487	16.2082	8.3215	15.744	15.807
3.599	3.0	3627	3.0789	16.9706	8.2425	16.3972	16.4067
3.429	4.0	4836	3.0492	17.2122	8.7398	16.7892	16.795
3.3279	5.0	6045	3.0384	17.5381	8.7438	17.0764	17.1831
3.2518	6.0	7254	3.0343	17.0966	8.5622	16.7016	16.8022
3.2084	7.0	8463	3.0255	16.7713	8.0472	16.3159	16.4091
3.1839	8.0	9672	3.0255	17.469	8.5134	17.1167	17.2481

Framework versions

• Transformers 4.18.0
• Pytorch 1.10.0+cu111
• Datasets 2.1.0
• Tokenizers 0.12.1