mt5-small-finetuned-amazon-en-es

This model is a fine-tuned version of google/mt5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.0171
• Rouge1: 16.778
• Rouge2: 8.0849
• Rougel: 16.5329
• Rougelsum: 16.4302

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
3.4297	1.0	1209	3.1211	17.6479	8.1669	17.1554	17.0276
3.4217	2.0	2418	3.0394	16.4501	8.3991	16.2225	16.2214
3.2701	3.0	3627	3.0427	16.3473	7.5173	16.1924	16.098
3.1888	4.0	4836	3.0283	15.3718	6.8591	15.0889	14.9769
3.1204	5.0	6045	3.0256	17.5963	8.331	17.1812	17.0733
3.072	6.0	7254	3.0189	16.5811	8.1764	16.28	16.207
3.0386	7.0	8463	3.0171	17.1018	8.4785	16.8196	16.7681
3.0193	8.0	9672	3.0171	16.778	8.0849	16.5329	16.4302

Framework versions

• Transformers 4.21.2
• Pytorch 1.12.1+cu113
• Datasets 2.4.0
• Tokenizers 0.12.1