mt5-small-finetuned-amazon-en-es

This model is a fine-tuned version of google/mt5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.0884
• Rouge1: 17.1847
• Rouge2: 8.3579
• Rougel: 16.7437
• Rougelsum: 16.7013

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
7.0911	1.0	1209	3.3147	13.5672	5.2491	13.2852	13.2424
3.9482	2.0	2418	3.1565	16.6427	7.9417	15.9819	15.9061
3.6511	3.0	3627	3.1186	17.3001	8.7502	16.7138	16.7334
3.5201	4.0	4836	3.0969	16.7251	7.8414	16.0046	15.9171
3.456	5.0	6045	3.0884	17.1847	8.3579	16.7437	16.7013

Framework versions

• Transformers 4.28.1
• Pytorch 2.0.0+cu118
• Datasets 2.11.0
• Tokenizers 0.13.3