mt5-small-finetuned-amazon-en-es

This model is a fine-tuned version of google/mt5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.0319
• Rouge1: 16.5297
• Rouge2: 8.0695
• Rougel: 16.2097
• Rougelsum: 16.1294

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
3.412	1.0	1209	3.1340	17.7742	8.512	17.0142	17.0204
3.3927	2.0	2418	3.0678	16.8885	8.3578	16.5111	16.4484
3.2559	3.0	3627	3.0782	15.7851	7.5518	15.4435	15.3263
3.1684	4.0	4836	3.0287	15.6152	7.421	15.4312	15.3636
3.1107	5.0	6045	3.0490	16.8609	8.0654	16.6694	16.6019
3.0603	6.0	7254	3.0357	16.0716	7.5671	15.9592	15.8328
3.024	7.0	8463	3.0307	16.7427	8.067	16.412	16.3441
3.0099	8.0	9672	3.0319	16.5297	8.0695	16.2097	16.1294

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.15.0
• Tokenizers 0.15.0