mt5-small-finetuned-amazon-en-es

This model is a fine-tuned version of google/mt5-small on the pubmed-summarization dataset. It achieves the following results on the evaluation set:

• Loss: 2.3381
• Rouge1: 14.1074
• Rouge2: 5.3407
• Rougel: 11.9593
• Rougelsum: 12.9286

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 6

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
3.0498	1.0	2500	2.4883	12.7167	5.1639	10.969	11.902
2.8737	2.0	5000	2.4022	13.812	5.1042	11.7056	12.6907
2.7603	3.0	7500	2.3895	13.6588	5.1146	11.6214	12.5331
2.6946	4.0	10000	2.3523	13.7167	5.2024	11.669	12.5419
2.6527	5.0	12500	2.3383	14.082	5.2787	11.9031	12.875
2.6303	6.0	15000	2.3381	14.1074	5.3407	11.9593	12.9286

Framework versions

• Transformers 4.28.1
• Pytorch 2.0.0+cu118
• Datasets 2.11.0
• Tokenizers 0.13.3