mt5-small-finetuned-amazon-en-es

This model is a fine-tuned version of google/mt5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.0279
• Rouge1: 16.4284
• Rouge2: 7.8601
• Rougel: 16.0029
• Rougelsum: 16.0246

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
4.4194	1.0	1209	3.3097	14.9867	6.4886	14.4174	14.4646
3.8132	2.0	2418	3.1602	16.1474	7.9815	15.5342	15.6445
3.5412	3.0	3627	3.0789	17.4468	8.8014	16.9142	17.002
3.3861	4.0	4836	3.0775	15.903	7.4423	15.4008	15.3871
3.2952	5.0	6045	3.0480	15.8646	7.3936	15.3989	15.4395
3.2155	6.0	7254	3.0354	16.5887	8.0624	16.2377	16.2562
3.1896	7.0	8463	3.0273	17.1092	8.5391	16.6507	16.7272
3.1594	8.0	9672	3.0279	16.4284	7.8601	16.0029	16.0246

Framework versions

• Transformers 4.33.1
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3