mt5-small-finetuned-amazon-en-fr

This model is a fine-tuned version of google/mt5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.9738
• Rouge1: 16.2618
• Rouge2: 8.4157
• Rougel: 15.7746
• Rougelsum: 15.6448

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
6.4338	1.0	1399	3.2788	12.6697	4.9248	12.0308	12.0007
3.8734	2.0	2798	3.1052	14.3438	7.2643	13.744	13.6593
3.5793	3.0	4197	3.0230	15.8565	8.5311	15.2736	15.2018
3.4243	4.0	5596	2.9943	16.1882	8.3288	15.6948	15.5725
3.3277	5.0	6995	2.9845	16.5005	8.6609	16.0231	15.9789
3.2652	6.0	8394	2.9793	15.8014	7.9576	15.3678	15.2699
3.2344	7.0	9793	2.9707	16.529	8.2051	15.9864	15.8459
3.1853	8.0	11192	2.9738	16.2618	8.4157	15.7746	15.6448

Framework versions

• Transformers 4.33.2
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3