mt5-small-finetuned-amazon-en-de

This model is a fine-tuned version of google/mt5-small on the Amazon reviews multi dataset. It achieves the following results on the evaluation set:

• Loss: 3.2896
• Rouge1: 14.7163
• Rouge2: 6.6341
• Rougel: 14.2052
• Rougelsum: 14.2318

Model description

the model can summarize texts for english and deutsch

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

the training was done on google colab ( using it's free GPU)

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
7.2925	1.0	1276	3.5751	13.6254	6.0527	13.109	13.1438
4.0677	2.0	2552	3.4031	13.5907	6.068	13.3526	13.2471
3.7458	3.0	3828	3.3434	14.7229	6.8482	14.1443	14.2218
3.5831	4.0	5104	3.3353	14.8696	6.6371	14.1342	14.2907
3.4841	5.0	6380	3.3037	14.233	6.2318	13.9218	13.9781
3.4142	6.0	7656	3.2914	13.7344	5.9446	13.5476	13.6362
3.3587	7.0	8932	3.2959	14.2007	6.1905	13.5255	13.5237
3.3448	8.0	10208	3.2896	14.7163	6.6341	14.2052	14.2318

Framework versions

• Transformers 4.20.0
• Pytorch 1.11.0+cu113
• Datasets 2.3.2
• Tokenizers 0.12.1