mt5-small-finetuned-amazon-en-de

This model is a fine-tuned version of google/mt5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.6490
• Rouge1: 17.4502
• Rouge2: 10.6692
• Rougel: 17.1908
• Rougelsum: 17.2891

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
8.5972	1.0	651	3.1283	14.8141	7.1026	14.6207	14.5524
4.1148	2.0	1302	2.8710	14.5174	6.9202	14.1471	14.2004
3.7447	3.0	1953	2.8028	13.9172	7.2051	13.8524	13.8559
3.537	4.0	2604	2.7263	17.4003	10.2674	17.2983	17.3666
3.4368	5.0	3255	2.6791	17.722	10.4401	17.4949	17.5327
3.3467	6.0	3906	2.6559	17.2255	10.2332	17.1067	17.1191
3.3096	7.0	4557	2.6530	17.9659	11.0035	17.7655	17.8441
3.268	8.0	5208	2.6490	17.4502	10.6692	17.1908	17.2891

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.15.0
• Tokenizers 0.15.0