mt5-small-test-amazon

This model is a fine-tuned version of google/mt5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.9515
• Rouge1: 30.3066
• Rouge2: 3.3019
• Rougel: 30.1887
• Rougelsum: 30.0314

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
10.0147	1.0	1004	2.9904	7.3703	0.2358	7.3703	7.4292
3.4892	2.0	2008	2.4061	23.4178	2.4764	23.2901	23.3097
2.724	3.0	3012	2.1630	26.6706	2.8302	26.6509	26.5723
2.4395	4.0	4016	2.0815	26.7296	2.9481	26.6313	26.533
2.2881	5.0	5020	2.0048	30.1887	3.3019	30.0708	29.9135
2.1946	6.0	6024	1.9712	29.4811	2.9481	29.4025	29.3042
2.1458	7.0	7028	1.9545	29.8153	3.3019	29.717	29.5204
2.1069	8.0	8032	1.9515	30.3066	3.3019	30.1887	30.0314

Framework versions

• Transformers 4.20.0
• Pytorch 1.11.0+cu113
• Datasets 2.3.2
• Tokenizers 0.12.1