mt5-small-finetuned-amazon-en-zh

This model is a fine-tuned version of google/mt5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.3029
• Rouge1: 16.5328
• Rouge2: 7.1687
• Rougel: 15.8574
• Rougelsum: 16.1506

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
6.5452	1.0	1676	3.5344	13.2336	4.9489	12.9231	13.033
3.803	2.0	3352	3.3963	14.814	6.2391	14.2581	14.4316
3.4981	3.0	5028	3.3262	16.6892	7.7862	15.9537	16.1894
3.3061	4.0	6704	3.3111	15.4591	7.1128	14.6644	14.7762
3.2023	5.0	8380	3.3039	15.7126	7.1817	14.895	15.1448
3.1173	6.0	10056	3.3094	16.1592	7.609	15.4639	15.5988
3.0803	7.0	11732	3.2999	16.2882	7.1004	15.7029	15.8878
3.0539	8.0	13408	3.3029	16.5328	7.1687	15.8574	16.1506

Framework versions

• Transformers 4.27.4
• Pytorch 1.13.0
• Datasets 2.1.0
• Tokenizers 0.13.2