mt5-small-summarizer-finetuned

This model is a fine-tuned version of google/mt5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.0158
• Rouge1: 17.7167
• Rouge2: 8.7443
• Rougel: 17.4487
• Rougelsum: 17.4013

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
6.8514	1.0	1209	3.2992	14.7682	6.17	14.2741	14.2171
3.8706	2.0	2418	3.1206	16.1753	7.7142	15.436	15.5325
3.5426	3.0	3627	3.0748	17.9388	8.786	17.3878	17.3805
3.3773	4.0	4836	3.0461	16.79	7.9251	16.4337	16.3482
3.2734	5.0	6045	3.0438	17.201	8.2405	16.9939	16.9181
3.194	6.0	7254	3.0227	17.3508	8.4746	17.0519	17.0831
3.1556	7.0	8463	3.0201	17.6119	8.686	17.3536	17.3228
3.1256	8.0	9672	3.0158	17.7167	8.7443	17.4487	17.4013

Framework versions

• Transformers 4.21.1
• Pytorch 1.12.0+cu113
• Datasets 2.4.0
• Tokenizers 0.12.1