mt5-small-finetuned-samsum-01

This model is a fine-tuned version of google/mt5-small on the samsum dataset. It achieves the following results on the evaluation set:

• Loss: 1.8784
• Rouge1: 39.8989
• Rouge2: 18.4549
• Rougel: 34.2186
• Rougelsum: 37.3438

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
4.1361	1.0	1842	2.1341	35.9845	15.146	30.819	33.4492
2.5514	2.0	3684	2.0119	37.8344	16.359	32.2541	35.4021
2.3851	3.0	5526	1.9674	38.8153	17.1048	33.2513	36.2178
2.2878	4.0	7368	1.9211	39.0649	17.5803	33.5863	36.4784
2.2202	5.0	9210	1.9016	39.5536	18.199	34.1462	37.1727
2.181	6.0	11052	1.8829	39.8724	18.1549	34.1958	37.2659
2.153	7.0	12894	1.8821	39.871	18.4563	34.2479	37.4525
2.138	8.0	14736	1.8784	39.8989	18.4549	34.2186	37.3438

Framework versions

• Transformers 4.29.2
• Pytorch 2.0.1+cu118
• Datasets 2.12.0
• Tokenizers 0.13.3