mt5-small-finetuned-QMSum-01

This model is a fine-tuned version of google/mt5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.8022
• Rouge1: 18.0578
• Rouge2: 4.3867
• Rougel: 14.448
• Rougelsum: 16.1248

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
5.8153	1.0	548	3.0610	11.8231	3.1478	9.9085	10.7642
3.5393	2.0	1096	2.9173	16.1634	4.0585	13.0541	14.6676
3.2879	3.0	1644	2.8507	16.6082	4.1563	13.4818	14.9447
3.163	4.0	2192	2.8268	16.9681	4.1602	13.7462	15.0741
3.0699	5.0	2740	2.8256	17.8647	4.5317	14.4077	15.8516
3.0156	6.0	3288	2.8175	17.7178	4.3329	14.3377	15.8622
2.9692	7.0	3836	2.7987	18.3523	4.6726	14.6873	16.413
2.9531	8.0	4384	2.8022	18.0578	4.3867	14.448	16.1248

Framework versions

• Transformers 4.29.2
• Pytorch 2.0.1+cu118
• Datasets 2.12.0
• Tokenizers 0.13.3