t5-small-finetuned-xsum

This model is a fine-tuned version of t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2715
• Rouge1: 0.8783
• Rouge2: 0.8348
• Rougel: 0.8739
• Rougelsum: 0.8746

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
No log	1.0	21	0.6229	0.7109	0.6524	0.7061	0.7071
No log	2.0	42	0.4551	0.7062	0.6477	0.7008	0.7017
No log	3.0	63	0.3653	0.7854	0.7293	0.7818	0.7823
No log	4.0	84	0.3170	0.8117	0.7606	0.8076	0.8101
No log	5.0	105	0.3047	0.8384	0.7893	0.8346	0.834
No log	6.0	126	0.2916	0.8489	0.8022	0.8454	0.8454
No log	7.0	147	0.2852	0.8512	0.8085	0.8479	0.8478
No log	8.0	168	0.2778	0.869	0.8249	0.8645	0.8651
No log	9.0	189	0.2762	0.8702	0.8258	0.8657	0.8663
No log	10.0	210	0.2760	0.8734	0.8294	0.8681	0.8693
No log	11.0	231	0.2749	0.8734	0.8294	0.8681	0.8693
No log	12.0	252	0.2747	0.8739	0.8303	0.8688	0.8699
No log	13.0	273	0.2743	0.8735	0.8295	0.8681	0.8694
No log	14.0	294	0.2747	0.8773	0.8335	0.8726	0.8731
No log	15.0	315	0.2748	0.8773	0.8335	0.8726	0.8731
No log	16.0	336	0.2734	0.8779	0.8344	0.8734	0.874
No log	17.0	357	0.2733	0.8779	0.8343	0.8733	0.8739
No log	18.0	378	0.2729	0.8779	0.8344	0.8734	0.874
No log	19.0	399	0.2718	0.8793	0.8357	0.8745	0.875
No log	20.0	420	0.2716	0.8793	0.8357	0.8745	0.875
No log	21.0	441	0.2721	0.8779	0.8343	0.8733	0.8739
No log	22.0	462	0.2720	0.8779	0.8343	0.8733	0.8739
No log	23.0	483	0.2724	0.8779	0.8344	0.8734	0.874
0.2699	24.0	504	0.2725	0.8783	0.8348	0.8739	0.8746
0.2699	25.0	525	0.2721	0.8783	0.8348	0.8739	0.8746
0.2699	26.0	546	0.2719	0.8783	0.8348	0.8739	0.8746
0.2699	27.0	567	0.2716	0.8783	0.8348	0.8739	0.8746
0.2699	28.0	588	0.2715	0.8783	0.8348	0.8739	0.8746
0.2699	29.0	609	0.2716	0.8783	0.8348	0.8739	0.8746
0.2699	30.0	630	0.2715	0.8783	0.8348	0.8739	0.8746

Framework versions

• Transformers 4.26.0.dev0
• Pytorch 1.13.0
• Datasets 2.8.0
• Tokenizers 0.13.2