t5-small-finetuned-xlsum-10-epoch

This model is a fine-tuned version of t5-small on the xlsum dataset. It achieves the following results on the evaluation set:

• Loss: 2.2204
• Rouge1: 31.6534
• Rouge2: 10.0563
• Rougel: 24.8104
• Rougelsum: 24.8732
• Gen Len: 18.7913

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
2.6512	1.0	19158	2.3745	29.756	8.4006	22.9753	23.0287	18.8245
2.6012	2.0	38316	2.3183	30.5327	9.0206	23.7263	23.7805	18.813
2.5679	3.0	57474	2.2853	30.9771	9.4156	24.1555	24.2127	18.7905
2.5371	4.0	76632	2.2660	31.0578	9.5592	24.2983	24.3587	18.7941
2.5133	5.0	95790	2.2498	31.3756	9.7889	24.5317	24.5922	18.7971
2.4795	6.0	114948	2.2378	31.4961	9.8935	24.6648	24.7218	18.7929
2.4967	7.0	134106	2.2307	31.44	9.9125	24.6298	24.6824	18.8221
2.4678	8.0	153264	2.2250	31.5875	10.004	24.7581	24.8125	18.7809
2.46	9.0	172422	2.2217	31.6413	10.0311	24.8063	24.8641	18.7951
2.4494	10.0	191580	2.2204	31.6534	10.0563	24.8104	24.8732	18.7913

Framework versions

• Transformers 4.13.0
• Pytorch 1.13.1+cpu
• Datasets 2.8.0
• Tokenizers 0.10.3