t5-small-finetuned-xsum-updated

This model is a fine-tuned version of t5-small on the xsum dataset. It achieves the following results on the evaluation set:

• Loss: 2.0767
• Rouge1: 33.2945
• Rouge2: 12.0165
• Rougel: 26.9804
• Rougelsum: 26.9729
• Gen Len: 18.7853

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
2.5219	1.0	12753	2.3054	30.4745	9.435	24.263	24.2522	18.823
2.4191	2.0	25506	2.2385	31.2305	10.0552	24.9345	24.9254	18.7562
2.3564	3.0	38259	2.1961	31.8234	10.6556	25.6109	25.6023	18.7708
2.3028	4.0	51012	2.1692	32.2053	11.0513	26.0184	26.0056	18.772
2.2737	5.0	63765	2.1452	32.3716	11.1779	26.1423	26.1363	18.7731
2.2432	6.0	76518	2.1304	32.5413	11.2517	26.2119	26.2098	18.8007
2.2266	7.0	89271	2.1193	32.8983	11.5683	26.5995	26.5958	18.8108
2.1863	8.0	102024	2.1058	32.9046	11.6564	26.6466	26.6473	18.8008
2.1583	9.0	114777	2.0987	32.9622	11.7285	26.7161	26.7116	18.7798
2.1653	10.0	127530	2.0900	33.1259	11.8525	26.8461	26.8419	18.7999
2.1403	11.0	140283	2.0880	33.0949	11.8135	26.7863	26.7765	18.7629
2.1212	12.0	153036	2.0825	33.1671	11.8939	26.9072	26.8982	18.7825
2.1021	13.0	165789	2.0793	33.1375	11.9119	26.8466	26.8386	18.8076
2.0877	14.0	178542	2.0774	33.2516	11.9574	26.9391	26.9327	18.7989
2.0984	15.0	191295	2.0767	33.2945	12.0165	26.9804	26.9729	18.7853

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.15.0
• Tokenizers 0.15.0