t5-small-finetuned-xsum

This model is a fine-tuned version of t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.3008
• Rouge1: 12.6103
• Rouge2: 9.5926
• Rougel: 12.6021
• Rougelsum: 12.6283
• Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	250	0.6205	5.6859	2.5527	5.1657	5.2341	18.991
1.5338	2.0	500	0.4386	9.4172	5.941	9.0685	9.1082	19.0
1.5338	3.0	750	0.3853	11.8647	8.8342	11.8639	11.8666	19.0
0.5244	4.0	1000	0.3544	11.9705	8.9387	11.9542	11.9761	19.0
0.5244	5.0	1250	0.3351	12.241	9.2923	12.2495	12.269	19.0
0.4437	6.0	1500	0.3227	12.4208	9.4373	12.4165	12.46	19.0
0.4437	7.0	1750	0.3115	12.3875	9.363	12.3873	12.4121	19.0
0.4122	8.0	2000	0.3055	12.5748	9.5329	12.5643	12.5861	19.0
0.4122	9.0	2250	0.3022	12.656	9.6538	12.6477	12.6745	19.0
0.397	10.0	2500	0.3008	12.6103	9.5926	12.6021	12.6283	19.0

Framework versions

• Transformers 4.38.2
• Pytorch 2.1.0+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2