t5-base-finetuned-newssum

This model is a fine-tuned version of t5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4711
• Rouge1: 40.715
• Rouge2: 32.036
• Rougel: 40.3437
• Rougelsum: 40.4235
• Gen Len: 8.4108

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.002
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 6

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	403	0.4862	38.2219	28.7394	37.9058	37.8425	8.2259
0.8381	2.0	806	0.4148	40.5379	30.8609	40.1883	40.1758	8.3742
0.3367	3.0	1209	0.4000	41.5718	32.8396	41.2338	41.2688	8.646
0.2095	4.0	1612	0.4131	40.6043	32.2201	40.2509	40.2628	8.3354
0.1339	5.0	2015	0.4433	41.0629	32.4348	40.7525	40.7657	8.5436
0.1339	6.0	2418	0.4711	40.715	32.036	40.3437	40.4235	8.4108

Framework versions

• Transformers 4.34.1
• Pytorch 2.1.0+cu118
• Datasets 2.14.6
• Tokenizers 0.14.1