t5-small_rouge_finetuned_sbic

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0640
• Rouge1: 77.8685
• Rouge2: 74.5217
• Rougel: 77.8247
• Rougelsum: 77.8156
• Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.0809	0.5	1000	0.0727	76.9035	72.8715	76.8453	76.84	19.0
0.0745	1.01	2000	0.0685	77.4372	73.7627	77.4008	77.3853	19.0
0.0739	1.51	3000	0.0659	77.8732	74.4368	77.8305	77.8222	19.0
0.0725	2.01	4000	0.0647	77.8991	74.4654	77.8536	77.8484	19.0
0.0696	2.51	5000	0.0640	77.8685	74.5217	77.8247	77.8156	19.0

Framework versions

• Transformers 4.34.1
• Pytorch 2.1.0+cu118
• Datasets 2.14.6
• Tokenizers 0.14.1