t5-tweet-sum

This model is a fine-tuned version of t5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.6148
• Rouge1: 31.4255
• Rouge2: 15.9836
• Rougel: 27.4891
• Rougelsum: 29.7407
• Gen Len: 18.8972

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
2.3357	1.0	750	1.5571	31.627	15.933	27.6339	29.8606	19.0
1.574	2.0	1500	1.5342	32.2002	16.72	28.6385	30.6897	18.9944
1.2704	3.0	2250	1.6148	31.4255	15.9836	27.4891	29.7407	18.8972

Framework versions

• Transformers 4.28.0
• Pytorch 2.0.0+cu118
• Datasets 2.11.0
• Tokenizers 0.13.3