long-t5-tglobal-base-samsum

This model is a fine-tuned version of google/long-t5-tglobal-base on an samsum dataset. It achieves the following results on the evaluation set:

• Loss: 1.3565
• Rouge1: 46.191
• Rouge2: 22.3147
• Rougel: 38.4272
• Rougelsum: 42.0471
• Gen Len: 16.7350

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 2

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.6898	1.0	7366	1.3795	45.766	21.7455	38.1609	41.7016	16.7265
1.5544	2.0	14732	1.3565	46.191	22.3147	38.4272	42.0471	16.7350

Framework versions

• Transformers 4.26.0
• Pytorch 1.13.1+cu117
• Datasets 2.9.0
• Tokenizers 0.13.2