flan-t5-base-samsum

This model is a fine-tuned version of google/flan-t5-base on the samsum dataset. It achieves the following results on the evaluation set:

• Loss: 1.4073
• Rouge1: 47.785
• Rouge2: 22.4604
• Rougel: 39.9337
• Rougelsum: 43.8529
• Gen Len: 17.4

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	0.96	15	1.4247	47.5374	22.6497	39.7898	43.5814	17.29
No log	1.98	31	1.4103	47.7422	22.905	40.249	43.749	17.52
No log	2.94	46	1.4079	46.9881	22.0884	39.4011	43.1689	17.44
No log	3.97	62	1.4074	47.8586	22.572	40.0117	43.918	17.42
No log	4.8	75	1.4073	47.785	22.4604	39.9337	43.8529	17.4

Framework versions

• Transformers 4.28.0
• Pytorch 2.0.1+cu118
• Datasets 2.12.0
• Tokenizers 0.13.3