flan-t5-base-samsum

This model is a fine-tuned version of google/flan-t5-base on the samsum dataset. It achieves the following results on the evaluation set:

• Loss: 1.3538
• Rouge1: 47.6993
• Rouge2: 24.0887
• Rougel: 40.2819
• Rougelsum: 43.8375
• Gen Len: 17.0842

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.4327	1.0	1841	1.3620	47.1926	23.4593	39.7385	43.2514	17.0623
1.3235	2.0	3683	1.3563	46.7874	23.1964	39.4248	42.9616	16.9585
1.2477	3.0	5524	1.3538	47.6993	24.0887	40.2819	43.8375	17.0842
1.208	4.0	7366	1.3555	47.6355	24.0054	40.1665	43.6581	17.0965
1.193	5.0	9205	1.3563	47.6582	23.9906	40.1561	43.7082	17.1477

Framework versions

• Transformers 4.36.1
• Pytorch 2.1.0
• Datasets 2.15.0
• Tokenizers 0.15.0