flan-t5-base-flant5

This model is a fine-tuned version of google/flan-t5-base on the samsum dataset. It achieves the following results on the evaluation set:

• Loss: 1.3709
• Rouge1: 46.9086
• Rouge2: 23.3531
• Rougel: 39.5835
• Rougelsum: 43.1465
• Gen Len: 17.2027

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.4403	1.0	1842	1.3829	46.5072	23.1696	39.4293	42.8838	17.0977
1.3534	2.0	3684	1.3732	47.0676	23.4885	39.6014	43.2614	17.4554
1.2795	3.0	5526	1.3709	46.9086	23.3531	39.5835	43.1465	17.2027
1.2313	4.0	7368	1.3736	47.4405	23.7952	40.0159	43.6213	17.2198
1.1934	5.0	9210	1.3772	47.41	23.971	40.0597	43.7149	17.3162

Framework versions

• Transformers 4.25.1
• Pytorch 1.13.0+cu116
• Datasets 2.8.0
• Tokenizers 0.13.2