flan-t5-large-extraction-cnndm_2000-all

This model was trained from scratch on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.8064
• Rouge1: 35.397
• Rouge2: 15.255
• Rougel: 30.1755
• Rougelsum: 30.1503
• Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 24
• seed: 1799
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.5259	0.8	200	1.8107	34.9007	14.9425	29.8237	29.8034	18.986
1.4407	1.6	400	1.8064	35.397	15.255	30.1755	30.1503	19.0
1.3694	2.4	600	1.8813	34.9925	15.2	30.3208	30.3286	18.99
1.3414	3.2	800	1.8937	34.8974	15.1983	30.3056	30.3014	19.0
1.3025	4.0	1000	1.8598	35.8005	15.6921	30.4112	30.4205	18.998
1.388	4.8	1200	1.8488	35.4537	15.1981	30.3749	30.3421	19.0
1.3503	5.6	1400	1.8335	35.7368	15.5233	30.6031	30.629	18.998

Framework versions

• Transformers 4.18.0
• Pytorch 1.10.0+cu111
• Datasets 2.5.1
• Tokenizers 0.12.1