flan-t5-large-extraction-cnndm_10000-all

This model is a fine-tuned version of google/flan-t5-large on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.7044
• Rouge1: 34.8618
• Rouge2: 15.5978
• Rougel: 29.7948
• Rougelsum: 29.7581
• Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 24
• seed: 1799
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
2.1668	0.16	200	1.8280	33.7941	14.3114	28.7743	28.7968	19.0
1.9736	0.32	400	1.7818	34.8351	15.5548	29.8974	29.8557	18.99
1.904	0.48	600	1.7513	35.465	15.8566	30.7139	30.6596	18.986
1.8938	0.64	800	1.7440	34.6193	15.5473	30.0661	30.0019	18.99
1.8471	0.8	1000	1.7366	34.553	15.2214	29.8807	29.8419	18.99
1.8621	0.96	1200	1.7486	34.9309	15.1932	29.8973	29.8774	18.99
1.8082	1.12	1400	1.7311	35.3395	16.0976	30.2748	30.293	18.99
1.7448	1.28	1600	1.7155	35.1387	15.7462	29.924	29.9287	18.99
1.7655	1.44	1800	1.7239	35.3603	15.6355	30.3944	30.3766	19.0
1.7283	1.6	2000	1.7132	34.7368	15.4073	29.9027	29.8971	19.0
1.7463	1.76	2200	1.7171	35.0545	15.726	30.0364	30.0056	19.0
1.7462	1.92	2400	1.7044	34.8618	15.5978	29.7948	29.7581	19.0
1.719	2.08	2600	1.7285	34.9598	15.5237	29.5593	29.5803	19.0
1.6828	2.24	2800	1.7179	35.0944	15.7333	29.8381	29.7784	19.0
1.7	2.4	3000	1.7047	35.1766	15.7758	29.818	29.7859	19.0

Framework versions

• Transformers 4.18.0
• Pytorch 1.10.0+cu111
• Datasets 2.5.1
• Tokenizers 0.12.1