fine-tuned-flan-t5-20-epochs-2048-input-256-output

This model is a fine-tuned version of google/flan-t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.1471
• Rouge1: 0.1308
• Rouge2: 0.023
• Rougel: 0.1183
• Rougelsum: 0.1188
• Gen Len: 103.96

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	301	5.5699	0.0309	0.0079	0.0275	0.0279	167.41
8.7004	2.0	602	5.0463	0.0629	0.0101	0.0632	0.0638	135.78
8.7004	3.0	903	4.0270	0.0471	0.0049	0.0468	0.0463	205.06
6.1746	4.0	1204	3.7187	0.0739	0.0101	0.0691	0.0702	88.45
5.1998	5.0	1505	3.3997	0.0564	0.0097	0.0511	0.0519	174.76
5.1998	6.0	1806	3.1995	0.0963	0.0195	0.0878	0.0884	108.71
4.6352	7.0	2107	3.1787	0.0978	0.0159	0.089	0.0893	143.4
4.6352	8.0	2408	3.1274	0.1123	0.0184	0.1037	0.1035	133.42
4.0979	9.0	2709	2.9934	0.0885	0.0169	0.0818	0.0811	136.61
3.7568	10.0	3010	2.9458	0.121	0.0154	0.1134	0.1122	141.13
3.7568	11.0	3311	2.9357	0.1232	0.0186	0.1119	0.1122	136.52
3.5713	12.0	3612	2.9760	0.1127	0.0199	0.1011	0.1009	96.31
3.5713	13.0	3913	2.9262	0.0962	0.0135	0.0854	0.0848	136.75
3.2308	14.0	4214	2.9597	0.1213	0.0248	0.1118	0.1122	125.09
3.0663	15.0	4515	3.0330	0.1054	0.019	0.0941	0.0934	130.3
3.0663	16.0	4816	3.0490	0.126	0.0203	0.1125	0.1137	123.51
2.9285	17.0	5117	3.0463	0.1215	0.0151	0.1086	0.1087	106.23
2.9285	18.0	5418	3.1519	0.1278	0.0195	0.1142	0.1137	108.3
2.6943	19.0	5719	3.1072	0.1338	0.017	0.1204	0.1206	105.96
2.7837	20.0	6020	3.1471	0.1308	0.023	0.1183	0.1188	103.96

Framework versions

• Transformers 4.36.2
• Pytorch 1.12.1+cu113
• Datasets 2.16.1
• Tokenizers 0.15.0