fine-tuned-bart-20-epochs-1024-input-160-output

This model is a fine-tuned version of bart-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.3882
• Rouge1: 0.1555
• Rouge2: 0.0334
• Rougel: 0.128
• Rougelsum: 0.1281
• Gen Len: 32.08

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	151	6.3727	0.0	0.0	0.0	0.0	10.49
No log	2.0	302	1.6983	0.107	0.0313	0.0918	0.0918	33.69
No log	3.0	453	1.4585	0.0292	0.0038	0.0258	0.0255	9.54
4.3803	4.0	604	1.3546	0.1501	0.0361	0.1189	0.1182	40.5
4.3803	5.0	755	1.3132	0.1543	0.0283	0.123	0.1237	32.81
4.3803	6.0	906	1.2906	0.1518	0.0289	0.1263	0.1246	31.78
1.0689	7.0	1057	1.2756	0.1467	0.0342	0.1154	0.1145	38.24
1.0689	8.0	1208	1.2746	0.1517	0.04	0.1168	0.117	32.48
1.0689	9.0	1359	1.2791	0.1675	0.0297	0.1339	0.1334	35.91
0.7243	10.0	1510	1.2832	0.1739	0.0426	0.1337	0.1348	35.1
0.7243	11.0	1661	1.2935	0.1827	0.0337	0.1429	0.1431	34.79
0.7243	12.0	1812	1.3085	0.1709	0.0278	0.1298	0.1296	40.29
0.7243	13.0	1963	1.3220	0.1815	0.0352	0.1387	0.1388	32.13
0.4908	14.0	2114	1.3308	0.1564	0.0272	0.1204	0.1206	38.63
0.4908	15.0	2265	1.3496	0.1662	0.0284	0.1292	0.1286	29.51
0.4908	16.0	2416	1.3737	0.1613	0.0316	0.132	0.133	29.14
0.3443	17.0	2567	1.3631	0.1835	0.0338	0.1415	0.1422	34.59
0.3443	18.0	2718	1.3836	0.1594	0.0329	0.1263	0.1265	32.76
0.3443	19.0	2869	1.3819	0.1587	0.0325	0.1251	0.1246	35.47
0.271	20.0	3020	1.3882	0.1555	0.0334	0.128	0.1281	32.08

Framework versions

• Transformers 4.36.2
• Pytorch 1.12.1+cu113
• Datasets 2.16.1
• Tokenizers 0.15.1