fine-tuned-BioBARTv2-20-epochs-1024-input-288-output

This model is a fine-tuned version of checkpoint_global_step_200000 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.8737
• Rouge1: 0.1792
• Rouge2: 0.0323
• Rougel: 0.1391
• Rougelsum: 0.1399
• Gen Len: 36.3

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	151	6.4453	0.0289	0.0003	0.0283	0.0278	21.32
No log	2.0	302	1.0350	0.1165	0.0335	0.0969	0.0965	34.35
No log	3.0	453	0.8905	0.0745	0.0151	0.0539	0.0545	32.84
4.1419	4.0	604	0.8387	0.1225	0.0334	0.0911	0.0909	31.01
4.1419	5.0	755	0.8139	0.17	0.0335	0.1343	0.1339	53.7
4.1419	6.0	906	0.8070	0.1165	0.0242	0.0936	0.0916	26.25
0.7039	7.0	1057	0.7982	0.1367	0.0299	0.0998	0.1007	43.94
0.7039	8.0	1208	0.7926	0.1689	0.0408	0.1265	0.1276	47.35
0.7039	9.0	1359	0.8005	0.1603	0.0336	0.1338	0.1335	31.24
0.4936	10.0	1510	0.8062	0.1641	0.0358	0.1256	0.1254	33.11
0.4936	11.0	1661	0.8085	0.1934	0.0437	0.1527	0.1542	42.14
0.4936	12.0	1812	0.8143	0.1699	0.0398	0.1304	0.1301	49.52
0.4936	13.0	1963	0.8348	0.1619	0.0272	0.1263	0.1262	31.8
0.352	14.0	2114	0.8365	0.2093	0.0485	0.1657	0.1667	42.77
0.352	15.0	2265	0.8455	0.168	0.0345	0.1298	0.131	35.95
0.352	16.0	2416	0.8532	0.1953	0.048	0.1546	0.1561	37.21
0.2569	17.0	2567	0.8604	0.1834	0.0359	0.1431	0.1442	39.24
0.2569	18.0	2718	0.8702	0.1628	0.029	0.1209	0.1217	35.94
0.2569	19.0	2869	0.8712	0.1792	0.04	0.1382	0.139	37.38
0.211	20.0	3020	0.8737	0.1792	0.0323	0.1391	0.1399	36.3

Framework versions

• Transformers 4.36.2
• Pytorch 1.12.1+cu113
• Datasets 2.16.1
• Tokenizers 0.15.1