fine-tuned-BioBARTv2-20-epochs-1024-input-320-output

This model is a fine-tuned version of checkpoint_global_step_200000 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.7867
• Rouge1: 0.1624
• Rouge2: 0.0352
• Rougel: 0.1185
• Rougelsum: 0.1188
• Gen Len: 37.64

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	151	6.3285	0.0632	0.0007	0.0552	0.0555	37.84
No log	2.0	302	0.9332	0.1075	0.0282	0.0825	0.0825	62.51
No log	3.0	453	0.8092	0.0826	0.0196	0.0629	0.0622	28.26
4.0641	4.0	604	0.7617	0.1106	0.0346	0.0814	0.0814	32.19
4.0641	5.0	755	0.7385	0.1359	0.0266	0.1043	0.1048	35.85
4.0641	6.0	906	0.7296	0.1507	0.0296	0.1099	0.1112	45.66
0.6482	7.0	1057	0.7225	0.1315	0.026	0.0978	0.0992	36.35
0.6482	8.0	1208	0.7165	0.1573	0.0302	0.1218	0.1222	42.68
0.6482	9.0	1359	0.7191	0.1445	0.0307	0.1155	0.1156	30.12
0.4567	10.0	1510	0.7281	0.1827	0.0423	0.1403	0.1408	47.87
0.4567	11.0	1661	0.7320	0.1603	0.0311	0.1193	0.1193	33.69
0.4567	12.0	1812	0.7395	0.1697	0.0357	0.1267	0.1267	46.9
0.4567	13.0	1963	0.7515	0.1442	0.0297	0.1064	0.1065	31.28
0.3275	14.0	2114	0.7549	0.1767	0.0306	0.1255	0.1259	49.23
0.3275	15.0	2265	0.7680	0.1475	0.0327	0.1054	0.1057	37.28
0.3275	16.0	2416	0.7760	0.1525	0.0337	0.1065	0.1072	38.87
0.2407	17.0	2567	0.7797	0.1543	0.039	0.1163	0.1168	38.67
0.2407	18.0	2718	0.7845	0.1794	0.0382	0.13	0.1305	38.47
0.2407	19.0	2869	0.7860	0.1645	0.0372	0.1218	0.1219	36.26
0.1982	20.0	3020	0.7867	0.1624	0.0352	0.1185	0.1188	37.64

Framework versions

• Transformers 4.36.2
• Pytorch 1.12.1+cu113
• Datasets 2.16.1
• Tokenizers 0.15.1