fine-tuned-BioBART-20-epochs-wang-lab

This model is a fine-tuned version of GanjinZero/biobart-v2-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.0360
• Rouge1: 0.308
• Rouge2: 0.1254
• Rougel: 0.2791
• Rougelsum: 0.2797
• Gen Len: 15.85

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	301	0.7404	0.2534	0.1025	0.2218	0.221	15.29
0.851	2.0	602	0.7099	0.2907	0.1069	0.255	0.2557	15.04
0.851	3.0	903	0.6956	0.2652	0.1049	0.2313	0.23	16.04
0.5436	4.0	1204	0.7288	0.3168	0.129	0.286	0.2872	15.11
0.3356	5.0	1505	0.7856	0.2869	0.1041	0.2569	0.2601	15.4
0.3356	6.0	1806	0.8174	0.2863	0.1214	0.2447	0.2444	15.95
0.1807	7.0	2107	0.8477	0.3048	0.1169	0.2787	0.2797	15.82
0.1807	8.0	2408	0.8816	0.3118	0.1227	0.2726	0.2733	15.66
0.1072	9.0	2709	0.9081	0.2988	0.1169	0.2665	0.2668	14.73
0.0649	10.0	3010	0.9342	0.2869	0.1175	0.2531	0.2535	15.63
0.0649	11.0	3311	0.9588	0.3094	0.1212	0.2722	0.2747	15.77
0.0376	12.0	3612	0.9761	0.3147	0.1197	0.2867	0.2882	15.62
0.0376	13.0	3913	0.9870	0.3144	0.1172	0.2843	0.285	16.09
0.0244	14.0	4214	0.9918	0.3217	0.1267	0.2931	0.2942	15.94
0.0145	15.0	4515	1.0044	0.3102	0.1196	0.2801	0.2815	15.91
0.0145	16.0	4816	1.0152	0.3094	0.1316	0.2796	0.2804	16.01
0.0094	17.0	5117	1.0290	0.317	0.1133	0.2838	0.2857	15.77
0.0094	18.0	5418	1.0337	0.3006	0.1216	0.2712	0.272	15.9
0.0066	19.0	5719	1.0346	0.307	0.1254	0.2785	0.2797	15.85
0.0047	20.0	6020	1.0360	0.308	0.1254	0.2791	0.2797	15.85

Framework versions

• Transformers 4.36.2
• Pytorch 1.12.1+cu113
• Datasets 2.15.0
• Tokenizers 0.15.0