fine-tuned-BioBART-20-epochs-test

This model is a fine-tuned version of checkpoint_global_step_200000 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.1997
• Rouge1: 0.0956
• Rouge2: 0.0145
• Rougel: 0.0591
• Rougelsum: 0.0593
• Gen Len: 217.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.7878	1.0	1201	0.2002	0.0888	0.0237	0.0689	0.0691	224.36
0.2064	2.0	2402	0.1817	0.02	0.0	0.02	0.02	8.0
0.1708	3.0	3603	0.1638	0.03	0.0	0.03	0.03	5.0
0.136	4.0	4804	0.1576	0.0228	0.0036	0.0232	0.023	10.0
0.1346	5.0	6005	0.1559	0.0631	0.018	0.0592	0.0591	11.0
0.097	6.0	7206	0.1573	0.0928	0.0177	0.0784	0.079	20.0
0.086	7.0	8407	0.1607	0.0638	0.0086	0.0522	0.0523	21.0
0.0638	8.0	9608	0.1649	0.0228	0.0036	0.0232	0.023	10.0
0.0425	9.0	10809	0.1690	0.064	0.0198	0.0578	0.0579	20.0
0.0359	10.0	12010	0.1726	0.1024	0.0157	0.0817	0.0817	49.0
0.0262	11.0	13211	0.1771	0.0868	0.0198	0.0787	0.0792	20.0
0.0204	12.0	14412	0.1819	0.0977	0.0104	0.0748	0.075	43.0
0.0156	13.0	15613	0.1852	0.066	0.0094	0.0509	0.051	43.0
0.0131	14.0	16814	0.1885	0.1068	0.018	0.0726	0.0725	135.0
0.0105	15.0	18015	0.1915	0.0967	0.0248	0.0784	0.0787	30.0
0.009	16.0	19216	0.1950	0.104	0.0221	0.0791	0.079	73.0
0.0081	17.0	20417	0.1962	0.0967	0.0248	0.0784	0.0787	30.0
0.0077	18.0	21618	0.1978	0.0903	0.0084	0.0567	0.0567	174.0
0.0068	19.0	22819	0.1991	0.0896	0.0107	0.055	0.055	174.0
0.0064	20.0	24020	0.1997	0.0956	0.0145	0.0591	0.0593	217.0

Framework versions

• Transformers 4.36.2
• Pytorch 1.12.1+cu113
• Datasets 2.16.1
• Tokenizers 0.15.0