fine-tuned-BioBART-20-epochs-1048-output

This model is a fine-tuned version of checkpoint_global_step_200000 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2283
• Rouge1: 0.087
• Rouge2: 0.0117
• Rougel: 0.0565
• Rougelsum: 0.0561
• Gen Len: 191.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.0073	1.0	1201	0.2033	0.0963	0.0192	0.0665	0.0663	167.0
0.014	2.0	2402	0.1983	0.0642	0.0194	0.0578	0.0578	20.0
0.018	3.0	3603	0.2010	0.1027	0.0117	0.0849	0.0855	25.0
0.0119	4.0	4804	0.2012	0.0932	0.0182	0.0649	0.0647	167.0
0.0109	5.0	6005	0.2059	0.1115	0.0203	0.0829	0.0828	52.0
0.009	6.0	7206	0.2083	0.0817	0.0132	0.0649	0.0651	29.0
0.0083	7.0	8407	0.2091	0.0785	0.0134	0.0592	0.0592	72.0
0.0081	8.0	9608	0.2113	0.095	0.0118	0.0623	0.0622	191.0
0.0072	9.0	10809	0.2142	0.0945	0.01	0.0619	0.0617	169.0
0.0072	10.0	12010	0.2163	0.0957	0.0182	0.0844	0.0845	27.0
0.0066	11.0	13211	0.2170	0.1006	0.0166	0.0652	0.0651	97.0
0.0062	12.0	14412	0.2189	0.0852	0.0122	0.0529	0.0526	206.0
0.0062	13.0	15613	0.2208	0.0967	0.0195	0.0855	0.086	24.0
0.0059	14.0	16814	0.2218	0.0783	0.0113	0.063	0.0629	43.0
0.0057	15.0	18015	0.2212	0.0961	0.0246	0.0786	0.0786	30.0
0.0054	16.0	19216	0.2248	0.1014	0.0211	0.0761	0.0763	79.0
0.0052	17.0	20417	0.2271	0.0874	0.0171	0.0775	0.0774	21.0
0.0051	18.0	21618	0.2268	0.0914	0.0138	0.0595	0.0592	160.0
0.0049	19.0	22819	0.2285	0.091	0.014	0.0594	0.0591	160.0
0.0048	20.0	24020	0.2283	0.087	0.0117	0.0565	0.0561	191.0

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.0