distilbart-cnn-arxiv-pubmed-pubmed-v3-e16

This model is a fine-tuned version of theojolliffe/distilbart-cnn-arxiv-pubmed-pubmed on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.8306
• Rouge1: 56.4519
• Rouge2: 41.6818
• Rougel: 44.7833
• Rougelsum: 54.6359
• Gen Len: 141.9815

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 16
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	398	1.1157	50.9487	31.3005	34.0145	48.6057	141.8519
1.3569	2.0	796	0.9688	53.0653	34.1855	37.0759	50.5942	141.2963
0.8704	3.0	1194	0.9053	53.9684	36.0388	38.6674	51.9604	142.0
0.6287	4.0	1592	0.8515	54.2379	36.4915	39.1393	51.6991	141.4074
0.6287	5.0	1990	0.8274	53.6806	34.8373	37.7369	51.239	141.6481
0.465	6.0	2388	0.8486	55.2534	39.1757	41.6366	53.2989	141.9259
0.3432	7.0	2786	0.8116	54.539	37.6314	40.5531	52.1997	141.3889
0.2577	8.0	3184	0.7976	54.8212	36.8347	40.6768	52.7785	142.0
0.204	9.0	3582	0.8010	53.9302	37.3523	40.135	52.139	141.7778
0.204	10.0	3980	0.8168	54.3151	38.0665	42.4112	52.4682	142.0
0.1663	11.0	4378	0.8171	54.7027	38.3117	42.0196	52.8821	142.0
0.135	12.0	4776	0.8202	54.1035	37.9154	40.7676	52.2509	142.0
0.1102	13.0	5174	0.8204	56.223	41.0947	44.0131	54.3353	142.0
0.0928	14.0	5572	0.8280	56.1637	41.0408	44.2931	54.5488	142.0
0.0928	15.0	5970	0.8273	56.2608	41.3855	44.4432	54.5778	142.0
0.0847	16.0	6368	0.8306	56.4519	41.6818	44.7833	54.6359	141.9815

Framework versions

• Transformers 4.18.0
• Pytorch 1.11.0+cu113
• Datasets 2.2.0
• Tokenizers 0.12.1