bart-cnn-pubmed-arxiv-pubmed-arxiv-arxiv-v3-e3

This model is a fine-tuned version of theojolliffe/bart-cnn-pubmed-arxiv-pubmed-arxiv-arxiv on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.8311
• Rouge1: 53.458
• Rouge2: 34.076
• Rougel: 37.3287
• Rougelsum: 50.7849
• Gen Len: 142.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	398	0.8697	52.6579	33.307	35.8099	49.9687	142.0
0.8264	2.0	796	0.8293	52.6738	33.7202	36.1502	50.0501	141.9815
0.5471	3.0	1194	0.8311	53.458	34.076	37.3287	50.7849	142.0

Framework versions

• Transformers 4.19.2
• Pytorch 1.11.0+cu113
• Datasets 2.2.2
• Tokenizers 0.12.1