Edit model card

distill-pegasus-cnn-arxiv-pubmed-v3-e16

This model is a fine-tuned version of theojolliffe/distill-pegasus-cnn-arxiv-pubmed on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 1.4922
  • Rouge1: 53.3238
  • Rouge2: 36.6165
  • Rougel: 38.9255
  • Rougelsum: 50.4853
  • Gen Len: 125.7407

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 1
  • eval_batch_size: 1
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 16
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum Gen Len
2.7655 1.0 795 2.1110 49.0541 29.7039 33.8403 44.2825 126.1296
2.2882 2.0 1590 1.9469 48.4651 30.1425 33.9702 44.3518 125.7778
2.1958 3.0 2385 1.8079 49.2302 31.0952 34.4448 45.5764 125.7778
2.0221 4.0 3180 1.7501 48.1928 29.9098 33.0587 44.6023 125.3148
1.9078 5.0 3975 1.6677 49.697 31.671 34.3162 46.5108 125.5185
1.8624 6.0 4770 1.6393 49.6517 31.7371 35.2019 46.2846 125.6852
1.7853 7.0 5565 1.6038 50.3151 33.0952 36.0028 47.3894 125.6852
1.7513 8.0 6360 1.5717 50.299 33.038 35.6841 47.4086 124.5556
1.7026 9.0 7155 1.5570 51.6216 34.7609 37.5598 48.5247 124.7037
1.6999 10.0 7950 1.5365 51.0888 34.2642 37.0603 48.5712 125.3519
1.6832 11.0 8745 1.5249 51.3422 34.2941 37.7111 48.556 124.9259
1.6093 12.0 9540 1.5092 51.4622 34.6397 38.1768 48.6346 124.8889
1.6049 13.0 10335 1.5002 52.2463 35.4629 38.2049 49.4066 124.7963
1.5904 14.0 11130 1.4957 51.6498 34.9739 38.4215 48.9704 125.0185
1.5963 15.0 11925 1.4920 52.769 35.9563 38.4861 49.9185 125.6481
1.5742 16.0 12720 1.4922 53.3238 36.6165 38.9255 50.4853 125.7407

Framework versions

  • Transformers 4.18.0
  • Pytorch 1.11.0+cu113
  • Datasets 2.1.0
  • Tokenizers 0.12.1
Downloads last month
0
Hosted inference API
This model can be loaded on the Inference API on-demand.