Edit model card

pegasus-xsum-finetuned-paws-parasci

This model is a fine-tuned version of domenicrosati/pegasus-xsum-finetuned-paws on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 3.2256
  • Rouge1: 61.8854
  • Rouge2: 43.1061
  • Rougel: 57.421
  • Rougelsum: 57.4417

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 16
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • training_steps: 4000
  • mixed_precision_training: Native AMP
  • label_smoothing_factor: 0.1

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum
No log 0.05 1000 3.8024 49.471 24.8024 43.4857 43.5552
No log 0.09 2000 3.6533 49.1046 24.4038 43.0189 43.002
No log 0.14 3000 3.5867 49.5026 24.748 43.3059 43.2923
No log 0.19 4000 3.5613 49.4319 24.5444 43.2225 43.1965

Framework versions

  • Transformers 4.18.0
  • Pytorch 1.11.0
  • Datasets 2.1.0
  • Tokenizers 0.12.1
Downloads last month
7
Hosted inference API
This model can be loaded on the Inference API on-demand.