Edit model card

bart-base-cnn-xsum-wiki-swe

This model is a fine-tuned version of Gabriel/bart-base-cnn-xsum-swe on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 2.3884
  • Rouge1: 26.8917
  • Rouge2: 11.8254
  • Rougel: 22.6089
  • Rougelsum: 26.1492
  • Gen Len: 19.3468

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 32
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 500
  • num_epochs: 9
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum Gen Len
2.4993 1.0 2985 2.3834 25.8959 10.9373 21.8329 25.2002 19.1416
2.2397 2.0 5970 2.2939 26.1166 11.4087 22.2444 25.4752 19.2351
2.0318 3.0 8955 2.2687 26.5222 11.6512 22.567 25.851 19.2384
1.879 4.0 11940 2.2750 26.7637 11.7676 22.6674 26.0753 19.2622
1.7532 5.0 14925 2.2923 26.8104 11.8724 22.6794 26.0907 19.3063
1.6315 6.0 17910 2.3190 26.7758 11.7989 22.5925 26.032 19.3136
1.5409 7.0 20895 2.3517 26.8762 11.8552 22.6694 26.1329 19.3275
1.4711 8.0 23880 2.3679 26.899 11.9185 22.6764 26.1574 19.2994
1.4105 9.0 26865 2.3884 26.8917 11.8254 22.6089 26.1492 19.3468

Framework versions

  • Transformers 4.22.2
  • Pytorch 1.12.1+cu113
  • Datasets 2.5.1
  • Tokenizers 0.12.1
Downloads last month
11

Space using Gabriel/bart-base-cnn-xsum-wiki-swe 1