Edit model card

switch-base-8-finetuned-samsum

This model is a fine-tuned version of google/switch-base-8 on the samsum dataset. It achieves the following results on the evaluation set:

  • Loss: 1.4606
  • Rouge1: 46.5651
  • Rouge2: 23.2378
  • Rougel: 39.4484
  • Rougelsum: 43.1011
  • Gen Len: 17.0183

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 5

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum Gen Len
1.8829 1.0 3683 1.5154 46.3805 23.0982 39.0612 43.0142 17.6296
1.6207 2.0 7366 1.4578 47.7434 24.9471 40.6481 44.351 17.2066
1.442 3.0 11049 1.4360 47.6903 24.9954 40.713 44.3487 17.0501
1.3103 4.0 14732 1.4396 48.4517 25.7725 41.5212 45.1211 16.9071
1.2393 5.0 18415 1.4445 48.4002 25.8727 41.5361 45.0467 16.9804

Framework versions

  • Transformers 4.25.1
  • Pytorch 1.13.1+cu116
  • Datasets 2.8.0
  • Tokenizers 0.13.2
Downloads last month
6
Safetensors
Model size
619M params
Tensor type
F32
ยท

Finetuned from

Dataset used to train emre/switch-base-8-finetuned-samsum

Space using emre/switch-base-8-finetuned-samsum 1

Evaluation results