t5-small-mlsum / README.md
kamranshah's picture
End of training
587c08c verified
metadata
license: apache-2.0
base_model: t5-small
tags:
  - generated_from_trainer
datasets:
  - mlsum
metrics:
  - rouge
model-index:
  - name: t5-small-mlsum
    results:
      - task:
          name: Sequence-to-sequence Language Modeling
          type: text2text-generation
        dataset:
          name: mlsum
          type: mlsum
          config: tu
          split: None
          args: tu
        metrics:
          - name: Rouge1
            type: rouge
            value: 14.4732

t5-small-mlsum

This model is a fine-tuned version of t5-small on the mlsum dataset. It achieves the following results on the evaluation set:

  • Loss: 2.6372
  • Rouge1: 14.4732
  • Rouge2: 6.6752
  • Rougel: 13.4183
  • Rougelsum: 13.8427
  • Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 5

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum Gen Len
No log 1.0 13 2.7607 14.4395 6.4679 13.2562 13.6373 19.0
No log 2.0 26 2.7068 14.4214 6.4106 13.4536 13.7502 19.0
No log 3.0 39 2.6689 14.7941 6.5511 13.6862 14.1839 19.0
No log 4.0 52 2.6450 14.3539 6.6061 13.281 13.7636 19.0
No log 5.0 65 2.6372 14.4732 6.6752 13.4183 13.8427 19.0

Framework versions

  • Transformers 4.41.2
  • Pytorch 2.3.0+cu121
  • Datasets 2.19.1
  • Tokenizers 0.19.1