metadata

license: apache-2.0
tags:
  - generated_from_trainer
  - summarization
datasets:
  - xsum
  - autoevaluate/xsum-sample
metrics:
  - rouge
model-index:
  - name: summarization
    results:
      - task:
          type: text2text-generation
          name: Sequence-to-sequence Language Modeling
        dataset:
          name: xsum
          type: xsum
          args: default
        metrics:
          - type: rouge
            value: 23.9405
            name: Rouge1
      - task:
          type: summarization
          name: Summarization
        dataset:
          name: autoevaluate/xsum-sample
          type: autoevaluate/xsum-sample
          config: autoevaluate--xsum-sample
          split: test
        metrics:
          - type: rouge
            value: 18.3841
            name: ROUGE-1
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOTYyOTMyYjUyZDU5MWUzOWNkYTcxYWRhMDJiMGUyYmJkNTc4Mjc4YjM2ODQ1ZTU3YWFhNzU0ZTcxYjVhODgyYSIsInZlcnNpb24iOjF9.Cgyt2MvDbRO7n7Yv4LCUp0hd_zunJFWB5q59p63-qHQdojgY83pnp6mK4Do2OCdvWW9PZQCBhC6PBc4FLrrmBw
          - type: rouge
            value: 3.0683
            name: ROUGE-2
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNDEyYWE4YjI3YTQzNGNhMDZkNGZhNjgwMTAzOGFlMjNiN2Y1ZWU1MGFmZWQ1ZjZjMmFiNDAyYTY2ZWZiZTg3MyIsInZlcnNpb24iOjF9.Yu8URVvUyAfIkXUekW2i_sA2JQuy-DTzuSmd4x9ZvrnrwTgMp0j5-Y5DxkgBVmMkx6ALciWtGB7rEvsxPZXmDg
          - type: rouge
            value: 14.8213
            name: ROUGE-L
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZmU5NjQ4NzVhMTVmZWQwZWI0MjA5MDJkMzQ3MTA2YzJlNTBiYzczYjIzY2MzZGIyYTU2Y2ViMTA1NTc5YWI1OSIsInZlcnNpb24iOjF9.Y63H8nXf3HpaCcekMrlr62zCginIFK5HHfZ_4vsYG8vTvhazvNAOB1ahdgg4gwZZtotdgNOv2jEu-LY1QTApCQ
          - type: rouge
            value: 14.8475
            name: ROUGE-LSUM
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZjAxM2QyMzQ5OWJhNGNmYzU0YjMyMzkxNjk4MTdmMDhmODNmZjVlOTU2NTU0N2M1MTFkYWU3ZmNkNzM0YjJjMSIsInZlcnNpb24iOjF9.otRM0NSyExwRs2RomqaMVGRJEE3wM34FGlnmY_2k1T1bQSQHPdHXTcRSh6vyYW-hCmCE1K0ILV2kyIR8pu7FDA
          - type: loss
            value: 3.0104541778564453
            name: loss
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMjc0YjFmZTBhZTNkMDczYzczZDk5YTdiNzg4MDUwODI0NDQyODg2MGIxYzRkMzBkMTU4ZDQxYjJlZTRjZGMwMyIsInZlcnNpb24iOjF9.dl4Nqw9EAQSbnc9JqLDu6sCt3P73G_53zc0Yh9d8Um4up3PISjkZNQDxqKQ68N_vgU0s0IGHQLxxJYgUkSgwBg
          - type: gen_len
            value: 18.05
            name: gen_len
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZWU3N2NkMmRhZjQ1YzYyOTNiNzJiMmNkMzlmYjYxMGVhNjhkMGQyZjliYmUyNGU0MTEwOGM3NWFmYmZjODhmOSIsInZlcnNpb24iOjF9.vgT2OjClJ21_tnYr933is4DqCgxKNQ0GlhrTWpUqFhypHkX_w1lZZi-DnIT8GdyLZBXpZkCFwWYi6jeXDGKnCQ

summarization

This model is a fine-tuned version of t5-small on the xsum dataset. It achieves the following results on the evaluation set:

Loss: 2.6690
Rouge1: 23.9405
Rouge2: 5.0879
Rougel: 18.4981
Rougelsum: 18.5032
Gen Len: 18.7376

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 16
eval_batch_size: 16
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
training_steps: 1000
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
2.9249	0.08	1000	2.6690	23.9405	5.0879	18.4981	18.5032	18.7376

Framework versions

Transformers 4.19.2
Pytorch 1.11.0+cu113
Datasets 2.2.2
Tokenizers 0.12.1