Hugging Face's logo

facebook
/
bart-large-xsum

Summarization
Transformers
PyTorch
TensorFlow
JAX
Rust
English
bart
text2text-generation
Eval Results
Inference Endpoints
Model card Files Community
11
bart-large-xsum / README.md
autoevaluator's picture
autoevaluator HF staff
Add verifyToken field to verify evaluation results are produced by Hugging Face's automatic model evaluator
636e504
preview code
|
raw
history blame
9.24 kB
metadata
language:
  - en
license: mit
tags:
  - summarization
model-index:
  - name: facebook/bart-large-xsum
    results:
      - task:
          type: summarization
          name: Summarization
        dataset:
          name: cnn_dailymail
          type: cnn_dailymail
          config: 3.0.0
          split: test
        metrics:
          - type: rouge
            value: 25.2697
            name: ROUGE-1
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMTM3ZWYzNDIyZWRlYzI2MDRkNmQwNzg4NTBhMzMzMmEwNGI5Mjg4ZGNkYzc0ODJjMWNjM2VkMDczNzk4M2ZhYiIsInZlcnNpb24iOjF9.FhfTibmxB-KfZdA0QA-dlaW2s837Y34litHb4SomxCTctYAuwwuFXhRjaYd1a3Q0RurJAOS5v31-LyQVnBiOBw
          - type: rouge
            value: 7.6638
            name: ROUGE-2
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNGI5NWVhYzAxYjRlMjRjNDlmZTRiNmY0Y2ZhYjJhNDA0MWRlZDUwZGIyZDg3MmViOTQ1MmQ3YjQxZjg4MWZhMyIsInZlcnNpb24iOjF9.CR6lwIak_ku4EiobhSpyAhtJmHdqJaBldAgJLGgrI1FZ4fQGWVcz1ugfD5O0amFeA5vYGO4_mppjuRhGR2ZMAA
          - type: rouge
            value: 17.1808
            name: ROUGE-L
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZGFhMmQwMWQyYzZhM2ZmOGMzYzI1YTg5MGQ3YWNmYTZjOWNhYjg1YzQ3MDA4MzA4Y2QxZWVlMjgyOWNkZjE3ZCIsInZlcnNpb24iOjF9.bbTmYKsUG57-9gCbk4f789A1GzvUpzjrAGI_GBGgg9TK-Lu56x38scURnsAENrKmEqOjBSTqROkEMZSJQ0bRAg
          - type: rouge
            value: 21.7933
            name: ROUGE-LSUM
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMzQ0ODY2M2Y4YjMxZTMxNTczYWVmOTQ0MDY1Y2Y5YWM3ZTU1ZTE2YWQyNWM4ZDIyNGZlZDYyM2VkNjQ2MjI2MyIsInZlcnNpb24iOjF9.28iOKCynvIt6kK5mhM6ZzKJsnwjVv_CDMG8veAB0JYeZ4yyrM-tyQUzaorFAbEyb9JBJnpn7YJR9ntGTP3YcCw
          - type: loss
            value: 3.5042972564697266
            name: loss
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNmQxMTc2OWU5MWFjODNkNGRjYzU5NWRkOWVlYzY3OGRkMGY2N2NlYmM4MTJkZDRkZTBkMDI0ZWJmYTUxYzZjMSIsInZlcnNpb24iOjF9.PbOoV_245iT1FuAOf03tKAIkBVFRop4XsB26v5qF4dPPeZRP2M2pXJOZli0hILH6hUX0-D3IGDV-a8segNCICw
          - type: gen_len
            value: 27.4462
            name: gen_len
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiN2Q1YmRkNTE5OTVlYjdlYzc4YTIyYjhjNjIyOWUxMTc5MDBiNTVjYzQ3NWEyODdhZGFjNzUzNjIzMTcxNzhlYSIsInZlcnNpb24iOjF9.szl4fWDoBqVXKbBQxXV9DFgk9UbFLedmiZmGBI1sKoN69jw8IZopOs4VYtyY5TbpzsVGzpHoZnRCCrVQG8V3AQ
      - task:
          type: summarization
          name: Summarization
        dataset:
          name: xsum
          type: xsum
          config: default
          split: test
        metrics:
          - type: rouge
            value: 45.4525
            name: ROUGE-1
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYTIxOWFlMWQxYzBiMzg1ODkxMmI3NTQwYmFiYzlkZDc4YTc3OWNhNDc3YTNmNjdhOGQyNGQ3NWExYTJhMGVjZiIsInZlcnNpb24iOjF9.CEDDzEKRIdxVndsQ5R9P6ROu70YMTUFtKQcDNI2BaHpoux3uqR20xBnd9xVJbaihnTm8Rn3Gz3FSEU6HloteBg
          - type: rouge
            value: 22.3455
            name: ROUGE-2
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOTBjNTgzNmY4ZGU0MzcyYzNmOGVmYTAwODgyNjNkNjMwNTA2YWRkNDM2NzM2NDQxN2IzOWMxMGFjZDkwY2I0YiIsInZlcnNpb24iOjF9.30NFIQgNYUdqCgIni38Nd0mPjkFAqEQqCnB0p58Csiukp8oZ9NSRUJJHKsxdQ_3mcmkwz4l8C87AdarL-X2wBw
          - type: rouge
            value: 37.2302
            name: ROUGE-L
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNTg5Yzg3OGM0MTM0ODI0NTJiMmQwM2Q5M2ExZWMwZDU2YjZjNzJmZGVmNWVmMWExYzYxMzRhOTg1MzUyMTY0ZCIsInZlcnNpb24iOjF9.7Os0OXC-gX5s0kcEhiKdSv9j40g_EOIrOXGSMPAZJz3NDP1EkKddSPpha8mwFFkphxYbryg69Z6b1NE3TBa5AQ
          - type: rouge
            value: 37.2323
            name: ROUGE-LSUM
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMzAyMDUzN2RlZDE5MjdiZGM1NjU2ZjQ1NGJlZTZmNjNlMjc2ZWUxZWU3YmY2OGY0YmJiZWViOTUyMDg2MDdhMiIsInZlcnNpb24iOjF9.M18WnqZ79MnAwz17NRHQ2iqv2_JneQ2SIV2sx10Pi3ACLYLordzcYAeQAGcfh38qQow-TyPXS-MC2Alwjvj3Bw
          - type: loss
            value: 2.3128726482391357
            name: loss
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYzAwZGZkYTVkZGM1MTNhNjA3NzBmNDMxNGJiODA0MzVhYmZkYjIzYzc1OGYxMjNiMDFhZjkxOGZmYTk1YjUyYyIsInZlcnNpb24iOjF9.i3pCisuPdpYFNLjniPejbseEeh3j6elXWze19As4pUJb3Gxp8uStckPR5rhmV_r-FDP7wKFY2GrqJZrWGGsVBg
          - type: gen_len
            value: 25.5435
            name: gen_len
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZTE3ODlkZDhhMTEwNTlhNzVjMWMxMGQyZDc0OTc0NWY0MDBlMzUzNGI3MGQwNmJmNzQ3NTQ5MjhhNDhiYTM5YSIsInZlcnNpb24iOjF9.e7nHzg3OH3zkWiCj3iZVAAQG6Zy0E16_MJzBEEyGTlSVuPGMziNfcjRvLD6WeY_6lXUonEwc9lur0X-qUvB7Aw
      - task:
          type: summarization
          name: Summarization
        dataset:
          name: samsum
          type: samsum
          config: samsum
          split: train
        metrics:
          - type: rouge
            value: 24.7852
            name: ROUGE-1
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiY2Y0NDZlNGM4M2IzMDNhZmI4MTI2ZTI0MmQ4ODYxYTk1Zjc5ZGYyYWQzNmMyN2Q2MDViNDVjNDBlYjVkNDM0MiIsInZlcnNpb24iOjF9.oSFDaNHNgTQ5WnrJqn01jXG2u-5HPhIrmPtyc_LT2kGwGs3dBe89HVsRFs3c6oAumvstKadMJ4TcQy6pDdg1Dw
          - type: rouge
            value: 5.2533
            name: ROUGE-2
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOTc1ZTJhY2U1MDNlM2EyN2Q0NWJkMDZlYTdkYzM3M2ZmY2JlMzU4ZDQwYWMzZjhhMTU5Y2VlMTIwNWYzNWM1MiIsInZlcnNpb24iOjF9.meJyYKPZRtmT2YLBiBWBxPf1XiZVFDjbFrc5_SVJv3EWvNuXlTZy0qUVbZgE9rnKLA1ND_0Yj1o-qlY8G6iVDg
          - type: rouge
            value: 18.6792
            name: ROUGE-L
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYzYxMjIzNWIzOTRlYWYwMmQ5YjFiNmM1NDRkMzVhZGI0OTJkMjE0OThmNGEwOWRkZjJiMGYyMTc0Zjc3NjUyNSIsInZlcnNpb24iOjF9.VlXVgQSBVdDjduKV_kg2TRoinJn7kkfsTcLJa_iwDTn2Lw0ZyyOBTcGdfWFcXeteee9m0-iA7uZBGkiaKtQkBw
          - type: rouge
            value: 20.629
            name: ROUGE-LSUM
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMzQ3NmM1ZTcyMGM3ZDRkZmJmN2IyZmNmOTFkYzYzZmFjNTJmYWVhYmViZGIwY2U0YzIzNWFjYzZjYjc2NGZhMyIsInZlcnNpb24iOjF9.POIXnLpVaPYKk07apBROnvbevoI4LNfs9LAelqJmL5aZsQrvb9w_mUj_y8cr_JtWMcYioKvMQfCNqweMR0QlCw
          - type: loss
            value: 3.746837854385376
            name: loss
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMDUxODlhM2NmNTNkY2E5OWE0MmVhMzEzOWFlZDUzMGUzNjQ0OWJmYmUxYWY2NzU3ZjQwZjYyMGQ0MDlkMDA2MyIsInZlcnNpb24iOjF9.dtkcai-opGLauvudNLIxw0GtkNF5DlcUG7A7h2xi42ymyUNigrAg0PcjcjuUt8uW4SEf4oTON5nmlLu924m_Dg
          - type: gen_len
            value: 23.1206
            name: gen_len
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNWMzOTVkOWYyYmJkZjY1MDJkMDdkMDhiMmEwNWJjNDU2ZDliMDg4ZTY0ODllM2VlMDUzNWY1NGViZjMyODZlNSIsInZlcnNpb24iOjF9.sLobtAdG5opp6UgM0sMzOQdXKQRbPNFFYng0Yv62loV8Ihwz5aDr8v0rlmmmvjUI45zXKpFdll0MP2mEjD91Aw
      - task:
          type: summarization
          name: Summarization
        dataset:
          name: samsum
          type: samsum
          config: samsum
          split: test
        metrics:
          - type: rouge
            value: 24.9158
            name: ROUGE-1
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYmZmYzYxNDU1YWVmYTNhZTAzYWNhNjE4YmQzNzUwMmJmZjM0OTdlZGY4NDJiNmU1OGM1NGUyZGU3MjEyNmZjZCIsInZlcnNpb24iOjF9.4_VqXLFvNv4EMJSg-vMYoj1BGp5ayLay8soylnHwEqicYeLyYNjeN1aYO4HFn9juBejXWLb1Yhe5n3nET8tRBg
          - type: rouge
            value: 5.5837
            name: ROUGE-2
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYzU2YWVjOWY3YzAzNjc2NWYxZTkzZDgyMzA4NzY0YWJmNjdlZTVlYWY0MmRkNjhiMjc1ZWZmNGRiYTJiNjNhNiIsInZlcnNpb24iOjF9.10Zo_slj3TtOkQ1ve2w2As8NrLUU1tjqyku7UMREqtYigd56p4SRuFZtr-cfZW1nrVfXrOw0BQlWMF1LevDECA
          - type: rouge
            value: 18.8935
            name: ROUGE-L
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYWYwMWY3NmEwZDYxYTMwYmM2NjY1YmFmOWI3Mjg3NmI2MGM5MDQwNzAyMGY0ZWVjNzc0MjJiY2IwNDhjNmEyZSIsInZlcnNpb24iOjF9.sbb6WadT2lIB1JgWxGgMg2hzkSp5sTn5qBbUfUZFupv1ugvpGEE6bCo7fNuYCQRu0qOYvWGNyibhYIAoJqymDQ
          - type: rouge
            value: 20.76
            name: ROUGE-LSUM
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOTcyNzJhMTU4YjM1ZDgyZWJiMTAyYzAyYTU4Y2E0M2M1YjVmODQzM2JlMmM2YmM3MWZlZTVhNWJkMzdjODhmNCIsInZlcnNpb24iOjF9.YQJDobcKtaOIro8g7Y7opjpfKZ081aJvYKCpzkBhDA5di1GIKtIjGkHqdulqtcGog_L5IcEfr9QBmwIGRFNqCw
          - type: loss
            value: 3.775235891342163
            name: loss
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiM2ViMmVhNGFkZmFiYmU5NmQxZmIzMTBkYTYwOGQ5NzA3MWQxMmZmZjljMmNkNjNkZGFlMTI0MGY2ZDQ3MjAxNSIsInZlcnNpb24iOjF9.YH3xzE3aQCPUXm1591TdRyJo2UM62QcP1705EKxHmg7BzS5VJmZI0-fpEMxegB1aMzNiEr7WSJ7pOWFG_1MwBQ
          - type: gen_len
            value: 23.0928
            name: gen_len
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZWZmZTQ0NjNkZWVlNWUwODNhZWMzMzQ2MGM5NGY2ZDY1OGUzM2JmYzRlN2JjNTczMmRlNWI0MjRhNGM5NjJmYyIsInZlcnNpb24iOjF9.9jC1XMqvVzK6bjwltfHJPswBWIwqbiguGX3onycpTSgbONtx1nsvB163sOfwzRppfGcLPC8E_lmEYqvgej7eCg

Bart model finetuned on xsum

docs: https://huggingface.co/transformers/model_doc/bart.html

finetuning: examples/seq2seq/ (as of Aug 20, 2020)

Metrics: ROUGE > 22 on xsum.

variants: search for distilbart

paper: https://arxiv.org/abs/1910.13461