autoevaluator's picture
Add verifyToken field to verify evaluation results are produced by Hugging Face's automatic model evaluator
a2ffb34
|
raw
history blame
No virus
4.23 kB
metadata
language: en
license: apache-2.0
tags:
  - bart
  - seq2seq
  - summarization
datasets:
  - samsum
widget:
  - text: |
      Hannah: Hey, do you have Betty's number?
      Amanda: Lemme check
      Amanda: Sorry, can't find it.
      Amanda: Ask Larry
      Amanda: He called her last time we were at the park together
      Hannah: I don't know him well
      Amanda: Don't be shy, he's very nice
      Hannah: If you say so..
      Hannah: I'd rather you texted him
      Amanda: Just text him πŸ™‚
      Hannah: Urgh.. Alright
      Hannah: Bye
      Amanda: Bye bye
model-index:
  - name: bart-large-xsum-samsum
    results:
      - task:
          type: abstractive-text-summarization
          name: Abstractive Text Summarization
        dataset:
          name: >-
            SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive
            Summarization
          type: samsum
        metrics:
          - type: rouge-1
            value: 54.3921
            name: Validation ROUGE-1
          - type: rouge-2
            value: 29.8078
            name: Validation ROUGE-2
          - type: rouge-l
            value: 45.1543
            name: Validation ROUGE-L
          - type: rouge-1
            value: 53.3059
            name: Test ROUGE-1
          - type: rouge-2
            value: 28.355
            name: Test ROUGE-2
          - type: rouge-l
            value: 44.0953
            name: Test ROUGE-L
      - task:
          type: summarization
          name: Summarization
        dataset:
          name: samsum
          type: samsum
          config: samsum
          split: train
        metrics:
          - type: rouge
            value: 46.2492
            name: ROUGE-1
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZjFlZjk0MTQxMDk5ODVlNzA4MjYyNjJiMzlkOGI5MjU0MzM1ZDgxMWFlY2YyODk1Y2QxNDk2ZGZkMmU0YmYyNyIsInZlcnNpb24iOjF9.-ZraFEyEy1BY0h3frazROc1W6DmPtkb0Rvvs_A7KeWUQQlwd4felknl2dLGS3N6K-SZ89yGd6V9QJhAGeUCNDg
          - type: rouge
            value: 21.346
            name: ROUGE-2
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiODhjYjUzMGIzNTBmNzg3NGIwNzliYmUwZjM3ZTdlNjMyYzg4MjU5NDE1NjUwM2Q1MGQ0N2NiZWFkOGUwN2ExMCIsInZlcnNpb24iOjF9.SsmyHQ3u9ATihMR3lyNPGaB6bpe5xLG0pDeWJRyXtda4KUefVE3B2SpvluGTjOcF7ikKHPwNMs65IcRh9PTuDg
          - type: rouge
            value: 37.2787
            name: ROUGE-L
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiN2VhNDllNGI1ODQ2OGM1MWFmOGMwY2JhOTc0NzYxMWI4N2VhM2JkZGY3OGNhMTU2YmQ0MmMzMTk4NWM4NWJmYSIsInZlcnNpb24iOjF9.vLIlnNYOv8ObzVma5-tXPhKgB0ClcSBzRzn2qKep_YWMYkWCLk-AbPZKLTimmuJvzfv7naVXLtJomZlVAzQ_Ag
          - type: rouge
            value: 42.1317
            name: ROUGE-LSUM
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMTRiNTEzZTFjYTljZmY4N2VhYWQ1MDNhNGViZDg3OTVhOTg5NDFlMmY1NzNmNWMwNTk3MTEwODY5NjQwYjVlMSIsInZlcnNpb24iOjF9.25YRNHi8K1JXnSUwmNs0VNcsFmjhFTMK9_FglOWYcs-_UeW44SURYyQlvdTSYJl0f4fBdf6TYe2nTEWJH0_oBA
          - type: loss
            value: 1.6859958171844482
            name: loss
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMjlkMDQ0NjYyMjBiYTVmMmUzMDQ4NzZmMjczZTdhNzY4NWI5Mzk0ZTc2NTM4NjE2ZjAzMDI4MWJkYTUyYjVjMyIsInZlcnNpb24iOjF9.ks0DwFnTsSu05CEJ6Wlm-41yVFyWdXzzAJlURdxjExPziPCWGXGEMVdZ07Nc4ANsKjlUD508Qyb3c_a-fIjEBA
          - type: gen_len
            value: 23.7103
            name: gen_len
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMDg3YTNhNWJkZDZmN2Q2ZmY4MjhmZjNjYWViNDY5ODZiNTVhNjY1NTk2YzI1NjQ4ZDFjYzZkZDYxZmMwMGU5MCIsInZlcnNpb24iOjF9._unExkTq29yj3ZBx1XzLs38T-k294vGaq5bsTnpPDTx3mR6h1JN-hBepuRJUBdIr5jIsTsPfsMh_xlrQ3JzuAA

bart-large-xsum-samsum

This model was obtained by fine-tuning facebook/bart-large-xsum on Samsum dataset.

Usage

from transformers import pipeline
summarizer = pipeline("summarization", model="knkarthick/bart-large-xsum-samsum")
conversation = '''Hannah: Hey, do you have Betty's number?
Amanda: Lemme check
Amanda: Sorry, can't find it.
Amanda: Ask Larry
Amanda: He called her last time we were at the park together
Hannah: I don't know him well
Amanda: Don't be shy, he's very nice
Hannah: If you say so..
Hannah: I'd rather you texted him
Amanda: Just text him πŸ™‚
Hannah: Urgh.. Alright
Hannah: Bye
Amanda: Bye bye                                       
'''
summarizer(conversation)