metadata

language:
  - en
license: apache-2.0
tags:
  - generated_from_trainer
  - summarization
datasets:
  - samsum
metrics:
  - rouge
widget:
  - text: >-
      Olivia: Hey Carter, are you still developing that restaurant business?
      Carter: Hi Olivia Carter: Yes, we want to launch next month :) Olivia:
      Next month? That's soon! Congrats :) Carter: thanks, I'm a bit nervous but
      I seriously believe we're delivering something innovative and needed
      Olivia: I think it's a great concept and I am sure you'll do great!
      Olivia: I am currently involved with a new restaurant in the city centre
      Carter: Which one? Olivia: Spicy and chilled Carter: I heard about it :)
      Is it any good? ;) Olivia: I love the restaurant and really like working
      there Carter: good for you! Olivia: and here's the question - are you
      still looking for restaurant to include in your discount app? Carter:
      sure, but I think it would be better to discuss it in person - would you
      like to meet up? Olivia: That would be great!
    example_title: Dialogue 1
  - text: >-
      Chad: Elton John is goat Eva: what do you mean by goat? Frank: greatest of
      all time Chad: indeed Eva: ahh... it makes sense now :P
    example_title: Dialogue 2
  - text: >-
      Astonishing X-Men is the name of four X-Men comic book series from Marvel
      Comics, the first two of which were limited series. The third volume, an
      ongoing series, began in 2004, with its first run written by Joss Whedon
      and art by John Cassaday. It was then written by Warren Ellis with art by
      Simone Bianchi and Phil Jimenez.[1] Daniel Way and Christos Gage then took
      over the title writing alternating stories. They were followed by James
      Asmus who wrote one issue, then Greg Pak, who took over for four issues in
      November 2011.[2] Marjorie Liu wrote the final 21 issues of the series
      until its end at issue #68 in 2013. The title's fourth volume and second
      ongoing series launched in 2017 during the "ResurrXion" storyline.[3] The
      first run was written by Charles Soule and illustrated by a rotating cast
      of artists. Matthew Rosenberg and artist Greg Land would then take over
      the series before its end in 2018. The original Astonishing X-Men was a
      four-issue limited series that replaced Uncanny X-Men during the 1995
      alternate universe storyline "Age of Apocalypse", in which all X-titles
      were given new names and issue numbers. In the storyline, Professor X was
      murdered 20 years in the past by his own son, Legion. Magneto, witnessing
      his friend's death, committed himself to Xavier's dream and created his
      own team of X-Men. However, he was unable to prevent the rise of the
      despotic Apocalypse and hence the series primarily dealt with the X-Men's
      battle against him. Astonishing X-Men, written by Scott Lobdell and
      illustrated by Joe Madureira, featured a team of X-Men led by Rogue and
      consisted of Sunfire, Blink, Morph, Sabretooth and Wildchild. source:
      https://en.wikipedia.org/wiki/Astonishing_X-Men
    example_title: Wikipedia Article
model-index:
  - name: flan-t5-base-samsum
    results:
      - task:
          type: text2text-generation
          name: Sequence-to-sequence Language Modeling
        dataset:
          name: samsum
          type: samsum
          config: samsum
          split: test
          args: samsum
        metrics:
          - type: rouge
            value: 46.8876
            name: Rouge1
      - task:
          type: summarization
          name: Summarization
        dataset:
          name: samsum
          type: samsum
          config: samsum
          split: test
        metrics:
          - type: rouge
            value: 47.1604
            name: ROUGE-1
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYzAzNjBhZmU3ZWE1Nzg2OGNmNWQxZTRkMWI3MGJmY2U3NzdiN2NhMzA2ZGY2N2VmOGQzNThkZDg5YmI1NTQzMCIsInZlcnNpb24iOjF9.fj5dvLTJmdTud-r9NBx468b_q7128WFc84Oa_ogUq1YuHfgK9KRBJl0V8YVP-UrVOB-5Mwcy_kVo2gqUq2fQCA
          - type: rouge
            value: 23.5947
            name: ROUGE-2
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiY2ExZTYyMDMzYjQyZWU0NjY4YWZiN2NjMjAyNzUwMzU3ZjQxOTdjZDdhNjE0MDE1NDVmY2Y5MDEyZTI5ODA5ZCIsInZlcnNpb24iOjF9.4XKnhKi4PtU0KnyXnBDRop-tWwDvAgJqbWkuPAVUPThcCjVrpjLiSgTWP49NEK-l3QBaLluoh7M-OF8OTwasBQ
          - type: rouge
            value: 39.7299
            name: ROUGE-L
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYWZiMDU1ODY4Y2ViOWJlZjhhZTAzNjY4NDhjYzdlYzg1MDRmZDM2ZDFkZGVhNjQzMmZjZDA3OWEzYjUzOTU0NCIsInZlcnNpb24iOjF9.EctQIDlK_ksR7NiCtHsxnWWzUF8WNmZ58JIsTUTjQPqmf8Igm82tihK78S4nit7IF24lug5_Ua7X5gWzMHDvDA
          - type: rouge
            value: 43.3052
            name: ROUGE-LSUM
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMzQwYTMyOGNlNzJiNDEzMjQ5NzEwMzMyZmRhZDAxOGNhMWNkZjA0YWEyM2NkZGU3ODU3ZDU4ZWFhODkyNzNkOCIsInZlcnNpb24iOjF9.nsQAnUdVTySov7ZkNYJjMbIjb7V87D1w0HFLdOzSq5gaKuZmkAXmh14c_bL4Fbyf3AV_skLCDCJZEnsJHN7mDQ
          - type: loss
            value: 1.3786224126815796
            name: loss
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNDgzMTkxY2EwOWU5MDgyODM3ZjE3MzBiN2Q0YmQ5MDI2MjI2NWNmMjUwZDY4MjZkZDg4ODcwMzVkN2Q4NTRmNSIsInZlcnNpb24iOjF9.vV700h6j3hdlzf-CEDIR3C9XND1jH3nW0r6Njgw0qB3Avfsq6zywr8ip2sxoo6aFCCQcmmcnmHiy7x1_xdwYAA
          - type: gen_len
            value: 17.3443
            name: gen_len
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNTNjYjFiYjgzNjBlMDY2MWUwZTVmY2Y1OWMwNGZkZTg0Mzc5ZmU2MWIwOWZlYWMzZGM1YWI0NTJjOTFhOTU2YiIsInZlcnNpb24iOjF9.-RshHr8uVG0B4qGh5Tr3bgqqai9R_Xho0M9iQyd5g0fyQJlYhIT12cUkcy2_NKUJEqu_JxSul723UWpiZgBHAQ

flan-t5-base-samsum

This model is a fine-tuned version of google/flan-t5-base on the samsum dataset. It achieves the following results on the evaluation set:

Loss: 1.3709
Rouge1: 46.8876
Rouge2: 23.2689
Rougel: 39.5369
Rougelsum: 43.1602
Gen Len: 17.2027

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.4403	1.0	1842	1.3829	46.5321	23.0912	39.4008	42.8993	17.0977
1.3534	2.0	3684	1.3732	47.1111	23.4456	39.5462	43.2534	17.4554
1.2795	3.0	5526	1.3709	46.8876	23.2689	39.5369	43.1602	17.2027
1.2313	4.0	7368	1.3736	47.4418	23.701	39.9856	43.6294	17.2198
1.1934	5.0	9210	1.3772	47.4656	23.9199	40.0284	43.7039	17.3162

Framework versions

Transformers 4.26.1
Pytorch 1.13.1+cu116
Datasets 2.9.0
Tokenizers 0.13.2