Commit
•
45d200e
1
Parent(s):
6284a4a
Add evaluation results on the samsum config and test split of samsum
Browse filesBeep boop, I am a bot from Hugging Face's automatic model evaluator 👋!\
Your model has been evaluated on the samsum config and test split of the [samsum](https://huggingface.co/datasets/samsum) dataset by
@achimoraites
, using the predictions stored [here](https://huggingface.co/datasets/autoevaluate/autoeval-eval-samsum-samsum-e62894-41756145079).\
Accept this pull request to see the results displayed on the [Hub leaderboard](https://huggingface.co/spaces/autoevaluate/leaderboards?dataset=samsum).\
Evaluate your model on more datasets [here](https://huggingface.co/spaces/autoevaluate/model-evaluator?dataset=samsum).
README.md
CHANGED
@@ -1,4 +1,6 @@
|
|
1 |
---
|
|
|
|
|
2 |
license: apache-2.0
|
3 |
tags:
|
4 |
- generated_from_trainer
|
@@ -8,21 +10,49 @@ datasets:
|
|
8 |
metrics:
|
9 |
- rouge
|
10 |
widget:
|
11 |
-
- text:
|
12 |
-
|
13 |
-
|
14 |
-
|
15 |
-
|
16 |
-
|
17 |
-
|
18 |
-
|
19 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
20 |
model-index:
|
21 |
- name: flan-t5-base-samsum
|
22 |
results:
|
23 |
- task:
|
24 |
-
name: Sequence-to-sequence Language Modeling
|
25 |
type: text2text-generation
|
|
|
26 |
dataset:
|
27 |
name: samsum
|
28 |
type: samsum
|
@@ -30,11 +60,48 @@ model-index:
|
|
30 |
split: test
|
31 |
args: samsum
|
32 |
metrics:
|
33 |
-
-
|
34 |
-
type: rouge
|
35 |
value: 46.8876
|
36 |
-
|
37 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
38 |
---
|
39 |
|
40 |
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
|
|
1 |
---
|
2 |
+
language:
|
3 |
+
- en
|
4 |
license: apache-2.0
|
5 |
tags:
|
6 |
- generated_from_trainer
|
|
|
10 |
metrics:
|
11 |
- rouge
|
12 |
widget:
|
13 |
+
- text: 'Olivia: Hey Carter, are you still developing that restaurant business? Carter:
|
14 |
+
Hi Olivia Carter: Yes, we want to launch next month :) Olivia: Next month? That''s
|
15 |
+
soon! Congrats :) Carter: thanks, I''m a bit nervous but I seriously believe we''re
|
16 |
+
delivering something innovative and needed Olivia: I think it''s a great concept
|
17 |
+
and I am sure you''ll do great! Olivia: I am currently involved with a new restaurant
|
18 |
+
in the city centre Carter: Which one? Olivia: Spicy and chilled Carter: I heard
|
19 |
+
about it :) Is it any good? ;) Olivia: I love the restaurant and really like working
|
20 |
+
there Carter: good for you! Olivia: and here''s the question - are you still looking
|
21 |
+
for restaurant to include in your discount app? Carter: sure, but I think it would
|
22 |
+
be better to discuss it in person - would you like to meet up? Olivia: That would
|
23 |
+
be great!'
|
24 |
+
example_title: Dialogue 1
|
25 |
+
- text: 'Chad: Elton John is goat Eva: what do you mean by goat? Frank: greatest of
|
26 |
+
all time Chad: indeed Eva: ahh... it makes sense now :P'
|
27 |
+
example_title: Dialogue 2
|
28 |
+
- text: 'Astonishing X-Men is the name of four X-Men comic book series from Marvel
|
29 |
+
Comics, the first two of which were limited series. The third volume, an ongoing
|
30 |
+
series, began in 2004, with its first run written by Joss Whedon and art by John
|
31 |
+
Cassaday. It was then written by Warren Ellis with art by Simone Bianchi and Phil
|
32 |
+
Jimenez.[1] Daniel Way and Christos Gage then took over the title writing alternating
|
33 |
+
stories. They were followed by James Asmus who wrote one issue, then Greg Pak,
|
34 |
+
who took over for four issues in November 2011.[2] Marjorie Liu wrote the final
|
35 |
+
21 issues of the series until its end at issue #68 in 2013. The title''s fourth
|
36 |
+
volume and second ongoing series launched in 2017 during the "ResurrXion" storyline.[3]
|
37 |
+
The first run was written by Charles Soule and illustrated by a rotating cast
|
38 |
+
of artists. Matthew Rosenberg and artist Greg Land would then take over the series
|
39 |
+
before its end in 2018. The original Astonishing X-Men was a four-issue limited
|
40 |
+
series that replaced Uncanny X-Men during the 1995 alternate universe storyline
|
41 |
+
"Age of Apocalypse", in which all X-titles were given new names and issue numbers.
|
42 |
+
In the storyline, Professor X was murdered 20 years in the past by his own son,
|
43 |
+
Legion. Magneto, witnessing his friend''s death, committed himself to Xavier''s
|
44 |
+
dream and created his own team of X-Men. However, he was unable to prevent the
|
45 |
+
rise of the despotic Apocalypse and hence the series primarily dealt with the
|
46 |
+
X-Men''s battle against him. Astonishing X-Men, written by Scott Lobdell and illustrated
|
47 |
+
by Joe Madureira, featured a team of X-Men led by Rogue and consisted of Sunfire,
|
48 |
+
Blink, Morph, Sabretooth and Wildchild. source: https://en.wikipedia.org/wiki/Astonishing_X-Men'
|
49 |
+
example_title: Wikipedia Article
|
50 |
model-index:
|
51 |
- name: flan-t5-base-samsum
|
52 |
results:
|
53 |
- task:
|
|
|
54 |
type: text2text-generation
|
55 |
+
name: Sequence-to-sequence Language Modeling
|
56 |
dataset:
|
57 |
name: samsum
|
58 |
type: samsum
|
|
|
60 |
split: test
|
61 |
args: samsum
|
62 |
metrics:
|
63 |
+
- type: rouge
|
|
|
64 |
value: 46.8876
|
65 |
+
name: Rouge1
|
66 |
+
- task:
|
67 |
+
type: summarization
|
68 |
+
name: Summarization
|
69 |
+
dataset:
|
70 |
+
name: samsum
|
71 |
+
type: samsum
|
72 |
+
config: samsum
|
73 |
+
split: test
|
74 |
+
metrics:
|
75 |
+
- type: rouge
|
76 |
+
value: 47.1604
|
77 |
+
name: ROUGE-1
|
78 |
+
verified: true
|
79 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYzAzNjBhZmU3ZWE1Nzg2OGNmNWQxZTRkMWI3MGJmY2U3NzdiN2NhMzA2ZGY2N2VmOGQzNThkZDg5YmI1NTQzMCIsInZlcnNpb24iOjF9.fj5dvLTJmdTud-r9NBx468b_q7128WFc84Oa_ogUq1YuHfgK9KRBJl0V8YVP-UrVOB-5Mwcy_kVo2gqUq2fQCA
|
80 |
+
- type: rouge
|
81 |
+
value: 23.5947
|
82 |
+
name: ROUGE-2
|
83 |
+
verified: true
|
84 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiY2ExZTYyMDMzYjQyZWU0NjY4YWZiN2NjMjAyNzUwMzU3ZjQxOTdjZDdhNjE0MDE1NDVmY2Y5MDEyZTI5ODA5ZCIsInZlcnNpb24iOjF9.4XKnhKi4PtU0KnyXnBDRop-tWwDvAgJqbWkuPAVUPThcCjVrpjLiSgTWP49NEK-l3QBaLluoh7M-OF8OTwasBQ
|
85 |
+
- type: rouge
|
86 |
+
value: 39.7299
|
87 |
+
name: ROUGE-L
|
88 |
+
verified: true
|
89 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYWZiMDU1ODY4Y2ViOWJlZjhhZTAzNjY4NDhjYzdlYzg1MDRmZDM2ZDFkZGVhNjQzMmZjZDA3OWEzYjUzOTU0NCIsInZlcnNpb24iOjF9.EctQIDlK_ksR7NiCtHsxnWWzUF8WNmZ58JIsTUTjQPqmf8Igm82tihK78S4nit7IF24lug5_Ua7X5gWzMHDvDA
|
90 |
+
- type: rouge
|
91 |
+
value: 43.3052
|
92 |
+
name: ROUGE-LSUM
|
93 |
+
verified: true
|
94 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMzQwYTMyOGNlNzJiNDEzMjQ5NzEwMzMyZmRhZDAxOGNhMWNkZjA0YWEyM2NkZGU3ODU3ZDU4ZWFhODkyNzNkOCIsInZlcnNpb24iOjF9.nsQAnUdVTySov7ZkNYJjMbIjb7V87D1w0HFLdOzSq5gaKuZmkAXmh14c_bL4Fbyf3AV_skLCDCJZEnsJHN7mDQ
|
95 |
+
- type: loss
|
96 |
+
value: 1.3786224126815796
|
97 |
+
name: loss
|
98 |
+
verified: true
|
99 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNDgzMTkxY2EwOWU5MDgyODM3ZjE3MzBiN2Q0YmQ5MDI2MjI2NWNmMjUwZDY4MjZkZDg4ODcwMzVkN2Q4NTRmNSIsInZlcnNpb24iOjF9.vV700h6j3hdlzf-CEDIR3C9XND1jH3nW0r6Njgw0qB3Avfsq6zywr8ip2sxoo6aFCCQcmmcnmHiy7x1_xdwYAA
|
100 |
+
- type: gen_len
|
101 |
+
value: 17.3443
|
102 |
+
name: gen_len
|
103 |
+
verified: true
|
104 |
+
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNTNjYjFiYjgzNjBlMDY2MWUwZTVmY2Y1OWMwNGZkZTg0Mzc5ZmU2MWIwOWZlYWMzZGM1YWI0NTJjOTFhOTU2YiIsInZlcnNpb24iOjF9.-RshHr8uVG0B4qGh5Tr3bgqqai9R_Xho0M9iQyd5g0fyQJlYhIT12cUkcy2_NKUJEqu_JxSul723UWpiZgBHAQ
|
105 |
---
|
106 |
|
107 |
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|