lewtun HF staff commited on
Commit
55b936f
β€’
1 Parent(s): 2a0bb57

Add evaluation results on samsum dataset

Browse files

Beep boop, I am a bot from Hugging Face's automatic model evaluator πŸ‘‹!\
Your model has been evaluated on the [samsum](https://huggingface.co/datasets/samsum) dataset by

@lewtun

, using the predictions stored [here](https://huggingface.co/datasets/autoevaluate/autoeval-staging-eval-project-6fbfec76-7855041).\
Accept this pull request to see the results displayed on the [Hub leaderboard](https://huggingface.co/spaces/autoevaluate/leaderboards?dataset=samsum).\
Evaluate your model on more datasets [here](https://huggingface.co/spaces/autoevaluate/model-evaluator?dataset=samsum).

Files changed (1) hide show
  1. README.md +58 -34
README.md CHANGED
@@ -8,48 +8,72 @@ license: apache-2.0
8
  datasets:
9
  - samsum
10
  widget:
11
- - text: |
12
- Hannah: Hey, do you have Betty's number?
13
- Amanda: Lemme check
14
- Amanda: Sorry, can't find it.
15
- Amanda: Ask Larry
16
- Amanda: He called her last time we were at the park together
17
- Hannah: I don't know him well
18
- Amanda: Don't be shy, he's very nice
19
- Hannah: If you say so..
20
- Hannah: I'd rather you texted him
21
- Amanda: Just text him πŸ™‚
22
- Hannah: Urgh.. Alright
23
- Hannah: Bye
24
- Amanda: Bye bye
25
  model-index:
26
  - name: bart-large-xsum-samsum
27
  results:
28
- - task:
29
  name: Abstractive Text Summarization
30
  type: abstractive-text-summarization
31
  dataset:
32
- name: "SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization"
33
  type: samsum
34
  metrics:
35
- - name: Validation ROGUE-1
36
- type: rogue-1
37
- value: 54.3921
38
- - name: Validation ROGUE-2
39
- type: rogue-2
40
- value: 29.8078
41
- - name: Validation ROGUE-L
42
- type: rogue-l
43
- value: 45.1543
44
- - name: Test ROGUE-1
45
- type: rogue-1
46
- value: 53.3059
47
- - name: Test ROGUE-2
48
- type: rogue-2
49
- value: 28.355
50
- - name: Test ROGUE-L
51
- type: rogue-l
52
- value: 44.0953
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
53
  ---
54
  ## `bart-large-xsum-samsum`
55
  This model was obtained by fine-tuning `facebook/bart-large-xsum` on [Samsum](https://huggingface.co/datasets/samsum) dataset.
 
8
  datasets:
9
  - samsum
10
  widget:
11
+ - text: "Hannah: Hey, do you have Betty's number?\nAmanda: Lemme check\nAmanda: Sorry,\
12
+ \ can't find it.\nAmanda: Ask Larry\nAmanda: He called her last time we were at\
13
+ \ the park together\nHannah: I don't know him well\nAmanda: Don't be shy, he's\
14
+ \ very nice\nHannah: If you say so..\nHannah: I'd rather you texted him\nAmanda:\
15
+ \ Just text him \U0001F642\nHannah: Urgh.. Alright\nHannah: Bye\nAmanda: Bye bye\n"
 
 
 
 
 
 
 
 
 
16
  model-index:
17
  - name: bart-large-xsum-samsum
18
  results:
19
+ - task:
20
  name: Abstractive Text Summarization
21
  type: abstractive-text-summarization
22
  dataset:
23
+ name: 'SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization'
24
  type: samsum
25
  metrics:
26
+ - name: Validation ROGUE-1
27
+ type: rogue-1
28
+ value: 54.3921
29
+ - name: Validation ROGUE-2
30
+ type: rogue-2
31
+ value: 29.8078
32
+ - name: Validation ROGUE-L
33
+ type: rogue-l
34
+ value: 45.1543
35
+ - name: Test ROGUE-1
36
+ type: rogue-1
37
+ value: 53.3059
38
+ - name: Test ROGUE-2
39
+ type: rogue-2
40
+ value: 28.355
41
+ - name: Test ROGUE-L
42
+ type: rogue-l
43
+ value: 44.0953
44
+ - task:
45
+ type: summarization
46
+ name: Summarization
47
+ dataset:
48
+ name: samsum
49
+ type: samsum
50
+ config: samsum
51
+ split: test
52
+ metrics:
53
+ - name: ROUGE-1
54
+ type: rouge
55
+ value: 46.4965
56
+ verified: true
57
+ - name: ROUGE-2
58
+ type: rouge
59
+ value: 20.7785
60
+ verified: true
61
+ - name: ROUGE-L
62
+ type: rouge
63
+ value: 37.378
64
+ verified: true
65
+ - name: ROUGE-LSUM
66
+ type: rouge
67
+ value: 42.0645
68
+ verified: true
69
+ - name: loss
70
+ type: loss
71
+ value: 1.6920803785324097
72
+ verified: true
73
+ - name: gen_len
74
+ type: gen_len
75
+ value: 23.6911
76
+ verified: true
77
  ---
78
  ## `bart-large-xsum-samsum`
79
  This model was obtained by fine-tuning `facebook/bart-large-xsum` on [Samsum](https://huggingface.co/datasets/samsum) dataset.