lewtun HF staff commited on
Commit
d9180b5
•
1 Parent(s): 5d4c0d8

Add evaluation results on samsum dataset

Browse files

Beep boop, I am a bot from Hugging Face's automatic model evaluator 👋!\
Your model has been evaluated on the [samsum](https://huggingface.co/datasets/samsum) dataset by

@lewtun

, using the predictions stored [here](https://huggingface.co/datasets/autoevaluate/autoeval-staging-eval-project-6fbfec76-7855042).\
Accept this pull request to see the results displayed on the [Hub leaderboard](https://huggingface.co/spaces/autoevaluate/leaderboards?dataset=samsum).\
Evaluate your model on more datasets [here](https://huggingface.co/spaces/autoevaluate/model-evaluator?dataset=samsum).

Files changed (1) hide show
  1. README.md +57 -27
README.md CHANGED
@@ -8,41 +8,71 @@ license: apache-2.0
8
  datasets:
9
  - samsum
10
  widget:
11
- - text: |
12
- Jeff: Can I train a 🤗 Transformers model on Amazon SageMaker?
13
- Philipp: Sure you can use the new Hugging Face Deep Learning Container.
14
- Jeff: ok.
15
- Jeff: and how can I get started?
16
- Jeff: where can I find documentation?
17
- Philipp: ok, ok you can find everything here. https://huggingface.co/blog/the-partnership-amazon-sagemaker-and-hugging-face
18
  model-index:
19
  - name: bart-base-samsum
20
  results:
21
- - task:
22
  name: Abstractive Text Summarization
23
  type: abstractive-text-summarization
24
  dataset:
25
- name: "SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization"
26
  type: samsum
27
  metrics:
28
- - name: Validation ROGUE-1
29
- type: rogue-1
30
- value: 46.6619
31
- - name: Validation ROGUE-2
32
- type: rogue-2
33
- value: 23.3285
34
- - name: Validation ROGUE-L
35
- type: rogue-l
36
- value: 39.4811
37
- - name: Test ROGUE-1
38
- type: rogue-1
39
- value: 44.9932
40
- - name: Test ROGUE-2
41
- type: rogue-2
42
- value: 21.7286
43
- - name: Test ROGUE-L
44
- type: rogue-l
45
- value: 38.1921
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
46
  ---
47
  ## `bart-base-samsum`
48
  This model was obtained by fine-tuning `facebook/bart-base` on Samsum dataset.
 
8
  datasets:
9
  - samsum
10
  widget:
11
+ - text: "Jeff: Can I train a \U0001F917 Transformers model on Amazon SageMaker? \n\
12
+ Philipp: Sure you can use the new Hugging Face Deep Learning Container. \nJeff:\
13
+ \ ok.\nJeff: and how can I get started? \nJeff: where can I find documentation?\
14
+ \ \nPhilipp: ok, ok you can find everything here. https://huggingface.co/blog/the-partnership-amazon-sagemaker-and-hugging-face\n"
 
 
 
15
  model-index:
16
  - name: bart-base-samsum
17
  results:
18
+ - task:
19
  name: Abstractive Text Summarization
20
  type: abstractive-text-summarization
21
  dataset:
22
+ name: 'SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization'
23
  type: samsum
24
  metrics:
25
+ - name: Validation ROGUE-1
26
+ type: rogue-1
27
+ value: 46.6619
28
+ - name: Validation ROGUE-2
29
+ type: rogue-2
30
+ value: 23.3285
31
+ - name: Validation ROGUE-L
32
+ type: rogue-l
33
+ value: 39.4811
34
+ - name: Test ROGUE-1
35
+ type: rogue-1
36
+ value: 44.9932
37
+ - name: Test ROGUE-2
38
+ type: rogue-2
39
+ value: 21.7286
40
+ - name: Test ROGUE-L
41
+ type: rogue-l
42
+ value: 38.1921
43
+ - task:
44
+ type: summarization
45
+ name: Summarization
46
+ dataset:
47
+ name: samsum
48
+ type: samsum
49
+ config: samsum
50
+ split: test
51
+ metrics:
52
+ - name: ROUGE-1
53
+ type: rouge
54
+ value: 45.0148
55
+ verified: true
56
+ - name: ROUGE-2
57
+ type: rouge
58
+ value: 21.6861
59
+ verified: true
60
+ - name: ROUGE-L
61
+ type: rouge
62
+ value: 38.1728
63
+ verified: true
64
+ - name: ROUGE-LSUM
65
+ type: rouge
66
+ value: 41.2794
67
+ verified: true
68
+ - name: loss
69
+ type: loss
70
+ value: 1.597476601600647
71
+ verified: true
72
+ - name: gen_len
73
+ type: gen_len
74
+ value: 17.6606
75
+ verified: true
76
  ---
77
  ## `bart-base-samsum`
78
  This model was obtained by fine-tuning `facebook/bart-base` on Samsum dataset.