autoevaluator HF staff commited on
Commit
65e5f44
1 Parent(s): 9a6c586

Add verifyToken field to verify evaluation results are produced by Hugging Face's automatic model evaluator

Browse files

Beep boop, I am a bot from Hugging Face's automatic model evaluator 👋! We've added a new `verifyToken` field to your evaluation results to verify that they are produced by the model evaluator. Accept this PR to ensure that your results remain listed as **verified** on the [Hub leaderboard](https://huggingface.co/spaces/autoevaluate/leaderboards).

Files changed (1) hide show
  1. README.md +200 -153
README.md CHANGED
@@ -1,4 +1,7 @@
1
  ---
 
 
 
2
  tags:
3
  - summarization
4
  - led
@@ -7,9 +10,6 @@ tags:
7
  - booksum
8
  - long-document
9
  - long-form
10
- license:
11
- - apache-2.0
12
- - bsd-3-clause
13
  datasets:
14
  - kmfoda/booksum
15
  metrics:
@@ -28,39 +28,38 @@ widget:
28
  deviation of the average recurrence interval, the more specific could be the long
29
  term prediction of a future mainshock.
30
  example_title: earthquakes
31
- - text: " A typical feed-forward neural field algorithm. Spatiotemporal coordinates\
32
- \ are fed into a neural network that predicts values in the reconstructed domain.\
33
- \ Then, this domain is mapped to the sensor domain where sensor measurements are\
34
- \ available as supervision. Class and Section Problems Addressed Generalization\
35
- \ (Section 2) Inverse problems, ill-posed problems, editability; symmetries. Hybrid\
36
- \ Representations (Section 3) Computation & memory efficiency, representation\
37
- \ capacity, editability: Forward Maps (Section 4) Inverse problems Network Architecture\
38
- \ (Section 5) Spectral bias, integration & derivatives. Manipulating Neural Fields\
39
- \ (Section 6) Edit ability, constraints, regularization. Table 2: The five classes\
40
- \ of techniques in the neural field toolbox each addresses problems that arise\
41
- \ in learning, inference, and control. (Section 3). We can supervise reconstruction\
42
- \ via differentiable forward maps that transform Or project our domain (e.g, 3D\
43
- \ reconstruction via 2D images; Section 4) With appropriate network architecture\
44
- \ choices, we can overcome neural network spectral biases (blurriness) and efficiently\
45
- \ compute derivatives and integrals (Section 5). Finally, we can manipulate neural\
46
- \ fields to add constraints and regularizations, and to achieve editable representations\
47
- \ (Section 6). Collectively, these classes constitute a 'toolbox' of techniques\
48
- \ to help solve problems with neural fields There are three components in a conditional\
49
- \ neural field: (1) An encoder or inference function \u20AC that outputs the conditioning\
50
- \ latent variable 2 given an observation 0 E(0) =2. 2 is typically a low-dimensional\
51
- \ vector, and is often referred to aS a latent code Or feature code_ (2) A mapping\
52
- \ function 4 between Z and neural field parameters O: Y(z) = O; (3) The neural\
53
- \ field itself $. The encoder \u20AC finds the most probable z given the observations\
54
- \ O: argmaxz P(2/0). The decoder maximizes the inverse conditional probability\
55
- \ to find the most probable 0 given Z: arg- max P(Olz). We discuss different encoding\
56
- \ schemes with different optimality guarantees (Section 2.1.1), both global and\
57
- \ local conditioning (Section 2.1.2), and different mapping functions Y (Section\
58
- \ 2.1.3) 2. Generalization Suppose we wish to estimate a plausible 3D surface\
59
- \ shape given a partial or noisy point cloud. We need a suitable prior over the\
60
- \ sur- face in its reconstruction domain to generalize to the partial observations.\
61
- \ A neural network expresses a prior via the function space of its architecture\
62
- \ and parameters 0, and generalization is influenced by the inductive bias of\
63
- \ this function space (Section 5)."
64
  example_title: scientific paper
65
  - text: ' the big variety of data coming from diverse sources is one of the key properties
66
  of the big data phenomenon. It is, therefore, beneficial to understand how data
@@ -105,50 +104,62 @@ widget:
105
  in their business An important area of data analytics on the edge of corporate
106
  IT and the Internet is Web Analytics.'
107
  example_title: data science textbook
108
- - text: "Transformer-based models have shown to be very useful for many NLP tasks.\
109
- \ However, a major limitation of transformers-based models is its O(n^2)O(n 2)\
110
- \ time & memory complexity (where nn is sequence length). Hence, it's computationally\
111
- \ very expensive to apply transformer-based models on long sequences n > 512n>512.\
112
- \ Several recent papers, e.g. Longformer, Performer, Reformer, Clustered attention\
113
- \ try to remedy this problem by approximating the full attention matrix. You can\
114
- \ checkout \U0001F917's recent blog post in case you are unfamiliar with these\
115
- \ models.\nBigBird (introduced in paper) is one of such recent models to address\
116
- \ this issue. BigBird relies on block sparse attention instead of normal attention\
117
- \ (i.e. BERT's attention) and can handle sequences up to a length of 4096 at a\
118
- \ much lower computational cost compared to BERT. It has achieved SOTA on various\
119
- \ tasks involving very long sequences such as long documents summarization, question-answering\
120
- \ with long contexts.\nBigBird RoBERTa-like model is now available in \U0001F917\
121
- Transformers. The goal of this post is to give the reader an in-depth understanding\
122
- \ of big bird implementation & ease one's life in using BigBird with \U0001F917\
123
- Transformers. But, before going into more depth, it is important to remember that\
124
- \ the BigBird's attention is an approximation of BERT's full attention and therefore\
125
- \ does not strive to be better than BERT's full attention, but rather to be more\
126
- \ efficient. It simply allows to apply transformer-based models to much longer\
127
- \ sequences since BERT's quadratic memory requirement quickly becomes unbearable.\
128
- \ Simply put, if we would have \u221E compute & \u221E time, BERT's attention\
129
- \ would be preferred over block sparse attention (which we are going to discuss\
130
- \ in this post).\nIf you wonder why we need more compute when working with longer\
131
- \ sequences, this blog post is just right for you!\nSome of the main questions\
132
- \ one might have when working with standard BERT-like attention include:\nDo all\
133
- \ tokens really have to attend to all other tokens? Why not compute attention\
134
- \ only over important tokens? How to decide what tokens are important? How to\
135
- \ attend to just a few tokens in a very efficient way? In this blog post, we will\
136
- \ try to answer those questions.\nWhat tokens should be attended to? We will give\
137
- \ a practical example of how attention works by considering the sentence 'BigBird\
138
- \ is now available in HuggingFace for extractive question answering'. In BERT-like\
139
- \ attention, every word would simply attend to all other tokens.\nLet's think\
140
- \ about a sensible choice of key tokens that a queried token actually only should\
141
- \ attend to by writing some pseudo-code. Will will assume that the token available\
142
- \ is queried and build a sensible list of key tokens to attend to.\n>>> # let's\
143
- \ consider following sentence as an example >>> example = ['BigBird', 'is', 'now',\
144
- \ 'available', 'in', 'HuggingFace', 'for', 'extractive', 'question', 'answering']\n\
145
- >>> # further let's assume, we're trying to understand the representation of 'available'\
146
- \ i.e. >>> query_token = 'available' >>> # We will initialize an empty `set` and\
147
- \ fill up the tokens of our interest as we proceed in this section. >>> key_tokens\
148
- \ = [] # => currently 'available' token doesn't have anything to attend Nearby\
149
- \ tokens should be important because, in a sentence (sequence of words), the current\
150
- \ word is highly dependent on neighboring past & future tokens. This intuition\
151
- \ is the idea behind the concept of sliding attention."
 
 
 
 
 
 
 
 
 
 
 
 
152
  example_title: bigbird blog intro
153
  - text: 'The majority of available text summarization datasets include short-form
154
  source documents that lack long-range causal and temporal dependencies, and often
@@ -186,30 +197,36 @@ model-index:
186
  config: kmfoda--booksum
187
  split: test
188
  metrics:
189
- - name: ROUGE-1
190
- type: rouge
191
  value: 33.4536
 
192
  verified: true
193
- - name: ROUGE-2
194
- type: rouge
195
  value: 5.2232
 
196
  verified: true
197
- - name: ROUGE-L
198
- type: rouge
199
  value: 16.2044
 
200
  verified: true
201
- - name: ROUGE-LSUM
202
- type: rouge
203
  value: 29.9765
 
204
  verified: true
205
- - name: loss
206
- type: loss
207
  value: 3.1985862255096436
 
208
  verified: true
209
- - name: gen_len
210
- type: gen_len
211
  value: 191.9783
 
212
  verified: true
 
213
  - task:
214
  type: summarization
215
  name: Summarization
@@ -219,30 +236,36 @@ model-index:
219
  config: samsum
220
  split: test
221
  metrics:
222
- - name: ROUGE-1
223
- type: rouge
224
- value: 32.0
225
  verified: true
226
- - name: ROUGE-2
227
- type: rouge
228
  value: 10.0781
 
229
  verified: true
230
- - name: ROUGE-L
231
- type: rouge
232
  value: 23.6331
 
233
  verified: true
234
- - name: ROUGE-LSUM
235
- type: rouge
236
  value: 28.7831
 
237
  verified: true
238
- - name: loss
239
- type: loss
240
  value: 2.903024673461914
 
241
  verified: true
242
- - name: gen_len
243
- type: gen_len
244
  value: 60.7411
 
245
  verified: true
 
246
  - task:
247
  type: summarization
248
  name: Summarization
@@ -252,30 +275,36 @@ model-index:
252
  config: 3.0.0
253
  split: test
254
  metrics:
255
- - name: ROUGE-1
256
- type: rouge
257
  value: 30.5046
 
258
  verified: true
259
- - name: ROUGE-2
260
- type: rouge
261
  value: 13.2577
 
262
  verified: true
263
- - name: ROUGE-L
264
- type: rouge
265
  value: 19.0306
 
266
  verified: true
267
- - name: ROUGE-LSUM
268
- type: rouge
269
  value: 28.3421
 
270
  verified: true
271
- - name: loss
272
- type: loss
273
  value: 3.9484164714813232
 
274
  verified: true
275
- - name: gen_len
276
- type: gen_len
277
  value: 231.0762
 
278
  verified: true
 
279
  - task:
280
  type: summarization
281
  name: Summarization
@@ -285,30 +314,36 @@ model-index:
285
  config: default
286
  split: test
287
  metrics:
288
- - name: ROUGE-1
289
- type: rouge
290
  value: 36.8502
 
291
  verified: true
292
- - name: ROUGE-2
293
- type: rouge
294
  value: 15.9147
 
295
  verified: true
296
- - name: ROUGE-L
297
- type: rouge
298
  value: 23.4762
 
299
  verified: true
300
- - name: ROUGE-LSUM
301
- type: rouge
302
  value: 30.9597
 
303
  verified: true
304
- - name: loss
305
- type: loss
306
  value: 3.878790855407715
 
307
  verified: true
308
- - name: gen_len
309
- type: gen_len
310
  value: 131.3622
 
311
  verified: true
 
312
  - task:
313
  type: summarization
314
  name: Summarization
@@ -318,30 +353,36 @@ model-index:
318
  config: y
319
  split: test
320
  metrics:
321
- - name: ROUGE-1
322
- type: rouge
323
  value: 33.7585
 
324
  verified: true
325
- - name: ROUGE-2
326
- type: rouge
327
  value: 9.4101
 
328
  verified: true
329
- - name: ROUGE-L
330
- type: rouge
331
  value: 18.8927
 
332
  verified: true
333
- - name: ROUGE-LSUM
334
- type: rouge
335
  value: 28.5051
 
336
  verified: true
337
- - name: loss
338
- type: loss
339
  value: 5.162865161895752
 
340
  verified: true
341
- - name: gen_len
342
- type: gen_len
343
  value: 222.6626
 
344
  verified: true
 
345
  - task:
346
  type: summarization
347
  name: Summarization
@@ -351,30 +392,36 @@ model-index:
351
  config: default
352
  split: test
353
  metrics:
354
- - name: ROUGE-1
355
- type: rouge
356
  value: 38.7332
 
357
  verified: true
358
- - name: ROUGE-2
359
- type: rouge
360
  value: 11.0072
 
361
  verified: true
362
- - name: ROUGE-L
363
- type: rouge
364
  value: 18.6018
 
365
  verified: true
366
- - name: ROUGE-LSUM
367
- type: rouge
368
  value: 34.5911
 
369
  verified: true
370
- - name: loss
371
- type: loss
372
  value: 3.5744354724884033
 
373
  verified: true
374
- - name: gen_len
375
- type: gen_len
376
  value: 192.0014
 
377
  verified: true
 
378
  ---
379
 
380
  # Longformer Encoder-Decoder (LED) for Narrative-Esque Long Text Summarization
 
1
  ---
2
+ license:
3
+ - apache-2.0
4
+ - bsd-3-clause
5
  tags:
6
  - summarization
7
  - led
 
10
  - booksum
11
  - long-document
12
  - long-form
 
 
 
13
  datasets:
14
  - kmfoda/booksum
15
  metrics:
 
28
  deviation of the average recurrence interval, the more specific could be the long
29
  term prediction of a future mainshock.
30
  example_title: earthquakes
31
+ - text: ' A typical feed-forward neural field algorithm. Spatiotemporal coordinates
32
+ are fed into a neural network that predicts values in the reconstructed domain.
33
+ Then, this domain is mapped to the sensor domain where sensor measurements are
34
+ available as supervision. Class and Section Problems Addressed Generalization
35
+ (Section 2) Inverse problems, ill-posed problems, editability; symmetries. Hybrid
36
+ Representations (Section 3) Computation & memory efficiency, representation capacity,
37
+ editability: Forward Maps (Section 4) Inverse problems Network Architecture (Section
38
+ 5) Spectral bias, integration & derivatives. Manipulating Neural Fields (Section
39
+ 6) Edit ability, constraints, regularization. Table 2: The five classes of techniques
40
+ in the neural field toolbox each addresses problems that arise in learning, inference,
41
+ and control. (Section 3). We can supervise reconstruction via differentiable forward
42
+ maps that transform Or project our domain (e.g, 3D reconstruction via 2D images;
43
+ Section 4) With appropriate network architecture choices, we can overcome neural
44
+ network spectral biases (blurriness) and efficiently compute derivatives and integrals
45
+ (Section 5). Finally, we can manipulate neural fields to add constraints and regularizations,
46
+ and to achieve editable representations (Section 6). Collectively, these classes
47
+ constitute a ''toolbox'' of techniques to help solve problems with neural fields
48
+ There are three components in a conditional neural field: (1) An encoder or inference
49
+ function that outputs the conditioning latent variable 2 given an observation
50
+ 0 E(0) =2. 2 is typically a low-dimensional vector, and is often referred to aS
51
+ a latent code Or feature code_ (2) A mapping function 4 between Z and neural field
52
+ parameters O: Y(z) = O; (3) The neural field itself $. The encoder € finds the
53
+ most probable z given the observations O: argmaxz P(2/0). The decoder maximizes
54
+ the inverse conditional probability to find the most probable 0 given Z: arg-
55
+ max P(Olz). We discuss different encoding schemes with different optimality guarantees
56
+ (Section 2.1.1), both global and local conditioning (Section 2.1.2), and different
57
+ mapping functions Y (Section 2.1.3) 2. Generalization Suppose we wish to estimate
58
+ a plausible 3D surface shape given a partial or noisy point cloud. We need a suitable
59
+ prior over the sur- face in its reconstruction domain to generalize to the partial
60
+ observations. A neural network expresses a prior via the function space of its
61
+ architecture and parameters 0, and generalization is influenced by the inductive
62
+ bias of this function space (Section 5).'
 
63
  example_title: scientific paper
64
  - text: ' the big variety of data coming from diverse sources is one of the key properties
65
  of the big data phenomenon. It is, therefore, beneficial to understand how data
 
104
  in their business An important area of data analytics on the edge of corporate
105
  IT and the Internet is Web Analytics.'
106
  example_title: data science textbook
107
+ - text: 'Transformer-based models have shown to be very useful for many NLP tasks.
108
+ However, a major limitation of transformers-based models is its O(n^2)O(n 2) time
109
+ & memory complexity (where nn is sequence length). Hence, it''s computationally
110
+ very expensive to apply transformer-based models on long sequences n > 512n>512.
111
+ Several recent papers, e.g. Longformer, Performer, Reformer, Clustered attention
112
+ try to remedy this problem by approximating the full attention matrix. You can
113
+ checkout 🤗''s recent blog post in case you are unfamiliar with these models.
114
+
115
+ BigBird (introduced in paper) is one of such recent models to address this issue.
116
+ BigBird relies on block sparse attention instead of normal attention (i.e. BERT''s
117
+ attention) and can handle sequences up to a length of 4096 at a much lower computational
118
+ cost compared to BERT. It has achieved SOTA on various tasks involving very long
119
+ sequences such as long documents summarization, question-answering with long contexts.
120
+
121
+ BigBird RoBERTa-like model is now available in 🤗Transformers. The goal of this
122
+ post is to give the reader an in-depth understanding of big bird implementation
123
+ & ease one''s life in using BigBird with 🤗Transformers. But, before going into
124
+ more depth, it is important to remember that the BigBird''s attention is an approximation
125
+ of BERT''s full attention and therefore does not strive to be better than BERT''s
126
+ full attention, but rather to be more efficient. It simply allows to apply transformer-based
127
+ models to much longer sequences since BERT''s quadratic memory requirement quickly
128
+ becomes unbearable. Simply put, if we would have compute & time, BERT''s attention
129
+ would be preferred over block sparse attention (which we are going to discuss
130
+ in this post).
131
+
132
+ If you wonder why we need more compute when working with longer sequences, this
133
+ blog post is just right for you!
134
+
135
+ Some of the main questions one might have when working with standard BERT-like
136
+ attention include:
137
+
138
+ Do all tokens really have to attend to all other tokens? Why not compute attention
139
+ only over important tokens? How to decide what tokens are important? How to attend
140
+ to just a few tokens in a very efficient way? In this blog post, we will try to
141
+ answer those questions.
142
+
143
+ What tokens should be attended to? We will give a practical example of how attention
144
+ works by considering the sentence ''BigBird is now available in HuggingFace for
145
+ extractive question answering''. In BERT-like attention, every word would simply
146
+ attend to all other tokens.
147
+
148
+ Let''s think about a sensible choice of key tokens that a queried token actually
149
+ only should attend to by writing some pseudo-code. Will will assume that the token
150
+ available is queried and build a sensible list of key tokens to attend to.
151
+
152
+ >>> # let''s consider following sentence as an example >>> example = [''BigBird'',
153
+ ''is'', ''now'', ''available'', ''in'', ''HuggingFace'', ''for'', ''extractive'',
154
+ ''question'', ''answering'']
155
+
156
+ >>> # further let''s assume, we''re trying to understand the representation of
157
+ ''available'' i.e. >>> query_token = ''available'' >>> # We will initialize an
158
+ empty `set` and fill up the tokens of our interest as we proceed in this section.
159
+ >>> key_tokens = [] # => currently ''available'' token doesn''t have anything
160
+ to attend Nearby tokens should be important because, in a sentence (sequence of
161
+ words), the current word is highly dependent on neighboring past & future tokens.
162
+ This intuition is the idea behind the concept of sliding attention.'
163
  example_title: bigbird blog intro
164
  - text: 'The majority of available text summarization datasets include short-form
165
  source documents that lack long-range causal and temporal dependencies, and often
 
197
  config: kmfoda--booksum
198
  split: test
199
  metrics:
200
+ - type: rouge
 
201
  value: 33.4536
202
+ name: ROUGE-1
203
  verified: true
204
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYmEzYjNkZTUxZjA0YTdmNTJkMjVkMTg2NDRjNTkzN2ZlNDlhNTBhMWQ5MTNiYWE4Mzg5YTMyMTM5YmZjNDI3OSIsInZlcnNpb24iOjF9.OWjM_HCQLQHK4AV4em70QGT3lrVk25WyZdcXA8ywest_XSx9KehJbsIMDKtXxOOMwxvkogKnScy4tbskYMQqDg
205
+ - type: rouge
206
  value: 5.2232
207
+ name: ROUGE-2
208
  verified: true
209
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOTVhOTdjZjc5YTdhMmVjZGE1NTA5MmJkYmM3Y2U3OGVlMjZmOGVlMTUzYTdiZGRhM2NmZjAzMjFkZjlkMzJmOCIsInZlcnNpb24iOjF9.qOlwWEe8dfBunmwImhbkcxzUW3ml-ESsuxjWN1fjn_o36zaUlDqlrXovMcL9GX9mVdvZDhx9W82rAR8h6410AQ
210
+ - type: rouge
211
  value: 16.2044
212
+ name: ROUGE-L
213
  verified: true
214
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNzkwOTEwYjkxYzlhMWE4ZjhlZDVjZWEwMWY2YzgwY2Q2YzJkYWFhMTQ4ODFlZmVkY2I1OWVhMTFmZThlOGY4NCIsInZlcnNpb24iOjF9.fJSr9wRQ07YIPMpb2_xv14EkHRz3gsPdZH-4LzpdviLOjVhlK1Y4gSZjp3PTEbu4Hua0umvNTMrhii8hp3DFBA
215
+ - type: rouge
216
  value: 29.9765
217
+ name: ROUGE-LSUM
218
  verified: true
219
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYWRkYjcwMTYwODRjN2E4MDliZWQyNjczNDU1NGZkMDRkNDlhNDA1YzZiOTk1MWJjZDkyMDg3MGMxYmVhOTA5MyIsInZlcnNpb24iOjF9.tUkVmhT0bl9eY_BzAzdzEI1lo3Iyfv6HBrrsVsRHqPFh4C0Q9Zk3IXbR-F_gMDx9vDiZIkpfG7SfsIZXwhDkBw
220
+ - type: loss
221
  value: 3.1985862255096436
222
+ name: loss
223
  verified: true
224
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiM2RmYzQ1NTFiYjk3YTZjMTI3NDJlMDY0MTgyZDZlZDRmZDcwOWE1YjU0OGYyZTJlY2RkZTEzZDFlNDk2ZjgyNSIsInZlcnNpb24iOjF9.Pc5Tfu8IXYeB5ETK2JMIL4gpRIvvYXVS6w1AZdfq9dD1dm9Te2xaNhzGBHviqgEfFI9APNSJB28wna1OpYP0Dg
225
+ - type: gen_len
226
  value: 191.9783
227
+ name: gen_len
228
  verified: true
229
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNmMyMDI5MzFlNzNjODNmOWQ0ZTM3MzVkNTNkYzIxNTIwZDQzMTU2MTM0YjYzNjJiMGRhOTQ0OWFhN2U4N2NjYyIsInZlcnNpb24iOjF9.AfsX-O1YwfbPxUwAD7rd1Ub7SXth7FFpTo2iNSOUWFhYmDUECkf6qtJ5pVHXXZwnpidAlfPTPg-5y3dx_BBGCA
230
  - task:
231
  type: summarization
232
  name: Summarization
 
236
  config: samsum
237
  split: test
238
  metrics:
239
+ - type: rouge
240
+ value: 32
241
+ name: ROUGE-1
242
  verified: true
243
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYmNhZjk3NjFlZDBhZjU2YzgzOTdhZTNkZjBkYjNjZDk2YjE2NDBmMDhiY2Y5M2EwNGI5Njk1NWU3ZDYyMzk2ZSIsInZlcnNpb24iOjF9.htkMQQLjIeFFjnpAJOwwxAdgzGZX10Und6RONubeeydXqQqb562EHqAw0K1ZlqltC4GBGKK3xslGOWXQ5AV6CA
244
+ - type: rouge
245
  value: 10.0781
246
+ name: ROUGE-2
247
  verified: true
248
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMWYzZDA1YmU5YTkzMjEwN2IzMTNhZmZmOTU2ZGUyNzdlNWQ0OGQ1Y2UxOGQ0NWUyOWVmZmZkYzFkODE3OTliNiIsInZlcnNpb24iOjF9.WVE3fmYLkOW32_neYYj4TNJ5lhrG-27DnoJd4YDUzpHYvGWGoFU9CUuIFraQFnojRr02f3KqVY7T33DG5mpzBg
249
+ - type: rouge
250
  value: 23.6331
251
+ name: ROUGE-L
252
  verified: true
253
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYTYyOTE0ODY2Mjk0YTk5ZTY5NTZkM2JkOGZhNjQ3NjNiMjVhNTc4ZmMwYzg1ZGIxOTA2MDQxNmU3Yjc5YWY0MSIsInZlcnNpb24iOjF9.yQ8WpdsyGKSuTG8MxHXqujEAYOIrt_hoUbuHc8HnS-GjS9xJ-rKO6pP6HYbi0LC9Xqh2_QPveCpNqr9ZQMGRCg
254
+ - type: rouge
255
  value: 28.7831
256
+ name: ROUGE-LSUM
257
  verified: true
258
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMzVkMDNlODA4NWI3OGI1OGFlNjFlNWE4YzY5ZDE1NDdhMjIwYjlkNDIxNDZjOGRiNTI1MGJkMmE0YWZiMDNhMiIsInZlcnNpb24iOjF9.qoxn2g70rbbX6sVCvm_cXzvYZf1UdTDU44vvEVdZL-4h36cJRCOx5--O1tZEVdyvlMVi-tYz1RSxLRwQd72FAw
259
+ - type: loss
260
  value: 2.903024673461914
261
+ name: loss
262
  verified: true
263
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZGM2M2NlY2Q3NjYxY2EyM2FkYmM5OGVhYzcyNjA3ZTFlYzc3M2M2ODNmNWVjNjZmMGNiODc4MWY5NWE2ZDMyNyIsInZlcnNpb24iOjF9.pC4UK75LbyVFFm0-fcStMtdQhbuHE37wkZHoVbSQOYSyxjI8yA46bQkPmgg5znby9FK_wIgGxC_4KOdEeN4jBw
264
+ - type: gen_len
265
  value: 60.7411
266
+ name: gen_len
267
  verified: true
268
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZWEwMDFiYjgyNzRhZDVmOWIzYzZlZWU5OTFkYmU4YzI2Mjk2OTg1ZDVlNzU0YzNhOWI1MmU2NTAxZWUzZmFlOCIsInZlcnNpb24iOjF9.Zepow4AFj1sQ6zyJGoy_Dl4ICKRtzZI2nVYWlTsDnGrBDT42ak9mFUuw-BjHR8dEVHJKmOZlLk6GJ09bL7tGAA
269
  - task:
270
  type: summarization
271
  name: Summarization
 
275
  config: 3.0.0
276
  split: test
277
  metrics:
278
+ - type: rouge
 
279
  value: 30.5046
280
+ name: ROUGE-1
281
  verified: true
282
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYmVmN2NkZDE1ZDAzZjhiYWJkNWJjZDIwNGNkY2EzOTVlNzA3OTZlYmEyNDE5NzQwNmI4NTdmM2M3YzlmMGJiYSIsInZlcnNpb24iOjF9.UbgnlgTUEd2yhULHeNKHQaVtAYwE3CijYGZc5mZSZkwXGIwJxwkDimhyo6XxMr8iCsu_hQLEsEtN9CWTn0SrDw
283
+ - type: rouge
284
  value: 13.2577
285
+ name: ROUGE-2
286
  verified: true
287
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYzU5MTg1OGJkYzAwNmUwMDdmYTBhODBhYjkyOTdjODRjMDJiNjU0ZjkzYzYyYWJhZTA1YjQ2NTliNWUyYmY3YiIsInZlcnNpb24iOjF9.fuvr3vrY8CSYpSluLeMz9VnxysWSlFFRMnnR3ZKZOxlh7_UNwtlMMHWCH6Yfy65LzglLNsRSnWNrwn5OXP4vAw
288
+ - type: rouge
289
  value: 19.0306
290
+ name: ROUGE-L
291
  verified: true
292
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMzMxYmY1ZmRlYzA4NzNkZDAzZjc0MzhkY2E4YTRiMmI0M2IwNDIwNzdiOWIzYTM1YTBmNWJkOWE3ODA3ZDM5NyIsInZlcnNpb24iOjF9.y-vzjHeER3iqyvSrjHUvy6Z_hom6aV0SRNV5CiB2efPmS7cL9nifoqpF2MJtip9RVn5nuuavlm-e3e2K0S5yDw
293
+ - type: rouge
294
  value: 28.3421
295
+ name: ROUGE-LSUM
296
  verified: true
297
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNGJjYTdhMDIxYWVhYzM0MDRkYTM4MTI4YmRhOGEwYjE5OGU1NWNiYjIzOTdkM2FlNTUxNTJiNzBmNWQ1NDgyOSIsInZlcnNpb24iOjF9.32seuR1CHAtUR_UCCx1nTiv-u88ETqypzWt5iItexmFTlVkZjPw7whgM7KXtgJsPdWfdcClYif5Qpnbq-NycDA
298
+ - type: loss
299
  value: 3.9484164714813232
300
+ name: loss
301
  verified: true
302
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYTgyMzJhNjY1OTU4YmVmMzVlYmI1N2IzZTdiNzYwMTA4YzRlZjY1ZjRhN2IxNTE5NjhkYjA1ZmMzMzVhNDk5NyIsInZlcnNpb24iOjF9.Cn8b20pksnlZF8LaJPxKrmrPMDIJ4CAPfjGifB86RaA6pLSTyY_wYsqEb2JfAczViquk4HtV8MvLnv0cioLODQ
303
+ - type: gen_len
304
  value: 231.0762
305
+ name: gen_len
306
  verified: true
307
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYTIxOTMyZDlhNjYwOTY5M2Q0ZTZiNWQ5YzAxMjE1OTllOGNhNWU4MjQ0OTBhZTE5NDZjNmEzZTZlOWRiMGY5NyIsInZlcnNpb24iOjF9.QK29Zwhj8jN3hWae54-qaF7vHdh1ijSa6Mq_4LdGcG8xHLCerVGg45H9B1XClCksMadp7auOzPa8CEjxYVpyBA
308
  - task:
309
  type: summarization
310
  name: Summarization
 
314
  config: default
315
  split: test
316
  metrics:
317
+ - type: rouge
 
318
  value: 36.8502
319
+ name: ROUGE-1
320
  verified: true
321
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYmE2ZjI4YmJkZGVjZDkzNzU5ZmI2MDYzNGZkNjE2OGM0Y2Y0Nzk1NTc1ZmUyZmFhYjIwY2RhMDVkMzQ1MWIxYyIsInZlcnNpb24iOjF9.SZjhhFkKwvRrI-Yl29psn17u1RCISsmmLVXxo2kxCjkhtMOma-EzC5YidjPDGQLb-J2nvqUworaC2pL_oeHxDQ
322
+ - type: rouge
323
  value: 15.9147
324
+ name: ROUGE-2
325
  verified: true
326
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiODgwOTJhOWIyZDQ4ZDA5YWMzYTJkZWFmMzlkNWYxNTg5OGFiNzY0MTExNTgyMTdlMTQ1N2EwYWY4OGZkNWY5YyIsInZlcnNpb24iOjF9.DS-X3eA1tGhVSuUL8uSPtJMNijODF3ugaKEtBglmPqF1OQZwIwQs-NExNYP4d6Y4Pa9d-DujD5yfyl9C8HBGCw
327
+ - type: rouge
328
  value: 23.4762
329
+ name: ROUGE-L
330
  verified: true
331
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYTYxNTA4YzhmYTQ0YmRjMWU5ZDliZWFhMjM4ZmUyNGUyOWJhNzA1MDBhZDliYmYyYzY3NjBmZTZlYWY3YTY3ZCIsInZlcnNpb24iOjF9.o0W7dqdz0sqMPKtJbXSRpyVNsREEUypW-bGv7TW5lfJFkijfDKhVITEClFLWu5n2tIV-sXAYxgQHDf5_hpY-Dw
332
+ - type: rouge
333
  value: 30.9597
334
+ name: ROUGE-LSUM
335
  verified: true
336
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNzEzOGNiYjk4NDkxNTFmMjA5YjM1YTQzZTk2N2JiZDgxNzAxYzFlYjliZjA3NmRjMzZlNGYyODBkNTI1NzVjNiIsInZlcnNpb24iOjF9.C_hobTR0ZY958oUZcGEKj2RoPOkyfMCTznwi4mUx-bfGRRAecMyn45bWVwwRq12glk1vThDetCjOMHA6jgSDCw
337
+ - type: loss
338
  value: 3.878790855407715
339
+ name: loss
340
  verified: true
341
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNmYyOWM0YWQ0MjAxZDg5ZWQyNDk3MGUwNzdkOWIwZDc0OGJjYTU3YjZmOWY0YTljNDI0OWRlNTI0ZDMwZWEzOCIsInZlcnNpb24iOjF9.P01Jzfa-5jyMeoEqEsEluKOydNmtRtNy8YhwfJuYHVJTVDzCIfzY8b7iNfqTfKFKwKkZ4eTwmA6vmsPZeASDAw
342
+ - type: gen_len
343
  value: 131.3622
344
+ name: gen_len
345
  verified: true
346
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYmJjN2Q5ZGNlZjQ2ODJiYTZlMzZmNWVmMzRlMGQ0ZTkxZWM3ZDQ4ZmQ1NmUyZjY4MTVhZGE5NDFiZTBhNDZiYSIsInZlcnNpb24iOjF9.DqYNc0ZCX_EqRi4zbSBAtb-js_JBHSWZkeGR9gSwEkJletKYFxPGZWd-B1ez88aj6PO775-qHd98xx3IWCHECQ
347
  - task:
348
  type: summarization
349
  name: Summarization
 
353
  config: y
354
  split: test
355
  metrics:
356
+ - type: rouge
 
357
  value: 33.7585
358
+ name: ROUGE-1
359
  verified: true
360
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiM2VmMGU5YWJlZWFlNjA3MDY2NTBmZWU3YWQxYTk3OGYzZmU5NmFmMTQ1NTVmNDQyZTJkNDMwY2E5NGRjMGU3MSIsInZlcnNpb24iOjF9.P6Rt9c3Xi_B-u8B1ug4paeZDoAO4ErGeNM0gELHGeOMj4XMjeSvyAW_-30cA9Wf23-0jGPOSZbN5pME4JpxfDA
361
+ - type: rouge
362
  value: 9.4101
363
+ name: ROUGE-2
364
  verified: true
365
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNDA0NzUxMjIwYTFjNGQ5YTA4YjE1NGU5YWMzYjhiOTk2NWE3ZGQxNDY4YTI3ZmI0ODBjYmJkZjcwYTM2OTg2MCIsInZlcnNpb24iOjF9.23hd2SuLoX3_Rygj2ykcSQccPeFsf4yLDAgvS189jx6JNln0MVR6YI2-3Yzo5g8LJk0MCbgkOp0my-nf7nMaDw
366
+ - type: rouge
367
  value: 18.8927
368
+ name: ROUGE-L
369
  verified: true
370
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiODhhMGZiZWFlNmZkYmYxZjJmODE1NWRiZjI2OGU1MTc4MDkyYjk1Mzk5ODFkYWVhY2ExNTViYjJmYzkzNWJhYiIsInZlcnNpb24iOjF9.SkKhf-l2cl2KcuC17oPrBtkBlZJaj2ujCgzRlfZy76rU9JtlW7N9bcy1ugnw-vRVUVVR6wUK08T45YorfuxqBg
371
+ - type: rouge
372
  value: 28.5051
373
+ name: ROUGE-LSUM
374
  verified: true
375
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMTgzYzA0NmQ0OTZmNzJkNGZiNTdmMzFmOTljMWE3YzM0NDg2MDY1ZDY5ZTE4MmQ5YzU1ZDFiNmE2ZjkwMjRjMiIsInZlcnNpb24iOjF9.p1TQINRxMatNe77_BMnusSg1K5FOD9f1_N4TBJDjJHNhYnyQDE4pKHfK8j6fsHGg58DHVQjmm8g96SK4uMF6DA
376
+ - type: loss
377
  value: 5.162865161895752
378
+ name: loss
379
  verified: true
380
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZWM1YTQ4MjVmMDkyZDI3OWJmODhmOWE2MDYyMDA4OGRmYzhiY2YzZjVmMTZkMTI4NjBlY2MwMDY3ZDE5ZjlmMyIsInZlcnNpb24iOjF9.Czh4TOG-QIqyc_-GJ3wc1TLuxc-KLwPelV5tiwEjNhZFyUZkjLH__ccOxBk9TYy2vunvh2AwdY3Mt6Fr8LhaDA
381
+ - type: gen_len
382
  value: 222.6626
383
+ name: gen_len
384
  verified: true
385
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiY2JjNzVkODhmOWQ5NWMwNDdlNzhkYjE5NjY3NTgwNWVmZDZlMzc4NDdmZjdlN2M2ODBkZGU5NGU0ZjMzM2Q5OCIsInZlcnNpb24iOjF9.z4hZ-uXg8PPn-THRHFrsWZpS3jgE8URk5yoLenwWtev5toTrZ2Y-DP8O30nPnzMkzA4yzo_NUKIACxoUdMqfCQ
386
  - task:
387
  type: summarization
388
  name: Summarization
 
392
  config: default
393
  split: test
394
  metrics:
395
+ - type: rouge
 
396
  value: 38.7332
397
+ name: ROUGE-1
398
  verified: true
399
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMGViMThhNTdlZDRiMTg5NTZjNGVmOThiMjI5NDEyZDMxYjU4MTU2ZTliZjZmMzAzMmRhNDIxYjViYjZmNWYwNSIsInZlcnNpb24iOjF9.SK_1Q9WlkNhu3mfsyir1l72pddjURZvJV3mcJ4jhBxS2k2q1NAR8JT_iT8v1thLiv8NUDmDr2o9Dig4A8svDBw
400
+ - type: rouge
401
  value: 11.0072
402
+ name: ROUGE-2
403
  verified: true
404
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMzkzMDU1ZGZlOWUwOGQyY2UwMWFjZTY1MDBmNzcyZGYzZTliNGVkNDZjZDVjZjA4NmE3OWVhMGIyZmE3NGE0NSIsInZlcnNpb24iOjF9.j0wvR0NPw0lqxW3ASbmBvxAbFHGikXw-Y7FjutojhzTfSs3BIs5Z8s5_h6eesvSGT5fS_qUrbnl9EEBwjrXqDg
405
+ - type: rouge
406
  value: 18.6018
407
+ name: ROUGE-L
408
  verified: true
409
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMjIwNTUzN2ZhZjU5OGFhYzRmZmEwY2NkZWVjYmYzZjRjMGIxNzNjZDY5YzIyMTg2NDJkMGYxYmViNTcwOTc5NCIsInZlcnNpb24iOjF9.rD_tFYRyb-o6VX7Z52fULvP_HQjqqshqnvbjAxWjuCM9hCn1J6oh0zAASPw0k1lWiURbiMCiaxIHxe_5BN_rAQ
410
+ - type: rouge
411
  value: 34.5911
412
+ name: ROUGE-LSUM
413
  verified: true
414
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiY2Q4MWY3NGFhNjE5YjE5NzIyODVhNTYxNWFmZDE5NjNiZTM1M2M3ZmIwNTZiOWEyMTc2MzQ0MWQ5YTdjYThlNyIsInZlcnNpb24iOjF9.R789HgYsv_k6OrjocVi0ywx0aCRlgOKpEWUiSUDca-AfoDS8ADJBtLYoEKg1wnRlR9yWoD4vtEWdKbyOOln1CA
415
+ - type: loss
416
  value: 3.5744354724884033
417
+ name: loss
418
  verified: true
419
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMzBjZTk0YWMwMzQxNDRlY2UxZDc4NTE1MmEzNDkwM2M3ZGZhNGMzNmI4ZDU2ZTVhZDkwMjNhYTkxZTIwN2E4MyIsInZlcnNpb24iOjF9.bDQ_3-CumosWKroMwBEMwKnDAj4ENQbUnbS387hU0zAY1K5g1NOy7fKBohxYZnRVolEfiuhszifUMW9zcLjqCA
420
+ - type: gen_len
421
  value: 192.0014
422
+ name: gen_len
423
  verified: true
424
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNDQxZmEwYmU5MGI1ZWE5NTIyMmM1MTVlMjVjNTg4MDQyMjJhNGE5NDJhNmZiN2Y4ZDc4ZmExNjBkMjQzMjQxMyIsInZlcnNpb24iOjF9.o3WblPY-iL1vT66xPwyyi1VMPhI53qs9GJ5HsHGbglOALwZT4n2-6IRxRNcL2lLj9qUehWUKkhruUyDM5-4RBg
425
  ---
426
 
427
  # Longformer Encoder-Decoder (LED) for Narrative-Esque Long Text Summarization