autoevaluator HF staff commited on
Commit
2b0367a
1 Parent(s): f4154e1

Add verifyToken field to verify evaluation results are produced by Hugging Face's automatic model evaluator

Browse files

Beep boop, I am a bot from Hugging Face's automatic model evaluator 👋! We've added a new `verifyToken` field to your evaluation results to verify that they are produced by the model evaluator. Accept this PR to ensure that your results remain listed as **verified** on the [Hub leaderboard](https://huggingface.co/spaces/autoevaluate/leaderboards).

Files changed (1) hide show
  1. README.md +182 -145
README.md CHANGED
@@ -1,13 +1,13 @@
1
  ---
 
 
 
2
  tags:
3
  - summarization
4
  - summary
5
  - booksum
6
  - long-document
7
  - long-form
8
- license:
9
- - apache-2.0
10
- - bsd-3-clause
11
  datasets:
12
  - kmfoda/booksum
13
  metrics:
@@ -26,39 +26,38 @@ widget:
26
  deviation of the average recurrence interval, the more specific could be the long
27
  term prediction of a future mainshock.
28
  example_title: earthquakes
29
- - text: " A typical feed-forward neural field algorithm. Spatiotemporal coordinates\
30
- \ are fed into a neural network that predicts values in the reconstructed domain.\
31
- \ Then, this domain is mapped to the sensor domain where sensor measurements are\
32
- \ available as supervision. Class and Section Problems Addressed Generalization\
33
- \ (Section 2) Inverse problems, ill-posed problems, editability; symmetries. Hybrid\
34
- \ Representations (Section 3) Computation & memory efficiency, representation\
35
- \ capacity, editability: Forward Maps (Section 4) Inverse problems Network Architecture\
36
- \ (Section 5) Spectral bias, integration & derivatives. Manipulating Neural Fields\
37
- \ (Section 6) Edit ability, constraints, regularization. Table 2: The five classes\
38
- \ of techniques in the neural field toolbox each addresses problems that arise\
39
- \ in learning, inference, and control. (Section 3). We can supervise reconstruction\
40
- \ via differentiable forward maps that transform Or project our domain (e.g, 3D\
41
- \ reconstruction via 2D images; Section 4) With appropriate network architecture\
42
- \ choices, we can overcome neural network spectral biases (blurriness) and efficiently\
43
- \ compute derivatives and integrals (Section 5). Finally, we can manipulate neural\
44
- \ fields to add constraints and regularizations, and to achieve editable representations\
45
- \ (Section 6). Collectively, these classes constitute a 'toolbox' of techniques\
46
- \ to help solve problems with neural fields There are three components in a conditional\
47
- \ neural field: (1) An encoder or inference function \u20AC that outputs the conditioning\
48
- \ latent variable 2 given an observation 0 E(0) =2. 2 is typically a low-dimensional\
49
- \ vector, and is often referred to aS a latent code Or feature code_ (2) A mapping\
50
- \ function 4 between Z and neural field parameters O: Y(z) = O; (3) The neural\
51
- \ field itself $. The encoder \u20AC finds the most probable z given the observations\
52
- \ O: argmaxz P(2/0). The decoder maximizes the inverse conditional probability\
53
- \ to find the most probable 0 given Z: arg- max P(Olz). We discuss different encoding\
54
- \ schemes with different optimality guarantees (Section 2.1.1), both global and\
55
- \ local conditioning (Section 2.1.2), and different mapping functions Y (Section\
56
- \ 2.1.3) 2. Generalization Suppose we wish to estimate a plausible 3D surface\
57
- \ shape given a partial or noisy point cloud. We need a suitable prior over the\
58
- \ sur- face in its reconstruction domain to generalize to the partial observations.\
59
- \ A neural network expresses a prior via the function space of its architecture\
60
- \ and parameters 0, and generalization is influenced by the inductive bias of\
61
- \ this function space (Section 5)."
62
  example_title: scientific paper
63
  - text: 'Is a else or outside the cob and tree written being of early client rope
64
  and you have is for good reasons. On to the ocean in Orange for time. By''s the
@@ -110,68 +109,82 @@ widget:
110
  the point of you of your model. This hidden data is complete by unseen. In other
111
  words, we solve our problem of validation.'
112
  example_title: transcribed audio - lecture
113
- - text: "Transformer-based models have shown to be very useful for many NLP tasks.\
114
- \ However, a major limitation of transformers-based models is its O(n^2)O(n 2)\
115
- \ time & memory complexity (where nn is sequence length). Hence, it's computationally\
116
- \ very expensive to apply transformer-based models on long sequences n > 512n>512.\
117
- \ Several recent papers, e.g. Longformer, Performer, Reformer, Clustered attention\
118
- \ try to remedy this problem by approximating the full attention matrix. You can\
119
- \ checkout \U0001F917's recent blog post in case you are unfamiliar with these\
120
- \ models.\nBigBird (introduced in paper) is one of such recent models to address\
121
- \ this issue. BigBird relies on block sparse attention instead of normal attention\
122
- \ (i.e. BERT's attention) and can handle sequences up to a length of 4096 at a\
123
- \ much lower computational cost compared to BERT. It has achieved SOTA on various\
124
- \ tasks involving very long sequences such as long documents summarization, question-answering\
125
- \ with long contexts.\nBigBird RoBERTa-like model is now available in \U0001F917\
126
- Transformers. The goal of this post is to give the reader an in-depth understanding\
127
- \ of big bird implementation & ease one's life in using BigBird with \U0001F917\
128
- Transformers. But, before going into more depth, it is important to remember that\
129
- \ the BigBird's attention is an approximation of BERT's full attention and therefore\
130
- \ does not strive to be better than BERT's full attention, but rather to be more\
131
- \ efficient. It simply allows to apply transformer-based models to much longer\
132
- \ sequences since BERT's quadratic memory requirement quickly becomes unbearable.\
133
- \ Simply put, if we would have \u221E compute & \u221E time, BERT's attention\
134
- \ would be preferred over block sparse attention (which we are going to discuss\
135
- \ in this post).\nIf you wonder why we need more compute when working with longer\
136
- \ sequences, this blog post is just right for you!\nSome of the main questions\
137
- \ one might have when working with standard BERT-like attention include:\nDo all\
138
- \ tokens really have to attend to all other tokens? Why not compute attention\
139
- \ only over important tokens? How to decide what tokens are important? How to\
140
- \ attend to just a few tokens in a very efficient way? In this blog post, we will\
141
- \ try to answer those questions.\nWhat tokens should be attended to? We will give\
142
- \ a practical example of how attention works by considering the sentence 'BigBird\
143
- \ is now available in HuggingFace for extractive question answering'. In BERT-like\
144
- \ attention, every word would simply attend to all other tokens.\nLet's think\
145
- \ about a sensible choice of key tokens that a queried token actually only should\
146
- \ attend to by writing some pseudo-code. Will will assume that the token available\
147
- \ is queried and build a sensible list of key tokens to attend to.\n>>> # let's\
148
- \ consider following sentence as an example >>> example = ['BigBird', 'is', 'now',\
149
- \ 'available', 'in', 'HuggingFace', 'for', 'extractive', 'question', 'answering']\n\
150
- >>> # further let's assume, we're trying to understand the representation of 'available'\
151
- \ i.e. >>> query_token = 'available' >>> # We will initialize an empty `set` and\
152
- \ fill up the tokens of our interest as we proceed in this section. >>> key_tokens\
153
- \ = [] # => currently 'available' token doesn't have anything to attend Nearby\
154
- \ tokens should be important because, in a sentence (sequence of words), the current\
155
- \ word is highly dependent on neighboring past & future tokens. This intuition\
156
- \ is the idea behind the concept of sliding attention."
 
 
 
 
 
 
 
 
 
 
 
 
157
  example_title: bigbird blog intro
158
- - text: "To be fair, you have to have a very high IQ to understand Rick and Morty.\
159
- \ The humour is extremely subtle, and without a solid grasp of theoretical physics\
160
- \ most of the jokes will go over a typical viewer's head. There's also Rick's\
161
- \ nihilistic outlook, which is deftly woven into his characterisation- his personal\
162
- \ philosophy draws heavily from Narodnaya Volya literature, for instance. The\
163
- \ fans understand this stuff; they have the intellectual capacity to truly appreciate\
164
- \ the depths of these jokes, to realise that they're not just funny- they say\
165
- \ something deep about LIFE. As a consequence people who dislike Rick & Morty\
166
- \ truly ARE idiots- of course they wouldn't appreciate, for instance, the humour\
167
- \ in Rick's existential catchphrase 'Wubba Lubba Dub Dub,' which itself is a cryptic\
168
- \ reference to Turgenev's Russian epic Fathers and Sons. I'm smirking right now\
169
- \ just imagining one of those addlepated simpletons scratching their heads in\
170
- \ confusion as Dan Harmon's genius wit unfolds itself on their television screens.\
171
- \ What fools.. how I pity them. \U0001F602\nAnd yes, by the way, i DO have a Rick\
172
- \ & Morty tattoo. And no, you cannot see it. It's for the ladies' eyes only- and\
173
- \ even then they have to demonstrate that they're within 5 IQ points of my own\
174
- \ (preferably lower) beforehand. Nothin personnel kid \U0001F60E"
 
 
175
  example_title: Richard & Mortimer
176
  parameters:
177
  max_length: 64
@@ -194,30 +207,36 @@ model-index:
194
  config: samsum
195
  split: test
196
  metrics:
197
- - name: ROUGE-1
198
- type: rouge
199
  value: 30.0032
 
200
  verified: true
201
- - name: ROUGE-2
202
- type: rouge
203
  value: 7.2671
 
204
  verified: true
205
- - name: ROUGE-L
206
- type: rouge
207
  value: 21.8779
 
208
  verified: true
209
- - name: ROUGE-LSUM
210
- type: rouge
211
  value: 26.4371
 
212
  verified: true
213
- - name: loss
214
- type: loss
215
  value: 2.6383285522460938
 
216
  verified: true
217
- - name: gen_len
218
- type: gen_len
219
  value: 54.2357
 
220
  verified: true
 
221
  - task:
222
  type: summarization
223
  name: Summarization
@@ -227,30 +246,36 @@ model-index:
227
  config: plain_text
228
  split: test
229
  metrics:
230
- - name: ROUGE-1
231
- type: rouge
232
  value: 37.0538
 
233
  verified: true
234
- - name: ROUGE-2
235
- type: rouge
236
  value: 8.1512
 
237
  verified: true
238
- - name: ROUGE-L
239
- type: rouge
240
  value: 17.6645
 
241
  verified: true
242
- - name: ROUGE-LSUM
243
- type: rouge
244
  value: 33.4275
 
245
  verified: true
246
- - name: loss
247
- type: loss
248
  value: 2.6052205562591553
 
249
  verified: true
250
- - name: gen_len
251
- type: gen_len
252
  value: 201.5951
 
253
  verified: true
 
254
  - task:
255
  type: summarization
256
  name: Summarization
@@ -260,30 +285,36 @@ model-index:
260
  config: kmfoda--booksum
261
  split: test
262
  metrics:
263
- - name: ROUGE-1
264
- type: rouge
265
  value: 36.1423
 
266
  verified: true
267
- - name: ROUGE-2
268
- type: rouge
269
  value: 5.634
 
270
  verified: true
271
- - name: ROUGE-L
272
- type: rouge
273
  value: 16.3747
 
274
  verified: true
275
- - name: ROUGE-LSUM
276
- type: rouge
277
  value: 33.0665
 
278
  verified: true
279
- - name: loss
280
- type: loss
281
  value: 2.454127550125122
 
282
  verified: true
283
- - name: gen_len
284
- type: gen_len
285
  value: 239.4179
 
286
  verified: true
 
287
  - task:
288
  type: summarization
289
  name: Summarization
@@ -293,30 +324,36 @@ model-index:
293
  config: y
294
  split: test
295
  metrics:
296
- - name: ROUGE-1
297
- type: rouge
298
  value: 35.615
 
299
  verified: true
300
- - name: ROUGE-2
301
- type: rouge
302
  value: 8.2625
 
303
  verified: true
304
- - name: ROUGE-L
305
- type: rouge
306
  value: 19.9883
 
307
  verified: true
308
- - name: ROUGE-LSUM
309
- type: rouge
310
  value: 30.1801
 
311
  verified: true
312
- - name: loss
313
- type: loss
314
  value: 2.8106656074523926
 
315
  verified: true
316
- - name: gen_len
317
- type: gen_len
318
  value: 170.3483
 
319
  verified: true
 
320
  ---
321
  # pszemraj/long-t5-tglobal-base-16384-booksum-V12
322
 
 
1
  ---
2
+ license:
3
+ - apache-2.0
4
+ - bsd-3-clause
5
  tags:
6
  - summarization
7
  - summary
8
  - booksum
9
  - long-document
10
  - long-form
 
 
 
11
  datasets:
12
  - kmfoda/booksum
13
  metrics:
 
26
  deviation of the average recurrence interval, the more specific could be the long
27
  term prediction of a future mainshock.
28
  example_title: earthquakes
29
+ - text: ' A typical feed-forward neural field algorithm. Spatiotemporal coordinates
30
+ are fed into a neural network that predicts values in the reconstructed domain.
31
+ Then, this domain is mapped to the sensor domain where sensor measurements are
32
+ available as supervision. Class and Section Problems Addressed Generalization
33
+ (Section 2) Inverse problems, ill-posed problems, editability; symmetries. Hybrid
34
+ Representations (Section 3) Computation & memory efficiency, representation capacity,
35
+ editability: Forward Maps (Section 4) Inverse problems Network Architecture (Section
36
+ 5) Spectral bias, integration & derivatives. Manipulating Neural Fields (Section
37
+ 6) Edit ability, constraints, regularization. Table 2: The five classes of techniques
38
+ in the neural field toolbox each addresses problems that arise in learning, inference,
39
+ and control. (Section 3). We can supervise reconstruction via differentiable forward
40
+ maps that transform Or project our domain (e.g, 3D reconstruction via 2D images;
41
+ Section 4) With appropriate network architecture choices, we can overcome neural
42
+ network spectral biases (blurriness) and efficiently compute derivatives and integrals
43
+ (Section 5). Finally, we can manipulate neural fields to add constraints and regularizations,
44
+ and to achieve editable representations (Section 6). Collectively, these classes
45
+ constitute a ''toolbox'' of techniques to help solve problems with neural fields
46
+ There are three components in a conditional neural field: (1) An encoder or inference
47
+ function that outputs the conditioning latent variable 2 given an observation
48
+ 0 E(0) =2. 2 is typically a low-dimensional vector, and is often referred to aS
49
+ a latent code Or feature code_ (2) A mapping function 4 between Z and neural field
50
+ parameters O: Y(z) = O; (3) The neural field itself $. The encoder € finds the
51
+ most probable z given the observations O: argmaxz P(2/0). The decoder maximizes
52
+ the inverse conditional probability to find the most probable 0 given Z: arg-
53
+ max P(Olz). We discuss different encoding schemes with different optimality guarantees
54
+ (Section 2.1.1), both global and local conditioning (Section 2.1.2), and different
55
+ mapping functions Y (Section 2.1.3) 2. Generalization Suppose we wish to estimate
56
+ a plausible 3D surface shape given a partial or noisy point cloud. We need a suitable
57
+ prior over the sur- face in its reconstruction domain to generalize to the partial
58
+ observations. A neural network expresses a prior via the function space of its
59
+ architecture and parameters 0, and generalization is influenced by the inductive
60
+ bias of this function space (Section 5).'
 
61
  example_title: scientific paper
62
  - text: 'Is a else or outside the cob and tree written being of early client rope
63
  and you have is for good reasons. On to the ocean in Orange for time. By''s the
 
109
  the point of you of your model. This hidden data is complete by unseen. In other
110
  words, we solve our problem of validation.'
111
  example_title: transcribed audio - lecture
112
+ - text: 'Transformer-based models have shown to be very useful for many NLP tasks.
113
+ However, a major limitation of transformers-based models is its O(n^2)O(n 2) time
114
+ & memory complexity (where nn is sequence length). Hence, it''s computationally
115
+ very expensive to apply transformer-based models on long sequences n > 512n>512.
116
+ Several recent papers, e.g. Longformer, Performer, Reformer, Clustered attention
117
+ try to remedy this problem by approximating the full attention matrix. You can
118
+ checkout 🤗''s recent blog post in case you are unfamiliar with these models.
119
+
120
+ BigBird (introduced in paper) is one of such recent models to address this issue.
121
+ BigBird relies on block sparse attention instead of normal attention (i.e. BERT''s
122
+ attention) and can handle sequences up to a length of 4096 at a much lower computational
123
+ cost compared to BERT. It has achieved SOTA on various tasks involving very long
124
+ sequences such as long documents summarization, question-answering with long contexts.
125
+
126
+ BigBird RoBERTa-like model is now available in 🤗Transformers. The goal of this
127
+ post is to give the reader an in-depth understanding of big bird implementation
128
+ & ease one''s life in using BigBird with 🤗Transformers. But, before going into
129
+ more depth, it is important to remember that the BigBird''s attention is an approximation
130
+ of BERT''s full attention and therefore does not strive to be better than BERT''s
131
+ full attention, but rather to be more efficient. It simply allows to apply transformer-based
132
+ models to much longer sequences since BERT''s quadratic memory requirement quickly
133
+ becomes unbearable. Simply put, if we would have compute & time, BERT''s attention
134
+ would be preferred over block sparse attention (which we are going to discuss
135
+ in this post).
136
+
137
+ If you wonder why we need more compute when working with longer sequences, this
138
+ blog post is just right for you!
139
+
140
+ Some of the main questions one might have when working with standard BERT-like
141
+ attention include:
142
+
143
+ Do all tokens really have to attend to all other tokens? Why not compute attention
144
+ only over important tokens? How to decide what tokens are important? How to attend
145
+ to just a few tokens in a very efficient way? In this blog post, we will try to
146
+ answer those questions.
147
+
148
+ What tokens should be attended to? We will give a practical example of how attention
149
+ works by considering the sentence ''BigBird is now available in HuggingFace for
150
+ extractive question answering''. In BERT-like attention, every word would simply
151
+ attend to all other tokens.
152
+
153
+ Let''s think about a sensible choice of key tokens that a queried token actually
154
+ only should attend to by writing some pseudo-code. Will will assume that the token
155
+ available is queried and build a sensible list of key tokens to attend to.
156
+
157
+ >>> # let''s consider following sentence as an example >>> example = [''BigBird'',
158
+ ''is'', ''now'', ''available'', ''in'', ''HuggingFace'', ''for'', ''extractive'',
159
+ ''question'', ''answering'']
160
+
161
+ >>> # further let''s assume, we''re trying to understand the representation of
162
+ ''available'' i.e. >>> query_token = ''available'' >>> # We will initialize an
163
+ empty `set` and fill up the tokens of our interest as we proceed in this section.
164
+ >>> key_tokens = [] # => currently ''available'' token doesn''t have anything
165
+ to attend Nearby tokens should be important because, in a sentence (sequence of
166
+ words), the current word is highly dependent on neighboring past & future tokens.
167
+ This intuition is the idea behind the concept of sliding attention.'
168
  example_title: bigbird blog intro
169
+ - text: 'To be fair, you have to have a very high IQ to understand Rick and Morty.
170
+ The humour is extremely subtle, and without a solid grasp of theoretical physics
171
+ most of the jokes will go over a typical viewer''s head. There''s also Rick''s
172
+ nihilistic outlook, which is deftly woven into his characterisation- his personal
173
+ philosophy draws heavily from Narodnaya Volya literature, for instance. The fans
174
+ understand this stuff; they have the intellectual capacity to truly appreciate
175
+ the depths of these jokes, to realise that they''re not just funny- they say something
176
+ deep about LIFE. As a consequence people who dislike Rick & Morty truly ARE idiots-
177
+ of course they wouldn''t appreciate, for instance, the humour in Rick''s existential
178
+ catchphrase ''Wubba Lubba Dub Dub,'' which itself is a cryptic reference to Turgenev''s
179
+ Russian epic Fathers and Sons. I''m smirking right now just imagining one of those
180
+ addlepated simpletons scratching their heads in confusion as Dan Harmon''s genius
181
+ wit unfolds itself on their television screens. What fools.. how I pity them.
182
+ 😂
183
+
184
+ And yes, by the way, i DO have a Rick & Morty tattoo. And no, you cannot see it.
185
+ It''s for the ladies'' eyes only- and even then they have to demonstrate that
186
+ they''re within 5 IQ points of my own (preferably lower) beforehand. Nothin personnel
187
+ kid 😎'
188
  example_title: Richard & Mortimer
189
  parameters:
190
  max_length: 64
 
207
  config: samsum
208
  split: test
209
  metrics:
210
+ - type: rouge
 
211
  value: 30.0032
212
+ name: ROUGE-1
213
  verified: true
214
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNjk2MTRiNDljZTM4NzliNDdmMTdkZGY3MGY4OTVmMzFhOTdjNGFjYjJhYTBjYTI4Y2VkOGMxYWI5M2M3YWEyZSIsInZlcnNpb24iOjF9.cZtcCwB1Bnnn1g4x8Ia_8oTSK89feGF80r20jwjSb-xy5Xt3eR3dOVjJyjurfN0UOGyEe7inTpneJhcAoRwwBg
215
+ - type: rouge
216
  value: 7.2671
217
+ name: ROUGE-2
218
  verified: true
219
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNThiYmJhN2NkYmU0MmZmZGY5MGU2NmEzZGQwNjM0MDEwNzlhNDgzY2E2MzkxMWVkZTUwMWFlZmFhYWEwN2M5ZSIsInZlcnNpb24iOjF9.IaaaHiOxUdh6IDGbb2vCCEcL-YhXCtaFlZnIpcgQwsC3KRgfrpQi5vdhyaaIJSieA2pzbFjUO--WqjylvpysCA
220
+ - type: rouge
221
  value: 21.8779
222
+ name: ROUGE-L
223
  verified: true
224
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNTc1N2YwODk4YmU1Mjk3NGQ2ZDVkYWVjN2Y1ZDVlOTNkMjU5MjcyYjY0ZWY5NjJkNzZjNjMwZWUxNWY0NTY1ZiIsInZlcnNpb24iOjF9.HhYA0t2Ee3YhtBDPneU7hzEEz5c4FeBcTo-3TSSClltG3A5E3RIgbxUbQNbldRAL9Y44Z8uzEHfe676eL22vBg
225
+ - type: rouge
226
  value: 26.4371
227
+ name: ROUGE-LSUM
228
  verified: true
229
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOTJmZmJhZTBiZDczYmNkNWQ0MGQ3ZTIyNzc2NGExMGY1MGNkOThlNDg0OWQ3YWFmNDRmYTUxZTYzN2U5Yzc4MCIsInZlcnNpb24iOjF9.fgr8NNlhDCvtXMudOce1pf_slujIhXAEC3a6fH6AAlgIvzxg1oGV5QiUcrPDNhyFD2XazZ39Xk1GhoMk4AnxAQ
230
+ - type: loss
231
  value: 2.6383285522460938
232
+ name: loss
233
  verified: true
234
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNjRiMjAyMjJkM2M5NGZjYzRiZGFlNTJhM2UyNjExODlmNjM4NjRmZTRlMWEzMTUzYTI2NjYzYTAyNmVlYjJjMCIsInZlcnNpb24iOjF9.wKAqpXyvHNGDpxwLmR6mzI4gRwVQI88uFJZJoRAWQD_d-H97y5cpP4VSBes_YfVpFpYzEF8miN9fv660xukiBA
235
+ - type: gen_len
236
  value: 54.2357
237
+ name: gen_len
238
  verified: true
239
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNzA1Y2IxN2Q4OGU0N2FkNDFmNTFmMjQwZDA4MTczMDJmNWIyMjdhYzhkNTE5ZjI4M2NjZTdkMmUwMTFjMzk1ZCIsInZlcnNpb24iOjF9.JuADjJNIcaqmZTw1RFnklHJYEYfTEKQ0YnmvL1TmvSihIVJORbK-3cFkJLVJdyaaRq40HjhQRw6mmpur9Lq1CQ
240
  - task:
241
  type: summarization
242
  name: Summarization
 
246
  config: plain_text
247
  split: test
248
  metrics:
249
+ - type: rouge
 
250
  value: 37.0538
251
+ name: ROUGE-1
252
  verified: true
253
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMzViY2Y2ZWIwMDdhNDEzMDU3MmE4ZTBlZjQ2MDI2YTVjOGZjZDM5NzhiZDk2MWJhZWY5MDUwY2NhZTY2OTc5ZSIsInZlcnNpb24iOjF9.p2z_oZD9uVTnBtf7vRRKvisW-rXWVibpU0QQ-S_16CIYLc2kTJRZMLzaMJqbi1d8icBTeG5PdIzKcAVwu7JKCA
254
+ - type: rouge
255
  value: 8.1512
256
+ name: ROUGE-2
257
  verified: true
258
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMWUzZGM0ZGJiMDYwM2ZmYjI5Mzk5MTU2N2JlZGVlOGRjMTJjY2QwOWIwMjgyMjM0ZjIzY2Q4MzJjNDkxZmVhMCIsInZlcnNpb24iOjF9.z6pMF8l4uMQIEcdyU1kgDc1v3rCn-0TVxntKP3hmOEwRJqfbeqDmhhAROWadYTPNewpfsCpShVHGJt9DvH55BQ
259
+ - type: rouge
260
  value: 17.6645
261
+ name: ROUGE-L
262
  verified: true
263
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZWNkYzY2NGY4YmFiNWRhODAwZmFmOTkzM2M3MGY0ZTQzZTUwNmExNDc5ZDdhZWVhZjFhYTUyYjFlZjQ3ZDA4ZCIsInZlcnNpb24iOjF9.XbVCDhR_l7OalwF2DsHJSZ39z_HHdG3PlwKL0Ls9lBvRo4E8sk00vrQy4IRCqPF8hPJusl2Nb65V3CvgIldqAA
264
+ - type: rouge
265
  value: 33.4275
266
+ name: ROUGE-LSUM
267
  verified: true
268
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNDdiYzI0MDlmYjg0MWFjZDBmMmIyZWUyNzNhYTUyNTU1ZDdhODE4ZTlmMTg5MDY1MDhhMGRlMGU1OTA3YzM4ZSIsInZlcnNpb24iOjF9.pDHKUDMXHihmLSQzYq6bxclcLyajcRf6Q5ImhpvpoepG8du5ggwb1q_2anGfDjJ0kkFa-Iwtbl8KmdqD7TTCAQ
269
+ - type: loss
270
  value: 2.6052205562591553
271
+ name: loss
272
  verified: true
273
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNjk0YWNjMjkxZjUwMDBlODNkNjE0ZWRkYzYxZmRjNjBhMmVjNTE2OWFkZTU1OTYzMzMxNzdkMGFlODVjOWVkNCIsInZlcnNpb24iOjF9.n-p8JJBe9nOsKwvS2CHO6HBiI6b-0dUZuVaL9aQgX_qFhETvwR_gHggWXU6sCiLCzkElH6ZpGpcMw9AogJWkCw
274
+ - type: gen_len
275
  value: 201.5951
276
+ name: gen_len
277
  verified: true
278
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMzMyYWViNDNjMzY2NmQyZjI5MWU2ZjMwMmYyOGFkMzM0YzgwMzg5ZDhmYzYzYzg0OTMzOWY5ZDRiM2NkNWViOSIsInZlcnNpb24iOjF9.6T6C1dimUVOHNbqm5drVZmiWVrQEC0VBc7nSAiyLm2K3WE99FisSByk4zhBtUf_CntT_TZm1dBpfTaAUVPDOAQ
279
  - task:
280
  type: summarization
281
  name: Summarization
 
285
  config: kmfoda--booksum
286
  split: test
287
  metrics:
288
+ - type: rouge
 
289
  value: 36.1423
290
+ name: ROUGE-1
291
  verified: true
292
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOTZkYTA5N2FhNjVhMzg1ZDRjOThhZjcwMjdmYzQ1MGE5N2RhNTM0MmNjMzVkYjNlYmZjOGZjMDFlZDBkMGM5MSIsInZlcnNpb24iOjF9.odQ-NMcQ06o2mqzXOfGY1c967_RUfg93YfGnMTpKUXPM5dGawkdVYGO8rPCHt5bttPvYlBmRgNl6Z7H_OhgnCA
293
+ - type: rouge
294
  value: 5.634
295
+ name: ROUGE-2
296
  verified: true
297
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZmFkODViOTg2MDYxZDhlMjZiOTNjZWE2ZTI5YmVhYWRiNGM1OTAzZDEzN2Y1ODI4OWI3NzU2ZmZlMGJjNGIyZiIsInZlcnNpb24iOjF9.4-VpnxVDiC0AG-de1dFr6VHNNbK2qZhAMQ62EpVU7Et-n25w8GPcoyr9l4AXIodQpU6p0H0pdntEUqQwJOHaDg
298
+ - type: rouge
299
  value: 16.3747
300
+ name: ROUGE-L
301
  verified: true
302
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYzkzYWY1NmEyMWNkODQ2N2ExYzMwNWExZDgwNTkxMTg5OTNjYjU5NjMwNWU3NzZhZDYwYzA4M2I0ZmU3Yjg2NiIsInZlcnNpb24iOjF9.tY2mQ0bZU9GMYYTJPot_vgvmiAoubdYWAzEQSQskigleh7AWtsXbO2CnhBsE_7UpsLPVWGccP0IWkHdHRg9zAA
303
+ - type: rouge
304
  value: 33.0665
305
+ name: ROUGE-LSUM
306
  verified: true
307
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZTEyZGZlNmRhNjllMGExZTJhOWE0NDQwN2Q3MjQyZmM5OGZjZDQwMGE4MGRiMjJmMWVmNjc2ZTQwOWFlMTdmNyIsInZlcnNpb24iOjF9.W1bgFs6XhmbeWJlX_6IvWx6MX-yUj5ErdBU1cGAAZRrEA0elBa_-FdbRkwnLDcBNmBm16vtxPAQfQgJQXmIcDA
308
+ - type: loss
309
  value: 2.454127550125122
310
+ name: loss
311
  verified: true
312
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMTQ0OGMyZGNmZjVlMDYzOTA1NjdlZjZhOThhN2M3ZTZjNWM5N2Y2MjQwZjg4Y2E4MjhiOWUzODFiMzY1YzU0NyIsInZlcnNpb24iOjF9.TOjsyBEWqDD5N9FzJPE9Z7Poj0oXefGryUy7rgj4uXbbWb8DMsMXMcxNVEKixG_vbGyFyASSmgyeW6bAFHaPCw
313
+ - type: gen_len
314
  value: 239.4179
315
+ name: gen_len
316
  verified: true
317
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMGZmOWY5NmMyNjUzZDM2NmNjNzBjMzU2OTMxYWE2MGFhM2JiMmFmNzQwOTg4NGY5Yzc1NmZjNGZmZjM5NWQzNyIsInZlcnNpb24iOjF9.piE6u39D58dKz2HimpE4Fng7cHELJPuSpZaoEU3gOXSXYw_lx2KQhi2VfFg-mUasmLuQn4bBvMJcWXyBTY8YBw
318
  - task:
319
  type: summarization
320
  name: Summarization
 
324
  config: y
325
  split: test
326
  metrics:
327
+ - type: rouge
 
328
  value: 35.615
329
+ name: ROUGE-1
330
  verified: true
331
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMWM4ZWQxMjBmNzFlYWMwODg5YTEzOWRmYzBiNmI4ZjBmNmFiZjk2NWQxNDFmY2QzNTA3ZTc5ODZkNmJkZGE4NSIsInZlcnNpb24iOjF9.MABjYbSyTQrT0QxzXM9VRpdDb5dchk1GI_TD_NSB27ozZdWEXyZ-dp44jR-M9mJTSsGk60czxmCF1gq-e4YhAQ
332
+ - type: rouge
333
  value: 8.2625
334
+ name: ROUGE-2
335
  verified: true
336
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNTk3MmI3ZmQyOTlmYzc4YTkwNjBjOTM3YmE5NjQxOGVkMDFlODc4YjgxMzlhNGRkYThkMzQ5OTU4YWFjYTg0NiIsInZlcnNpb24iOjF9.KHipwLhPWwc55GQpvNe3bSrKOgaAs4sFvLEGvzVa4HWWyvz4oX2ZaytYnURH9Xid7d9nTr7zWYYiwQ7TmSXPDA
337
+ - type: rouge
338
  value: 19.9883
339
+ name: ROUGE-L
340
  verified: true
341
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOTlhZDk5ZmEyYzgxY2IyNWI1MTk1Nzg2YmVlNmRhMjcyZmFmMWZkNGQ4OWEwYjQwYTk3YzllODdiNzRkN2M5ZCIsInZlcnNpb24iOjF9.ah1-tJ5rUuUToNUHUMf9v9_TGJdhffBMdPDthvo3fmKcFtUQFAMwIloGLp0ePcCS_h8IMEyrtpMwqcDc7jrgAw
342
+ - type: rouge
343
  value: 30.1801
344
+ name: ROUGE-LSUM
345
  verified: true
346
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMzViMzBiY2I2NWNkMjJmMmZhOTk2YzY3NTFhZTIxOTAzY2ZmNmJlYTlmZDI4YjAyYmRiNDRlNTk0MWJjMmY1MCIsInZlcnNpb24iOjF9.KUPyHMK77clPtJHyXR5WirKcy5O5hZP-MBZE-gFRy21S_sIsHpZNnBuGTJ6AMVi_38MNvDgLQWwSE-4y9eG8Dg
347
+ - type: loss
348
  value: 2.8106656074523926
349
+ name: loss
350
  verified: true
351
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYjA1ZTk2NzA5NDUwMjQ1ZDcxZTA0ZTA3YzdjYzhhZWM1ZjI3MTllYTg2YzAxOTk0Nzk1Yjc0OTRiNzIyOWExZSIsInZlcnNpb24iOjF9.q2sdYyFeFxpjGPKGpJDnoOmzTznwA1Z99GBWOHA-9YUI5q_w_kbV8JdfbiQ9GsaN8EqDlmkCL2kv5lC3xvvUAA
352
+ - type: gen_len
353
  value: 170.3483
354
+ name: gen_len
355
  verified: true
356
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiY2MxNWFjYTg1Yjc3YmNjMjViYjM5ZDdmY2NhNjFjMWQxYWQwOWI3NTczY2M5ZWVmMGM2MmQ0ZmY3M2Y0MDEwZiIsInZlcnNpb24iOjF9.J80uRlSZCVIsvyVkO8rqQ4vyZrgBMu1YpOckAzIaj_jTWKGaOPM3kj6sSePiEN8OLZYwDueqLsKkPa0B6ZXIBw
357
  ---
358
  # pszemraj/long-t5-tglobal-base-16384-booksum-V12
359