Hugging Face's logo

lmqg
/
bart-large-squad-qg

Text2Text Generation
Transformers
PyTorch
English
bart
question generation
Eval Results
Inference Endpoints
Model card Files Community
5
asahi417 commited on
Commit
5b4435f
•
1 Parent(s): 838a4fe

model update

Browse files
Files changed (1) hide show
  1. README.md +116 -126
README.md CHANGED
@@ -33,60 +33,60 @@ model-index:
33
  metrics:
34
  - name: BLEU4
35
  type: bleu4
36
- value: 0.26168385362299557
37
  - name: ROUGE-L
38
  type: rouge-l
39
- value: 0.5384959163821219
40
  - name: METEOR
41
  type: meteor
42
- value: 0.27073122286541956
43
  - name: BERTScore
44
  type: bertscore
45
- value: 0.9100413219045603
46
  - name: MoverScore
47
  type: moverscore
48
- value: 0.6499011626820898
49
- - name: QAAlignedF1Score (BERTScore)
50
- type: qa_aligned_f1_score_bertscore
51
- value: 0.9553719667577645
52
- - name: QAAlignedRecall (BERTScore)
53
- type: qa_aligned_recall_bertscore
54
- value: 0.9548501701962565
55
- - name: QAAlignedPrecision (BERTScore)
56
- type: qa_aligned_precision_bertscore
57
- value: 0.9559103034487555
58
- - name: QAAlignedF1Score (MoverScore)
59
- type: qa_aligned_f1_score_moverscore
60
- value: 0.708244616864839
61
- - name: QAAlignedRecall (MoverScore)
62
- type: qa_aligned_recall_moverscore
63
- value: 0.7053901691540012
64
- - name: QAAlignedPrecision (MoverScore)
65
- type: qa_aligned_precision_moverscore
66
- value: 0.7112501965782075
67
  - task:
68
  name: Text2text Generation
69
  type: text2text-generation
70
  dataset:
71
  name: lmqg/qg_squadshifts
72
- type: reddit
73
- args: reddit
74
  metrics:
75
  - name: BLEU4
76
  type: bleu4
77
- value: 0.059525104157825456
78
  - name: ROUGE-L
79
  type: rouge-l
80
- value: 0.22365090580055863
81
  - name: METEOR
82
  type: meteor
83
- value: 0.21499800504546457
84
  - name: BERTScore
85
  type: bertscore
86
- value: 0.9095144685254328
87
  - name: MoverScore
88
  type: moverscore
89
- value: 0.6059332247878408
90
  - task:
91
  name: Text2text Generation
92
  type: text2text-generation
@@ -114,71 +114,71 @@ model-index:
114
  name: Text2text Generation
115
  type: text2text-generation
116
  dataset:
117
- name: lmqg/qg_subjqa
118
- type: tripadvisor
119
- args: tripadvisor
120
  metrics:
121
  - name: BLEU4
122
  type: bleu4
123
- value: 8.380171318718442e-07
124
  - name: ROUGE-L
125
  type: rouge-l
126
- value: 0.1402922852924756
127
  - name: METEOR
128
  type: meteor
129
- value: 0.1372146070365174
130
  - name: BERTScore
131
  type: bertscore
132
- value: 0.8891002409937424
133
  - name: MoverScore
134
  type: moverscore
135
- value: 0.5604572211470809
136
  - task:
137
  name: Text2text Generation
138
  type: text2text-generation
139
  dataset:
140
  name: lmqg/qg_squadshifts
141
- type: nyt
142
- args: nyt
143
  metrics:
144
  - name: BLEU4
145
  type: bleu4
146
- value: 0.08117757543966063
147
  - name: ROUGE-L
148
  type: rouge-l
149
- value: 0.25292097720734297
150
  - name: METEOR
151
  type: meteor
152
- value: 0.25254205113198686
153
  - name: BERTScore
154
  type: bertscore
155
- value: 0.9249009759439454
156
  - name: MoverScore
157
  type: moverscore
158
- value: 0.6406329128556304
159
  - task:
160
  name: Text2text Generation
161
  type: text2text-generation
162
  dataset:
163
  name: lmqg/qg_subjqa
164
- type: restaurants
165
- args: restaurants
166
  metrics:
167
  - name: BLEU4
168
  type: bleu4
169
- value: 1.1301750984972448e-06
170
  - name: ROUGE-L
171
  type: rouge-l
172
- value: 0.13083168975354642
173
  - name: METEOR
174
  type: meteor
175
- value: 0.12419733006916912
176
  - name: BERTScore
177
  type: bertscore
178
- value: 0.8797711839570719
179
  - name: MoverScore
180
  type: moverscore
181
- value: 0.5542757411268555
182
  - task:
183
  name: Text2text Generation
184
  type: text2text-generation
@@ -207,24 +207,24 @@ model-index:
207
  type: text2text-generation
208
  dataset:
209
  name: lmqg/qg_subjqa
210
- type: books
211
- args: books
212
  metrics:
213
  - name: BLEU4
214
  type: bleu4
215
- value: 0.006278914808207679
216
  - name: ROUGE-L
217
  type: rouge-l
218
- value: 0.12368226019088967
219
  - name: METEOR
220
  type: meteor
221
- value: 0.11576293675813865
222
  - name: BERTScore
223
  type: bertscore
224
- value: 0.8807110440044503
225
  - name: MoverScore
226
  type: moverscore
227
- value: 0.5555905941686486
228
  - task:
229
  name: Text2text Generation
230
  type: text2text-generation
@@ -253,71 +253,52 @@ model-index:
253
  type: text2text-generation
254
  dataset:
255
  name: lmqg/qg_subjqa
256
- type: grocery
257
- args: grocery
258
  metrics:
259
  - name: BLEU4
260
  type: bleu4
261
- value: 0.00528043272450429
262
  - name: ROUGE-L
263
  type: rouge-l
264
- value: 0.12343711316491492
265
  - name: METEOR
266
  type: meteor
267
- value: 0.15133496445452477
268
  - name: BERTScore
269
  type: bertscore
270
- value: 0.8778951253890991
271
  - name: MoverScore
272
  type: moverscore
273
- value: 0.5701949938103265
274
  - task:
275
  name: Text2text Generation
276
  type: text2text-generation
277
  dataset:
278
- name: lmqg/qg_squadshifts
279
- type: amazon
280
- args: amazon
281
  metrics:
282
  - name: BLEU4
283
  type: bleu4
284
- value: 0.06530369842068952
285
  - name: ROUGE-L
286
  type: rouge-l
287
- value: 0.25030985091008146
288
  - name: METEOR
289
  type: meteor
290
- value: 0.2229994442645732
291
  - name: BERTScore
292
  type: bertscore
293
- value: 0.9092814804525936
294
  - name: MoverScore
295
  type: moverscore
296
- value: 0.6086538514008419
297
  ---
298
 
299
  # Model Card of `lmqg/bart-large-squad`
300
- This model is fine-tuned version of [facebook/bart-large](https://huggingface.co/facebook/bart-large) for question generation task on the
301
- [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) (dataset_name: default) via [`lmqg`](https://github.com/asahi417/lm-question-generation).
302
-
303
 
304
- Please cite our paper if you use the model ([https://arxiv.org/abs/2210.03992](https://arxiv.org/abs/2210.03992)).
305
-
306
- ```
307
-
308
- @inproceedings{ushio-etal-2022-generative,
309
- title = "{G}enerative {L}anguage {M}odels for {P}aragraph-{L}evel {Q}uestion {G}eneration",
310
- author = "Ushio, Asahi and
311
- Alva-Manchego, Fernando and
312
- Camacho-Collados, Jose",
313
- booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing",
314
- month = dec,
315
- year = "2022",
316
- address = "Abu Dhabi, U.A.E.",
317
- publisher = "Association for Computational Linguistics",
318
- }
319
-
320
- ```
321
 
322
  ### Overview
323
  - **Language model:** [facebook/bart-large](https://huggingface.co/facebook/bart-large)
@@ -330,58 +311,68 @@ Please cite our paper if you use the model ([https://arxiv.org/abs/2210.03992](h
330
  ### Usage
331
  - With [`lmqg`](https://github.com/asahi417/lm-question-generation#lmqg-language-model-for-question-generation-)
332
  ```python
333
-
334
  from lmqg import TransformersQG
 
335
  # initialize model
336
- model = TransformersQG(language='en', model='lmqg/bart-large-squad')
 
337
  # model prediction
338
- question = model.generate_q(list_context=["William Turner was an English painter who specialised in watercolour landscapes"], list_answer=["William Turner"])
339
 
340
  ```
341
 
342
  - With `transformers`
343
  ```python
344
-
345
  from transformers import pipeline
346
- # initialize model
347
- pipe = pipeline("text2text-generation", 'lmqg/bart-large-squad')
348
- # question generation
349
- question = pipe('<hl> Beyonce <hl> further expanded her acting career, starring as blues singer Etta James in the 2008 musical biopic, Cadillac Records.')
350
 
351
  ```
352
 
353
- ## Evaluation Metrics
354
 
355
 
356
- ### Metrics
357
 
358
- | Dataset | Type | BLEU4 | ROUGE-L | METEOR | BERTScore | MoverScore | Link |
359
- |:--------|:-----|------:|--------:|-------:|----------:|-----------:|-----:|
360
- | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) | default | 0.262 | 0.538 | 0.271 | 0.91 | 0.65 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval/metric.first.sentence.paragraph_answer.question.lmqg_qg_squad.default.json) |
 
 
 
 
 
 
 
361
 
362
 
363
- ### Metrics (QAG)
364
 
365
- | Dataset | Type | QA Aligned F1 Score (BERTScore) | QA Aligned F1 Score (MoverScore) | Link |
366
- |:--------|:-----|--------------------------------:|---------------------------------:|-----:|
367
- | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) | default | 0.955 | 0.708 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval/metric.first.answer.paragraph.questions_answers.lmqg_qg_squad.default.json) |
368
-
 
 
 
 
369
 
370
 
371
- ### Out-of-domain Metrics
372
 
373
- | Dataset | Type | BLEU4 | ROUGE-L | METEOR | BERTScore | MoverScore | Link |
374
- |:--------|:-----|------:|--------:|-------:|----------:|-----------:|-----:|
375
- | [lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) | reddit | 0.06 | 0.224 | 0.215 | 0.91 | 0.606 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_squadshifts.reddit.json) |
376
- | [lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) | new_wiki | 0.111 | 0.297 | 0.273 | 0.932 | 0.662 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_squadshifts.new_wiki.json) |
377
- | [lmqg/qg_subjqa](https://huggingface.co/datasets/lmqg/qg_subjqa) | tripadvisor | 0.0 | 0.14 | 0.137 | 0.889 | 0.56 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_subjqa.tripadvisor.json) |
378
- | [lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) | nyt | 0.081 | 0.253 | 0.253 | 0.925 | 0.641 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_squadshifts.nyt.json) |
379
- | [lmqg/qg_subjqa](https://huggingface.co/datasets/lmqg/qg_subjqa) | restaurants | 0.0 | 0.131 | 0.124 | 0.88 | 0.554 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_subjqa.restaurants.json) |
380
- | [lmqg/qg_subjqa](https://huggingface.co/datasets/lmqg/qg_subjqa) | electronics | 0.009 | 0.16 | 0.153 | 0.878 | 0.563 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_subjqa.electronics.json) |
381
- | [lmqg/qg_subjqa](https://huggingface.co/datasets/lmqg/qg_subjqa) | books | 0.006 | 0.124 | 0.116 | 0.881 | 0.556 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_subjqa.books.json) |
382
- | [lmqg/qg_subjqa](https://huggingface.co/datasets/lmqg/qg_subjqa) | movies | 0.0 | 0.125 | 0.119 | 0.875 | 0.553 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_subjqa.movies.json) |
383
- | [lmqg/qg_subjqa](https://huggingface.co/datasets/lmqg/qg_subjqa) | grocery | 0.005 | 0.123 | 0.151 | 0.878 | 0.57 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_subjqa.grocery.json) |
384
- | [lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) | amazon | 0.065 | 0.25 | 0.223 | 0.909 | 0.609 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_squadshifts.amazon.json) |
385
 
386
 
387
  ## Training hyperparameters
@@ -407,7 +398,6 @@ The full configuration can be found at [fine-tuning config file](https://hugging
407
 
408
  ## Citation
409
  ```
410
-
411
  @inproceedings{ushio-etal-2022-generative,
412
  title = "{G}enerative {L}anguage {M}odels for {P}aragraph-{L}evel {Q}uestion {G}eneration",
413
  author = "Ushio, Asahi and
33
  metrics:
34
  - name: BLEU4
35
  type: bleu4
36
+ value: 26.17
37
  - name: ROUGE-L
38
  type: rouge-l
39
+ value: 53.85
40
  - name: METEOR
41
  type: meteor
42
+ value: 27.07
43
  - name: BERTScore
44
  type: bertscore
45
+ value: 91.0
46
  - name: MoverScore
47
  type: moverscore
48
+ value: 64.99
49
+ - name: QAAlignedF1Score (BERTScore) [Gold Answer]
50
+ type: qa_aligned_f1_score_bertscore_gold_answer
51
+ value: 95.54
52
+ - name: QAAlignedRecall (BERTScore) [Gold Answer]
53
+ type: qa_aligned_recall_bertscore_gold_answer
54
+ value: 95.49
55
+ - name: QAAlignedPrecision (BERTScore) [Gold Answer]
56
+ type: qa_aligned_precision_bertscore_gold_answer
57
+ value: 95.59
58
+ - name: QAAlignedF1Score (MoverScore) [Gold Answer]
59
+ type: qa_aligned_f1_score_moverscore_gold_answer
60
+ value: 70.82
61
+ - name: QAAlignedRecall (MoverScore) [Gold Answer]
62
+ type: qa_aligned_recall_moverscore_gold_answer
63
+ value: 70.54
64
+ - name: QAAlignedPrecision (MoverScore) [Gold Answer]
65
+ type: qa_aligned_precision_moverscore_gold_answer
66
+ value: 71.13
67
  - task:
68
  name: Text2text Generation
69
  type: text2text-generation
70
  dataset:
71
  name: lmqg/qg_squadshifts
72
+ type: amazon
73
+ args: amazon
74
  metrics:
75
  - name: BLEU4
76
  type: bleu4
77
+ value: 0.06530369842068952
78
  - name: ROUGE-L
79
  type: rouge-l
80
+ value: 0.25030985091008146
81
  - name: METEOR
82
  type: meteor
83
+ value: 0.2229994442645732
84
  - name: BERTScore
85
  type: bertscore
86
+ value: 0.9092814804525936
87
  - name: MoverScore
88
  type: moverscore
89
+ value: 0.6086538514008419
90
  - task:
91
  name: Text2text Generation
92
  type: text2text-generation
114
  name: Text2text Generation
115
  type: text2text-generation
116
  dataset:
117
+ name: lmqg/qg_squadshifts
118
+ type: nyt
119
+ args: nyt
120
  metrics:
121
  - name: BLEU4
122
  type: bleu4
123
+ value: 0.08117757543966063
124
  - name: ROUGE-L
125
  type: rouge-l
126
+ value: 0.25292097720734297
127
  - name: METEOR
128
  type: meteor
129
+ value: 0.25254205113198686
130
  - name: BERTScore
131
  type: bertscore
132
+ value: 0.9249009759439454
133
  - name: MoverScore
134
  type: moverscore
135
+ value: 0.6406329128556304
136
  - task:
137
  name: Text2text Generation
138
  type: text2text-generation
139
  dataset:
140
  name: lmqg/qg_squadshifts
141
+ type: reddit
142
+ args: reddit
143
  metrics:
144
  - name: BLEU4
145
  type: bleu4
146
+ value: 0.059525104157825456
147
  - name: ROUGE-L
148
  type: rouge-l
149
+ value: 0.22365090580055863
150
  - name: METEOR
151
  type: meteor
152
+ value: 0.21499800504546457
153
  - name: BERTScore
154
  type: bertscore
155
+ value: 0.9095144685254328
156
  - name: MoverScore
157
  type: moverscore
158
+ value: 0.6059332247878408
159
  - task:
160
  name: Text2text Generation
161
  type: text2text-generation
162
  dataset:
163
  name: lmqg/qg_subjqa
164
+ type: books
165
+ args: books
166
  metrics:
167
  - name: BLEU4
168
  type: bleu4
169
+ value: 0.006278914808207679
170
  - name: ROUGE-L
171
  type: rouge-l
172
+ value: 0.12368226019088967
173
  - name: METEOR
174
  type: meteor
175
+ value: 0.11576293675813865
176
  - name: BERTScore
177
  type: bertscore
178
+ value: 0.8807110440044503
179
  - name: MoverScore
180
  type: moverscore
181
+ value: 0.5555905941686486
182
  - task:
183
  name: Text2text Generation
184
  type: text2text-generation
207
  type: text2text-generation
208
  dataset:
209
  name: lmqg/qg_subjqa
210
+ type: grocery
211
+ args: grocery
212
  metrics:
213
  - name: BLEU4
214
  type: bleu4
215
+ value: 0.00528043272450429
216
  - name: ROUGE-L
217
  type: rouge-l
218
+ value: 0.12343711316491492
219
  - name: METEOR
220
  type: meteor
221
+ value: 0.15133496445452477
222
  - name: BERTScore
223
  type: bertscore
224
+ value: 0.8778951253890991
225
  - name: MoverScore
226
  type: moverscore
227
+ value: 0.5701949938103265
228
  - task:
229
  name: Text2text Generation
230
  type: text2text-generation
253
  type: text2text-generation
254
  dataset:
255
  name: lmqg/qg_subjqa
256
+ type: restaurants
257
+ args: restaurants
258
  metrics:
259
  - name: BLEU4
260
  type: bleu4
261
+ value: 1.1301750984972448e-06
262
  - name: ROUGE-L
263
  type: rouge-l
264
+ value: 0.13083168975354642
265
  - name: METEOR
266
  type: meteor
267
+ value: 0.12419733006916912
268
  - name: BERTScore
269
  type: bertscore
270
+ value: 0.8797711839570719
271
  - name: MoverScore
272
  type: moverscore
273
+ value: 0.5542757411268555
274
  - task:
275
  name: Text2text Generation
276
  type: text2text-generation
277
  dataset:
278
+ name: lmqg/qg_subjqa
279
+ type: tripadvisor
280
+ args: tripadvisor
281
  metrics:
282
  - name: BLEU4
283
  type: bleu4
284
+ value: 8.380171318718442e-07
285
  - name: ROUGE-L
286
  type: rouge-l
287
+ value: 0.1402922852924756
288
  - name: METEOR
289
  type: meteor
290
+ value: 0.1372146070365174
291
  - name: BERTScore
292
  type: bertscore
293
+ value: 0.8891002409937424
294
  - name: MoverScore
295
  type: moverscore
296
+ value: 0.5604572211470809
297
  ---
298
 
299
  # Model Card of `lmqg/bart-large-squad`
300
+ This model is fine-tuned version of [facebook/bart-large](https://huggingface.co/facebook/bart-large) for question generation task on the [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) (dataset_name: default) via [`lmqg`](https://github.com/asahi417/lm-question-generation).
 
 
301
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
302
 
303
  ### Overview
304
  - **Language model:** [facebook/bart-large](https://huggingface.co/facebook/bart-large)
311
  ### Usage
312
  - With [`lmqg`](https://github.com/asahi417/lm-question-generation#lmqg-language-model-for-question-generation-)
313
  ```python
 
314
  from lmqg import TransformersQG
315
+
316
  # initialize model
317
+ model = TransformersQG(language="en", model="lmqg/bart-large-squad")
318
+
319
  # model prediction
320
+ questions = model.generate_q(list_context="William Turner was an English painter who specialised in watercolour landscapes", list_answer="William Turner")
321
 
322
  ```
323
 
324
  - With `transformers`
325
  ```python
 
326
  from transformers import pipeline
327
+
328
+ pipe = pipeline("text2text-generation", "lmqg/bart-large-squad")
329
+ output = pipe("<hl> Beyonce <hl> further expanded her acting career, starring as blues singer Etta James in the 2008 musical biopic, Cadillac Records.")
 
330
 
331
  ```
332
 
333
+ ## Evaluation
334
 
335
 
336
+ - ***Metric (Question Generation)***: [raw metric file](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval/metric.first.sentence.paragraph_answer.question.lmqg_qg_squad.default.json)
337
 
338
+ | | Score | Type | Dataset |
339
+ |:-----------|--------:|:--------|:---------------------------------------------------------------|
340
+ | BERTScore | 91 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
341
+ | Bleu_1 | 58.79 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
342
+ | Bleu_2 | 42.79 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
343
+ | Bleu_3 | 33.11 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
344
+ | Bleu_4 | 26.17 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
345
+ | METEOR | 27.07 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
346
+ | MoverScore | 64.99 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
347
+ | ROUGE_L | 53.85 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
348
 
349
 
350
+ - ***Metric (Question & Answer Generation)***: QAG metrics are computed with *the gold answer* and generated question on it for this model, as the model cannot provide an answer. [raw metric file](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval/metric.first.answer.paragraph.questions_answers.lmqg_qg_squad.default.json)
351
 
352
+ | | Score | Type | Dataset |
353
+ |:--------------------------------|--------:|:--------|:---------------------------------------------------------------|
354
+ | QAAlignedF1Score (BERTScore) | 95.54 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
355
+ | QAAlignedF1Score (MoverScore) | 70.82 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
356
+ | QAAlignedPrecision (BERTScore) | 95.59 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
357
+ | QAAlignedPrecision (MoverScore) | 71.13 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
358
+ | QAAlignedRecall (BERTScore) | 95.49 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
359
+ | QAAlignedRecall (MoverScore) | 70.54 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
360
 
361
 
362
+ - ***Metrics (Question Generation, Out-of-Domain)***
363
 
364
+ | Dataset | Type | BERTScore| Bleu_4 | METEOR | MoverScore | ROUGE_L | Link |
365
+ |:--------|:-----|---------:|-------:|-------:|-----------:|--------:|-----:|
366
+ | [lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) | amazon | 90.93 | 6.53 | 22.3 | 60.87 | 25.03 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_squadshifts.amazon.json) |
367
+ | [lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) | new_wiki | 93.23 | 11.12 | 27.32 | 66.23 | 29.68 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_squadshifts.new_wiki.json) |
368
+ | [lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) | nyt | 92.49 | 8.12 | 25.25 | 64.06 | 25.29 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_squadshifts.nyt.json) |
369
+ | [lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) | reddit | 90.95 | 5.95 | 21.5 | 60.59 | 22.37 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_squadshifts.reddit.json) |
370
+ | [lmqg/qg_subjqa](https://huggingface.co/datasets/lmqg/qg_subjqa) | books | 88.07 | 0.63 | 11.58 | 55.56 | 12.37 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_subjqa.books.json) |
371
+ | [lmqg/qg_subjqa](https://huggingface.co/datasets/lmqg/qg_subjqa) | electronics | 87.83 | 0.87 | 15.35 | 56.35 | 16.02 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_subjqa.electronics.json) |
372
+ | [lmqg/qg_subjqa](https://huggingface.co/datasets/lmqg/qg_subjqa) | grocery | 87.79 | 0.53 | 15.13 | 57.02 | 12.34 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_subjqa.grocery.json) |
373
+ | [lmqg/qg_subjqa](https://huggingface.co/datasets/lmqg/qg_subjqa) | movies | 87.49 | 0.0 | 11.86 | 55.29 | 12.51 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_subjqa.movies.json) |
374
+ | [lmqg/qg_subjqa](https://huggingface.co/datasets/lmqg/qg_subjqa) | restaurants | 87.98 | 0.0 | 12.42 | 55.43 | 13.08 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_subjqa.restaurants.json) |
375
+ | [lmqg/qg_subjqa](https://huggingface.co/datasets/lmqg/qg_subjqa) | tripadvisor | 88.91 | 0.0 | 13.72 | 56.05 | 14.03 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_subjqa.tripadvisor.json) |
376
 
377
 
378
  ## Training hyperparameters
398
 
399
  ## Citation
400
  ```
 
401
  @inproceedings{ushio-etal-2022-generative,
402
  title = "{G}enerative {L}anguage {M}odels for {P}aragraph-{L}evel {Q}uestion {G}eneration",
403
  author = "Ushio, Asahi and