asahi417 commited on
Commit
53c61a7
1 Parent(s): 225d4e3

model update

Browse files
Files changed (1) hide show
  1. README.md +17 -17
README.md CHANGED
@@ -21,7 +21,7 @@ widget:
21
  - text: "Beyonce further expanded her acting career, starring as blues singer Etta James in the 2008 musical biopic, <hl> Cadillac Records <hl> ."
22
  example_title: "Question Generation Example 3"
23
  model-index:
24
- - name: lmqg/bart-large-squad
25
  results:
26
  - task:
27
  name: Text2text Generation
@@ -296,7 +296,7 @@ model-index:
296
  value: 0.5604572211470809
297
  ---
298
 
299
- # Model Card of `lmqg/bart-large-squad`
300
  This model is fine-tuned version of [facebook/bart-large](https://huggingface.co/facebook/bart-large) for question generation task on the [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) (dataset_name: default) via [`lmqg`](https://github.com/asahi417/lm-question-generation).
301
 
302
 
@@ -314,7 +314,7 @@ This model is fine-tuned version of [facebook/bart-large](https://huggingface.co
314
  from lmqg import TransformersQG
315
 
316
  # initialize model
317
- model = TransformersQG(language="en", model="lmqg/bart-large-squad")
318
 
319
  # model prediction
320
  questions = model.generate_q(list_context="William Turner was an English painter who specialised in watercolour landscapes", list_answer="William Turner")
@@ -325,7 +325,7 @@ questions = model.generate_q(list_context="William Turner was an English painter
325
  ```python
326
  from transformers import pipeline
327
 
328
- pipe = pipeline("text2text-generation", "lmqg/bart-large-squad")
329
  output = pipe("<hl> Beyonce <hl> further expanded her acting career, starring as blues singer Etta James in the 2008 musical biopic, Cadillac Records.")
330
 
331
  ```
@@ -333,7 +333,7 @@ output = pipe("<hl> Beyonce <hl> further expanded her acting career, starring as
333
  ## Evaluation
334
 
335
 
336
- - ***Metric (Question Generation)***: [raw metric file](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval/metric.first.sentence.paragraph_answer.question.lmqg_qg_squad.default.json)
337
 
338
  | | Score | Type | Dataset |
339
  |:-----------|--------:|:--------|:---------------------------------------------------------------|
@@ -347,7 +347,7 @@ output = pipe("<hl> Beyonce <hl> further expanded her acting career, starring as
347
  | ROUGE_L | 53.85 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
348
 
349
 
350
- - ***Metric (Question & Answer Generation)***: QAG metrics are computed with *the gold answer* and generated question on it for this model, as the model cannot provide an answer. [raw metric file](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval/metric.first.answer.paragraph.questions_answers.lmqg_qg_squad.default.json)
351
 
352
  | | Score | Type | Dataset |
353
  |:--------------------------------|--------:|:--------|:---------------------------------------------------------------|
@@ -363,16 +363,16 @@ output = pipe("<hl> Beyonce <hl> further expanded her acting career, starring as
363
 
364
  | Dataset | Type | BERTScore| Bleu_4 | METEOR | MoverScore | ROUGE_L | Link |
365
  |:--------|:-----|---------:|-------:|-------:|-----------:|--------:|-----:|
366
- | [lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) | amazon | 90.93 | 6.53 | 22.3 | 60.87 | 25.03 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_squadshifts.amazon.json) |
367
- | [lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) | new_wiki | 93.23 | 11.12 | 27.32 | 66.23 | 29.68 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_squadshifts.new_wiki.json) |
368
- | [lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) | nyt | 92.49 | 8.12 | 25.25 | 64.06 | 25.29 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_squadshifts.nyt.json) |
369
- | [lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) | reddit | 90.95 | 5.95 | 21.5 | 60.59 | 22.37 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_squadshifts.reddit.json) |
370
- | [lmqg/qg_subjqa](https://huggingface.co/datasets/lmqg/qg_subjqa) | books | 88.07 | 0.63 | 11.58 | 55.56 | 12.37 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_subjqa.books.json) |
371
- | [lmqg/qg_subjqa](https://huggingface.co/datasets/lmqg/qg_subjqa) | electronics | 87.83 | 0.87 | 15.35 | 56.35 | 16.02 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_subjqa.electronics.json) |
372
- | [lmqg/qg_subjqa](https://huggingface.co/datasets/lmqg/qg_subjqa) | grocery | 87.79 | 0.53 | 15.13 | 57.02 | 12.34 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_subjqa.grocery.json) |
373
- | [lmqg/qg_subjqa](https://huggingface.co/datasets/lmqg/qg_subjqa) | movies | 87.49 | 0.0 | 11.86 | 55.29 | 12.51 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_subjqa.movies.json) |
374
- | [lmqg/qg_subjqa](https://huggingface.co/datasets/lmqg/qg_subjqa) | restaurants | 87.98 | 0.0 | 12.42 | 55.43 | 13.08 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_subjqa.restaurants.json) |
375
- | [lmqg/qg_subjqa](https://huggingface.co/datasets/lmqg/qg_subjqa) | tripadvisor | 88.91 | 0.0 | 13.72 | 56.05 | 14.03 | [link](https://huggingface.co/lmqg/bart-large-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_subjqa.tripadvisor.json) |
376
 
377
 
378
  ## Training hyperparameters
@@ -394,7 +394,7 @@ The following hyperparameters were used during fine-tuning:
394
  - gradient_accumulation_steps: 4
395
  - label_smoothing: 0.15
396
 
397
- The full configuration can be found at [fine-tuning config file](https://huggingface.co/lmqg/bart-large-squad/raw/main/trainer_config.json).
398
 
399
  ## Citation
400
  ```
21
  - text: "Beyonce further expanded her acting career, starring as blues singer Etta James in the 2008 musical biopic, <hl> Cadillac Records <hl> ."
22
  example_title: "Question Generation Example 3"
23
  model-index:
24
+ - name: lmqg/bart-large-squad-qg
25
  results:
26
  - task:
27
  name: Text2text Generation
296
  value: 0.5604572211470809
297
  ---
298
 
299
+ # Model Card of `lmqg/bart-large-squad-qg`
300
  This model is fine-tuned version of [facebook/bart-large](https://huggingface.co/facebook/bart-large) for question generation task on the [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) (dataset_name: default) via [`lmqg`](https://github.com/asahi417/lm-question-generation).
301
 
302
 
314
  from lmqg import TransformersQG
315
 
316
  # initialize model
317
+ model = TransformersQG(language="en", model="lmqg/bart-large-squad-qg")
318
 
319
  # model prediction
320
  questions = model.generate_q(list_context="William Turner was an English painter who specialised in watercolour landscapes", list_answer="William Turner")
325
  ```python
326
  from transformers import pipeline
327
 
328
+ pipe = pipeline("text2text-generation", "lmqg/bart-large-squad-qg")
329
  output = pipe("<hl> Beyonce <hl> further expanded her acting career, starring as blues singer Etta James in the 2008 musical biopic, Cadillac Records.")
330
 
331
  ```
333
  ## Evaluation
334
 
335
 
336
+ - ***Metric (Question Generation)***: [raw metric file](https://huggingface.co/lmqg/bart-large-squad-qg/raw/main/eval/metric.first.sentence.paragraph_answer.question.lmqg_qg_squad.default.json)
337
 
338
  | | Score | Type | Dataset |
339
  |:-----------|--------:|:--------|:---------------------------------------------------------------|
347
  | ROUGE_L | 53.85 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
348
 
349
 
350
+ - ***Metric (Question & Answer Generation)***: QAG metrics are computed with *the gold answer* and generated question on it for this model, as the model cannot provide an answer. [raw metric file](https://huggingface.co/lmqg/bart-large-squad-qg/raw/main/eval/metric.first.answer.paragraph.questions_answers.lmqg_qg_squad.default.json)
351
 
352
  | | Score | Type | Dataset |
353
  |:--------------------------------|--------:|:--------|:---------------------------------------------------------------|
363
 
364
  | Dataset | Type | BERTScore| Bleu_4 | METEOR | MoverScore | ROUGE_L | Link |
365
  |:--------|:-----|---------:|-------:|-------:|-----------:|--------:|-----:|
366
+ | [lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) | amazon | 90.93 | 6.53 | 22.3 | 60.87 | 25.03 | [link](https://huggingface.co/lmqg/bart-large-squad-qg/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_squadshifts.amazon.json) |
367
+ | [lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) | new_wiki | 93.23 | 11.12 | 27.32 | 66.23 | 29.68 | [link](https://huggingface.co/lmqg/bart-large-squad-qg/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_squadshifts.new_wiki.json) |
368
+ | [lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) | nyt | 92.49 | 8.12 | 25.25 | 64.06 | 25.29 | [link](https://huggingface.co/lmqg/bart-large-squad-qg/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_squadshifts.nyt.json) |
369
+ | [lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) | reddit | 90.95 | 5.95 | 21.5 | 60.59 | 22.37 | [link](https://huggingface.co/lmqg/bart-large-squad-qg/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_squadshifts.reddit.json) |
370
+ | [lmqg/qg_subjqa](https://huggingface.co/datasets/lmqg/qg_subjqa) | books | 88.07 | 0.63 | 11.58 | 55.56 | 12.37 | [link](https://huggingface.co/lmqg/bart-large-squad-qg/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_subjqa.books.json) |
371
+ | [lmqg/qg_subjqa](https://huggingface.co/datasets/lmqg/qg_subjqa) | electronics | 87.83 | 0.87 | 15.35 | 56.35 | 16.02 | [link](https://huggingface.co/lmqg/bart-large-squad-qg/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_subjqa.electronics.json) |
372
+ | [lmqg/qg_subjqa](https://huggingface.co/datasets/lmqg/qg_subjqa) | grocery | 87.79 | 0.53 | 15.13 | 57.02 | 12.34 | [link](https://huggingface.co/lmqg/bart-large-squad-qg/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_subjqa.grocery.json) |
373
+ | [lmqg/qg_subjqa](https://huggingface.co/datasets/lmqg/qg_subjqa) | movies | 87.49 | 0.0 | 11.86 | 55.29 | 12.51 | [link](https://huggingface.co/lmqg/bart-large-squad-qg/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_subjqa.movies.json) |
374
+ | [lmqg/qg_subjqa](https://huggingface.co/datasets/lmqg/qg_subjqa) | restaurants | 87.98 | 0.0 | 12.42 | 55.43 | 13.08 | [link](https://huggingface.co/lmqg/bart-large-squad-qg/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_subjqa.restaurants.json) |
375
+ | [lmqg/qg_subjqa](https://huggingface.co/datasets/lmqg/qg_subjqa) | tripadvisor | 88.91 | 0.0 | 13.72 | 56.05 | 14.03 | [link](https://huggingface.co/lmqg/bart-large-squad-qg/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_subjqa.tripadvisor.json) |
376
 
377
 
378
  ## Training hyperparameters
394
  - gradient_accumulation_steps: 4
395
  - label_smoothing: 0.15
396
 
397
+ The full configuration can be found at [fine-tuning config file](https://huggingface.co/lmqg/bart-large-squad-qg/raw/main/trainer_config.json).
398
 
399
  ## Citation
400
  ```