metadata

language:
  - en
tags:
  - question generation
  - question answer generation
license: mit
datasets:
  - squad
metrics:
  - bleu
  - meteor
  - rouge
  - bertscore
  - moverscore
widget:
  - text: >-
      <hl> Beyonce <hl> further expanded her acting career, starring as blues
      singer Etta James in the 2008 musical biopic, Cadillac Records.
    example_title: Example 1
  - text: >-
      Beyonce further expanded her acting career, starring as blues singer <hl>
      Etta James <hl> in the 2008 musical biopic, Cadillac Records.
    example_title: Example 2
  - text: >-
      Beyonce further expanded her acting career, starring as blues singer Etta
      James in the 2008 musical biopic,  <hl> Cadillac Records  <hl> .
    example_title: Example 3

BART LARGE fine-tuned for English Question Generation

BART LARGE Model fine-tuned on English question generation dataset (SQuAD) with an extensive hyper-parameter search.

Project Repository

Overview

Language model: facebook/bart-large Language: English (en)
Downstream-task: Question Generation
Training data: SQuAD
Eval data: SQuAD
Code: See our repository

Usage

In Transformers

from transformers import pipeline

model_path = 'asahi417/lmqg-t5-small-squad'
pipe = pipeline("text2text-generation", model_path)

paragraph = 'Beyonce further expanded her acting career, starring as blues singer Etta James in the 2008 musical biopic, Cadillac Records.'
# highlight an answer in the paragraph to generate question
answer = 'Etta James'
highlight_token = '<hl>'
input_text = paragraph.replace(answer, '{0} {1} {0}'.format(highlight_token, answer))
input_text = 'generate question: {}'.format(input_text)  # add task specific prefix
generation = pipe(input_text)
print(generation)
>>> [{'generated_text': 'What is the name of the biopic that Beyonce starred in?'}]

Evaluations

Evaluation on the test set of SQuAD QG dataset. The results are comparable with the leaderboard and previous works. All evaluations were done using our evaluation script.

BLEU 4	ROUGE L	METEOR	BERTScore	MoverScore
21.75	50.48	25.12	90.78	64.80

Fine-tuning Parameters

We ran grid search to find the best hyper-parameters and continued fine-tuning until the validation metric decrease. The best hyper-parameters can be found here, and fine-tuning script is released in our repository.

Citation

TBA