---
language: 
  - en
tags:
- question generation
- question answer generation
license: mit
datasets:
- squad
metrics:
- bleu
- meteor
- rouge
- bertscore
- moverscore
widget:
- text: "<hl> Beyonce <hl> further expanded her acting career, starring as blues singer Etta James in the 2008 musical biopic, Cadillac Records."
  example_title: "Example 1"
- text: "Beyonce further expanded her acting career, starring as blues singer <hl> Etta James <hl> in the 2008 musical biopic, Cadillac Records."
  example_title: "Example 2"
- text: "Beyonce further expanded her acting career, starring as blues singer Etta James in the 2008 musical biopic,  <hl> Cadillac Records  <hl> ."
  example_title: "Example 3"
---

# BART LARGE fine-tuned for English Question Generation
BART LARGE Model fine-tuned on English question generation dataset (SQuAD) with an extensive hyper-parameter search.

- [Project Repository](https://github.com/asahi417/lm-question-generation)

## Overview

**Language model:** facebook/bart-large
**Language:** English (en)    
**Downstream-task:** Question Generation  
**Training data:** SQuAD  
**Eval data:** SQuAD   
**Code:**  See [our repository](https://github.com/asahi417/lm-question-generation)

## Usage
### In Transformers
```python
from transformers import pipeline

model_path = 'asahi417/lmqg-t5-small-squad'
pipe = pipeline("text2text-generation", model_path)

paragraph = 'Beyonce further expanded her acting career, starring as blues singer Etta James in the 2008 musical biopic, Cadillac Records.'
# highlight an answer in the paragraph to generate question
answer = 'Etta James'
highlight_token = '<hl>'
input_text = paragraph.replace(answer, '{0} {1} {0}'.format(highlight_token, answer))
input_text = 'generate question: {}'.format(input_text)  # add task specific prefix
generation = pipe(input_text)
print(generation)
>>> [{'generated_text': 'What is the name of the biopic that Beyonce starred in?'}]
```

## Evaluations

Evaluation on the test set of [SQuAD QG dataset](https://huggingface.co/datasets/asahi417/qg_squad).
The results are comparable with the [leaderboard](https://paperswithcode.com/sota/question-generation-on-squad11) and previous works.
All evaluations were done using our [evaluation script](https://github.com/asahi417/lm-question-generation).


| BLEU 4 | ROUGE L  | METEOR | BERTScore | MoverScore |
| ------ | -------- | ------ | --------- | ---------- |
| 21.75  | 50.48    | 25.12  | 90.78     | 64.80      |

## Fine-tuning Parameters
We ran grid search to find the best hyper-parameters and continued fine-tuning until the validation metric decrease.
The best hyper-parameters can be found [here](https://huggingface.co/asahi417/lmqg-bart-large-squad/raw/main/trainer_config.json), and fine-tuning script is released in [our  repository](https://github.com/asahi417/lm-question-generation).

## Citation
TBA