what is procced function for this 
Question generation is the task of automatically generating questions from a text paragraph. The most straight-forward way for this is answer aware question generation. In answer aware question generation the model is presented with the answer and the passage and asked to generate a question for that answer by considering the passage context. While there are many papers available for QG task, it's still not as mainstream as QA. One of the reasons is most of the earlier papers use complicated models/processing pipelines and have no pre-trained models available. Few recent papers, specifically UniLM and ProphetNet have SOTA pre-trained weights availble for QG but the usage seems quite complicated.

This project is aimed as an open source study on question generation with pre-trained transformers (specifically seq-2-seq models) using straight-forward end-to-end methods without much complicated pipelines. The goal is to provide simplified data processing and training scripts and easy to use pipelines for inference.

Initial experiments
Initial experiments are conducted using the SQuADv1 dataset and T5 model with different input processing formats as described below.

answer aware question generation
For answer aware models the input text can be processed in two ways.

1. prepend format:

Here the answer is simply added before the context and seperated by sep token. For example

42 [SEP] 42 is the answer to life, the universe and everything.

for T5 model the input is processed like this

answer: 42  context: 42 is the answer to life, the universe and everything.

2. highlight format

Here the answer span is highlighted within the text with special highlight tokens.

<hl> 42 <hl> is the answer to life, the universe and everything.

This idea is proposed in the "A Recurrent BERT-based Model for Question Generation" paper. See section 4.3

answer extraction models
As the answer aware models need answers for generating question, we need something which can extract answer like spans from the text. This can be done using various methods like NER, noun-phrase extarction etc. But here a model is trained to extract answer like spans, to see how it'll work. With T5, answer extarction is done using the text-to-format.

As the highlight format will need to know the position of extracted answer spans the input for answer extraction is processed as follows

split the text into senteces.
for each sentence that has answers, highlight the sentence with <hl> tokens.
for the target text join the answers in that sentence with <sep> tokens.
For example for this text

Python is a programming language. Created by Guido van Rossum and first released in 1991.

following examples will be created

Input text: <hl> Python is a programming language. <hl> Created by Guido van Rossum and first released in 1991.

target text: Python <sep>

and

Input text: Python is a programming language. <hl> Created by Guido van Rossum and first released in 1991 <hl>.

target text: Guido van Rossum <sep> 1991 <sep>

At inference time the text is split into sentences and each sentence is highlighted.

Multitask QA-QG
For answer aware question generation we usually need 3 models, first which will extract answer like spans, second model will generate question on that answer and third will be a QA model which will take the question and produce an answer, then we can compare the two answers to see if the generated question is correct or not.

Having 3 models for single task is lot of complexity, so goal is to create a multi-task model which can do all of these 3 tasks

extract answer like spans
generate question based on the answer
QA
T5 model is fine-tuned in multi-task way using task prefixes as described in the paper.



End-to-End question generation (answer agnostic)
In end-to-end question generation the model is aksed to generate questions without providing the answers. This paper discusses these ideas in more detail. Here the T5 model is trained to generate multiple questions simultaneously by just providing the context. The questions are seperated by the <sep> token. Here's how the examples are processed

input text: Python is a programming language. Created by Guido van Rossum and first released in 1991.

target text: Who created Python ? <sep> When was python released ? <sep>

All the training details can be found in this wandb project