M ix QG : Neural Question Generation with Mixed Answer Types (2022)

Abstract: Asking good questions is an essential ability for both human and machine intelligence. However, existing neural question generation approaches mainly focus on short factoid type of answers. In this paper, we introduce a neural question generator, MixQG, to bridge this gap. We combine nine question answering datasets with diverse answer types, including yes/no, multiple-choice, extractive, and abstractive answers, to train a single generative model. We show with empirical results that our model outperforms existing work in both seen and unseen domains, and can generate questions with different cognitive levels when conditioned on different answer types. We run a human evaluation study to assess the quality of generated questions and find that MixQG outperforms the next best model by 10%. Our code and model checkpoints will be released and integrated with the HuggingFace library to facilitate various downstream applications.

## introduction question generation (qg) aims to automatically create questions from a given text passage or document with or without answers. it has a wide range of applications such as improving question answering (qa) systems @xcite and search engines @xcite through data augmentation, making chatbots more engaging @xcite , enabling automatic evaluation @xcite and fact verification @xcite , and facilitating educational applications @xcite . earlier qg approaches relied on syntactic rules that incorporated linguistic features into the qg process @xcite . @xcite pointed out some of the limitations of such rule-based systems and formulated the task of question generation as a sequence-to-sequence learning problem. based on this formulation, recent works rely on pre-trained context: in the late 17th century, robert boyle proved that air is necessary for combustion. english chemist john mayow (1641-1679) refined this work by showing that fire requires only a part of air that he called spiritus nitroaereus or just nitroaereus. in one experiment he found that placing either a mouse or a lit candle in a closed container over water caused the water to rise and replace one-fourteenth of the air's volume before extinguishing the subjects. from this he surmised that nitroaereus is consumed in both respiration and combustion. question: who proved that air is necessary for combustion? ext. short answer: robert boyle question: how did john mayow find that spiritus nitroaereus is consumed in both respiration and combustion? abs. short answer: through an experiment question: does fire need air to burn? yes/no answer: yes question: what did john mayow discover about nitroaereus? ext. long answer: in the late 17th century . . . in both respiration and combustion. question: why was the mouse used in the experiment? abs. long answer: the mouse was used in the experiment to test the consumption of nitroaereus during respiration. transformer-based models to generate answeraware questions @xcite @xcite . however, the majority of qg research so far has been performed on the squad dataset @xcite , and as a result, it mainly focuses on factoid short answer questions @xcite @xcite . in reality, answers can come in a variety of types and forms, e.g., short/long, multiple-choice, yes-no, and extractive/abstractive answers. we hypothesize that answer types are as important as question types, and that different answer types have their unique qg challenges and result in questions with different cognitive levels. mixqg combines nine qa datasets with varied answer types to build a more robust and versatile qg model. we use pretrained generative language models like t5 @xcite and bart @xcite without question-specific or domain-specific prefixes to generate the questions. figure 1 illustrates the dataset type source train examples dev. examples ## conclusion in this paper, we present mixqg, a question generation model pre-trained on a collection of qa datasets with a mix of answer types. we show through experiments that the resulting model is a strong starting point for further fine-tuning which achieves state-of-the-art results on target datasets in commonly-used similarity metrics as well as our designed human evaluation. we release our code and the model checkpoints to facilitate qg research and downstream applications.