Learning to Generate Question by Asking Question: A Primal-Dual Approach with Uncommon Word Generation (2022)

Abstract: Automatic question generation (AQG) is the task of generating a question from a given passage and an answer. Most existing AQG methods aim at encoding the passage and the answer to generate the question. However, limited work has focused on modeling the correlation between the target answer and the generated question. Moreover, unseen or rare word generation has not been studied in previous works. In this paper, we propose a novel approach which incorporates question generation with its dual problem, question answering, into a unified primal-dual framework. Specifically, the question generation component consists of an encoder that jointly encodes the answer with the passage, and a decoder that produces the question. The question answering component then re-asks the generated question on the passage to ensure that the target answer is obtained. We further introduce a knowledge distillation module to improve the model generalization ability. We conduct an extensive set of experiments on SQuAD and HotpotQA benchmarks. Experimental results demonstrate the superior performance of the proposed approach over several state-of-the-art methods.

## introduction question answering @xcite plays a crucial role in both the growth of human beings and the improvement of artificial intelligent systems. as a dual task of question answering, automatic question generation (aqg) @xcite based on a passage and a target answer has attracted much attention in recent years. one of its key applications is to generate questions for educational materials @xcite . another application is automatically producing question-answer pairs to enhance machine reading comprehension systems @xcite . besides, aqg is also widely used in building web answering system @xcite ; wang figure 1 : an example of generated questions from human, base models and our model. the purple text in the passage indicates the target answer. our model generates a more desirable question compared to the questions generated from the base models. our model is able to generate the uncommon word, attend, that does not appear in the passage. @xcite , conversational dialog systems @xcite @xcite and chatbots @xcite such as siri, cortana, alexa and google assistant, helping them to start and continue a conversation with human users. automatic question generation is a challenging task due to the unstructured nature of textual data. early research @xcite on aqg focuses on generating questions that are grammatically correct and answerable from the passage, but not specific to any answer in the passage. there are fundamental limitations of these methods as they are not able to produce useful question-answer pairs for downstream tasks. for example, in a conversational dialog system, the next question should be generated according to the user's previous answers or conversations but not just the current context. on the other hand, specifying the answer is necessary for generating natural questions because there could be multiple target answers in the passage. for example, in figure 1 , there are various candidate questions to be asked on the passage, such as the city "new york city", the country "united states", and the year "2012". recent aqg models incorporate the target answer information in generating the question @xcite @xcite @xcite . these approaches focus on encoding the passage, the answer and their correlation using complex networks and then generate the question with a decoder. however, there are two main limitations: first, most of these methods generate the question in one single pass, and do not explicitly model the correlation between the target answer and the generated question. a natural question arises: would the target answer be retrieved when asking the generated question on the passage? for instance, in figure 1 , the question generated by the qg+ssm+api model is grammatically correct but is not answerable. on the other hand, the generated question from refinenet is both grammatically correct and answerable, but not specific to the answer. if asking both generated questions on the passage, it is clear that the target answer will not be obtained. second, very limited research has focused on new or unseen word generation. however, these uncommon words not only increase the diversity of the generated questions, but also improve the question quality in terms of naturalness and answerability. for example, in figure 1 , the word "attend" does not appear in the input passage, which is also uncommon in the training data. but it is an important word in this case as people use the phrase "attend school" naturally in their daily conversations. a quantitative study of uncommon words generation is shown in figure 2 . we calculate the average generalizability score (ags) using a normalized idf (inverse document frequency) metric as gs(q) = max w q t / ∈p 1 1+log(1+df (w q t )) . here df (w q t ) is the document frequency indicating how many training questions contain the word w q t . in-tuitively, ags measures the rarity of the words in the generated questions. in this paper, we propose a novel primal-dual approach, question generation by asking question with uncommon word generation (qg+aq+ug), which integrates question generation with its dual problem, question answering, into a unified learning framework. in particular, the question generation component consists of an encoder that encodes the answer with the passage, and a decoder that produces the question based on the output of the encoder. the question answering component, which shares the same encoder, then asks the generated question on the passage to ensure that the target answer is obtained. a knowledge distillation module is introduced for better uncommon word generation, which transfers the knowledge from a large pre-trained model to the primal-dual framework. we conduct an extensive set of experiments on the squad and hotpotqa benchmarks, which shows superior performance of the proposed approach over several state-of-the-art methods. we summarize the main contributions as follows: ## related work early works on automatic question generation are essentially rule based systems @xcite @xcite . several aqg models have been proposed to generate questions from the passage alone @xcite . these methods usually aim at generating questions that are grammatically correct and answerable from the passage, but not specific to any answer in the passage, which have fundamental limitations for downstream tasks. recent models for aqg are based on the encodeattend-decode paradigm and they generate questions from the passage and a target answer @xcite @xcite . over the past few years, several variants @xcite @xcite @xcite of the encode-attend-decode model have been proposed. to generate more plausible questions, zhou et al. @xcite utilize answer positions to make the model aware of the target answer. song et al. @xcite apply the multi-perspective context matching algorithm of @xcite to employ the interaction between the target answer and the passage. both works employ a copy mechanism @xcite to handle the missing words. @xcite ) develop an answer separation technique which masks out the answer in the passage to generate more reasonable questions. huang et al. @xcite propose an entity guided question generation model with additional question type information. there has also been some work on generating questions from images @xcite , knowledge bases @xcite and products @xcite . there are several aqg methods @xcite that try to leverage both question answering and question generation. nema et al. @xcite augment the basic encoder-decoder model with a reward based refinement network, which re-evaluates the generated question in a second pass. this method requires an additional reward mechanism to obtain the fluency and answerability scores. another closely related work is @xcite , which linearly combines the question generation loss with the question answering loss in a multi-task setting. ## conclusions automatic question generation is an important task in the improvement of artificial intelligent systems. in this work, we propose a novel primal-dual approach for question generation. it integrates question generation with its dual problem question answering into a unified framework. a knowledge distillation module is introduced into the framework to improve model generalization on uncommon word generation. experimental results on two benchmarks demonstrate the effectiveness of the primal-dual modeling.