Answer-driven Deep Question Generation based on Reinforcement Learning (2020)

Abstract: Deep question generation (DQG) aims to generate complex questions through reasoning over multiple documents. The task is challenging and underexplored. Existing methods mainly focus on enhancing document representations, with little attention paid to the answer information, which may result in the generated question not matching the answer type and being answerirrelevant. In this paper, we propose an Answer-driven Deep Question Generation (ADDQG) model based on the encoder-decoder framework. The model makes better use of the target answer as a guidance to facilitate question generation. First, we propose an answer-aware initialization module with a gated connection layer which introduces both document and answer information to the decoder, thus helping to guide the choice of answer-focused question words. Then a semantic-rich fusion attention mechanism is designed to support the decoding process, which integrates the answer with the document representations to promote the proper handling of answer information during generation. Moreover, reinforcement learning is applied to integrate both syntactic and semantic metrics as the reward to enhance the training of the ADDQG. Extensive experiments on the HotpotQA dataset show that ADDQG outperforms state-of-the-art models in both automatic and human evaluations.

## introduction neural question generation (qg) aims at generating specific answer related questions from a given document with a target answer based on deep neural networks. its key applications include generating questions for reading comprehension @xcite , enhancing question answering systems as a strategy of data augmentation @xcite and helping digital assistants (e.g., alexa, cortana, siri and google assistant) to start and continue a conversation. various methods have been proposed for general qg @xcite @xcite @xcite . however, most existing methods focus on generating questions relevant to only one fact without deep comprehension and reasoning. for example, @xcite find that more than 80% of the questions in the widely adopted squad dataset @xcite are shallow and only relevant to information confined to a single sentence. generating deep question which requires higher cognitive skills is rarely studied @xcite . these skills include a thorough understanding of the input sources and the ability to reason over disjoint and relevant contexts. this paper focuses on the task of deep question generation (dqg), which focuses on generating deep questions with multi-hop reasoning over document-level contexts. previous work mainly focuses on the enhancement of document representations and obtains good performance. however, the answer information is also important since the generated questions should match the answer type and be answerfocused, and several common problems in qg are caused by the lack or improper use of the answer information: 1) the generated questions may be irrelevant to the answer. as shown in figure 1 , with a wrongly chosen question word, inappropriate question 1 is asking about time information but not place. @xcite was an armenian -sc ( born 15 september 1940 ) is an english n do chris menges and aram avakian share ? ## conclusion and future work deep question generation aims to generate complex questions that require reasoning over multiple pieces of information. in this paper, we propose an answer-driven end-to-end deep question generation model (addqg) based on reinforcement learning. an answer-aware initialization module with a gated connection layer and a semantic-rich fusion attention mechanism are designed to incorporate document and answer information into the generation process. reinforcement learning is further applied to integrate both syntactic and semantic metrics as the reward to enhance the training of addqg. experiments show that addqg outperforms the state-of-the-art systems on the challenging dqg dataset. ablation studies have demonstrated the effectiveness of our designs, and human evaluations show that our model can produce more coherent and answer-focused questions. future research can be carried out in several directions. first, we will try deep graph convolutional encoders @xcite to get deeper semantic information and explore more elaborate mechanisms for the integration of document and answer information. second, we will apply a pre-trained multi-hop question answering model to generate the reward to optimize addqg, thus further enhancing the reasoning ability of this dqg model.