A Question Type Driven and Copy Loss Enhanced Frameworkfor Answer-Agnostic Neural Question Generation (2020)

Abstract: The answer-agnostic question generation is a significant and challenging task, which aims to automatically generate questions for a given sentence but without an answer. In this paper, we propose two new strategies to deal with this task: question type prediction and copy loss mechanism. The question type module is to predict the types of questions that should be asked, which allows our model to generate multiple types of questions for the same source sentence. The new copy loss enhances the original copy mechanism to make sure that every important word in the source sentence has been copied when generating questions. Our integrated model outperforms the state-of-the-art approach in answer-agnostic question generation, achieving a BLEU-4 score of 13.9 on SQuAD. Human evaluation further validates the high quality of our generated questions. We will make our code public available for further research.

## introduction question generation (qg) has been investigated for many years because of its huge potential benefits to various fields, especially for education @xcite @xcite . qg can also act as an essential component of other comprehensive task, such as dialogue systems @xcite . besides, it can supplement question answering task by automatically constructing a large question set @xcite . traditional methods for automatic question generation are mostly rule-based, which need a set of complex empirical rules @xcite @xcite . recently, with the flourish of deep learning, especially the sequence-tosequence (seq2seq) frame @xcite with attention mechanism @xcite , many neural models have been proposed to solve the qg task and achieve rapid progress @xcite @xcite @xcite @xcite . however, most of the previous works are devoted to deal with answer-aware question generation. that is, given a text and also an answer span, the system is required to generate questions. but in a real application for educational purpose, people or machines are often required to generate questions for natural sentences without explicitly annotated answer. comparing with the answer-aware qg, the answer-agnostic qg (ag-qg) task is more challenging and attractive. unfortunately, ag-qg has been much less studied. du's work @xcite is the first one to tackle this problem, and @xcite achieve the state-of-the-art by employing an extended transformer network. for the ag-qg task, where the input is only a sentence but without any answer, multiple questions might be asked from various perspectives. according to our statistics on squad @xcite , nearly 34% of the source sentences are offered multiple gold reference questions, and nearly 20% of the source sentences are offered different types of questions. to enable the model to ask different types of questions given the same input sentence, we propose a question type driven framework for ag-qg task. specially, our model firstly predicts the probability of different question types distribution on the input sentence, which allows us to choose the best k question types with higher possibility. then these different question types will be embedded into different vectors, which will guide the decoder to pay attention to informative parts with respect to different questions. meanwhile, according to the statistics on squad, on average there are 3.09 non-stop words copying from the source sentence for each reference question. those non-stop words appearing in both questions and sentences are regarded as keywords, since they act as the connection of these two parts. to increase the probability of copying keywords from source sentences, we design a new copy loss to enhance the traditional copy mechanism. in our model, by minimizing the new copy loss, the model will be forced to copy these keywords at least once during decoding. we conduct experiments on squad. both the question type module and the new copy loss improve performance over the baseline model, and our full model combining two modules obtains a new state-of-the-art performance with a bleu-4 of 13.9. moreover, our model can ask different types of questions for a given sentence. we conclude the contributions as follows: answer-agnostic question generation. ag-qg is more challenging than answer-aware qg. du's work @xcite is the first one to tackle this problem, and they achieve better performance than rule-based approaches by employing a sequence-to-sequence attention model. @xcite aim to automatically find questionworthy sentences from a paragraph and then generate questions. @xcite treat qg as a two-stage task: answer phrase extraction and answer-aware question generation. @xcite propose a multi-agent communication framework, using a local extraction agent to extract question-worthy phrases, and then taking extracted phrases as assistance to generate questions. @xcite employ the transformer network @xcite and extend it with the placeholder strategy, copy mechanism and contextualized embedding. question word prediction. question word is one of the most important components of a question. @xcite study multi-types visual question generation, by feeding the encoded representation to a multi-layer perception to calculate the question words distribution. @xcite propose an answer-focused and position-aware model to generate the first question word. @xcite ) propose an answer-separated sequence-tosequence model to identify the proper question word. they replace the answer span in the source sentence with a special token to make better use of the context information. multi-types question generation. the multitypes qg has been much less researched. in ma's work @xcite , in order to generate different types of questions, they use question type embedding at the first step of decoding. however, because of the difficulty in automatically predicting question types, their model fails to outperform the previous works. the question type driven framework has also been tried for visual question generation @xcite , where they concatenate the question type embedding with the encoded representation of input. ## results and analysis in this section, we report the automatic evaluation results of our proposed model and do ablation study to prove the effectiveness of different parts of the model. then we conduct human evaluation and case study to test the quality of generated questions. furthermore, we give a detailed analysis on multiple questions generation. ## conclusion in this paper, we propose two new strategies to deal with the answer-agnostic qg: question type module and copy loss mechanism. these proposed modules improve the performance over the base-line model, achieving the state-of-the-art. moreover, our model has the ability and flexibility to generate multiple questions for one source sentence. hopefully, the idea of question type module and copy loss mechanism can also be used to do answer-aware qg task or other similar text generation tasks.