Learning to Ask More: Semi-Autoregressive Sequential Question Generation under Dual-Graph Interaction (2020)

Abstract: Traditional Question Generation (TQG) aims to generate a question given an input passage and an answer. When there is a sequence of answers, we can perform Sequential Question Generation (SQG) to produce a series of interconnected questions. Since the frequently occurred information omission and coreference between questions, SQG is rather challenging. Prior works regarded SQG as a dialog generation task and recurrently produced each question. However, they suffered from problems caused by error cascades and could only capture limited context dependencies. To this end, we generate questions in a semi-autoregressive way. Our model divides questions into different groups and generates each group of them in parallel. During this process, it builds two graphs focusing on information from passages, answers respectively and performs dual-graph interaction to get information for generation. Besides, we design an answer-aware attention mechanism and the coarse-to-fine generation scenario. Experiments on our new dataset containing 81.9K questions show that our model substantially outperforms prior works.

## introduction question generation (qg) aims to teach machines to ask human-like questions from a range of inputs such as natural language texts @xcite , images @xcite and knowledge bases @xcite . in recent years, qg has received increasing attention due to its wide applications. asking questions in dialog systems can enhance the interactiveness and persistence of humanmachine interactions @xcite . qg benefits question answering (qa) models through data augmentation @xcite and joint learning @xcite . it also plays an important role in education @xcite and clinical @xcite systems. traditional question generation (tqg) is defined as the reverse task of qa, i.e., a passage and an answer (often a certain span from the passage) are provided as inputs, and the output is a question grounded in the input passage targeting on the given answer. when there is a sequence of answers, we can perform sequential question generation (sqg) to produce a series of interconnected questions. sqg is a challenging task in two aspects. first, information omissions between questions lead to complex context dependencies. second, there are frequently occurred coreference between questions. prior works regarded sqg as a dialog generation task (namely conversational qg) where questions are generated autoregressively (recurrently), i.e., a new question is produced based on previous outputs. although many powerful dialog generation models can be adopted to address the challenges mentioned above, there are two major obstacles. first, these models suffer from problems caused by error cascades. empirical results from experiments reveal that the later generated questions tend to become shorter with lower quality, especially becoming more irrelevant to given answers, e.g., "why?", "what else?". second, models recurrently generating each question struggle to capture complex context dependencies, e.g., long-distance coreference. essentially, sqg is rather different from dialog generation since all answers are given in advance and they act as strict semantic constraints during text generation. prior works performed sqg on coqa @xcite , a high-quality dataset for conversational qa. as will be further illustrated, a number of data in coqa are not suitable for sqg. some researchers @xcite directly discarded these data, but the remaining questions may become incoherent, e.g., the antecedent words for many pronouns are unclear. to this end, we build a new dataset from coqa containing 81.9k relabeled questions. above all, the main contributions of our work are: dataset for this paper is available at https:// github.com/chaiz-pku/sequential-qg. ## dataset as the reverse task of qa, qg is often performed on existing qa datasets, e.g., squad @xcite , newsqa @xcite , etc. however, questions are independent in most qa datasets, making tqg the only choice. in recent years, the appearance of large-scale conversational qa datasets like coqa @xcite and quac @xcite makes it possible to train data-driven sqg models, and the coqa dataset was widely adopted by prior works. since the test set of coqa is not released to the public, its training set (7.2k passages with 108.6k questions) was split into new training and validation set, and its validation set (0.5k passages with 8.0k questions) was used as the new test set. different from traditional qa datasets where the answers are certain spans from given passages, answers in coqa are free-form text foot_0 with cor-responding evidence highlighted in the passage. this brings a big trouble for qg. as an example, consider the yes/no questions counting for 19.8% among all questions. given the answer "yes" and a corresponding evidence "...the group first met on july 5 , 1967 on the campus of the ohio state university...", there are many potential outputs, e.g., "did the group first met in july?", "was the group first met in ohio state?". when considering the context formed by previous questions, the potential outputs become even more (the original question in coqa is "was it founded the same year?"). when there are too many potential outputs with significantly different semantic meanings, training a converged qg model becomes extremely difficult. for this reason, @xcite directly discarded questions that cannot be answered by spans from passages. however, the remaining questions can become incoherent, e.g., antecedent words for many pronouns become unclear. to this end, we build a new dataset from coqa by preserving all 7.7k passages and rewriting all questions and answers. more specifically, we first discarded questions that are unsuitable for sqg. to do so, three annotators were hired to vote for the preservation/deletion of each question. a question is preserved if and only if it can be answered by a certain span from the input passage foot_1 . as a result, most deleted questions were yes/no questions and unanswerable questions. besides, the kappa score between results given by different annotators was 0.83, indicating that there was a strong interagreement between annotators. for the remaining qa-pairs, we preserved their original order and replaced all answers by spans from input passages. after that, we rewrote all questions to make them coherent. to avoid over-editing, annotators were asked to modify as little as possible. it turned out that in most cases, they only needed to deal with coreference since the prototype of pronouns were no longer existed. to further guarantee the annotation quality, we hired another project manager who daily examined 10% of the annotations from each annotator and provided feedbacks. the annotation was considered valid only when the accuracy of examined results surpasses 95%. our annotation process took 2 months, and we finally got a dataset containing 7.7k passage with 81.9k qa-pairs. 1 4 model in this section, we formalize the sqg task and introduce our model in details. as shown in figure 1 , the model first builds a passage-info graph and an answer-info graph by its passage-info encoder and answer-info encoder respectively. after that, it performs dual-graph interaction to get representations for the decoder. finally, different groups of questions are generated in parallel under a coarse-to-fine scenario. both encoders and decoder take the form of transformer architecture @xcite . ## experiments in this section, we first introduce the three kinds of baselines. after that, we compare and analyse the results of different models under both automatic and human evaluation metrics. ## conclusion in this paper, we focus on sqg which is an important yet challenging task. different from prior works regarding sqg as a dialog generation task, we propose the first semi-autoregressive sqg model, which divides questions into different groups and further generates each group of closely-related questions in parallel. during this process, we first build a passage-info graph, an answer-info graph, and then perform dual-graph interaction to get representations capturing the context dependencies between passages and questions. these representations are further used during our coarse-to-fine generation process. to perform experiments, we analyze the limitation of existing datasets and create the first dataset specially used for sqg containing 81.9k questions. experimental results show that our model outperforms previous works by a substantial margin. for future works, the major challenge is generating more meaningful, informative but concise questions. besides, more powerful question clustering and coarse-to-fine generation scenarios are also worth exploration. finally, performing sqg on other types of inputs, e.g., images and knowledge graphs, is an interesting topic.