CQG : A Simple and Effective Controlled Generation Framework for Multi-hop Question Generation (2022)

Abstract: Multi-hop question generation focuses on generating complex questions that require reasoning over multiple pieces of information of the input passage. Current models with state-of-the-art performance have been able to generate the correct questions corresponding to the answers. However, most models can not ensure the complexity of generated questions, so they may generate shallow questions that can be answered without multi-hop reasoning. To address this challenge, we propose the CQG, which is a simple and effective controlled framework. CQG employs a simple method to generate the multi-hop questions that contain key entities in multi-hop reasoning chains, which ensure the complexity and quality of the questions. In addition, we introduce a novel controlled Transformer-based decoder to guarantee that key entities appear in the questions. Experiment results show that our model greatly improves performance, which also outperforms the state-of-the-art model about 25% by 5 BLEU points on HotpotQA.

## introduction question generation (qg) aims to endow machines with the ability to ask relevant and to-the-point questions about a document. qg plays a vital role in question answering (qa), dialogue systems, and automated tutoring applications: -by enriching the training qa corpora @xcite , helping chatbots start conversations with intriguing questions @xcite , and automatically generating assessment questions @xcite , respectively. most prior research on qg has focused on shallow factoid-based questions where answering the question simply by extracting the span of the text from a single input document @xcite , 2018; @xcite @xcite . recently, motivated by building the nlp systems that are capable of understanding and reasoning @xcite , there is an increasing interest in developing systems that are capable of more complex multi-hop question generation, where answering the questions requires reasoning over multiple documents @xcite @xcite @xcite . compared with shallow qg, there are two challenges for multi-hop qg (mqg). at first, generating multi-hop questions requires the model to understand the relationship between disjointed pieces of information in multiple context documents @xcite . secondly, multihop questions must have complex chains of connecting the mentioned entities, which ensure the complexity of multi-hop questions, as such, multi-hop questions are also called deep questions @xcite . to address the first challenge, existing research on mqg relies on the graph-to-sequence (g2s) architecture @xcite @xcite . these methods construct a semantic-level graph or entity-graph to capture the information among multiple context documents that employ a graph neural network(gnn) and then feed it to the decoder. however, these models can not handle the second challenge because they can not ensure the complexity of generated questions; thus, they may generate shallow questions that can be answered without multi-hop reasoning chains. we show an example in figure 1 , where the uncontrolled model generates a shallow question that can be answered by a single sentence but ignores the other sentences and entities. to solve this issue, we propose the cqg, a simple and effective controlled framework. @xcite claim that the reasoning chains can be captured by propagating information along the edges in an entity graph using a gnn. motivated by this, we construct the entity graph from the input documents first and then employ the graph attention network (gat) to extract the key entities that appear in multihop reasoning chains. intuitively, all these key entities should appear in the generated questions to ensure the generated questions have complex and complete reasoning chains. we introduce the flag tag @xcite , a lexical constraint for generation at each decoding step, which will assist the controlled generation. in detail, in decoding progressing, each input token is provided a flag tag that indicates whether the constraint of this token has been satisfied. three possible types of flag tags exist for each token, is not a constrain, does not appear in question and appear in question. as shown in figure 2 , the flag tag of six updates to appear in question at the fourth step because six is generated at this step. we represent the three flag tags by training the embedding and injecting them into the transformer generator. the flag tag can explicitly inform the generator to satisfy as many as possible constraints. in the training stage, when the generation is stopped, the flag tags for each token are either not a constrain or satisfied. it is a strong signal for the model to try to satisfy all constrains. we conduct experiments on hotpotqa @xcite : a challenging dataset in which the questions are generated by reasoning over text from separate wikipedia pages. results show that our model greatly improves performance; it outperforms the state-of-the-art about 25% by 5 bleu @xcite points. our main contributions are summarized as follows: ## methodology in this section, we formalize the multi-hop question generation (mqg) task and introduce our cqg. in particular, we first describe our graph attention network (gat) based key entities extractor. following this, we describe the flag tag and finally we introduce our novel controlled transformer-based generator with flag tag. ## conclusion the mqg task is more challenging and worthy of exploration compared with conventional shallow qg. to address the complexity control problem of mqg, we propose a simple control framework cqg, which consists of a gat-based key entity extractor and a controlled generated. cqg greatly improves the performance and we hope our model