# Multi-Hop Question Generation with Knowledge Graph-Enhanced Language Model
## 1 Introduction  
Question generation (QG) endeavors to automatically generate high-quality questions through reasoning and inference from context and answers. QG can provide a wealth of training data for question answering (QA) tasks [1] or serve as a starting point for dialogue systems [2]. Much of the research in text-based QG focuses on single-hop reasoning (e.g., the SQuAD dataset) [3; 4; 5; 6], where each question-answer pair is usually derived from a single sentence. For example, in the SQuAD dataset, a question such as \"In what country is Normandy located?\" would be generated after reading the sentence \"The Normans were the people who in the 10 th and 11 th centuries gave their name to Normandy, a region in France\". This type of single-hop QG, although useful, is considered limited in terms of its ability to mimic human-like intelligence. A human, on the other hand, can easily ask insightful questions after reading a book with a complex narrative. For instance, one might ask \"What caused [PERSON] to become so evil, or is his villainy 'inherent'?\" after reading the Harry Potter series. This type of multi-hop QG constitutes a significant challenge that must be addressed in the pursuit of advanced AI. To address the multi-hopquestion generation task, we adopt a straightforward learning paradigm that consists of two phases [7]. The first phase involves learning what to ask by identifying and integrating crucial evidence in sentences. The second phase involves learning how to ask by generating logical and fluent questions based on the identified evidence. Our model faces three main challenges in this learning paradigm. The first challenge is to effectively identify and integrate crucial evidence from sentences for learning what to ask. The copy mechanism [6] has been widely used for finding words related to questions in context. [PERSON] et al. [8] addressed this challenge by fusing document-level and graph-level representations through an attention-based gate graph neural network to select answer-related evidence information. [PERSON] et al. [9] employed multi-task learning with an auxiliary loss to predict sentence-level evidence. To aggregate potential evidence related to questions, [PERSON] et al. [10] leveraged an answer-aware dynamic entity graph constructed from mentioned entities in the answer and input paragraphs. The second challenge in the multi-hop QG task is to perform complex reasoning over scattered evidence and form a reasoning chain that connects the answer to the question. To address this challenge, several approaches have been proposed, such as the dynamic multi-hop reasoning-based model proposed by [PERSON] et al. [11], which collects evidence using relational paths. The integration of external commonsense knowledge was also explored as a means of enhancing reasoning ability [12; 13]. The third challenge is to generate fluent and logically consistent questions. Several pre-trained language models, such as GPT-2 [14], BART [15], and UniLM [16], have achieved success in various language generation tasks. However, these models often struggle with multi-hop reasoning, as they are designed with a self-attention module that only considers single-hop (or obvious) relationships among words, leading to shallow generation that fails to reflect human-like multi-hop reasoning processes. It is worth noting that some research has addressed one or two of the three challenges, but there is currently no single approach that addresses all three simultaneously. For instance, RNN-based QG models [3; 6] are prone to generating questions with grammatical errors, and pre-trained models such as GPT-2 [14] often generate non-logical or irrelevant questions, as demonstrated in Figure 1.  
This paper proposes a novel knowledge graph-enhanced language model to cope with the challenges noted above by a straightforward learning paradigm \"learn what to ask and learn how to ask\". Our model split the learning paradigm into three steps. Figure 1: Examples of multi-hop QG (**left**) and single-hop QG (**right**).