LFKQG : A Controlled Generation Framework with Local Fine-tuning for Question Generation over Knowledge Bases (2022)

Abstract: Question generation over knowledge bases (KBQG) aims at generating natural questions about a subgraph, which can be answered by a given answer entity. Existing KBQG models still face two main challenges: (1) Most models often focus on the most relevant part of the answer entity, while neglecting the rest of the subgraph. (2) There are a large number of out-of-vocabulary (OOV) predicates in real-world scenarios, which are hard to adapt for most KBQG models. To address these challenges, we propose LFKQG, a controlled generation framework for Question Generation over Knowledge Bases. (1) LFKQG employs a simple controlled generation method to generate the questions containing the critical entities in the subgraph, ensuring the question is relevant to the whole subgraph. (2) We propose an optimization strategy called local fine-tuning, which can make good use of the rich information hidden in the pre-trained model to improve the ability of the model to adapt the OOV predicates. Extensive experiments show that our method outperforms existing methods significantly on three widely-used benchmark datasets SimpleQuestion, PathQuestions, and WebQuestions.

## introduction question generation (qg) aims to endow machines with the ability to ask relevant and to-the-point questions for a given form of data such as text @xcite , image @xcite , table @xcite and knowledge bases (kb) @xcite . kbqg aims to generate natural language questions given a subgraph in the kb, i.e. a set of connected triples of the @xmath0, predicate, object>. kbqg is an effective approach to generating highquality qa pairs that can significantly address the data scarcity issue for knowledge base question answering (kbqa). in addition, kbqg can be applied for educational purposes by producing figure 1 : the examples of two main challenges in kbqg where the questions are generated from the subgraph and the red texts are the answer entity. q1 only focus on the second triple but neglect the first.q2 does not contain any semantic of discoverer or inventor, which is an oov predicatge. current kbqg systems follow an attentionbased sequence-to-sequence structure @xcite . to make use of the rich structure information hidden in the subgraph, @xcite proposes the graphto-sequence framework. however, these models face two main challenges: (1) most models often focus on the most relevant part of the answer entity, while neglecting the rest of the subgraph @xcite ). (2) there are many out-ofvocabulary (oov) predicates in the real knowledge bases, which are unseen at the training time. most kbqg models are hard to adapt to oov predicates, which makes these models difficult to use in realworld scenarios @xcite . figure 1 illustrates the examples of these two problems. q1 only focuses on the second triple but neglects the first. as for q2, discoverer of inventor is an oov predicate, and the model is unable to handle this type of predicate, so the q2 does not contain any semantic relating to it. to address these challenges, we propose lfkqg, a controlled generation framework. @xcite claim that the entity words in the given input play a decisive role in the semantics of the whole question. we can see that q1 only focuses on the second triple and miss the critical entity alice betty stern in the first triple. intuitively, all the critical entities should appear in the generated questions to ensure the generated questions contains the semantics of the whole subgraph. to this end, we introduce the controlled generation method to kbqg task. we use flag tag @xcite , a simple but effective lexical constraint for generation at each decoding step, to achieve the controlled generation. in detail, in decoding progressing, each input token is provided with a flag tag that indicates whether the constraint of this token has been satisfied. it is a strong incentive for the model to try to satisfy all constrains. furthermore, the fine-tuning method distorts the pre-trained features for oov samples, because the model over-fits the features for training data while removing the oov features that were originally hidden in the pre-trained models @xcite . to address the oov problem in kbqg, we propose a novel optimization strategy called local fine-tuning, which can retain the oov features in the pre-trained model. extensive experiments show that our lfkqg model outperforms existing methods greatly on three widely-used benchmark datasets simple-question, pathquestions, and webquestions. in addition, we conduct experiments on oov data, and the results show that our local fine-tuning greatly improves the performance in this challenging scenario. our main contributions are summarized as follows: ## methodology in this section, we formalize the question generation over knowledge bases (kbqg) task and introduce our methodology. in particular, we describe our controlled generation framework for kbqg and show the overall architecture in figure 2 . following this, we describe our local fine-tuning method and we show it in figure 4 . ## experiment in this section, we conduct extensive experiments to evaluate the effectiveness of our proposed model for the kbqg task. ## conclusion the kbqg task is challenging and worthy of exploration. to address the two main challenges of kbqg, we propose lfkqg, including the controlled generation framework and local fine-tuning method. the controlled generation framework makes the given question word, and critical entity in the subgraph appear in the question to control the semantic and the type of question. the local fine-tuning method can retain the oov features hidden in the pre-trained models. in addition, we find that the phenomenon that fine-tuning method distorts the pre-trained features also appears in the kbqg task. it may be an exciting way to study the pre-trained generation models.