Knowledge-enriched, Type-constrained and Grammar-guided Question Generation over Knowledge Bases (2020)

Abstract: Question generation over knowledge bases (KBQG) aims at generating natural-language questions about a subgraph, i.e. a set of triples. Two main challenges still face the current crop of encoder-decoder-based methods, especially on small subgraphs: (1) low diversity and poor fluency due to the limited information contained in the subgraphs, and (2) semantic drift due to the decoder’s oblivion of the semantics of the answer entity. We propose an innovative knowledge-enriched, type-constrained and grammar-guided KBQG model, named KTG, to addresses the above challenges. In our model, the encoder is equipped with auxiliary information from the KB, and the decoder is constrained with word types during QG. Specifically, entity domain and description, as well as relation hierarchy information are considered to construct question contexts, while a conditional copy mechanism is incorporated to modulate question semantics according to current word types. Besides, a novel reward function featuring grammatical similarity is designed to improve both generative richness and syntactic correctness via reinforcement learning. Extensive experiments show that our proposed model outperforms existing methods by a significant margin on two widely-used benchmark datasets SimpleQuestion and PathQuestion.

## introduction question generation over knowledge bases (kbqg) aims to generate natural-language questions given a subgraph in the kb, i.e. a set of connected triples of the @xmath0, predicate, object>. kbqg has a wide range of applications and is increasingly attracting attention from both academia and industry. for example, kbqg can improve factoid-based question answering (qa) systems by either dual training of qa and qg or by data augmentation for training corpora. as another example, kbqg can play a critical role in developing a chat-bot to ask kb-based questions under conversational settings. ## methodology in this section, we present the details of our model. the overall architecture of our model is shown in figure 1 . our model consists of a knowledge-augmented fact encoder, a typed decoder, as well as a grammar-guided evaluator in the reinforcement learning framework. the knowledge-augmented fact encoder takes the given entities, relations, and corresponding auxiliary knowledge, i.e. entity description and relation domain, as input and learns a knowledge-augmented fact representation. the learned representation is passed to a typed decoder for question generation. for each token the decoder outputs, the evaluator rewards the generated question using the grammatical similarity between it and the groundtruth question. based on the reward assigned by the evaluator, our encoder-decoder module updates and improves its current generation. ## experiment in this section we present the evaluation of our kbqg model ktg. the main experiments compare our model to a number of baseline models in two settings: automatic evaluation using standard metrics, as well as human evaluation over a number of criteria. we also conduct an ablation analysis to examine the effect of various components on model performance. ## conclusion in this paper, we tackle two crucial challenges: insufficient source input and semantic drift problem for the task of question generation over knowledge bases (kbqg). we enrich encoder input with auxiliary knowledge, including entity descriptions and predicate domains to improve question diversity. we employ a typed decoder with a conditional copy mechanism to further improve the semanticconsistency of generated questions. we further optimize model performance through reinforcement learning and design a novel reward function based on grammatical similarity but not n-gram overlap. this reward ensures the generation of syntactically and semantically valid questions while allowing more diversity and fluency. experimental results on two benchmark datasets show that our model achieves significant improvements over state-of-the-art models on all automatic and human evaluation metrics. the source code will be released to encourage reproducibility and further research https://github.com/bisheng/ktg4kbqg .