DSM : Question Generation over Knowledge Base via Modeling Diverse Subgraphs with Meta-learner (2022)

Abstract: Existing methods on knowledge base question generation (KBQG) learn a one-size-fits-all model by training together all subgraphs without distinguishing the diverse semantics of subgraphs. In this work, we show that making use of the past experience on semantically similar subgraphs can reduce the learning difficulty and promote the performance of KBQG models. To achieve this, we propose a novel approach to model diverse subgraphs with meta-learner (DSM). Specifically, we devise a graph contrastive learning-based retriever to identify semantically similar subgraphs, so that we can construct the semantics-aware learning tasks for the meta-learner to learn semantics-specific and semantics-agnostic knowledge on and across these tasks. Extensive experiments on two widely-adopted benchmarks for KBQG show that DSM derives new state-of-the-art performance and benefits the question answering tasks as a means of data augmentation.

## introduction in recent years, knowledge base question generation (kbqg) has attracted substantial research interests as it shows great promise to improve the quality of question answering (qa). specifically, kbqg can augment training data for qa systems @xcite , and it can also motivate the machines to actively ask questions in human-machine conversations @xcite . concretely, kbqg generates natural language questions according to a set of facts extracted from kb, where each fact is typically specified as a triplet (e, r, e ) meaning entity e has relation r with entity e . previous efforts on kbqg can be categorized into template-based models @xcite and neural network-based (nn-based) models @xcite . the former ones heavily depend on hand-crafted templates, resulting in low scalability as these templates are limited to narrow domains. alternatively, nn-based models address this issue via inputting the set of triplets about a certain answer into a seq2seq architecture to automatically generate the question. in fact, for generating a question, triplets about a certain answer can naturally form a subgraph as illustrated in figure 1 . we observe that subgraphs differ in their semantics, which is especially shown in the relations that express the triplets foot_0 as well as the structural patterns such as chain, star, and triangle foot_1 . existing efforts do not distinguish the semantics of different subgraphs but learn a one-sizefits-all model by training together all subgraphs, which increases the learning difficulty. inspired by humans who solve a problem by searching the relevant problems that they have encountered in the past and adjusting the solution of these problems to the new one @xcite , we avoid directly learning a model on the entire data but try to leverage the past experience from similar kbqg cases to supervise generation from the current subgraph. to achieve the goal, we propose a kbqg approach which models diverse subgraph with metalearner (dsm). dsm retrieves semantically similar subgraphs which share similar relations and structures to construct semantics-aware learning tasks, so that the model can carefully learn potential question generation (qg) patterns over each kind of subgraphs. with multiple learning tasks, we employ a model-agnostic meta-learning @xcite -like (maml-like) meta-learner to capture semantics-specific and semantics-agnostic knowl- lisa bob chinese tea mm ate sp ea k tom lucy english teammate speak which teammate of bob can speak chinese? which language can tom's teammate speak? marry jack french wo rkm ate spe ak mike lily paris s p o u s e li v e _ in live_in (a) star-style subgraph with teammate, speak (b) chain-style subgraph with teammate, speak which workmate of jack can speak french? (c) star-style subgraph with workmate, speak (e) triangle-style query graph with spouse, live_in teammate speak relation paths q 1 : a 1 : lisa g1: p1: π1 × 1 π2 × 1 english teammate speak relation paths π1 × 1 π2 × 1 teammate speak workmate speak relation paths π2 × 1 hinton cambridge turing gra du ate aw ard who is graduated from cambridge and awarded turing? (d) star-style subgraph with graduate, award graduate award relation paths where do mike and his wife live? ## preliminary a knowledge base (kb) organizes the factual information as a set of triplets, i.e., @xmath0, r, e )|e, @xmath1, @xmath2, where e and r denote the entity set and the relation set respectively. from a kb, we can extract any subgraph g i ⊂ kb. for convenience, we employ a i to denote the adjacent matrix of g i and use n i to represent the number of nodes in g i . ## dsm we introduce our kbqg approach which models diverse subgraph with meta-learner (dsm). ## experimental evaluation we conduct extensive experiments to mainly answer the four questions: (1) does dsm take effect in improving the kbqg performance? (2) can the gcl-based subgraph retriever result in more effective support set for the meta-learner? (3) how do the positive sample size for contrastive learning and the support set size for the meta-learner affect dsm? (4) can dsm benefit the qa tasks as a means of data augmentation? 2018), for evaluating the proposed dsm. to be specific, wq combines instances from webques-tionssp @xcite and complexwebquestions @xcite . evaluation metrics. we adopt bleu-n (@xmath3. bleu-n @xcite and rouge-l @xcite compute the ratios of the common n-grams between the generated question and the ground truth question, where the former can be viewed as precision and the latter focuses on recall. for qa performance, we use hits@1 to evaluate the accuracy of the top-1 predicted answer. since some questions have multiple answers, we also evaluate the f1 score. we also hire three people to evaluate the relevance and fluency of the generated questions. ## conclusion this work pilots studies on kbqg. we propose dsm to exploit semantic knowledge of diverse subgraphs. instead of training on different subgraphs together, we construct semantics-specific learning tasks to reduce the learning difficulty. specifically, we devise a gcl-based retriever to flexibly construct semantics-specific learning tasks. besides, a maml-like meta-learner is employed to learn on the different learning tasks, such that we can learn the semantics-specific and the semanticsagnostic knowledge shared on and across tasks. our model shows competitive performance across the widely used benchmarks. we believe that using the maml-like meta-learner could be inspiring for learning on datasets with high diversity.