Meta- CQG : A Meta-Learning Framework for Complex Question Generation over Knowledge Bases (2022)

Abstract: Complex question generation over knowledge bases (KB) aims to generate natural language questions involving multiple KB relations or functional constraints. Existing methods train one encoder-decoder-based model to fit all questions. However, such a one-size-fits-all strategy may not perform well since complex questions exhibit an uneven distribution in many dimensions, such as question types, involved KB relations, and query structures, resulting in insufficient learning for long-tailed samples under different dimensions. To address this problem, we propose a meta-learning framework for complex question generation. The meta-trained generator can acquire universal and transferable meta-knowledge and quickly adapt to long-tailed samples through a few most related training samples. To retrieve similar samples for each input query, we design a self-supervised graph retriever to learn distributed representations for samples, and contrastive learning is leveraged to improve the learned representations. We conduct experiments on both WebQuestionsSP and ComplexWebQuestion, and results on long-tailed samples of different dimensions have been significantly improved, which demonstrates the effectiveness of the proposed framework.

## introduction question generation (qg) over knowledge base (kb) aims to generate natural language questions with structured kb queries, which has been widely used to improve the performance of question answering (qa) by data augmentation for the training corpora. it can also help chatbots ask questions during human-computer interaction. traditional methods @xcite rely on hand-crafted rules and templates to convert kb queries into questions, leading to poor generalization. recently, neural approaches @xcite figure 1 : the distribution of question type (typ-con as type constraint, ent-con as entity constraint, com-con as comparative constraint), relation type (the involved kb relation in the query) and query structure in training set of webqsp. we also give the illustration of three types of query structure leveraged one encoder-decoder-based model to fit the entire training set, then used the trained model to generate questions in the testing phase. however, such a one-size-fits-all strategy may not perform well for generating complex questions which contain multiple kb relations or functional constraints, such as comparison and sorting, and have complex semantic structures. this is due to the uneven distribution of the training set. take a widely used dataset webquestionssp (webqsp) @xcite as an example, questions are unevenly distributed across multiple dimensions, including question types, involved kb relations, and query structures. as illustrated in figure 1 , single-hop questions are the most common type of questions in the dataset, e.g., the question "what kind of money to take to bahamas?" relies on a single-hop kb query "(bahamas, cur-rency_used, ?x)". in contrast, questions with comparative or sorting constraints are the long-tailed samples, e.g., the question " what was the first who played meg? meg character actor ?a ?x bang cast who is the oldest actor that played jack in the bang? transfer jack character actor ?a ?x age ?num argmax 1 ent_1 character actor ?a ?x which is the longest river in american? to deal with the problem of data imbalance, we resort to the process of human cognition. as shown in figure 2 , when faced with a rare query, humans can write the corresponding question according to the previously learned query patterns. in this paper, we collectively refer to these universal and transferable query patterns as meta-knowledge, which can help generate long-tailed questions. to learn such meta-knowledge, we propose a meta-learning framework for kbqg, namely meta-cqg. during model training, each sample in the training set is viewed as a query set, and its similar samples are retrieved to form the support set. our generator adapts to each query set by trials and the supervision signals on the support set. through the modelagnostic meta-learning (maml) @xcite algorithm, the qg model can learn to generalize over varied samples to acquire the meta-knowledge, instead of fitting all samples. to select similar samples and construct the support sets, we design a self-supervised graph retriever that takes into account the similarity between different samples in different dimensions, including question types, involved kb relations, and query structures. specifically, the graph retriever encodes the input queries into distributed representations, and the cosine similarities between these vectors denote the similarities between different queries. due to lack of supervision, we train the graph retriever in a self-supervised way, and contrastive learning is leveraged to improve the learned representations. to demonstrate the effectiveness of the proposed framework, we conduct new delhi sri lanka adjoins_s adjoins capital country type ?a ?x select ?a where { sri lanka location.location.adjoin_s ?x . ?x ns:location.adjoining_relationship.adjoins ?a . ?a ns:common.topic.notable_types country . ?a ns:location.country.capital new delhi . } extensive experiments and ablation studies on two widely-used datasets, and the results on long-tailed samples under different dimensions have been significantly improved. in general, our main contributions are listed as follows: ## preliminary we aim to generate complex questions from queries, which can be executed on the knowledge base to get the answers to the generated questions. as the query is always displayed in the form of graph, we represent the query q with a query graph g. then we translate the g to the corresponding complex question. knowledge base. a knowledge base k is a collection of triples in the form of (s, r, o), where s, r, and o denote subject, relation, and object respectively. query graph. as described in @xcite , query graph g is a graph representation of sparql query. as shown in figure 3 , our query graph consists of two types of nodes: variable nodes and non-variable nodes. variable nodes represent ungrounded kb nodes or values. a nonvariable node can be a grounded kb entity or kb type, such as sri lanka and country. complex question. while a simple question can be answered by a single kb triple, complex questions require more information and even functional operations, such as comparison, aggregation, and sorting. ## methodology in this section, we will describe the proposed framework, meta-cqg. figure 4 gives an overview of our framework, which mainly consists of two parts: query-agnostic meta learning (qaml) and the graph retriever. qaml trains a unique generator for each target query by learning the potential features of retrieved similar samples. the graph retriever selects a few most similar samples and construct the support set. ## experiments in this section, we evaluate meta-cqg on two widely-used benchmark datasets and show the effectiveness of our method. we first introduce datasets and training settings. then, we evaluate the proposed model with the state-of-the-art models on both datasets. in addition, we investigate the performance on different dimensions and the influence of different sampling strategies. finally, we conduct human evaluation to verify the effectiveness of our method. ## conclusion in this paper, we focus on the task of complex question generation over knowledge bases. we propose a meta-learning framework for complex question generation, namely meta-cqg, to deal with the data imbalance problem. to consider the imbalance of all dimensions, we adopt the maml method to train a unique generator for each sam-ple to be generated via a few most similar training samples. specially, we design a self supervised graph retriever to flexibly retrieve most similar samples. we evaluate the effectiveness of meta-cqg on two widely-used benchmark datasets, and it outperforms all the baselines.