Unseen Entity Handling in Complex Question Answering over Knowledge Base via Language Generation (2021)

Abstract: Complex question answering over knowledge base remains as a challenging task because it involves reasoning over multiple pieces of information, including intermediate entities/relations and other constraints. Previous methods simplify the SPARQL query of a question into such forms as a list or a graph, missing such constraints as “filter” and “order_by”, and present models specialized for generating those simplified forms from a given question. We instead introduce a novel approach that directly generates an executable SPARQL query without simplification, addressing the issue of generating unseen entities. We adapt large scale pre-trained encoder-decoder models and show that our method significantly outperforms the previous methods and also that our method has higher interpretability and computational efficiency than the previous methods.

## introduction answering user's questions via correct relation paths over a knowledge base may facilitate machine-human interaction to understand how the machine gets the answer. the relation path of a question is defined as the sequence of relations from the topic entity mentioned in a question to its answer entity in a knowledge base, which corresponds to the semantics of the question. while answering simple questions whose relation path has only one relation (or edge) without any other constraint has been largely resolved @xcite , answering complex questions over a knowledge base (called complex kbqa) whose relation path contains more than one relation and/or other constraints remains as a difficult task @xcite @xcite . previous works on complex kbqa cast it as a graph searching task. @xcite , @xcite , and @xcite identify the relation path of a question, by comparing the question with each candidate relation path. they should restrict the set of candidate relation paths (e.g. those with up to two relations), excluding any other constraints (e.g. filter, order_by), due to too big search space of all potential candidate relation paths. the methods thus show limited coverage for such datasets as complexwebquestions, whose relation paths have up to three relations and other constraints. @xcite instead identify intermediate entities in the relation path iteratively until reaching the answer entity. however, the methods predict only one answer entity for a question and thus show low recall for questions with multiple answer entities. @xcite , @xcite , and lan and jiang (2020) extend the previous methods @xcite @xcite by iteratively generating a query graph instead of ranking candidate relation paths. the methods predict one of the actions 'extend', 'connect' and 'aggregate' to grow a query graph by one more pair of edge and node, but yet do not cover such constraints as "filter" and "order_by". please refer to appendix a for detailed discussion of the previous works. inspired by the recent progress of adapting natural language generation (nlg) for various natural language processing (nlp) applications @xcite , we approach complex kbqa as a language generation task, fine-tuning large-scale pre-trained encoder-decoder models to generate executable sparql query from question. an issue of this approach is to generate unseen entities for questions of test dataset. the sparql queries in the kbqa datasets represent entities with their ids (e.g. "ns:m.08x9_6"), but it is impractical to learn to generate unseen entity ids. to address the issue, we leverage language generation models to learn the correlation between entity text labels (e.g. "1980 nba finals") and questions during the training process so as to generate unseen entities' text labels in the inference process. specifically, our method learns to generate entity text labels instead of entity ids, by replacing each entity id in a sparql query with a placeholder (e.g. 'c1') and adding a string matching filter at the end of the sparql query (e.g. 'filter(str(?c1) = "1980 nba finals")'). the proposed approach has the following advantages over the previous works: 1) the proposed approach can optimize a model for the whole query sequence generation, while the iterative graph generation models are optimized for predicting one edge (or action) of query graph at a time; 2) the interpretability of sequence generation models is higher than that of iterative graph generation models (see section 3.4 for details); 3) our method can utilize a large-scale pre-trained language model for learning sparql query generation, while the previous works can utilize such a model only for representing texts (e.q. question, entity and relation text labels); and 4) our method can learn to generate any constraints, while the previous works should define a new action type to deal with another unaddressed constraint type. the language generation part of the proposed approach is in fact semantic parsing, which converts a question into a logical representation or an executable query (e.g. sql) @xcite @xcite . the key difference between complex kbqa and semantic parsing is that complex kbqa assumes a large knowledge base (e.g. freebase) for the whole dataset, while semantic parsing aims at learning dynamic correlation between a question and any given table or relational database. recent methods of semantic parsing @xcite ) learn the dynamic correlation by encoding the whole table together with the question. however, such knowledge base as freebase is too large to be represented by a single encoder (see we conduct experiments on three benchmark datasets: metaqa @xcite , com-plexwebquestions @xcite , and webquestionssp @xcite . evaluation results show that the proposed method significantly outperforms the state-of-the-art methods over all metrics on all three datasets. besides, our method also outperforms the previous methods in terms of interpretability and computational efficiency. we summarize the contributions that will be shown in this paper as follows: ## methodology our method first recognises topic entities in a given question (section 2.2), and then generates a list of sparql queries given the question and the category (or type) of each topic entity by training an encoder-decoder model (section 2.3), and finally identifies the best valid sparql query that locates at least one answer entity in a given knowledge base at a post-processing step (section 2.4). a question may mention multiple entities. our method considers them all as candidate topic entities of the question and generates sparql queries with each of the candidate topic entities. if a sparql query has multiple entities, the entity whose id is the first element of a triple (e.g. <entity id, predicate, ?@xmath0. we select one topic entity at a time, while the other entities are considered as constraint entities. our method is schematically described in appendix b.1, and figure 1 depicts how the method analyzes a question to generate an executable sparql query. ## experiments we conducted experiments on the three datasets of metaqa, webquestionssp (webqsp) and com-plexwebquestions (cwq) (see appendix c.1 for detailed descriptions and statistics of the datasets and their knowledge bases). ## conclusion we propose to improve complex kbqa by utilizing pre-trained encoder-decoder models to generate a normalized sparql query from questions. the proposed method outperforms previous models on all of three complex kbqa benchmarks and addresses unseen entities by translating entity ids to sparql queries. in the future, we will explore combining relation classification with the constraint generation to reduce the space of beam search.