Knowledge Informed Semantic Parsing for Conversational Question Answering (2021)

Abstract: Smart assistants are tasked to answer various questions regarding world knowledge. These questions range from retrieval of simple facts to retrieval of complex, multi-hops question followed by various operators (i.e., filter, argmax). Semantic parsing has emerged as the state-of-the-art for answering these kinds of questions by forming queries to extract information from knowledge bases (KBs). Specially, neural semantic parsers (NSPs) effectively translate natural questions to logical forms, which execute on KB and give desirable answers. Yet, NSPs suffer from non-executable logical forms for some instances in the generated logical forms might be missing due to the incompleteness of KBs. Intuitively, knowing the KB structure informs NSP with changes of the global logical forms structures with respect to changes in KB instances. In this work, we propose a novel knowledge-informed decoder variant of NSP. We consider the conversational question answering settings, where a natural language query, its context and its final answers are available at training. Experimental results show that our method outperformed strong baselines by 1.8 F1 points overall across 10 types of questions of the CSQA dataset. Especially for the “Logical Reasoning” category, our model improves by 7 F1 points. Furthermore, our results are achieved with 90.3% fewer parameters, allowing faster training for large-scale datasets.

## introduction knowledge base question answering (kbqa) has emerged as an important research topic over the past few years @xcite @xcite alongside with question answering over text corpora. in kbqa, world knowledge is given in the form of multi-relational graph databases * equal contribution @xcite with millions of entities and interrelations between them. when a natural language question arrives, kbqa systems analyse relevant facts in the knowledge bases and derive the answers. in the presence of knowledge bases, question answering results are often time more interpretable and modifiable. for example, the question "who started his career at manchester united in 1992?" can be answered by fact triples such as ("david beckham", member of sports team, "manchester united"). this fact can be updated as the world knowledge changes while it might be non-trivial to achieve the same effect on text corpora. likewise, kbqa systems face their own challenges @xcite , especially in the real-world, conversational settings. in real-world settings, kbqa systems need to perform multi-hop reasoning over chains of supporting facts and carry out various operations within the context of a conversation. for instance, answering the follow up question "when did he win his first championship?" might require identifying the player previously mentioned, all of his sport teams, the dates the sport teams won their championships. then, argmax and filter operators are applied on the returned dates, yielding answers, i.e., "1999" for "david beckham". semantic parsing provides a weak supervision framework to learn to perform all these reasoning steps from just the question answer pairs. semantic parsers define a set of rules (or grammar) for generating logical forms from natural language questions. candidate logical forms are executable queries on the knowledge bases that yield the corresponding answers. neural semantic parsers (nsps) @xcite @xcite employ a neural network to translate natural language questions into logical forms. nsps have shown good performance on kbqa tasks @xcite and further improved with reinforcement learning @xcite , multitask learning @xcite , and most recently meta-learning @xcite . most previous works place more emphasis on modeling the reasoning behavior given in the questions than on interactions with the kb. in this work, we propose a kb-aware nsp variant (kisp) to fill in this gap. one of the main challenges in learning kbqa systems is to adapt to structural changes of the relevant sub-knowledge base. different reasoning behaviors might apply to similar questions with respect to different sub-knowledge bases. for example, a similar question "when did tiger woods win his first championship?" would require a different reasoning chain since he didn't participate in a sports team. structural changes of the sub-kb is a common phenomenon due to the incompleteness nature of knowledge bases. in such cases, knowing the attributes and relations would inform nsps with changes in logical forms with respect to specific relevant kb entities. to address this problem, we propose a nsps with a kb-informed decoder that utilizes local knowledge base structure encoded in pre-trained kb embeddings. our model collects all relevant kb artifacts and integrates their embeddings into each decoding step, iteratively. we also introduce an attention layer on a set of associated kb random walks as an k-steps look ahead that prevents the decoder from going into kb regions where generated logical forms are not executable. pre-trained kb embeddings were shown to improve multi-hop kbqa where answers are entities and no operations are involved @xcite . in this paper, we demonstrate our work on the full kbqa settings with 10 question categories with no constraints on the answers @xcite . while @xcite evaluates 2-hop questions @xcite and 2 and 3-hop questions with limited relation types @xcite . our model is also the first nsp variant that utilizes pre-trained features for logical forms generation. carton @xcite uses an updated action grammar with stacked pointer networks. lasagne @xcite is an extension of carton which further includes a graph attention network to exploit correlations between entities, predicates. empirical results showed that our model improves upon the masp model @xcite , a strong baseline for csqa dataset, by an absolute 1.8 f1, 1.5% accuracy two sets of questions respectively. further, we find that by incorporating knowledge-graph information we can match the performance of much larger pre-trained encoder models while using 90.3% fewer parameters. ## background we first formally describe our task and the neural semantic parser (nsp) on which our work is based. knowledge graph: let @xmath0...e n } be a set of given entities, and let @xmath1...r m } be a set of relations. a knowledge graph g is a set of fact triples in e × r × e. a triple is represented as (h, r, t) where h, @xmath2. there is an extensive literature on representing the knowledge graph @xcite ) that encode its semantics and structures. in this work, we use the pre-trained knowledge graph embeddings from pytorch-biggraph @xcite . ## knowledge-informed decoder in this section, we introduce a knowledge-informed decoder that utilizes kg information to generate logical forms. we propose a knowledge injection layer that incorporates kg embeddings into the decoder state at each decoding step. to further inform the decoder with information about the expected structure of the kg, we propose an attention layer on random, k-hops knowledge walks from entities we encounter at each decoding step. ## conclusion we introduced a neural semantic parsing decoder that uses additional knowledge graph information for conversational qa. results show that kisp can significantly boost performance in complex multihop question types like logical reasoning questions. our method can help improve over strong baseline methods like masp. finally we presented a smaller version of our model that is approx 10x smaller without any performance degradation compared to a system that doesn't use kg informed decoding. victor zhong, caiming xiong, and richard socher.