End-to-End Entity Resolution and Question Answering Using Differentiable Knowledge Graphs (2021)

Abstract: Recently, end-to-end (E2E) trained models for question answering over knowledge graphs (KGQA) have delivered promising results using only a weakly supervised dataset. However, these models are trained and evaluated in a setting where hand-annotated question entities are supplied to the model, leaving the important and non-trivial task of entity resolution (ER) outside the scope of E2E learning. In this work, we extend the boundaries of E2E learning for KGQA to include the training of an ER component. Our model only needs the question text and the answer entities to train, and delivers a stand-alone QA model that does not require an additional ER component to be supplied during runtime. Our approach is fully differentiable, thanks to its reliance on a recent method for building differentiable KGs (Cohen et al., 2020). We evaluate our E2E trained model on two public datasets and show that it comes close to baseline models that use hand-annotated entities.

## introduction the conventional approach for question answering using a knowledge graph (kgqa) involves a set of loosely connected components; notably, an entity resolution component identifies entities mentioned in the question, and a semantic parsing component produces a structured representation of the question. the programs resulting from combining these components can be executed on a knowledge graph (kg) engine to retrieve the answers. while this approach can be effective, collecting training datasets for individual components can be challenging @xcite . for example, supervised semantic parsing requires training data pairing naturallanguage questions with structured queries, which is difficult to obtain. this has motivated many efforts in weakly supervised training @xcite . following recent breakthroughs in figure 1 : high-level architecture of the end-to-end model. one forward pass of roberta extracts contextual embeddings for all components. span detection and entity resolution happen jointly to derive seed entities vector x 0 . the inference module performs multihop reasoning to reach answer entities vector ŷ machine translation @xcite , a new goal is to directly optimize the entire chain of components end-to-end, without the need for intermediate annotations. however, entity resolution (er) is by and large a neglected component of e2e learning, and existing weakly supervised solutions mostly assume question entities are either given or extracted by an external system. in practice, there's a scarcity of quality training data for er on questions, and poor entity extraction by out-of-domain models affects the overall performance of a kgqa system @xcite . in this work, we present an end-to-end model for kgqa that learns to jointly perform entity resolution and inference. our work leverages the differentiable kg proposed in @xcite , which allows all the components of our model to be trained using a dataset of only questions and answers. this eliminates the need for labelled er data for questions and allows our model to run independently, without relying on external components. furthermore, the tight integration of er into our solution allows uncertainties about entities to be directly reflected in our confidence in answers. ## related work traditional approaches to kgqa rely on semantic parsing (sp) to translate natural language into a logical form. weakly supervised sp is a wellstudied topic with increasing interest in applying reinforcement learning (rl) @xcite . er is rarely considered in the scope of surveyed solutions, and if it is, it's treated as an independent component and not included in the weak supervision scope @xcite . in general, rl algorithms for qa are hard to tune and have large variances in their results. the exploration-exploitation issue also lets models settle on high-reward but spurious logical forms, leading to poor generalization @xcite . including er with a discrete output space as part of an e2e rl pipeline will further add to the challenges that rl-based solutions face. another line of work in kgqa uses embedding techniques to implicitly infer answers from knowledge graphs without explicit queries @xcite . while these embeddingbased approaches perform well, they are memory intensive and difficult to scale to large knowledge graphs. in addition, when new entities are added to the kg, they need to be retrained to learn updated embeddings. the differentiable kg we use in this work can incorporate new entities without affecting trained models, and can scale to billions of entities via horizontal scaling @xcite . the few relevant works on entity resolution for questions utilize complex models with many interworking modules (e.g. @xcite . elq @xcite ) is a more recent effort and simplifies the process by relying on a bi-encoder to jointly perform span detection and er in a multi-task setup. however, these solutions rely on direct supervision. our proposed method eliminates the need for labelled data for er. in fact, the weakly supervised er model presented here could be detached and used as a standalone er module after training. ## experiments we call our model rigel and evaluate 3 versions at different e2e learning boundaries. the baseline model , rigel-baseline, is given gold entities and no entity resolution is involved, demonstrating the performance of the inference module alone. rigel-er is given the gold spans, but still has to learn to disambiguate between candidate entities for that span. finally, in rigel-e2e, we provide the question text only, requiring the model to attend to the right span and disambiguate between candidates for each span. ## conclusion in this work, we proposed a solution for kgqa that jointly learns to perform entity resolution (er) and multi-hop inference. our model extends the boundaries for end-to-end learning and is weakly supervised using pairs of only questions and answers. this eliminates the need for external components and expensive domain-specific labelled data for er. we further demonstrate the feasibility of this approach on two open-domain qa datasets.