Efficient One-Pass End-to-End Entity Linking for Questions (2020)

Abstract: We present ELQ, a fast end-to-end entity linking model for questions, which uses a biencoder to jointly perform mention detection and linking in one pass. Evaluated on WebQSP and GraphQuestions with extended annotations that cover multiple entities per question, ELQ outperforms the previous state of the art by a large margin of +12.7% and +19.6% F1, respectively. With a very fast inference time (1.57 examples/s on a single CPU), ELQ can be useful for downstream question answering systems. In a proof-of-concept experiment, we demonstrate that using ELQ significantly improves the downstream QA performance of GraphRetriever.

## introduction entity linking (el), the task of identifying entities and mapping them to the correct entries in a database, is crucial for analyzing factoid questions and for building robust question answering (qa) systems. for instance, the question "when did shaq come to the nba?" can be answered by examining shaquille o'neal's wikipedia article @xcite , or its properties in a knowledge graph @xcite . however, real-world user questions are invariably noisy and ill-formed, lacking cues provided by casing and punctuation, which prove challenging to current end-to-end entity linking systems @xcite . while recent pre-trained models have proven highly effective for entity linking @xcite , they are only designed for entity disambiguation and require mention boundaries to be given in the input. additionally, such systems figure 1 : overview of our end-to-end entity linking system. we separately encode the question and entity. we use the question representations to jointly detect mentions and score candidate entities through innerproduct with the entity vector. in this work, we propose elq, a fast and accurate entity linking system that specifically targets questions. following the wikification setup @xcite , elq aims to identify the mention boundaries of entities in a given question and their corresponding wikipedia entity. we employ a biencoder based on bert @xcite as shown in figure 1 . the entity encoder computes entity embeddings for all entities in wikipedia, using their short descriptions. then, the question encoder derives token-level embeddings for the input question. we detect mention boundaries using these embeddings, and disambiguate each entity mention based on an inner product between the mention embeddings (averaged embedding over mention tokens) and the entity embeddings. our model ex-tends the work of @xcite but with one major difference: our system does not require prespecified mention boundaries in the input, and is able to jointly perform mention detection and entity disambiguation in just one pass of bert. thus, at inference time, we are able to identify multiple entities in the input question efficiently. we extend entity disambiguation annotations from @xcite to create an endto-end question entity linking benchmark. evaluated on this benchmark, we are able to outperform previous methods in both accuracy and run-time. elq has much faster end-to-end inference time than any other neural baseline (by 2×), while being more accurate than all previous models we evaluate against, suggesting that it is practically useful for downstream qa systems. we verify the applicability of elq to practical qa models in a proofof-concept experiment, by augmenting graphretriever @xcite to use our model, improving its downstream qa performance on three open-domain qa datasets (by up to 6%). ## related work much prior work on entity linking has focused on long, grammatically coherent documents that contain many entities. this setting does not accurately reflect the difficulties of entity linking on questions. while there has been some previous work on entity linking for questions @xcite @xcite , such works (mostly from the pre-bert era) utilize complex models with many interworking modules. for example, @xcite proposes a variable-context granularity (vcg) model to address the noise and lack of context in questions, which incorporates signals from various levels of granularity by using character-level, token-level, and knowledge-baselevel modules. they also rely on external systems as a part of the modeling pipeline. in this work, we take a much simpler approach that uses a biencoder. biencoder models have been used in a wide range of tasks @xcite @xcite . they enable fast inference time through maximum inner product search. moreover, as we find, biencoders can be decomposed into reusable question and entity encoders, and we can greatly expedite training by training one component independently of the other. ## problem definition & elq model we formally define our entity linking task as follows. given a question q and a set of entities @xmath0, each with titles t(e i ) and text descriptions d(e i ), our goal is to output a list of tuples, (e, [m s , m e ]), whereby @xmath1. in practice, we take the title and first 128 tokens of the entity's wikipedia article as its title t(e i ) and description d(e i ). we propose an end-to-end entity linking system that performs both mention detection and entity disambiguation on questions in one pass of bert. given an input question @xmath2, we first obtain question token representations based on bert @xcite : ## qa experiments to demonstrate the impact of improved entity linking on the end qa accuracy, we experiment with the task of textual open-domain question answering, using graphretriever (gretriever) @xcite . gretriever uses entity linking to construct a graph of passages in the retrieval step and deploys experiment elq (wiki) blink md + el 87.3 82.2 md only 94.6 92.9 el only 90.2 86.6 a reader model to answer the question. the original model uses tagme for entity linking; we replace tagme with elq and keep the other components the same, in order to isolate the impact of entity linking. 4 as an additional baseline, we also add the result of tf-idf, implemented by @xcite , a widely used retrieval system. results are shown in ## analysis mention detector vs. entity linker we set up experiments to disentangle the capability of elq's entity linker and mention detector. first, to test just the mention detector (md only), we measure just the mention boundary overlap between predicted and groundtruth mentions, ignoring the entity label. next, to test just the entity linker (el only), we give the entity linking component gold mention boundaries, and compute the resulting f1 score. we do this for both elq and blink. for comparability, we use the version of elq trained on wikipedia. results are reported in runtime to confirm that our biencoder's main bottleneck is the bert forward pass -and thus, investing in decreasing the number of bert forward passes is valuable -we separately time each component of elq during inference. we run examples from webqsp el test set one at a time through elq, on 1 cpu, and average runtimes across all examples. indeed, we find that the bert forward pass to be the slowest component of the model, taking 0.683s, over 6× slower than the next slowest component of the model, inner-product search (taking 0.107s). everything else takes a combined total of 5.08 × 10 -3 s. qualitative we manually examine all our model's errors on the webqsp el and graphq el dev sets. we identify four broad error categories: ## conclusion we proposed an end-to-end model for entity linking on questions that jointly performs mention detection and disambiguation with one pass through bert. we showed that it is highly efficient, and that it outperforms previous state-of-the-art models on two benchmarks. furthermore, when applied to a qa model, elq improves that model's end qa accuracy. despite being originally designed with questions in mind, we believe elq could also generalize to longer, well-formed documents.