Contextualized End-to-End Neural Entity Linking (2020)

Abstract: We propose an entity linking (EL) model that jointly learns mention detection (MD) and entity disambiguation (ED). Our model applies task-specific heads on top of shared BERT contextualized embeddings. We achieve state-of-the-art results across a standard EL dataset using our model; we also study our model’s performance under the setting when hand-crafted entity candidate sets are not available and find that the model performs well under such a setting too.

## introduction entity linking (el) 1 , in our context, refers to the joint task of recognizing named entity mentions in text through mention detection (md) and linking each mention to a unique entity in a knowledge base (kb) through entity disambiguation (ed) 2 . for example, in the sentence "the times began publication under its current name in 1788," the span the times should be detected as a named entity mention and then linked to the corresponding entity: the times, a british newspaper. however, an el model which disjointly applies md and ed might easily mistake this mention with the new york times, an american newspaper. since our model jointly learns md and ed from the same contextualized bert embeddings, its final el prediction is partially informed by both. as a result, it is able to generalize better. another common approach employed in previous el research is candidate generation, where for each detected mention, a set of candidate entities is generated and the entities within it are ranked by a model to find the best match. such sets are built using hand-crafted rules which define which entities make it in and which do not. this risks (1) skipping out on valid entities which should be in the candidate set and (2) inflating model performance since often times candidate sets contain only one or two items. these sets are almost always used at prediction time and sometimes even during training. our model has the option of not relying on them during prediction, and never uses them during training. we introduce two main contributions: ## related work neural-network based models have recently achieved strong results across standard el datasets. research has focused on learning better entity representations and extracting better local and global features through novel model architectures. entity representation. good kb entity representations are a key component of most ed and el models. representation learning has been addressed by @xcite , @xcite , @xcite and @xcite . @xcite and @xcite extend it to the cross-lingual setting. more recently, @xcite have suggested learning entity representations using bert which achieves state-of-the-art results in ed. ## model description given a document containing a sequence of n tokens @xmath0, ..., w n } with mention label indicators foot_0 y @xmath1, o, b} n and entity ids y @xmath2: @xmath3, k]} n which index a pre-trained entity embedding matrix @xmath4, the model is trained to tag each token with its correct mention indicator and link each mention with its correct entity id. ## conclusions and future work we propose an el model that jointly learns the md and ed task, achieving state-of-the-art results. we also show that training and inference without candidate sets is possible. we think that interesting future directions of study include a better understanding of how bert already comprehends entities in text without reference to external entity embeddings. finally, we think that moving forward, reducing the el community's dependence on candidate sets could be a good thing and requires more research. dropping candidate sets could make models more easily comparable.