Entity Linking via Explicit Mention-Mention Coreference Modeling (2022)

Abstract: Learning representations of entity mentions is a core component of modern entity linking systems for both candidate generation and making linking predictions. In this paper, we present and empirically analyze a novel training approach for learning mention and entity representations that is based on building minimum spanning arborescences (i.e., directed spanning trees) over mentions and entities across documents to explicitly model mention coreference relationships. We demonstrate the efficacy of our approach by showing significant improvements in both candidate generation recall and linking accuracy on the Zero-Shot Entity Linking dataset and MedMentions, the largest publicly available biomedical dataset. In addition, we show that our improvements in candidate generation yield higher quality re-ranking models downstream, setting a new SOTA result in linking accuracy on MedMentions. Finally, we demonstrate that our improved mention representations are also effective for the discovery of new entities via cross-document coreference.

## introduction natural language corpora, such as biomedical research papers @xcite , news articles @xcite , and, more generally, web page text @xcite , often contain ambiguous mentions of entities. resolving this ambiguity requires mentions to either be linked to a knowledge base (kb) of entities or discovered as a new kb concept if no suitable entry exists. grounded entity mentions are beneficial for tasks such as question-answering @xcite , semantic search @xcite , recommendation ranking @xcite , and kb construction @xcite . the task is made particularly challenging in zero-shot settings, where not every entity has labeled training data @xcite . in such settings, a common approach is to make use of entity descriptions, types, and aliases to form entity representations, which can then be used for making predictions. learned vector representations of entity mentions are an integral part of modern linking systems @xcite . these representations are used for (a) retrieving a short-list of entity candidates for a mention to use with a re-ranker @xcite , (b) making linking predictions directly @xcite @xcite , and (c) performing coreference by clustering mentions to form entities @xcite . in this work, we present a new objective and training procedure for learning mention and entity representations that explicitly model mention coreference relationships. our proposed method uses a supervised clustering training objective based on forming a directed minimum spanning tree, or arborescence, over mentions and entities. we hypothesize that such coreference links provide a useful inductive bias because the two tasks are inherently related @xcite . we thoroughly analyze the performance of the proposed procedure in each of the aforementioned use cases on medmentions @xcite and zeshel @xcite , two challenging datasets that require zero-shot generalization at inference. ## arborescence-based training for mention & entity representations in this section, we describe our approach for constructing training objectives for dual-encoders that model mention coreference relationships. ## experiments we are interested in investigating the following empirical research questions: experiment details our experiments are run on top of blink @xcite , a pytorch @xcite implementation of dual-and crossencoder architectures for entity linking, with model fine-tuning performed over only bert-base, since gains from pre-trained lm size are unrelated to our approach. more details are provided in appendix §a.1. ## conclusion we present a novel approach for learning mention and entity representations for use in entity linking candidate generation and prediction, as well as in the discovery of new entities. our method uses an objective that explicitly incorporates mention-tomention coreference relationships. we demonstrate its empirical effectiveness through analysis on two datasets -medmentions and the zero-shot entity linking dataset. as future work, we hope to further analyze these objectives with the lens of efficiency, distillation, and domain transfer.