MOLEMAN : Mention-Only Linking of Entities with a Mention Annotation Network (2021)

Abstract: We present an instance-based nearest neighbor approach to entity linking. In contrast to most prior entity retrieval systems which represent each entity with a single vector, we build a contextualized mention-encoder that learns to place similar mentions of the same entity closer in vector space than mentions of different entities. This approach allows all mentions of an entity to serve as “class prototypes” as inference involves retrieving from the full set of labeled entity mentions in the training set and applying the nearest mention neighbor’s entity label. Our model is trained on a large multilingual corpus of mention pairs derived from Wikipedia hyperlinks, and performs nearest neighbor inference on an index of 700 million mentions. It is simpler to train, gives more interpretable predictions, and outperforms all other systems on two multilingual entity linking benchmarks.

## introduction a contemporary approach to entity linking represents each entity with a textual description d e , encodes these descriptions and contextualized mentions of entities, m, into a shared vector space using dual-encoders f (m) and g(d e ), and scores each mention-entity pair as the inner-product between their encodings @xcite . by restricting the interaction between e and m to an inner-product, this approach permits the pre-computation of all g(d e ) and fast retrieval of top scoring entities using maximum inner-product search (mips). here we begin with the observation that many entities appear in diverse contexts, which may not be easily captured in a single high-level description. for example, actor tommy lee jones played football in college, but this fact is not captured in the entity description derived from his wikipedia page (see figure 1 ). furthermore, when new entities need to be added to the index in a zero-shot setting, it may be difficult to obtain a high quality description. we propose that both problems can be solved by allowing the entity mentions themselves to serve as exemplars. in addition, retrieving from the set of mentions can result in more interpretable predictions -since we are directly comparing two mentions -and allows us to leverage massively multilingual training data more easily, without forcing choices about which language(s) to use for the entity descriptions. we present a new approach (moleman foot_0 ) that maintains the dual-encoder architecture, but with the same mention-encoder on both sides. entity linking is modeled entirely as a mapping between mentions, where inference involves a nearest neighbor search against all known mentions of all entities in the training set. we build moleman using exactly the same mention-encoder architecture and training data as model f @xcite . we show that moleman significantly outperforms model f on both the mewsli-9 and @xcite datasets, particularly for low-coverage languages, and rarer entities. we also observe that moleman achieves high accuracy with just a few mentions for each entity, suggesting that new entities can be added or existing entities can be modified simply by labeling a small number of new mentions. we expect this update mechanism to be significantly more flexible than writing or editing entity descriptions. finally, we compare the massively multilingual moleman model to a much more expensive english-only dualencoder architecture @xcite on the wellstudied tackbp-2010 dataset @xcite and show that moleman is competitive even in this setting. figure 1 : illustration of hypothetical contextualized mention (m) and multilingual description (d) embeddings for the entities 'tommy lee jones (q170587)' and 'tom jones (q18152778). the query mention [ ] pertains to the former's college football career, which is unlikely to be captured by the high-level entity description. a retrieval against descriptions would get this query incorrect, but with indexed mentions gets it correct. note that prior dual-encoder models that use a single vector to represent each entity are forced to contort the embedding space to solve this problem. our goal is to learn a function φ(m) that maps an arbitrary mention-in-context token sequence m to a fixed vector h @xmath0 ## discussion and future work we have recast the entity linking problem as an application of a more generic mention encoding task. this approach is related to methods which perform clustering on test mentions in order to improve inference @xcite , and can also be viewed as a form of crossdocument coreference resolution @xcite @xcite . we also take inspiration from recent instance-based language modelling approaches @xcite . our experiments demonstrate that taking an instance-based approach to entity-linking leads to better retrieval performance, particularly on rare entities, for which adding a small number of mentions leads to superior performance than a single description. for future work, we would like to explore the application of this instance-based approach to entity knowledge related tasks @xcite , and to entity discovery @xcite .