M u VER : I mproving First-Stage Entity Retrieval with Multi-View Entity Representations (2021)

Abstract: Entity retrieval, which aims at disambiguating mentions to canonical entities from massive KBs, is essential for many tasks in natural language processing. Recent progress in entity retrieval shows that the dual-encoder structure is a powerful and efficient framework to nominate candidates if entities are only identified by descriptions. However, they ignore the property that meanings of entity mentions diverge in different contexts and are related to various portions of descriptions, which are treated equally in previous works. In this work, we propose Multi-View Entity Representations (MuVER), a novel approach for entity retrieval that constructs multi-view representations for entity descriptions and approximates the optimal view for mentions via a heuristic searching method. Our method achieves the state-of-the-art performance on ZESHEL and improves the quality of candidates on three standard Entity Linking datasets.

## introduction entity linking (el) refers to the task that disambiguates the mentions in textual input and retrieves the corresponding unique entity in large knowledge bases (kbs) @xcite . the majority of neural entity retrieval approaches consist of two steps: candidate generation @xcite , which nominates a small list of candidates from millions of entities with low-latency algorithms, and entity ranking @xcite @xcite , which ranks those candidates to select the best match with more sophisticated algorithms. in this paper, we focus on the candidate generation problem (a.k.a. the first-stage retrieval). prior works filter entities by alias tables @xcite or precalculated mention-entity prior probabilities, e.g., p(entity|mention) @xcite . @xcite and @xcite build entity embedding from the local context of hyperlinks in entity pages or entity-entity co-occurrences. those embedding-based methods were extended by blink @xcite and deer @xcite to two-tower dualencoders @xcite , which encode mentions and descriptions of entities into highdimensional vectors respectively. candidates are retrieved by nearest neighbor search @xcite for a given mention. solutions that require only entity descriptions @xcite are scalable, as descriptions are more readily obtainable than statistical or manually annotated resources. although description-based dual-encoders can compensate for the weakness of traditional methods and have better generalization ability to unseen domains, they aim to map mentions with divergent context to the same high-dimensional entity embedding. as shown in figure 1 , the description of "kobe bryant" mainly concentrates on his professional journey. as a result, the embedding of "kobe bryant" is close to the context which describes the career of kobe but is semantically distant from his helicopter accident. dual-encoders are trained to encode those semantically divergent contexts to representations that are close to the embedding of "kobe bryant". the evidence relies on the figure 2 (section 3.2) that the previous method @xcite ) is good at managing entities with short descriptions but seems troubling to retrieve entities with long descriptions, which contains too much information to be encoded into a single fixed-size vector. to tackle those issues, we propose to construct multi-view representations from descriptions. the contributions of our paper are as follows: description for entity 'kobe bryant' ... an nba basketball player bryant was the all-time leading scorer in lakers history. their second daughter, gainna maria-onore ("gigi") ... died in a helicopter crash ! " mention for entity 'kobe bryant' bryant would retire after 20 seasons with lakers all 9 people on the sikorsky s-76 were killed: bryant, his daughter gianna … 1 2 4 3 1 2 nth view for 'kobe bryant' views for negative entities n embedding of 'kobe bryant' 5 6 won five nba championships, was an 18-time all-star. we refer to each sentence as a view for descriptions to form a view set v (gray circles with number) and merge views to approximate the optimal views for mentions (points enclosed by ellipses). ## related work representing each entity with a fixed-sized vector has been a common approach in entity linking. @xcite defines a word-entity conditional distribution and samples positive words from it. the representations of those positive words aim to approximate the entity embeddings compared with random words. @xcite models the relatedness between entities into entity representations. ntee @xcite trains entity representations by predicting the relevant entities for a given context in dbpedia abstract corpus. ling et al. (2020) and yamada et al. (2020) pre-train variants of the transformer-based model by maximizing the consistency between the context of the mentions and the corresponding entities. those entity representations suffer from a cold-start problem that they cannot link mentions to unseen entities. another line of work is to generate entity representations using entity textual information, such as entity descriptions. @xcite introduces an el dataset in the zero-shot scenario to place more emphasis on reading entity descriptions. blink @xcite proposes a bi-encoder to encode the descriptions and enhance the bi-encoder by distilling the knowledge from the cross-encoder. @xcite repeats the position embedding to solve the long-range modeling problem in entity descriptions. @xcite demonstrates that hard negatives can enhance the contrast when training an el model. ## conclusion in this work, we propose a novel approach to construct multi-view representations from descriptions, which shows promising results on four el datasets. extensive results demonstrate the effectiveness of multi-view representations and the heuristic search strategy. in the future, we will explore more reliable and efficient approaches to construct views.