Improving Entity Linking through Semantic Reinforced Entity Embeddings (2020)

Abstract: Entity embeddings, which represent different aspects of each entity with a single vector like word embeddings, are a key component of neural entity linking models. Existing entity embeddings are learned from canonical Wikipedia articles and local contexts surrounding target entities. Such entity embeddings are effective, but too distinctive for linking models to learn contextual commonality. We propose a simple yet effective method, FGS2EE, to inject fine-grained semantic information into entity embeddings to reduce the distinctiveness and facilitate the learning of contextual commonality. FGS2EE first uses the embeddings of semantic type words to generate semantic embeddings, and then combines them with existing entity embeddings through linear aggregation. Extensive experiments show the effectiveness of such embeddings. Based on our entity embeddings, we achieved new sate-of-the-art performance on entity linking.

## introduction entity linking (el) or named entity disambiguation (ned) is to automatically resolve the ambiguity of entity mentions in natural language by linking them to concrete entities in a knowledge base (kb). for example, in figure 1 , mentions "congress" and "mr. mueller" are linked to the corresponding wikipedia entries, respectively. neural entity linking models use local and global scores to rank and select a set of entities for mentions in a document. entity embeddings are critical for the local and global score functions. but the current entity embeddings @xcite encoded too many details of entities, thus are too distinctive for linking models to learn contextual commonality. we hypothesize that fine-grained semantic types of entities can let the linking models learn contextual commonality about semantic relatedness. for example, rugby related documents would have entities of rugby player and rugby team. if a linking model learns the contextual commonality of rugby related entities, it can correctly select entities of similar types using the similar contextual information. in this paper, we propose a method fgs2ee to inject fine-grained semantic information into entity embeddings to reduce the distinctiveness and facilitate the learning of contextual commonality. fgs2ee uses the word embeddings of semantic words that represent the hallmarks of entities (e.g., writer, carmaker) to generate semantic embeddings. we find that the training converges faster when using semantic reinforced entity embeddings. our proposed fgs2ee consists of four steps: ## extracting fine-grained semantic types we first create a dictionary of fine-grained semantic types, then we extract fine-grained types for each entity. ## fgs2ee: injecting fine-grained semantic information into entity embeddings fgs2ee first uses semantic words of each entity to generate semantic entity embeddings, then combine them with existing entity embeddings to generate semantic reinforced entity embeddings. ## conclusion in this paper, we presented a simple yet effective method, fgs2ee, to inject fine-grained semantic information into entity embeddings to reduce the distinctiveness and facilitate the learning of contextual commonality. fgs2ee first uses the word embeddings of semantic type words to generate semantic embeddings, and then combines them with existing entity embeddings through linear aggregation. our entity embeddings draw entities of similar types closer, while entities of different types are drawn further. thus can facilitate the learning of semantic commonalities about entity-context and entity-entity relations. we have achieved new state-of-the-art performance using our entity embeddings. for the future work, we are planning to extract fine-grained semantic types from unlabelled documents and use the relatedness between the finegrained types and contexts as distant supervision for entity linking.