Improving Entity Linking by Encoding Type Information into Entity Embeddings (2021)

Abstract: Entity Linking (EL) refers to the task of linking entity mentions in the text to the correct entities inthe Knowledge Base (KB) in which entity embeddings play a vital and challenging role because of the subtle differences between entities. However existing pre-trained entity embeddings onlylearn the underlying semantic information in texts yet the fine-grained entity type informationis ignored which causes the type of the linked entity is incompatible with the mention context. In order to solve this problem we propose to encode fine-grained type information into entity embeddings. We firstly pre-train word vectors to inject type information by embedding wordsand fine-grained entity types into the same vector space. Then we retrain entity embeddings withword vectors containing fine-grained type information. By applying our entity embeddings to twoexisting EL models our method respectively achieves 0.82% and 0.42% improvement on average F1 score of the test sets. Meanwhile our method is model-irrelevant which means it can helpother EL models.

## introduction entity linking (el) refers to the task of linking entity mentions in the text to correct entities in the knowledge base (kb) to help text comprehension by making use of rich semantic information in kb. the state-of-the-art entity linking models ignore fine-grained entity type information which causes the type of the linked entity is incompatible with the mention context. for example, among the error cases in aida-conll @xcite dataset produced by @xcite , the mention "japan" in the sentence "japan began the defense of their asian cup" is linked to the entity "japan"(country) while it should be linked to the entity "japan national football team"(sports team). and such errors account for more than 1/2 of the total errors which show that their model cannot correctly distinguish different types of candidate entities. entity embeddings are especially important to entity linking task for the sake of distinguishing entities with similar characters. we hope the entity embeddings can have type information so as to differentiate entities in terms of categories. however, previous pre-trained entity embeddings are learned by using plenty of texts, which ignores fine-grained type information @xcite . as a result, the similar entity vectors in the vector space are of different types. in order to alleviate this problem, we firstly pre-train word vectors to inject type information by embedding words and fine-grained entity types into the same vector space. then, we follow @xcite to retrain the representation of each entity by making it closer to the word vectors of its context, using the word vectors containing type information. finally, we apply the entity embeddings into the existing el models without modifying the model architectures. compared with two baseline models, our method respectively achieves 0.82% and 0.42% improvement on average f1 score of the test sets and obtains new state-of-the-art results on multiple datasets. besides, detailed experimental analysis confirms the hypothesis that encoding type information in entity embeddings can reduce the type error cases. our contributions can be summarized as follows: ## our method our method mainly consists of four steps: 2 ) are also fused with the word vectors they use in the same way. finally, we apply the result word and entity embeddings to the existing entity linking models. the overall architecture of our method is illustrated in figure 1 . ## conclusion this paper proposes a simple yet effective method to encode fine-grained entity type information into the entity embeddings, enabling models to better capture the fine-grained entity type information. we obtain solid improvement compared with the previous models and the detailed experimental analysis shows our method can correct a fair portion of type error cases produced by the baseline model. moreover, our method doesn't need to modify the el model architecture, thus can be easily applied to other el models. in the future, we will consider injecting the entity information and fine-grained type information into c c l 2 0 2 1 the latest pre-trained language model (e.g., bert @xcite ) to obtain entity representation and further improve the performance of entity linking.