Improving Zero-Shot Entity Linking Candidate Generation with Ultra-Fine Entity Type Information (2022)

Abstract: Entity linking, which aims at aligning ambiguous entity mentions to their referent entities in a knowledge base, plays a key role in multiple natural language processing tasks. Recently, zero-shot entity linking task has become a research hotspot, which links mentions to unseen entities to challenge the generalization ability. For this task, the training set and test set are from different domains, and thus entity linking models tend to be overfitting due to the tendency of memorizing the properties of entities that appear frequently in the training set. We argue that general ultra-fine-grained type information can help the linking models to learn contextual commonality and improve their generalization ability to tackle the overfitting problem. However, in the zero-shot entity linking setting, any type information is not available and entities are only identified by textual descriptions. Thus, we first extract the ultra-fine entity type information from the entity textual descriptions. Then, we propose a hierarchical multi-task model to improve the high-level zero-shot entity linking candidate generation task by utilizing the entity typing task as an auxiliary low-level task, which introduces extracted ultra-fine type information into the candidate generation task. Experimental results demonstrate the effectiveness of utilizing the ultra-fine entity type information and our proposed method achieves state-of-the-art performance.

## introduction entity linking (el) is the task of assigning entity mentions in a text to corresponding entity records in a reference knowledge base. el plays a key role in the language understanding pipeline, underlying a variety of downstream applications, such as information extraction @xcite , semantic search @xcite and question answering @xcite @xcite . in general, el consists of two phases: candidate generation which generates a set of candidates for each mention from millions of entities, and candidate ranking which retrieves the matched entity for each mention from the set of candidates. as the final results in el are only generated from candidta sets, the accuracy of the whole el task is limited by the candidate generation phase. therefore, in this paper, we focus on the candidate generation phase to set a higher upper bound on el accuracy. traditional el approaches usually train models under the setting that linked entities in the test set are available in the training set. however, in many real-world scenarios, labeled data are not easily obtained in multiple domains. thus, there is a need for el models to have the capability of generalizing to new domains and new entities. to challenge the generalization ability, a zero-shot entity linking task @xcite has been proposed, where mentions need to be linked to unseen entities and only the textual information is available. for this task, the training and test sets share different distributions of entities, and thus entity linking models tend to be overfitting due to the tendency of memorizing the properties of entities that appear frequently in the training set. we argue that general ultra-fine-grained type information can help the linking models learn contextual commonality and improve their generalization ability to tackle the overfitting problem. if a linking model learns the contextual commonality of athlete related entities, it can use similar contextual information to correctly select entities of the same type. examples of entity linking with general ultra-finegrained entity type information are shown in figure 1 . a key observation is that the given ultra-fine entity types could have positive effect on the entity linking task. in this example, the type information can help the linking model link the michael in the text to michael jordan [actor, athlete, player, basketball player] instead of michael jackson @xcite . therefore, in this paper, we try to introduce ultrafine entity type information into the zero-shot entity linking candidate generation. however, in the zeroshot entity linking setting, any type information is not available and entities are only identified by textual descriptions. thus, we first extract ultra-fine types from textual descriptions of each entity. in general, more fine-grained entity type information can better help the linking models learn contextual commonality and improve their generalization ability. thus, we train an entity typing model by utilizing the ultra-fine entity typing dataset @xcite whose types are more fine-grained rather than other fine-grained entity typing datasets (e.g. figer @xcite and ontonotes @xcite ), and use the model to extract ultra-fine types from each entity textual description. then, we propose a hierarchical multi-task model, which jointly models the candidate generation task and ultra-fine entity typing task to introduce the type information extracted by the trained typing model into the candidate generation phase. the ultra-fine entity typing task is utilized as an auxiliary low-level task, providing corresponding type features for the high-level candidate generation task. our primary motivation is to discover helpful training signals from ultra-fine-grained type information to ensure a more robust zero-shot entity linking candidate generation model. to summarize, our major contributions are shown as follows: ## model figure 2 shows our proposed two-stage model, which consists of two parts: the entity typing model and the candidate generation model. in this section, we describe these two models in detail. ## experiments in this section, we compare our proposed method to other state-of-the-art methods to demonstrate the effectiveness of our model. we first introduce the datasets we used and the implementation details of our model. then we briefly introduce the baselines and present the overall performance of our model in comparison with others. ## conclusion in this paper, we focus on the zero-shot entity linking task, which links mentions to unseen entities and only the textual information is available. this task challenges the generalization ability and often leads to a tendency of overfitting for entity linking models. to tackle the problem, we introduce the ultra-fine entity type information into the candidate generation phase of this task. considering only entity description is available, we propose a two-stage model. we first extract ultra-fine entity types from each entity description, without depending on additional manually annotated data. then we present a hierarchical multi-task model for jointly modeling candidate generation and ultra-fine entity typing, which can help the model to learn contextual commonality of types about the gold entity to improve the generalization ability. the experimental results demonstrate the effectiveness of utilizing the ultra-fine entity type information and our proposed method achieves state-of-the-art performance.