Learn to Not Link: Exploring NIL Prediction in Entity Linking (2023)

Abstract: Entity linking models have achieved significant success via utilizing pretrained language models to capture semantic features. However, the NIL prediction problem, which aims to identify mentions without a corresponding entity in the knowledge base, has received insufficient attention. We categorize mentions linking to NIL into Missing Entity and Non-Entity Phrase, and propose an entity linking dataset NEL that focuses on the NIL prediction problem.NEL takes ambiguous entities as seeds, collects relevant mention context in the Wikipedia corpus, and ensures the presence of mentions linking to NIL by human annotation and entity masking. We conduct a series of experiments with the widely used bi-encoder and cross-encoder entity linking models, results show that both types of NIL mentions in training data have a significant influence on the accuracy of NIL prediction. Our code and dataset can be accessed at https://github.com/solitaryzero/NIL_EL .

## introduction entity linking (el) aims to map entity mentions in free texts to their corresponding entities in a given knowledge base (kb). entity linking acts as a bridge between unstructured text and structured knowledge, and benefits various downstream tasks like question answering @xcite and knowledge extraction @xcite . however, not all entity mentions correspond to a specific entity in the kb that suits the mention context @xcite . take there have been some earlier representation learning based researches that take nil prediction into consideration @xcite @xcite . they identify nil mentions by setting a vector similarity threshold or viewing nil as a special entity. recently, pretrained language model (plm) based models @xcite @xcite have achieved great success for their great transferability and expandability. however, these models generally assume that there always exists a correct entity for each mention in the knowledge base, which leaves the nil prediction problem without adequate attention. previous entity linking datasets have paid insufficient attention to the nil prediction problem. for example, some of the previous datasets like aida @xcite view it as an auxiliary task, while others like msnbc @xcite and wned-wiki @xcite does not require nil prediction at all. there does not yet exist a strong benchmark for the ability on nil prediction. in this paper, we propose an entity linking dataset nel that focuses on the nil prediction problem. about 30% of the mentions in nel do not have their corresponding entity in the candidate entity set, which requires models to identify these mentions rather than linking them to the wrong candidates. in nel construction, we take ambiguous entities as seeds, and build the dataset by mining mention contexts related to seed entities on the wikipedia corpus. then, human annotators are missing entity mention context eu rejects german call to boycott british lamb. peter blackburn brus-sels 1996-08-22 peter blackburn (bishop) peter blackburn (d.1616) was a scottish scholar and prelate. he was the second protestant bishop of aberdeen. peter blackburn (mp) peter blackburn (1811 -20 may 1870) was a british conservative party politician. peter blackburn (badminton) peter grant blackburn (born 25 march 1968) is an australian badminton player who affiliated with the ballarat badminton association. ## preliminary entity mentions @xmath0, w 2 , ..., w n ), where w i is either a plain token or a mention. each mention m i may correspond to an entity e @xmath1. definition 1. entity linking aims to find an optimal mapping γ : m ⇒ e, which maps entity mentions @xmath2, where e @xmath3. the nil prediction problem is to determine whether an entity mention m is absent from the knowledge base kb. when there does not exist a proper entity @xmath4, ## dataset construction there does not yet exist a strong benchmark on nil prediction. we manually annotated 300 examples with their mentions linking to nil from the widely used entity linking dataset aida 1 , and discover that about 10% of these mentions should actually link to an entity. for example, the mention "eu" in "eu rejects german" should be linked to the european union rather than nil (see appendix d for details). meanwhile, nil mentions in aida fall mostly in the missing entity category. the incorrect and imbalanced data for nil prediction indicates that the importance of nil prediction is currently underestimated. in this section, we propose an entity linking dataset named nel, which focuses on the nil prediction problem. the construction process is demonstrated in figure 1 . unlike normal entity linking data, there does not exist annotated references for mentions linking to nil, and the portion of nil data in the text corpus is unknown. hyperlinks in wikipedia can be viewed as mentions linking to non-nil entities, from which we can find the aliases of entities. we assume that when an alias does not appear as a hyperlink in a certain context, it may be identified as a mention linking to nil. in this way, we collect such contexts as the raw data. the raw data is then annotated by humans to ensure correctness, and we further mask out some answer entities in the candidate set to control the percentage of nil in answers. @xcite states that the title of creative works could be a place, a personal name, or a certain abstract concept like the choric embodiment of some collectivity (the clouds, the birds) or stock types (the alchemist, le misanthrope), which would naturally lead to the two situations where a mention links to nil. the absence of the referenced entity from the kb would lead to missing entity, while an abstract concept not viewed as an entity would lead to non-entity phrase. ## entity linking with nil prediction a mention links to nil when all of its candidate entities fail to match its context. a common paradigm of nil prediction is to compute the similarity scores between the mention context and candidates, and judge its linking result on the base of similarities. ## conclusion in this paper, we propose an entity linking dataset nel that focuses on the nil prediction problem. we observe that mentions linking to nil can be categorized into two patterns: missing entity and non-entity phrase, but the latter one has not been paid sufficient attention. we propose an entity linking dataset nel that focuses on nil prediction. the dataset is built upon the wikipedia corpus by choosing ambiguous entities as seeds and collecting relevant mention contexts. nel is humanannotated to ensure correctness, and entity masking is further performed to control the percentage of nil. we conducted a series of experiments to examine the performance of plm-based models on different datasets. experimental results indicate that the accuracy without considering nil prediction would be inflated. meanwhile, sufficient data of both nil types during training is essential to trigger the ability of nil prediction. in the future, we may further try to integrate entity types into the pretraining process and explore type transfer between datasets.