N amed E ntity R ecognition for E ntity L inking: W hat Works and What’s Next (2021)

Abstract: Entity Linking (EL) systems have achieved impressive results on standard benchmarks mainly thanks to the contextualized representations provided by recent pretrained language models. However, such systems still require massive amounts of data – millions of labeled examples – to perform at their best, with training times that often exceed several days, especially when limited computational resources are available. In this paper, we look at how Named Entity Recognition (NER) can be exploited to narrow the gap between EL systems trained on high and low amounts of labeled data. More specifically, we show how and to what extent an EL system can benefit from NER to enhance its entity representations, improve candidate selection, select more effective negative samples and enforce hard and soft constraints on its output entities. We release our software – code and model checkpoints – at https://github.com/Babelscape/ner4el .

## introduction entity linking (el), also known as named entity disambiguation (ned), is the task of associating an ambiguous textual mention with a named entity in a knowledge base. indeed, named entities may have several surface forms -their full names, partial names, aliases and abbreviations -making el a very challenging task in natural language processing (nlp). over the years, el systems have achieved impressive results in standard benchmarks, especially thanks to the advent of modern language models @xcite , and have found innumerable applications in a wide range of downstream tasks, including information extraction @xcite @xcite , question answering @xcite , knowledge base population @xcite and recommender systems @xcite @xcite , inter alia. in general, el systems are composed of two main components: a candidate generation module and a mention disambiguation module. the aim of the former is to select from a knowledge base (e.g. wikipedia) a suitable subset of named entities that can be associated with a given textual mention in an input text. this set of candidates is then given to the latter module whose objective is to choose and assign the most appropriate entity to the mention. recent studies @xcite @xcite have shown that learning better representations of mentions and entities is key to improving the two aforementioned components and enabling state-of-the-art results. however, one common issue with current el approaches is that they require massive amounts of training data -often millions of labeled items -in order to perform at their best, making the development of a high-performance el system viable only to a limited audience. in this paper, we study whether it is possible to narrow the performance gap between systems trained on limited and large amounts of data. in particular, we take a look at named entity recognition (ner) -the task of identifying specific words as belonging to predefined semantic types such as person, location, organization -and how this task can be exploited to improve a strong entity linking baseline in low-resource settings without requiring any additional data. with this as our aim, we introduce a fine-grained set of ner classes and propose multiple approaches to the exploitation of ner for el, showing how a state-of-the-art model can benefit from them. our main contributions can be summarized as follows: we hope that our work will provide a stepping stone for further studies on the interplay between entity linking and named entity recognition, and encourage further studies on high-performance el systems for scenarios in which only a small amount of labeled data is available. we release our software -code and model checkpoints -at https://github.com/babelscape/ner4el . ## ner for el in this section, we take inspiration from what has already been shown to work and propose several new methods for exploiting ner for el. to this end, we first describe a simple yet strong baseline into which we will plug our ner-focused contributions (section 3.1). in particular, we introduce a set of finer-grained ner classes (section 3.2) and use them to inject ner information into entity representations (section 3.3), devise a ner-enhanced candidate generation module (section 3.4), better select negative samples during training (section 3.5), and introduce a ner-constrained decoding technique (section 3.6). ## experiments in this section, we describe our experimental setup (section 4.1), the datasets we use to train and evaluate our ner-based approaches (section 4.2), the results of each contribution (section 4.3), followed by an analysis of the benefits of ner for el (section 4.4). ## conclusion and future work in recent years, entity linking systems based on contextualized embeddings from pretrained language models have achieved unprecedented results. however, such systems require training on millions of labeled samples, making them practically inaccessible to broad audiences and users and severely hampering the development of a high-performance el system. in this paper, instead, we presented various ner-based strategies which allow systems trained on limited amounts of data to narrow the performance gap with those systems trained on massive training corpora. to this end, we first introduced a new fine-grained set of ner classes to better cluster entities and then used these classes to enhance a strong el baseline with i) ner-enriched entity representations, ii) ner-enhanced candidate selection, iii) ner-based negative sampling, and iv) ner-constrained decoding. our experiments show that the integration of ner information can aid an el system trained on less than 20k instances in narrowing the gap with el systems trained on millions of samples. over the past few years, the field of ner has witnessed continuous growth, with many researchers studying more complex forms of ner, including nested and structured ner @xcite @xcite . although we focused on the benefits of traditional ner in el, we trust that more complex forms of ner can lead to even greater improvements in el. in conclusion, we believe that our work can encourage further developments on entity linking systems that require fewer and fewer training instances and still achieve strong results across indomain and out-of-domain evaluations.