SU - NLP at S em E val-2022 Task 11: Complex Named Entity Recognition with Entity Linking (2022)

Abstract: This paper describes the system proposed by Sabancı University Natural Language Processing Group in the SemEval-2022 MultiCoNER task. We developed an unsupervised entity linking pipeline that detects potential entity mentions with the help of Wikipedia and also uses the corresponding Wikipedia context to help the classifier in finding the named entity type of that mention. The proposed pipeline significantly improved the performance, especially for complex entities in low-context settings.

## introduction named entity recognition (ner) is a widely studied task of natural language processing. recent leading architectures achieved impressive performances, especially in formal contexts (like news articles, etc.) and for commonly worked named entity types @xcite . in multilingual complex named entity recognition task @xcite , organizers introduced a dataset which is a large collection of short texts that includes complex entities. as they stated, complex entities can be noun phrases, gerunds, or full clauses @xcite . the participants were challenged to develop ner systems for identifying these complex entities in short and lowcontext settings. recent contextual neural architectures like transformers work quite well on a variety of nlp tasks and datasets. however, they are strongly dependent on the context of the given text. in case of lack of context, these models alone may not gather enough evidence to make a correct prediction. when that is the case, getting help from knowledge bases can be useful. the dataset provided with this task has similar low-context characteristics together with some complex named entity types to be resolved. therefore, in this work, we investigated whether performing entity linking to provide additional context for * these authors contributed equally to this work. a transformer model can be useful for identifying complex named entities. we initially used an unsupervised entity linking approach to identify the possible entity mentions and their corresponding wikipedia pages. later on, we used the additional context retrieved from wikipedia to predict the types of the identified entities. performing entity linking prior to the classifier seems to be beneficial in the case of complex entities in short and lowcontext settings. our code is publicly available * . ## related work ner is an important and also well-studied task in nlp. similar to other nlp tasks, recent neural architectures provided significant improvements in ner as well. however, since these models are heavily dependent on context information, applying them to short, noisy texts such as very short social media posts or web queries results in significant performance degradation @xcite . to overcome the lack of context in these short texts, as well as the code-mixed terms, it has been prominent to use external knowledge in neural approaches. recent studies @xcite combining multilingual transformer models with gazetteers have shown remarkable improvement in performance. another study @xcite utilized the entity linking task to address the issues of twitter ner by detecting entity mentions from wikipedia. in this study, we also focused on external information due to the lack of context. however, instead of using a static auxiliary like gazetteers, we tried to detect entities dynamically from the continuously extending wikipedia by using a search engine architecture. we integrated the wikipedia content into the input representation in order to improve the performance of entity type classification. an issue we found with the dataset is that in some examples, a phrase is labeled as other although there are similar phrases that are labeled as a named entity. consider the following example: "nokia 1011, 1994'e kadar, nokia 2010 ve nokia 2110'un halefler olarak tanıtılmasıyla üretime devam etti." english: "nokia 1011 continued in production until 1994, with the introduction of the nokia 2010 and nokia 2110 as successors." the text includes three different cellphone models produced by nokia. two of these models (the blue ones) were annotated as product; however, nokia 1011 was mislabeled as other. a more major issue is when the same named entity occurs more than once in a sentence, but their annotations are different. an example is illustrated below: "whitney houston yine bu sene içinde new jack swing'e adını yazdırdı, i'm your baby tonight adlı şarkı whitney houston'a başarı üstüne başarı getirmiştir." english: "whitney houston made her name on new jack swing again this year, the song i'm your baby tonight brought whitney houston success after success." in the example, whitney houston in blue was labeled as person; but, the red whitney houston was labeled as other. these types of wrong annotations cause ambiguities and learning problems in supervised approaches. ## methodology in order to overcome the challenges described in previous sections, we propose an unsupervised entity linking pipeline followed by a classifier as our proposed ner architecture. the entity linking pipeline is designed to detect potential entity mentions and retrieve corresponding candidate documents for each mention. later, these detected mentions and the linked content is used together in a classifier to detect the entity type of the mention. a pre-trained bert model was also fine-tuned in order to create a strong baseline for comparison. ## results the official evaluation metric was chosen as macroaverage f1-score by the organizers. in our baseline bert, the one fine-tuned directly for ner, worked quite well on the validation set but performed badly on the test set. this substantial difference in these two sets shows that contextual models like bert do not perform well in lowcontext settings. on the other hand, although our proposed entity linking approaches fell slightly behind the baseline model in the development set, they performed significantly better than the baseline on test set. among the two entity linking methods, the el_multibert model with two classification heads performed better. while determining the named entity type, training with this auxiliary task even on the same data seems to be useful. ## conclusion in this work, we proposed a language-independent method for the complex ner task in low-context settings. our results showed that utilizing the use of entity linking for ner provides a significant improvement over state-of-the-art transformer models when there is not enough context. yet, since our resources are limited, we experimented with only the turkish part. in future work, we will extend this approach in a multilingual setting.