FII _ B etter at S em E val-2023 Task 2: M ulti C o NER II Multilingual Complex Named Entity Recognition (2023)

Abstract: This task focuses on identifying complex named entities (NEs) in several languages. In the context of SemEval-2023 competition, our team presents an exploration of a base transformer model’s capabilities regarding the task, focused more specifically on five languages (English, Spanish, Swedish, German, Italian). We take DistilBERT and BERT as two examples of basic transformer models, using DistilBERT as a baseline and BERT as the platform to create an improved model. The dataset that we are using, MultiCoNER II, is a large multilingual dataset used for NER, that covers domains like: Wiki sentences, questions and search queries across 12 languages. This dataset contains 26M tokens and it is assembled from public resources. MultiCoNER II defines a NER tag-set with 6 classes and 67 tags. We have managed to get moderate results in the English track (we ranked 17th out of 34), while our results in the other tracks could be further improved in the future (overall third to last).

## introduction named entity recognition (ner) is a subtask of natural language processing (nlp), being still a crucial learning problem @xcite , @xcite ). ner is used especially for relevant information extraction from text data @xcite , @xcite ). it is impressive that state-of-the-art ner systems rely heavily on hand-crafted features and domain-specific knowledge @xcite . the multiconer ii shared task @xcite aims at building ner systems for 12 languages, namely english, spanish, hindi, bangla, chinese, swedish, farsi, french, italian, portuguese, ukrainian and german. the task has 12 monolingual tracks and a multilingual one. ner helps one easily identify key elements in a text, like athletes, politicians, human settlements, clothing pieces, medications and more. extracting the main entities in a text helps sorting unstructured data and detecting important information, which is crucial if one has to deal with large datasets. the datasets mainly contain sentences from three domains: wikipedia, web questions and user queries which are usually short and low-context sentences @xcite , @xcite ). moreover, these short sentences usually contain semantically ambiguous and complex entities, which makes the problem more difficult. usually, retrieving knowledge related to such ambiguous concepts in any form (like an article or results of a web search) is a definite method of understanding and disambiguating them. thus, the ideal ner model would be capable of taking on hard samples if the option of additional context information was available. the rest of the paper is organized as follows: section 2 briefly presents studies related to ner, either in a multilingual context or not, section 3 presents the dataset, the required pre-processing and plausible methods for it, section 4 resumes the results of the conducted experiments, with their interpretations, followed by section 5 with the conclusions. ## related work currently, processing complex named entities is still a challenging nlp task. there is relatively little work on recognizing other types of entities than the traditional ones (persons, locations, organizations, date, currency). complex nes, like chemicals, ingredients, diseases or active substances are not simple nouns and are much more difficult to recognize @xcite . they can take the form of any linguistic constituent and have a very different surface from than traditional nes. their ambiguity makes it challenging to recognize them. additionally, more and more people share nowadays information about various topics online, making ner for these non-traditional entities more important in the context of data collected from social media, where people's interests are directly expressed @xcite . there has been made an attempt to investigate the ability of modern ner systems to generalize effectively over a variety of genres. this attempt also found out, as expected, that there is a strong correlation between the ner performance and the training corpus size, so by having a bigger corpus, the results may be more accurate @xcite . the job of handling nes by extracting them from the text has been done by transformers. in the last few years, new technologies have appeared, including a google research releasing mt5, their own version of transformer, which outperforms the previously released multilingual transformers @xcite . among those, bert is one of the most powerful unsupervised models. a multilingual variant of it, trained over 100 languages and enhanced with context-awareness thanks to a crf layer on top, has been leveraged before for such a task with promising results @xcite . lastly, the most successful approaches used in the previous multiconer semeval task were multilingual systems that revolved around improving upon the baseline model of xlm-roberta, adapted to the multilingual track and then specialized for each monolingual track, as well as an enhanced bil-stm network @xcite . ## dataset and methods although we explored a few options, we opted for the bert transformer model for our approach. in this section, we present statistics over the dataset, as well as the steps we went through before choosing the bert model and using the data for training. ## conclusion in this paper, we got the opportunity to explore a transformer model's capabilities at dealing with nlp tasks -in this case complex ner -and how to handle task-specific input. more specifically, we put the classic bert model to the test and found it to live up to its reputation of general-purpose transformer model by managing moderate results. we have learned a lot about the workings of the transformer model and now have a better understanding of what tackling such a task entails with regards to approaches and resource management. among the things that could improve the results of this particular model, one important thing would surely be a more versatile module for handling input test data. contrary to expectation, we should have put more focus on this part of the system. apart from that, parallelization of the system could have potentially made it available to us to harness more powerful transformer models.