DANGNT - SGU at S em E val-2022 Task 11: Using Pre-trained Language Model for Complex Named Entity Recognition (2022)

Abstract: In this paper, we describe a system that we built to participate in the SemEval 2022 Task 11: MultiCoNER Multilingual Complex Named Entity Recognition, specifically the track Mono-lingual in English. To construct this system, we used Pre-trained Language Models (PLMs). Especially, the Pre-trained Model base on BERT is applied for the task of recognizing named entities by fine-tuning method. We performed the evaluation on two test datasets of the shared task: the Practice Phase and the Evaluation Phase of the competition.

## introduction in natural language processing, named entity recognition (ner) is traditionally a fundamental task that aims to recognize groups of words used to identify people, organizations, places, times, etc. in documents. semeval-2022 task 11 focuses on detecting semantically complex and ambiguous entities @xcite . this is a challenging nlp task that has not sufficiently received attention from the research community. in practice, available named entity recognition models find it hard to recognize complex named entities, such as titles of works (movie/book/song/software titles) that are not simple nouns @xcite . to enable the recognition of such entities, we believe that it is necessary to make full use of pre-trained language models (plms) to increase efficiency as well as reduce training time, which is also the method we applied in this study. ## related work nowadays, there are many ner works and different approaches to solving this problem, such as lstm (long short-term memory) (nut @xcite , transformers (ashish @xcite . in particular, the bert model (jacob @xcite . many relevant studies also approach the direction of combining deep learning models and adjusting model parameters to improve the efficiency of named entity recognition in the text (zheng @xcite . however, entities with complex and ambiguous names appear continuously in increasing numbers. this has posed emerging challenges for the natural language processing field. many research show that clarifies that named substance acknowledgement is particularly troublesome in circumstances with a low context or in scenarios where the named substances are especially complex @xcite . the misrecognition of entities with complex and ambiguous names significantly affects the performance of natural language processing systems (andreas @xcite . in this paper, we will use plms to find ways to improve accuracy in such cases. plms are pre-trained language models with large datasets to be utilized in specific tasks. in particular, bert has drawn great attention and it is the favorite research direction of the nlp community after achieving stateof-the-art on 11 nlp tasks @xcite . based on bert, there have been many research directions from the nlp research community: distilbert @xcite , roberta @xcite , and so on. ## data in total, there are 13 tags: b-per, i-per, b-loc, i-loc, b-grp, i-grp, b-corp, i-corp, b-prod, i-prod, b-cw, i-cw, and o with b is the beginning of a named entity, i is all the words inside an entity and all words outside an entity @xcite ## methodology in this paper, we use the transformer library of huggingface @xcite . currently, the huggingface library is considered the most powerful and widely accepted pytorch interface to deal with bert. in addition to the support for a wide range of pre-trained language models, the library also includes prebuilt modifications of bert tailored to specific tasks. then we tested several models based on bert downloaded from huggingface as bert, roberta, and xml-rob-erta with different versions. ## conclusion in this paper, we proudly introduced our semantically complex and ambiguous entity recognition system. we tested on different plms to evaluate the effectiveness of entity recognition. we trained each model with the dataset provided by the contest @xcite : model bert with two versions bertbase-uncased and bert-large-uncased, model roberta with roberta-large version and model xml-roberta with xml-robertalarge, and we also tried to fine-tune the hyperparameters to increase recognition efficiency, but there were no significant changes. the evaluation results in the practice phase are quite positive. finally, based on the evaluation results, we used the xml-roberta-large model to participate in the evaluation phase of the competition, but the results were not good. in future work, we plan to continue testing on other plms. in addition, we will also extend the approach by applying deep learning techniques and theories of language to improve the accuracy in the task of recognizing possible entities, especially complex entities