VBD _ NLP at S em E val-2023 Task 2: Named Entity Recognition Systems Enhanced by B abel N et and W ikipedia (2023)

Abstract: We describe our systems participated in the SemEval-2023 shared task for Named Entity Recognition (NER) in English and Bangla. In order to address the challenges of the task, where a large number of fine-grained named entity types need to be detected with only a small amount of training data, we use a method to augment the training data based on BabelNet conceptsand Wikipedia redirections to automatically annotate named entities from Wikipedia articles. We build our NER systems based on the powerful mDeBERTa pretrained language model and trained on the augmented data. Our approach significantly enhances the performance of the fine-grained NER task in both English and Bangla subtracks, outperforming the baseline models. Specifically, our augmented systems achieve macro-f1 scores of 52.64% and 64.31%, representing improvements of 2.38% and 11.33% over the English and Bangla baselines, respectively.

## introduction the goal of the semeval-2023 task 2: multi-coner ii multilingual complex named entity recognition @xcite is to focus on extracting semantically ambiguous complex named entities based on the multiconer v2 @xcite . the first version of multiconer @xcite , which used the same taxonomy schema as introduced in wnut 2017 @xcite , was utilized for semeval-2022 task 11: multiconer @xcite , that had 13 subtracks in multilingual, codemixing, and 11 monolingual languages, namely: english, spanish, dutch, russian, turkish, korean, farsi, german, chinese, hindi, and bangla. in the second version, the the multiconer v2 taxonomy was changed to challenge the previous systems and evaluate their performance on noisy and fine-grained data. the second version provides 13 subtracks in multilingual and 12 monolingual languages, including english, spanish, farsi, german, chinese, hindi, bangla, swedish, french, italian, portuguese, and ukrainian. this task is derived from the multiconer shared task last year @xcite , which was a challenging task due to scarcity training instances, with approximately ten thousand training instances and testing data that was ten times larger than the training data. furthermore, the shared task this year has become more challenging when the number of entity types increases from 6 to 35 finegrained entity types. this increase in the number of entity types has caused issues for models that train on few training instances and detect entities in a large number of classes. however, this task presents an interesting opportunity to investigate techniques for extending training data to deal with these issues. as a step towards addressing the gap, we developed and evaluated our systems for the semeval-2023 task 2 in english and bangla by augmenting the training data from a knowledge base. while english is the most commonly used language worldwide, bangla represents a low-resource language that has not been well-investigated. we exploit a recent and powerful method for building ner data from the well-known babelnet @xcite , using a combination of knowledge base (kb)-based and model-based techniques.specifically, we annotated named entities from wikipedia articles based on babelnet concepts and wikipedia redirection links, and further corrected the annotations by checking the agreement between the kb and neural model predictions. for our ner models, we built a strong baseline system based on the mdeberta pretrained model. we achieved a significant improvement in performance for both english (2%) and bangla (12%) by utilizing the augmented data for training our systems, compared to the baseline. our extensive analysis also revealed the contribution of our method to each fine-grained entity type. ## dataset we present the data statistics for english and bangla in compared to the bangla subset, the english subset has almost twice as many sentences in the training phase. as a result, the english subset has more distinct entities and a larger vocabulary than the bangla subset (19,466 versus 6,081 in terms of the number of distinct entities; 34,140 versus 22,274 in terms of vocabulary size, respectively). on average, each sentence consists of approximately 13 to 15 tokens, which is similar for both subsets. in addition, more statistics about the fine-grained taxonomy of the train, dev, and test sets of both subtasks are shown in on the other hand, the test sets for both languages are extremely large in comparison to the train and dev sets. the reason for this, as stated in multiconer @xcite , is that the organizers want to evaluate systems' ability to generalize to unseen and complex entities, as well as their performance on cross-domain adaptation tasks. ## results and analyses meanwhile, the augmented model improve all scores by almost 10±2% in all scores for the bangla results ( ## conclusion in this paper, we have introduced our ner systems for the semeval-2023 task 2: multiconer ii multilingual complex named entity recognition. our ner systems are finetuned on the mde-berta pretrained language model and enhanced by augmented data from babelnet and wikipedia. specifically, babelnet concepts are utilized to annotate named entities from wikipedia articles. experimental results show that the augmented data significantly improve baseline systems trained on the limited training data, especially on the lowresource language such as bangla. for future work, we plan to improve the quality of augmented data by filter noisy annotations as well as extend our systems on other languages.