silpa_nlp at S em E val-2022 Tasks 11: Transformer based NER models for H indi and B angla languages (2022)

Abstract: We present Transformer based pretrained models, which are fine-tuned for Named Entity Recognition (NER) task. Our team participated in SemEval-2022 Task 11 MultiCoNER: Multilingual Complex Named Entity Recognition task for Hindi and Bangla. Result comparison of six models (mBERT, IndicBERT, MuRIL (Base), MuRIL (Large), XLM-RoBERTa (Base) and XLM-RoBERTa (Large) ) has been performed. It is found that among these models MuRIL (Large) model performs better for both the Hindi and Bangla languages. Its F1-Scores for Hindi and Bangla are 0.69 and 0.59 respectively.

## introduction named entity recognition (ner) is one of the hot topic in natural language processing (nlp). ner is a task of identification of named entities from given sentence and their classification into predefined classes like person, location, organisation, corporation etc. for below sentence: राम दल्ली में गू गल में काम करता है ।, राम is person, दल्ली is location and गू गल is corporation. the application of ner can be found in other nlp tasks such as text summarization @xcite , information retrieval, machine translation @xcite , question-answering @xcite . the researchers have come up with many approaches for ner task such as @xcite , feature-based supervised approach @xcite , unsupervised approach and deep learning based approach @xcite and transformer based approach @xcite . the transformer models are good at capturing features from lengthy sentences compared to recurrent neural networks @xcite . roberta model @xcite performed good for ner task for rich resource languages like english. for hindi and bangla languages, we applied xlm-roberta, which is a multilingual version of roberta pre-trained in 100 languages (including hindi and bangla). we also applied in-dicbert @xcite and mbert @xcite . at last, we applied muril @xcite which is specifically pre-trained in the text of 17 indic languages and it gave better result than above models for the ner task. this paper consists of a total of six sections apart from the introduction. section 2 briefly defines the problem definition and task provided by organizers. section 3 discusses the work done till now on the ner task. section 4 mentions the dataset being used in this paper. section 5 describes the general transformer architecture for the ner task, along with prepossessing and postprocessing. section 6 discusses the results obtained by used models on both hindi and bangla languages and the error analysis. finally, section 7 concludes this paperwork. ## problem definition the organisers @xcite have arranged 13 tasks according to language. they have provided a separate dataset for each task. each dataset is comprised of training, development and testing. respective named entity tags were provided in the training and development dataset. only tokens were provided in the testing dataset. participants were required to train the models using the training and development dataset and predict ner tags on the testing dataset. we have worked on hindi and bangla tasks. ## related work several works have been done on ner that can be categorized under two broad categories: traditional and deep learning methods. ## data the dataset @xcite for hindi and bangla contains six different ner entities, namely location (loc), person (per), production (prod), group (grp), corporation (corp) and creative work (cw). the dataset is in standard conll format, which uses bio (beginning-inside-outside) tagging. the dataset provided was of three types, namely training, development and testing. the training and development data contains tokens with tags, whereas testing data contains only tokens. for both hindi and bangla tracks, there were 15300 samples in training and 800 samples in development. in the test- ## methodology this work fine tuned 6 transformer @xcite based pre-trained model for the task. indicbert @xcite is albert based model which is pre-trained on 11 indic languages, including hindi and bangla. we also fine-tuned xlm-roberta (base) and xlm-roberta (large) @xcite , which is pre-trained on text in 100 languages. other models are mbert @xcite which is pretrained on text in 104 languages and muril base and muril large @xcite which is pre-trained on text in 17 languages with explicitly augmented monolingual text corpora with translated and transliterated document pairs. all the corpora describe above include hindi and bangla languages. figure 1 shows the architecture of this work, which is divided into 3 sections: preprocessing, fine tuning and post processing. ## conclusion results show that large models are better than their corresponding base models. muril (large) model is the best among all six models described above and the second-best model is muril (base).