azaad@ BND at S em E val-2023 Task 2: How to Go from a Simple Transformer Model to a Better Model to Get Better Results in Natural Language Processing (2023)

Abstract: In this article, which was prepared for the sameval2023 competition (task number 2), information about the implementation techniques of the transformer model and the use of the pre-trained BERT model in order to identify the named entity (NER) in the English language, has been collected and also the implementation method is explained. Finally, it led to an F1 score of about 57% for Fine-grained and 72% for Coarse-grained in the dev data. In the final test data, F1 score reached 50%.

## introduction the purpose of this competition is to extract the named entity (named-entity recognition) within a text. ner is basically a token classification task where each token is classified into one or more predefined categories. for example, persons, locations, corporations, etc. should be extracted and classified from within the text. @xcite in this competition, the basis of classification is divided into two categories: general classification and partial classification. for example, each location category has 4 sub-categories (subclassifications) including: facility, otherloc, humansettlement, station, and all classifications are available and accessible on the competition dataset page @xcite . the total classifications are as follows: ## related work this article is related to the previous year's semeval-2022 task 11 article @xcite , which is to identify the named entity in the general category [per, loc, corp, grp, prod, cw] . about 55 teams have participated in this task and the best team achieved the score of @xmath0.91 in the english language that we want. according to what is mentioned in the article, "most of the teams have used external databases such as wikipedia, gazetteer. also, they have been more interested in the pre-trained xlm-roberta model ." azaad@bnd at semeval-2023 task 2: how to go from a simple transformer model to a better model to get better results in natural language processing reza ahmadi, shiva arefi, mohammad jafarabad bandar abbas islamic azad university { reza.zx@live.com, shiva.arefi@outlook.com, tcsms@yahoo.com } @xmath1:['otherper','sportsmanager','cleric','politicia n','athlete','artist','scientist'], 'loc':['facility','otherloc','humansettlement','stat ion'], 'grp':['musicalgrp','publiccorp','privatecorp','oth ercorp','aerospacemanufacturer','sportsgrp','carm anufacturer','techcorp','org'], 'prod':['otherprod','drink','food','vehicle','clothi ng'], 'cw':['visualwork','musicalwork','writtenwork','a rtwork','software','othercw'], 'med':['medication/vaccine','medicalprocedure','a natomicalstructure','symptom','disease'] } ## data the data set used for training and developing the model is the same data that is provided in the dataset section @xcite of the competition, which are labeled with the conll format and the entities are labeled with the iob method . iob: inside-outside-beginning (tagging) is a common tagging for tagging tokens in computational linguistics .the b-prefix before a tag specify that the tag is the first of a chunk, and an i-prefix before a tag specify that the tag is inside a chunk. the b-tag specify that a tag is followed by a tag of the same type without o tokens between them. an o tag shows that a token belongs to no entity chunk . after checking, due to the imbalance of the classes used in the training data, in order for the machine to understand more about the data and classifications, techniques such as oversampling (increasing data with less density) should be used on the data in the pre-processing stage of the data. the balance of the categories could be maintained . also, in order to improve machine learning, we first calculated the ratio of the number of famous words to the length of the sentence, and according to the obtained ratio, we added the sentence to the dataset several time . here, experimentally, if obtained ratio was more than 90%, 6 times were repeated, if the ratio was more than 80%, 3 times were repeated, and for a ratio greater than 65%, one repetition was sufficient . ## conclusion conclusion using pre-trained models such as bert can give us much better output (prediction) about an nlp problem, you just need to fine-tune the model and do a good pre-processing on the data to be closer to the final goal. let's become and the work will go better. of course, in the future, we plan to implement these tests on the bert-large model, which requires more computing time .