OFFL ang O ne@ D ravidian L ang T ech- EACL 2021: Transformers with the Class Balanced Loss for Offensive Language Identification in D ravidian Code-Mixed text. (2021)

Abstract: The intensity of online abuse has increased in recent years. Automated tools are being developed to prevent the use of hate speech and offensive content. Most of the technologies use natural language and machine learning tools to identify offensive text. In a multilingual society, where code-mixing is a norm, the hate content would be delivered in a code-mixed form in social media, which makes the offensive content identification, further challenging. In this work, we participated in the EACL task to detect offensive content in the code-mixed social media scenario. The methodology uses a transformer model with transliteration and class balancing loss for offensive content identification. In this task, our model has been ranked 2nd in Malayalam-English and 4th in Tamil-English code-mixed languages.

## introduction language is a social phenomenon. it is through language that day-to-day interactions and interpersonal relations are possible @xcite . languages keep on changing and adapting. in a multilingual scenario, the languages influence each other in certain ways. normally, this interaction is reflected in language convergence, borrowing, and replacement. it also leads to the emergence of hybrid languages, such as pidgins, creoles, and other mixed languages. this form of language interaction is known as language contact @xcite . language contact is considered to be an important phenomenon, especially in multilingual societies. in bilingual or multilingual communities, speakers use their native tongue and their second language in different domains. this form of alternation of two or more languages is called 'code-mixing' (cm) @xcite . with the increase in social media access, offensive content and hateful material on the internet has increased in the recent past @xcite . the internet harbors a variety of hateful and offensive statements, and nowadays, social media is a hotbed of such conversations. recently, countries across the world have already begun to address hate speech and offensive content and how it affects society's functioning @xcite . research and technologies worldwide are utilizing natural language and machine learning tools to detect and curb the use of offensive content on social media. in a multilingual society, code-mixing has become a norm. the hateful and offensive content is delivered in a code-mixed form @xcite . automatic hate speech detection on code-mixed data has faced quite many challenges due to the non-standard variations in spelling and grammar @xcite . the typical hate-speech and offensive language tools developed for monolingual data will not work for codemixed data. so there is a need for more research and analysis to be done to identify the offensive content in code-mixed social media data. to encourage research on code-mixing and restrain the use of offensive texts on social media, the nlp community has organized several workshops such as workshops on computational approaches to linguistic code-switching, sen-timix @xcite , dravidian codemix (chakravarthi et al., 2020d), hasoc dravidian codemix foot_0 @xcite . similarly, the european association of computational linguistics 2021's dravidianlangtech @xcite was also devoted to identifying offensive content on kannada-english, tamil-english, and malayalam-english code-mixed languages. this task aims to classify the given code-mixed com-ments into one of the six predefined categories: not-offensive, offensive-untargeted, offensive-targeted-insult-individual, offensive-targeted-insult-group, offensive-targeted-insult-other, or not-in-indented-language. this paper presents a pre-trained bert model with the class balanced loss for offensive content identification on the dravidian code-mixed text. the paper is organized as follows. section 2 provides related work on offensive content identification on cm social media text. section 3 provides information on the task and datasets. section 4 describes the proposed work. section 5 presents the experimental setup and the performance of the model. section 6 concludes our work. ## related work this section describes the related work on hate speech detection and offensive content identification in the code-mixed scenario. @xcite created a dataset for hate speech detection from hindi-english tweets. they collected around 4575 hindi-english tweets and used traditional machine learning models with feature engineering for hateful and offensive content identification. @xcite created a hindi-english dataset for a hate speech and offensive content identification task (hasoc), organized at fire 2019. it consists of 4665 hindi-english annotated posts collected from social media sites. they used twitter api for crawling an unbiased dataset. the top models used the long short term memory (lstm) @xcite with attention mechanism, pre-trained bidirectional encoder representations with transformers (bert) @xcite ) models, and convolutional neural networks (cnn) for hate speech and offensive content identification. @xcite created the dataset for an aggression detection task. the dataset was annotated for a comparison task by @xcite . this data set consists of 3367 posts and tweets collected from social media sites. the authors used traditional machine learning classification models for hate speech detection. a shared task called dravidian @xcite was organized to identify the offensive language from comments/posts in code-mixed dravidian languages (tamil-english and malayalam-english) collected from social media. each comment/post is annotated with the offensive language label at the com-ment/post level. the data set has been collected from youtube comments and tweets. the dataset contains 4000 annotated youtube comments and 4952 annotated tweets for the malayalam language, and 4940 annotated tweets for the tamil language. the top models used deep learning models like lstm, cnn, and bert for hate speech and offensive content detection on the given dravidian data. ## task and dataset information the goal of offensive language identification is to identify the offensive language content of the code-mixed dataset of comments/posts in tamil-english (chakravarthi et al., 2020c), malayalam-english (chakravarthi et al., 2020a), and kannada-english @xcite ) dravidian languages collected from social media. each comment or post is annotated with offensive, not-offensive, and notin-intended-language labels. where the offensive label is fine-grained into further categories. the description of each label is given below • offensive-targeted-individual: offensive text delivered to an individual or person the dataset for offensive content identification into divided into train, development(dev), and test sets for the given languages. the details of the dataset are given in ## our work in this section, we start with pre-processing of codemixed text. later we describe the pre-trained multilingual bert model with the class balanced loss for offensive content identification. ## experiments the section presents the baselines, hyper-parameter settings, and analysis of observed results. the baselines used for the proposed work is: