JUNLP @ D ravidian L ang T ech- EACL 2021: Offensive Language Identification in D ravidian Langauges (2021)

Abstract: Offensive language identification has been an active area of research in natural language processing. With the emergence of multiple social media platforms offensive language identification has emerged as a need of the hour. Traditional offensive language identification models fail to deliver acceptable results as social media contents are largely in multilingual and are code-mixed in nature. This paper tries to resolve this problem by using IndicBERT and BERT architectures, to facilitate identification of offensive languages for Kannada-English, Malayalam-English, and Tamil-English code-mixed language pairs extracted from social media. The presented approach when evaluated on the test corpus provided precision, recall, and F1 score for language pair Kannada-English as 0.62, 0.71, and 0.66, respectively, for language pair Malayalam-English as 0.77, 0.43, and 0.53, respectively, and for Tamil-English as 0.71, 0.74, and 0.72, respectively.

## introduction social media platforms like question answering platforms, collaborative projects, social networks, news platforms provide discussion area for users, where content moderators are engaged to keep respectful conversations @xcite . moderators assure that the platform's discussion rules are adhered to, including the prohibition of offensive languages. moderators implement these rules by partly or entirely removing user comments. typically, platform rules are available to the user in the form of guidelines. however, all user doesn't follow the rules while commenting on a post. an increase of end-users in social platforms leads to the increasing number of comments that make the moderator restless to identify offensive and non-offensive. to intercept the circumstances recent trends of identification of offensive language have become a scientific asset @xcite . in the context of natural language processing (nlp), offensive language identification is a classification task that is aimed to identify and minimize offensive contents in social media @xcite . there have been advancements in this domain of research both in industrial and academia with increasing access to larger and richer social media data over the years. india is a linguistically diverse country with 22 official languages with common languages used being english and hindi. there has been a mixing of cultures and languages over the years thus leading to an increasing demand for offensive language identification on social media texts which are largely code-mixed. code-mixing refers to a prevalent phenomenon which exists in a multilingual community and the code-mixed texts are sometimes written in nonnative scripts. any system which is trained on monolingual datasets miserably fails when exposed to code-mixed data due to the complexity of codeswitching at different linguistic levels in the text @xcite . many researcher has proposed many different approaches to achieve the state-of-the-art results in code-mixing scenarios during the recent years. author @xcite has solved the problem of classification of the tweets in hindi-english offensive tweet (heot) dataset using transfer learning in which author has employed convolutional neural networks as pre-trained on tweets in english followed by retraining on hinglish tweets. heot dataset consists of hindi-english code switched language into three classes nonoffensive, abusive and hate-speech. the author @xcite proposes a supervised classification system that detects hate speech in the text using various character level, word level, and lexicon based features whereas @xcite dealt the task of identification of hate speech and offensive language from code-mixed social media text using two architecture namely sub-word level lstm model and hierarchical lstm model with attention based on phonemic sub-words. another real-world issue dataset have been released by the authors @xcite who have introduced kannada codemixed dataset (kancmd). the dataset is a multi-task learning dataset for sentiment analysis and offensive language identification. the kancmd dataset highlights two real-world issues from the social media text. firstly, it contains actual comments in code mixed text posted by users on youtube social media, rather than in monolingual text from the textbook. secondly, it has been annotated for two tasks, namely sentiment analysis and offensive language detection for underresourced kannada language. hence, kancmd was meant to stimulate research in under-resourced kannada language on real-world code-mixed social media text and multi-task learning. our proposed system is yet to explore the dataset. this paper aims to solve this research problem by using generalized deep learning architectures named indicbert @xcite and bert @xcite . the goal of this task is to identify offensive language content of the code-mixed dataset of comments or posts in dravidian languages collected from social media. the code-mixed language for the task was tamil-english (chakravarthi et al., 2020b), malayalam-english (chakravarthi et al., 2020a), and kannada-english @xcite provided by @xcite . the working system is available in github foot_0 . the rest of the paper has been organized as follows. section 2 describes the data that was used to build the proposed offensive language identification system. section 3 describes the proposed model used to build the system and will be followed by the evaluation of the model in section 4. ## data the shared task organized by dravidianlangtech-2021 provided the gold standard corpus for offensive language identification of code-mixed text for three different sets tamil-english, malayalam-english, and kannada-english in dravidian languages. the dataset was collected from social media. the average sentence length of comment or post of the corpora is 1. each comment or post is annotated at the comment or post level. depicting the real-life scenarios the corpora was provided with class imbalance problem. the corpora has 6 output labels: not offensive, offensive untargeted, offensive targeted individual, offensive targeted group, offensive targeted other, or not in indented language. the training corpus was first tokenized and the required tags were inserted. then the tokenized data was fed to the indicbert and bert models. we make use of pre-trained vectors for initializing the mentioned models and then fine-tuned using the training corpus. the indicbert model supports various indic languages. therefore for each language pair a seperate model had to be trained with corresponding language settings. the confidence scores for each class obtained from the indicbert model and the regular bert model are then concatenated to give the final scores for each of the classes. the score array is converted to a 1-d array in one-hot vector format containing 1's depicting not offensive category and 0's depicting offensive category, which was further classified for rest of the sub categories. the output vector consisted of the following labels in order: if the output vector had '1' for the first class, then rest of the classes were converted to '0' else the maximum among the classes leaving the first class was converted to '1'. finally the corresponding label is given as output. ## evaluation offensive language identification was evaluated using sklearn classification report @xcite . performance was measured in terms of precision, recall and f1-score across all the classes. the result of the evaluation are shown in ## conclusion in the current work, we attempted to solve the problem of offensive language identification, while participating in the dravidianlangtech-2021 shared task. our system was based on ensemble of indicbert model and generic bert model using multi-label classification approach. our system when evaluated by the organizers earned f1 score of 0.66, 0.54, and 0.72 for kannada-english, malayalam-english, and tamil-english language pair, respectively. as future work, we would like to increase this data, and use other state-of-the-art neural network architecture to improve the performance. also we would like to evaluate with the dataset released by author @xcite .