MUCS @ D ravidian L ang T ech- EACL 2021: COOLI -Code-Mixing Offensive Language Identification (2021)

Abstract: This paper describes the models submitted by the team MUCS for Offensive Language Identification in Dravidian Languages-EACL 2021 shared task that aims at identifying and classifying code-mixed texts of three language pairs namely, Kannada-English (Kn-En), Malayalam-English (Ma-En), and Tamil-English (Ta-En) into six predefined categories (5 categories in Ma-En language pair). Two models, namely, COOLI-Ensemble and COOLI-Keras are trained with the char sequences extracted from the sentences combined with words as features. Out of the two proposed models, COOLI-Ensemble model (best among our models) obtained first rank for Ma-En language pair with 0.97 weighted F1-score and fourth and sixth ranks with 0.75 and 0.69 weighted F1-score for Ta-En and Kn-En language pairs respectively.

## introduction along with the increasing developments on social media, social networks, and internet, the number of people using these are also increasing. the advantage of social media is that users' have the freedom of expressing their opinions without revealing their identity @xcite . this feature is exploited by miscreants for spreading offensive messages targeting an individual or a group. these messages will be usually code-mixed texts in native languages mixed with english words but written in roman script. the texts on social media do not adhere to the rules of any of the languages in which they are written. hence, the analysis of code-mixed texts is a more challenging task compared to analysis of texts in native scripts because of the inconsistent romanization conventions and non-standard grammars in code-mixed texts @xcite . these issues have created a demand for analyzing social media text which is becoming important day by day. generally, social media analysis and offensive language identification (oli) is fathomably important for social media platforms to monitor the texts including hateful or offensive content or advertising violence against people, communities, or religions. many studies have been carried out in this direction to identify offensive content in texts. but, most of these works focus on rich resource languages such as english, spanish, etc. giving less or no importance for low resource languages such as kannada, tamil, telugu, malayalam @xcite . however, most texts in social media are not written in any one language, but a mix of several languages making the oli task more challenging @xcite . further, the task becomes more complex due to the usage of non-native scripts. indians usually mix english language with their native language and use roman script mixed with their native language script to post messages. this type of mixing two or more languages in a text is called @xcite @xcite . due to lack of efficient keyboards for native languages or ease of using roman script and better adaptation of roman script in software and smartphones, code-mixing texts are increasing day-by-day. to promote analyzing codemixing texts to identify offensive language posts in dravidian languages, "offensive language identification in dravidian languages" @xcite shared task provides texts in three codemixing language pairs namely, kannada-english (kn-en), malayalam-english (ma-en), and tamil-english (ta-en). the rest of paper is organized as follows: section 2 describes the recent literature on code-mixed text processing, and section 3 presents details of submitted models followed by the results in section 4. conclusion and future plans are given in section 5. ## related work researchers have proposed many models for oli for several languages. however, very few works are reported for oli task in code-mixed texts. some of the recent literature related to oli in dravidian code-mixed texts @xcite are given here: a shared task on oli conducted by @xcite , on dravidian code-mixed texts consists of two message level subtasks namely, subtask a and subtask b. while the focus of subtask a was to classify youtube comments in malayalam-english code-mixed texts subtask b focused on the classification of romanized twitter comments in tamil-english and malayalam-english codemixed texts. the datasets used in this shared task are described in @xcite . @xcite proposed a binary classification model for subtask b to classify malayalam-english code-mixed texts into "offensive" and "not offensive" posts. their model con-2021 2 cooli stands for code-mixing offensive language identification sists of a text processing step which includes removing english stopwords, hashtags, urls, and emojis, converting text to lower case, and tokenization. using keras embedding they represented text as one-hot encoding with 50d and fed it to two lstm architectures. the first lstm architecture comprised of an lstm layer and recurrent dropout (0.2) followed by a dense layer that was configured with sigmoid activation and binary cross-entropy and the second lstm architecture is same as the first architecture with three dense layers and relu activation added in between lstm and dense layer. they obtained 0.53 and 0.48 macro f1-score using first and second architectures respectively. a universal language model fine-tuning (ulmfit) based on transfer learning (tl) approach submitted by @xcite trained a language model synthetically for malayalam -english code-mixed texts using fastai library from malayalam -english code-mixed data collected from wikipedia articles in native script as well as the translated and transliterated versions. fastai library was also used to build final classifiers after a step of fine-tuning the pre-trained language model using training set. they obtained 0.91 and 0.74 weighted f1-score on subtasks a and b respectively. another work in this shared task submitted by @xcite is based on tl and machine learning (ml) approaches for subtask a. in addition to the dataset provided by task organizers they used the olid dataset @xcite , a well-known dataset that has been used in the semeval-2019 task 6 (offenseval) @xcite in tl model and explored bert and xlm-roberta by adopting implementation from huggingface's transformer models @xcite . they trained ml classifiers namely, multinomial naive bayes (mnb), support vector machines (svm), and random forest (rf) on vectors obtained from bag-of-words feature engineering method. the reported results illustrate that rf with a 0.93 weighted f1-score outperformed other models. however, xlm-roberta based on tl obtained 0.89 weighted f1-score and 5 th place in subtask a. ## methodology the methodology includes two major steps for each proposed models, namely, feature extraction and classification models. details of the steps are given below: extracted features "yuvanvera level ya" (ta-en) yu, uv, va, an, n , v, ve, er, ra, a , l, le, ev, ve, el, l , y, ya, yuv, uva, van, an , ve, ver, era, ra , le, lev, eve, vel, el , ya, yuva, uvan, van , ver, vera, era , lev, leve, evel, vel , yuvan, uvan , vera, vera , leve, level, evel , yuvan , vera , level, level , yuvanvera, level, ya ## conclusion and future work this paper describes the two models, cooli-ensemble and cooli-keras submitted for "offensive language identification in dravidian languages-eacl 2021" shared task to classify code-mixed text in tamil-english, malayalam-english, and kannada-english. the analysis of results shows that cooli-ensemble (best among our models) on a feature set of char sequences, and words outperformed cooli-keras model for both ma-en and ta-en language pairs and also obtained first rank for ma-en language pair with 0.97 weighted f1-score and fourth and sixth ranks with 0.75 and 0.69 weighted f1-score for ta-en and kn-en language pairs respectively. the results also illustrate that feature selection applied on ka-en texts slightly improved the performance of models. as a future work, we planned to explore different feature sets and feature selection models along with various learning approaches to process code-mixed texts.