professionals@ D ravidian L ang T ech- EACL 2021: M alayalam Offensive Language Identification - A Minimalistic Approach (2021)

Abstract: The submission is being made as a working note as part of the Offensive Language Identification in Dravidian Languages shared task. The proposed model “DrOLIC” uses IndicBERT and a simple 4-layered MLP to do the multiclass classification problem and we achieved an F1 score of 0.85 on the Malayalam dataset.

## introduction offensive language identification is a very popular multi-class classification research problem in nlp. a proper solution to the research problem could give the world ways and means to identify and restrict offensive content on the internet, especially on public platforms and social media. the rising demand for a system to identify offensive language on social media and in multiple languages is a testimony to the relevance of the offensive language identification in dravidian languages task @xcite . dravidian languages are spoken in south asia, especially in the indian subcontinent and are a family of multiple languages including tamil, malayalam and kannada, and more. research on nlp in indian languages was largely focused on hindi which is spoken by a large section of people in india. dravidian languages have now recently started getting attention from the research community which is viewed as a positive change. in this work, we propose dravidian offensive language identifier classifier (drolic), a 4layered multi layer perceptron (mlp) to do the multi-class classification task of identifying offensive language for the malayalam language. the reason for choosing a simple mlp over more complex neural networks was because of the simplicity that we see in an mlp. our initial experimentation with mlp gave us pretty good results which were much better than what we were initially expecting out of a simple system like a multi layer perceptron. this made us want to stick to the system and explore how well it would perform under different conditions. ## related work a lot of work has been put into identification of offensive language task in recent years with an aim to enable machines and platforms to automatically filter content, especially in social media platforms. semeval task-6 @xcite is a notable shared task done in this direction where participants identified and categorized offensive language in social media. the dataset contained over 14,000 tweets in english and was divided into three sub-tasks. the sub-task a required participants to identify offensive language while sub-task b was an automatic categorization and sub-task c required participants to identify the offense target. the top teams experimented extensively with lstm @xcite , pretrained bert @xcite ) models, pretrained glove vectors @xcite to achieve the results. with the evolution of technology it has been made possible to prepare and circulate content in different languages and recent years have seen a lot of importance being given to indian languages. the availability of content in indian languages brought in a need to expand the task of offensive language identification into indian languages. this resulted in much work being done in the domain, but we can see most work clearly leaning towards hindi and a few other languages spoken mostly in the northern and central parts of india. one such work does an interesting comparative study of offensive and aggressive language in hindi, bangla and english @xcite . they have used svm, bert and its derivatives like albert @xcite and distilbert @xcite to develop the classifiers. the performance of the classifiers were judged based on f1 score where they managed to achieve an f1 score as high as 0.80 using bert. hasoc (hate speech and offensive content identification in indo-european languages) track at fire 2019 @xcite and fire 2020 @xcite ) is another such interesting initiative. the hasoc dataset was prepared from publicly available posts from twitter facebook and allows the participants to develop supervised models. the limitation to this task again lies in the fact that the task and the dataset are specific to three languages: english, hindi and german. ## system description we approach the problem as a multi-class classification of embeddings and for this we present the drolic architecture as depicted in figure 1 . for every comment represented as an indic-bert embedding, we pass it through a series of dense layers with relu activation to learn the representations. in order to make the layers do the learning more independently and avoid overfitting of the model, we introduce batch normalization and dropout layers respectively. to do the classification we introduce a dense softmax layer to output class probabilities. of the sentences belonging to the "not offensive" class. ## results and analysis the results of our method stands at 85% in the contest for the malayalam language, but we have fine tuned our model since then and the latest results are reported here. the original test set was used for evaluation. the loss and accuracy curves for the training can be seen in figures 4 and 5 respectively. we noticed that mlp stops learning after some epochs, which is bound to happen because mlp's don't take care of vanishing gradients unlike for eg. lstm. sr. no. model precision recall f1 1. m lp w/op r 0.88 0.83 0. 85 from ## conclusion this shared task is one of the first in the area of offensive language identification in dravidian languages. with the advent of a need for localization of the internet by introducing content in multiple languages, this shared task is a positive step. the most interesting aspect of our work lies in the fact that we have used a simple 4-layered mlp to train the multi-class classifier. while keeping the system, we managed to get an f1 score as high as 0.85 and achieve a shared 11 th position in the shared task. the sentences in the dataset were represented as indicbert embeddings. the use of indicbert gives us an added advantage by letting us use the same setup for around 12 popularly spoken indian languages. to overcome the challenges of using a codemixed dataset, we transliterated the sentences into malayalam before generating the indicbert embeddings. the system proposed by us can be further improved especially because, like we have mentioned earlier, this is a very simple setup. the first step that can be taken is replacing the mlp with an lstm or an svm which could give a significantly better result.