A mrita_ CEN _ NLP @ D ravidian L ang T ech- EACL 2021: Deep Learning-based Offensive Language Identification in M alayalam, T amil and K annada (2021)

Abstract: This paper describes the submission of the team Amrita_CEN_NLP to the shared task on Offensive Language Identification in Dravidian Languages at EACL 2021. We implemented three deep neural network architectures such as a hybrid network with a Convolutional layer, a Bidirectional Long Short-Term Memory network (Bi-LSTM) layer and a hidden layer, a network containing a Bi-LSTM and another with a Bidirectional Recurrent Neural Network (Bi-RNN). In addition to that, we incorporated a cost-sensitive learning approach to deal with the problem of class imbalance in the training data. Among the three models, the hybrid network exhibited better training performance, and we submitted the predictions based on the same.

## introduction in recent years, people from all walks of life use social platforms like twitter, instagram, facebook. so, it is demanding to monitor their behavior to avoid violence, hateful and offensive content @xcite . offensive content is any non-verbal or oral communication expressing disparity against a group or person based on their religion, age, sexual orientation, race, gender, nationality, and ethnicity @xcite . a substantial amount of work was done to identify offensive content in english, but much work is not done in dravidian languages @xcite @xcite . the dravidian languages were first documented in tamili script engraved on cave walls in tamil nadu's madurai and tirunelveli districts in the 6th century bce. india being a multilingual country, a lot of people use regional languages along with english. the usage of two languages to communicate is called code-mixing. it is even more challenging to identify hateful content from code-mixed language owing to the non-standard grammar and spelling @xcite , @xcite , @xcite . dravidianlangtech-eacl2021 is a task to identify offensive content from code-mixed tamil-english (tam-eng), malayalam-english (mal-eng), and kannada-english (kan-eng). in this task, we came up with a deep learning model to identify offensive content from malayalam-english, tamil-english, and kannada-english datasets. we employed three different deep learning models for solving the classification problem. a hybrid model that includes a convolutional layer followed by a bi-lstm @xcite , @xcite and a fully connected network attained the maximum scores while training. therefore, the labels for the test data were predicted using the aforementioned model. the rest of the contents are explained in the following sections: section 2 presents the literature review. dataset details are provided in section 3. section 4 explains the system description, and section 5 relates to experimental details and results. finally, the work is concluded in section 6. ## literature review different abusive and offense language identification problems and shared tasks have been explored in the literature ranging from aggression to cyberbullying, hate speech, toxic comments, and offensive language. below we discuss each of them briefly. in 2018, adithya et.al @xcite ) evolved a dataset consisting of 4500 hate and nonhate code-mixed hindi-english tweets. the dataset was congregated using twitter api and annotated by two linguists. machine learning models like random forest and svm and handcrafted features like character n-grams, punctuation count, emoticon count, negation words, word n-grams were used for classification. semeval @xcite conducted three tasks in 2019, of which one task is on offensive and non-offensive comments detection from english tweets. the dataset (olid) used for the task has 13240 tweets for training and 860 tweets for testing. several models like convolutional neural networks (cnn), bidirectional encoder representations from transformers (bert), long short term memory (lstm), lstm with attention, embeddings from language models (elmo) were used by various teams. even basic machine learning models like svm was a part of the assorted models used. semeval 2020 conducted a task on offensive language identification in multilingual languages (of-fenseval) such as english, arabic, danish, greek, and turkish. the same task was also conducted for indo-european languages in fire 2019 @xcite . in 2020, fire conducted a shared task called hate speech and offensive content identification from code-mixed posts in dravidian languages (malayalam-english and tamil-english) @xcite @xcite . different teams came up with diversified approaches of which include, the work by gaurav arora @xcite . he came up with an approach based on a pretraining ulm-fit on code-mixed data, which are generated synthetically. the code-mixed data was modeled as a markov process using markov chains. ## dataset description the dataset @xcite , @xcite , @xcite , @xcite ) consists of sentences from three code-mixed languages namely tamil-english, malayalam-english and kannada-english. the kannada-english and tamil-english dataset have sentences labeled to six classes and, the malayalam-english dataset has five labels. the labels for each language are given in ## system description this section describes the details of the model submitted to the shared task. we have experimented with various deep neural networks for identifying the underlying patterns in the text required for classification. ## results and discussion we experimented with three deep learning models for three subtasks in the shared task. the first model, model-1 is a hybrid of cnn, bi-lstm, and a fully connected layer apart from the output layer, the second model, model-2, has a bi-lstm layer, and the third model, model-3, is made up of a bi-rnn layer. we used validation data to evaluate the models to identify the best performing model. the performance was measured using metrics such as accuracy, precision, recall, and auc. among the three models, the network containing cnn+bi-lstm+dense layers achieved the best scores. even though all the models exhibited comparable accuracy, the decisive factor was the recall score. the hybrid model performed substantially better than the other two models in terms of recall and precision. besides that, it is also evident that the hybrid model could use the class weights effectively. this trend is visible in all three tasks. tables 4 , 5 , and 6 shows the training performance of the malayalam-english dataset, tamil-english dataset and kannada-english dataset, respectively. we submitted the predictions obtained by model-1 based on the training performance. the performance of the submitted model over the testing data is given in ## conclusion this paper presents the submission of amrita cen nlp to the shared task at eacl 2021 on offensive language identification from three dravidian languages, namely tamil-english (tam-eng), malayalam-english (mal-eng), and kannada-english (kan-eng). three deep learning architectures, such as a hybrid network with a convolutional layer, a bidirectional long short-term memory network (bi-lstm) layer, and a fully connected network, a network containing a bi-lstm, and another with a bidirectionalrecurrent neural network (bi-rnn) were implemented. the class imbalance problem was solved using the costsensitive learning approach. the hybrid of cnn, bi-lstm, and a fully-connected layer model gave the highest result of 90% accuracy for mal-eng, 64% accuracy for tam-eng, and 65% accuracy for kan-eng.