Beware Haters at C om MA @ ICON : Sequence and Ensemble Classifiers for Aggression, Gender Bias and Communal Bias Identification in I ndian Languages (2021)

Abstract: Aggressive and hate-filled messages are prevalent on the internet more than ever. These messages are being targeted against a person or an event online and making the internet a more hostile environment. Since this issue is widespread across many users and is not only limited to one language, there is a need for automated models with multilingual capabilities to detect such hostile messages on the online platform. In this paper, the performance of our classifiers is described in the Shared Task on Multilingual Gender Biased and Communal Language Identification at ICON 2021. Our team “Beware Haters” took part in Hindi, Bengali, Meitei, and Multilingual tasks. Our team used various models like Random Forest, Logistic Regression, Bidirectional Long Short Term Memory, and an ensemble model. Model interpretation tool LIME was used before integrating the models. The instance F1 score of our best performing models for Hindi, Bengali, Meitei, and Multilingual tasks are 0.289, 0.292, 0.322, and 0.294 respectively.

## introduction this project is a demonstration of the capabilities of sophisticated text based machine learning classifiers which contributed to identifying various hostile features of text data that are provided as a part of the competition for the icon conference. ever since social media has become mainstream and gained millions of active users everyday, it has played a pivotal role in various events of information exchange, like photos, comments, tweets etc. as social media platforms encourage free speech from all their users, people can express their opinions on everything they view. as the number of people and this interaction over the web has increased, incidents of aggression and related activities like trolling, cyberbullying, flaming, hate speech, etc. have also increased manifold across the globe @xcite . types of hate speech include biased comments against a specific gender, certain caste, race, community or just speech containing abusive language. so it became necessary to find automated solutions to identify hate and abusive text on these platforms to make them a safe place for everyone. in this paper, the performance of deep learning and ensemble classifiers are discussed and analyzed. our team "beware haters" participated in the shared task of building models to perform classification on multilingual data provided which contained text in three different indian languages namely hindi, bengali, meitei along with english. each row in the data contains three different labels belonging to communal bias, aggressive, gender bias. the task is to build and train models to perform classification over the data concerning these three labels. our team also participated in the individual tasks where the training and the testing data given are purely in one of the particular languages mentioned previously. this section gives a detailed description of the shared tasks along with the necessary datasets. this dataset @xcite will also contain the description of the datasets for all the languages. our team participated in bengali, hindi, meitei, and the multilingual track of the competition. hindi 2 is an indo-aryan language predominantly spoken in northern parts of india. bengali 3 is the national language of bangladesh and is also spoken in a few parts of india. the training dataset contains only texts in the indian varieties of bangla. meitei 4 is a tibeto-burman language mainly spoken in the northeastern state of manipur. hindi and bengali texts are written both in english and the respective language. the texts in meitei are written in english script. ## experimental setup only the dataset provided by the organizers were used for all the tasks that we participated in. the training and development datasets are both used for training the models. detailed description of datasets is provided in [fig. 5 ]. ## conclusion we participated in the shared task on multilingual gender biased and communal language identification. the sub tasks required building and testing models for multiclass classification task on aggression identification and binary classification tasks on gender bias identification and communal bias identification. the datasets have been provided in bengali, hindi, meitei and finally on multilingual data. ensemble classifier consisting of logistic regression, random forest and svm performed better compared to bi lstm for hindi, meitei, bengali in both subtask b and c. but for multilingual data, bi-lstm has performed better in these subtasks. however, in the final submission for subtask a, logistic regression has performed better compared to the other models tested. our team "beware haters" ranked 1 st in the leaderboard figure 18: lime explanations for misclassified example of gender bias identification for the meitei and bangla dataset and 3 rd for both hindi and multilingual datasets. we further aim to improve the performance at subtasks by using transformer models like xlm roberta which have been proven to perform better on multilingual datasets. we also aim to explore other deep learning models which might achieve better performance compared to what our models achieved.