Bias, Threat and Aggression Identification Using Machine Learning Techniques on Multilingual Comments (2022)

Abstract: In this paper, we presented our team " IIITRanchi ” for the Trolling, Aggression and Cyberbullying (TRAC-3) 2022 shared tasks. Aggression and its different forms on social media and other platforms had tremendous growth on the Internet. In this work we have tried upon different aspects of aggression, aggression intensity, bias of different forms and their usage online and its identification using different Machine Learning techniques. We have classified each sample at seven different tasks namely aggression level, aggression intensity, discursive role, gender bias, religious bias, caste/class bias and ethnicity/racial bias as specified in the shared tasks. Both of our teams tried machine learning classifiers and achieved the good results. Overall, our team " IIITRanchi ” ranked first position in this shared tasks competition.

## introduction social media is an open platform where users can interact, share, learn and behave openly with other online users. due to the high demand and popularity of these media, aggression and its manifestations in different forms have taken unprecedented proportions. users of these media are generally writing their posts in multilingual forms @xcite . so, identification of such kinds of aggression, threats and biases are not an easy task due to various reasons like these comments are unstructured, multilingual, short forms and highly contextual in nature. due to these challenges, the research communities are very much interested in such kinds of automated identification. we have tried to develop systems that could automatically identify and separate these posts from the normal posts on the aggression shared dataset @xcite . for the given tasks, we have attempted all seven different categories of the text and classified them into their classes using different machine learning classifiers. the main motive of the work is to develop an efficient machine learning system to detect the aggression, biased and threatening contents on the social media platform which can be removed and altered afterwards. this will prevent the negative impact on many users and hate that may spread in society. the proposed models have different machine learning algorithms and we have fine tuned the models with different hyper-parameters which we have found for testing and cross validation phases. we found better results for all the shared tasks and ranked first. our team (iiitranchi) ranked first on all the task1, task1 surprise tests, and also in task2. in the preceding section, we have discussed a detailed description of the some related works, dataset, the preprocessing steps involved, the initial challenges and the models which we used for our use case. ## related work the variety of aggression related works have been proposed by researchers in the last few years. aggression related shared tasks were proposed by the organising team of shared tasks on aggression identification in every second year 2018 @xcite , 2020 @xcite and 2022 @xcite . some of the recent works are discussed as: at first, we are discussing some of the important works on 2018 aggression dataset @xcite . the work @xcite used ensemble learning and data augmentation techniques. they augmented english training dataset with the help of machine translation using three languages (french, german and spanish) by preserving the meaning of comments with different wording. their system was not stable for hindi dataset across the platforms (facebook and twitter). their system is not stable, especially for hindi dataset for the same domain it was performed well, but for other domain, it fails to classify the tweets with good accuracy. aroyehun and gelbukh (aroyehun and gelbukh, 2018) used various deep learning models such as long short term memory (lstm), cnn, and fasttext as word representation and data augmentation techniques by machine-translating the original post into different languages and then translated back to the original language. their system was not clearly classified covertly aggressive comments from overtly aggressive comments with significant accuracy. julian and krestel @xcite and aroyehun and gelbukh (aroyehun and gelbukh, 2018) found that augmentation of training data gives a better result. raiyani et al. @xcite used dense system architecture and compared several models such as dense neural network, fasttext and voting-based ensemble model. they found that simple three-layer dense neural network was performing better than the other two (fasttext and voting-based ensemble classification) models. their system has continued to suffer from false-positive cases and has also overlooked words that are not available in their vocabulary. some important works on 2020 aggression dataset @xcite . julian and krestel @xcite uses transformer based multiple fine-tuned bert models based on bagging technique and found very good results. the work @xcite also used the transformer based bert models and achieved good performance. ## dataset in this section, we discuss brief descriptions about datasets @xcite and given shared tasks. ## preprocessing in this section we are going to describe the preprocessing steps we did for cleaning the dataset. ## models we have considered naïve bayes, decision tree based random forest, logistic regression and support vector machines (svm) for text classifications. we have used training data on each model by performing grid-searchcv for all the combinations of feature parameters. we have analyzed performance on the basis of a weighted average micro f1-score of the cross validation. ## results and discussion in this section, we present our findings and observations of this work. ## error analysis in this section, we presented error analysis of our models. the size of training data was sufficient for ml models as we came across a large number of vocabulary. since the metric used was micro f1-score and most of the tasks had only 2-3 classes we got good results as the micro f1-score came out to be above 0.70 on an average therefore class wise classification scores were also better. model performs well on most of the tasks. by good performance, we mean good class wise f1-score on all the respective classes. few of the tasks where there was a surplus of one class had a lesser macro average due to absence of context aware classification but mostly the model has outperformed all other techniques. the confusion matrix and classification reports of logistic regression model's performance on training data are given below: all the respective classes are encoded to one categorical numeral below is the dictionary for that. 'aggression': 'cag': 0, 'oag': 1, 'nag': 2, 'aggression intensity': 'ntag': 0, 'na': 1, 'cuag': 2, 'sth': 3, 'pth': 4, 'discursive role': 'na': 0,'cns': 1,'atk': 2,'ain': 3,'dfn': 4,'gsl': 5, 'gender bias': 'ngen': 0, 'gen': 1, 'gent': 2, 'communal bias': 'ncom': 0, 'com': 1, 'comt': 2, 'caste/class bias': 'ncas': 0, 'cas': 1, 'cast': 2, 'ethnicity/racial bias': 'neth': 0, 'eth': 1, 'etht': 2 8 conclusion our team secured the first position in the competition of the given shared tasks on bias, threat and aggression detection for given datasets. we found that the logistic regression classifier outperforms all the models on test data due to more generalization and better prediction nature in sparse data. the possible reason was that the dataset in sparse form was linearly separable, but the svm model being soft margin was not a generalized model for our case. svm model was overfitting on train data but logistic regression generalized the metrics and hence performed really well in our case. at the end, we would like to conclude with the possibility that there exists many of the techniques other than what we have presented in this paper. in order to improve the model's performance, we can go for ensemble techniques of the models which have performed well in order to increase the variance and make the model more generalized. ## acknowledgement we would like to thank mrinmoy mahato, amitesh patel , aman kapoor and ankit kumar of department