MUM at C om MA @ ICON : Multilingual Gender Biased and Communal Language Identification Using Supervised Learning Approaches (2021)

Abstract: Due to the rapid rise of social networks and micro-blogging websites, communication between people from different religion, caste, creed, cultural and psychological backgrounds has become more direct leading to the increase in cyber conflicts between people. This in turn has given rise to more and more hate speech and usage of abusive words to the point that it has become a serious problem creating negative impacts on the society. As a result, it is imperative to identify and filter such content on social media to prevent its further spread and the damage it is going to cause. Further, filtering such huge data requires automated tools since doing it manually is labor intensive and error prone. Added to this is the complex code-mixed and multi-scripted nature of social media text. To address the challenges of abusive content detection on social media, in this paper, we, team MUM, propose Machine Learning (ML) and Deep Learning (DL) models submitted to Multilingual Gender Biased and Communal Language Identification (ComMA@ICON) shared task at International Conference on Natural Language Processing (ICON) 2021. Word uni-grams, char n-grams, and emoji vectors are combined as features to train a ML Elastic-net regression model and multi-lingual Bidirectional Encoder Representations from Transformers (mBERT) is fine-tuned for a DL model. Out of the two, fine-tuned mBERT model performed better with an instance-F1 score of 0.326, 0.390, 0.343, 0.359 for Meitei, Bangla, Hindi, Multilingual texts respectively.

## introduction the advancement in internet technology and social media websites have made the information reach the wider audience within no time. these characteristics of social media sites are attracting more and more people towards them leading to exponential rise in the amount of user-generated content on social media. in addition to the exchange of constructive and useful information, few miscreants are taking advantage of the anonymity of users and spreading the abusive and potentially harmful content over the web. while the act of bullying and hate speech kind of things existed very much before the internet, the reach and influence of the internet have given these acts unprecedented power and influence to affect the lives of many people. according to the report by @xcite , these incidents have caused not only mental and psychological agony to the social media users, but have also forced some of them for suicidal attempts in the extreme cases. abusive, aggressive, communal hate speech and any other forms of potentially harmful content getting generated on social media needs to be filtered out almost instantaneously in order to stop the further spread and the damage it is going to create. filtering this harmful content manually is almost impossible due to the voluminous amount of data getting generated and also due to the increasing number of social media users. this has received the attention of the research community in recent years to automatically detect such content on social media @xcite . identifying the harmful content in social media data automatically is challenging as the social media data which is usually code-mixed do not adhere to the rules and regulations of any language. further, in a multilingual country like india people tend to pen comments using words from multiple languages making the analysis of social media data more challenging. to address some of the challenges in identifying gender biased and communal language in codemixed, multi-scripted, multilingual content on social media, this paper describes the models submitted to comma@icon 1 shared task at icon 65 2021 foot_1 . the shared task is a multi-label three level (level a, b and c) text classification (tc) task in code-mixed and multi-scripted texts in meitei, bangla, hindi, and also in multilingual (a combination of meitei, bangla and hindi). this shared task is addressed by constructing i) ml classifier with elastic-net penalty which is trained using word unigrams and char n-grams combined with emoji vectors and ii) fine-tuning a pre-trained multi-lingual bidirectional encoder representations from transformers (mbert) as a dl model. the rest of the paper is arranged as follows: the recent literature related to detection of abusive content in social media data is summarized in section 2 and the proposed methodology is described in section 3. experiments and results are presented in section 4 followed by conclusion and future work in section 5. ## related work several models have been developed by researchers to detect offensive and abusive content in social media text @xcite . the description of some of the recent works are mentioned below: @xcite have proposed ml approaches using logistic regression (lr) with elastic-net penalty and without penalty, naive bayes (nb), support vector machines (svm), and random forest (rf), all trained with word unigrams and bi-grams nd evaluated on the swedish twitter dataset containing the public opinion about ikea -a popular store. lr model with elastic-net penalty trained using word bi-grams outperformed other models with an accuracy of 0.81 and f1-score of 0.79. @xcite proposed a multilingual toxic text classifier integrating multiple pre-trained models and different loss functions and evaluated its performance on jigsaw multilingual toxic comment dataset. the proposed learning pipeline begins with a series of preprocessing steps, including translation, word segmentation, purification, digitization, and vectorization to convert word tokens into vectors suitable for tc. mbert and cross-lingual language model-robustly optimized bert (xlm-roberta) are employed for pre-training through masking language modeling and translation language modeling to incorporate the semantic and contextual information. the results of experiments show that fusion of loss func-tions and fusion of multilingual models outperform the mbert and xlm-r models by obtaining f1scores of 0.505 and 0.76 respectively, justifying the effectiveness and robustness of fusion strategy. anusha and shashirekha (2020) have described the work submitted to subtasks (a and b) of hate speech and offensive content identification (hasoc) shared task in forum for information retrieval evaluation (fire) 2020 to identify hate and offensive content in indo-european languages. they combine term frequency -inverse document frequency (tf-idf) vectors with additional textbased features to build an ensemble of gradient boosting, rf, and xgboost classifiers, with soft voting. for subtasks a and b, they obtained macroaveraged f1-score of 0.5046 and 0.2596 for english, 0.5106 and 0.2595 for german, and 0.5033 and 0.2488 for hindi respectively. tit ¸a and zubiaga (2021) aims to classify english and french text into "hateful" (hate speech data) and "non-hateful" (clean/neutral data) categories by fine-tuning mbert and xlm-roberta on task-specific datasets. the mbert and xlm-roberta models achieved weighted averages of 0.71 and 0.41 for english and 0.52 and 0.55 for french, respectively. tanase et al. (2020) fine-tuned bert, mbert and xlm-roberta -the pre-trained transformerbased architectures using different combinations of task-specific datasets for tackling the problem of aggressiveness detection in mex-a3t@iberlef2020 shared task. xlm-roberta model achieved an f1-score of 0.7969, the thirdbest score in the competition which proves that transformer-based models can be successfully used to detect aggressiveness in mexican spanish tweets. @xcite trained a set of multiclass classifiers such as lr, nb, decision trees, rf, and linear svm to categorize tweets into one of 'hate speech', 'offensive but not hate speech', and 'neither offensive nor hate speech' categories. their best-performing model achieved a precision of 0.91, recall of 0.90, and an f1-score of 0.90. @xcite trained a lr algorithm with linguistically motivated features and different types of n-grams to identify if a tweet is aggressive or not in the aggressive detection track at mex-a3t 2018. they applied several pre-processing steps to standardize tweets in order to capture relevant information and achieved 0.4285 f1-score. ## methodology the proposed methodology consists of preprocessing, feature extraction and model construction as explained below: