MUCIC at C om MA @ ICON : Multilingual Gender Biased and Communal Language Identification Using N-grams and Multilingual Sentence Encoders (2021)

Abstract: Social media analytics are widely being explored by researchers for various applications. Prominent among them are identifying and blocking abusive contents especially targeting individuals and communities, for various reasons. The increasing abusive contents and the increasing number of users on social media demands automated tools to detect and filter the abusive contents as it is highly impossible to handle this manually. To address the challenges of detecting abusive contents, this paper describes the approaches proposed by our team MUCIC for Multilingual Gender Biased and Communal Language Identification shared task (ComMA@ICON) at International Conference on Natural Language Processing (ICON) 2021. This shared task dataset consists of code-mixed multi-script texts in Meitei, Bangla, Hindi as well as in Multilingual (a combination of Meitei, Bangla, Hindi, and English). The shared task is modeled as a multi-label Text Classification (TC) task combining word and char n-grams with vectors obtained from Multilingual Sentence Encoder (MSE) to train the Machine Learning (ML) classifiers using Pre-aggregation and Post-aggregation of labels. These approaches obtained the highest performance in the shared task for Meitei, Bangla, and Multilingual texts with instance-F1 scores of 0.350, 0.412, and 0.380 respectively using Pre-aggregation of labels.

## introduction in the past few years, the spread of internet is gradually increasing the user-generated content over various platforms. consequently, aggressive and hateful content like trolling, cyberbullying, flaming, abusive language, etc. is also growing alarmingly @xcite . these abusive contents targeting individuals and communities for various reasons is creating negative impact on individuals as well as on the society @xcite . detection of such abusive contents on social media is a crucial task. filtering these contents manually is almost an impossible task due to the increasing number of social media users as well as increasing abusive contents. this demands an automated abusive content detection system that aims to reduce the abusive contents and discourage users from demonstrations of any form of aggression. recently, several shared tasks such as sexism identification in social networks @xcite , arabic misogyny identification @xcite , etc. have explored the detection of abusive contents in different languages. to tackle the challenges of detecting the abusive contents on social media, in this paper, we team mucic, present two ml approaches proposed for comma@icon shared task at icon 2021 @xcite . the shared task is defined as a threelevel (level a, b and c) multi-label tc task for code-mixed multi-script texts in three languages: meitei, bangla, hindi as well as in multilingual (a combination of meitei, bangla, hindi, and english). while level a is a multi-class classification task with three categories, level b and c are binary classifications. the shared task could be approached as three separate classification tasks or a multi-label classification task or a structured classification task. however, the final submission file must contain the labels for each of the three levels as one single predicted tuple. the shared task is modeled as a multi-label tc task combining word and char n-grams with vectors obtained from mse to train three ml classifiers using pre-aggregation and post-aggregation of labels. ml classifiers, namely: logistic regression (lr), support vector machine (svm) and random forest (rf) are ensembled as a soft voting classifier. the results released by shared task organizers show that the proposed approaches obtained highest performance for meitei, bangla, and multilingual texts using pre-aggregation of labels. the rest of the paper is organized as follows: section 2 throws some light on some of the recent works in detecting abusive contents in general, followed by the proposed methodology to detect gender biased and communal language identification in section 3. experiments and results are brought out in section 4 and the paper finally concludes in section 5. ## related work hateful content detection is a very challenging task. in the last few years, there have been several studies proposing several methods for the classification of offensive and hateful speech @xcite al. (2017); hardaker (2013); dadvar et al. (2013) ; @xcite . few researchers have also shown that taking context dependencies into account can improve hateful speech detection system considerably @xcite ; @xcite ; @xcite . several studies have looked into different types of abusive languages like hate speech, cyberbullying, and trolling. @xcite suggested classification of abusive language as a two-fold topology that considers whether (i) abusive content is either directed towards a specific individual or a general one and (ii) abusive language is explicit (unambiguous in its potential) or implicit (does not immediately apply or denote abuse). @xcite approached cyberbullying detection as a tc task using content-based, cyberbullying-specific and user-based features to train a svm to classify comments as bullying or non-bullying. this study proves that incorporating user's context such as comments history and characteristics can considerably improve the performance of cyberbullying detection tools. @xcite compiled the offensive language identification (oli) dataset in english with tweets annotated using a fine-grained threelayer annotation scheme to distinguish whether the language is offensive or not along with its type and target. among the experiments conducted using svm, bidirectional long short-term-memory (bilstm) and convolutional neural network (cnn), cnn models outperformed other models for oli, its type and target with macro-f1 scores of 0.80, 0.69, and 0.47 respectively. the hindi-english code-mixed dataset devel-oped by @xcite is crawled from the public pages of facebook and twitter consisting of the posts about the issues that are expected to be discussed more among the indians. with approximately 18k tweets and 21k facebook comments, the dataset was annotated with different levels and types of aggression, such as physical threat, sexual aggression, gender aggression, etc. using the crowdflower platform. @xcite developed a ml based method to detect hate speech in online user comments. data was sampled from comments posted on yahoo! finance and annotated by new yahoo's in-house trained raters. experiments were performed by training vowpal wabbit's regression model using n-grams, linguistic, syntactic and distributional semantics features as well as different types of embeddings combined with the standard natural language langauge (nlp) features. the models with a combination of all the features achieved best f1-scores of 0.795 and 0.817 for finance and news data respectively. in spite of several techniques to detect abusive language in code-mixed script, very few works focus on indian languages. this provides lot of scope to carry out experimentation on indian particularly low-resource languages and also multilingual text and script. ## methodology inspired by @xcite ; fazlourrahman and shashirekha (2021) in utilizing various types and combinations of n-grams for code-mixed multi-scripts tc tasks, this work transforms word and char n-grams in the range (1, 3) to term frequency-inverse document frequency (tf-idf) vectors and stacks them with vectors extracted from mse foot_0 . the stacked vectors are then used to train the ml classifiers. range of word and char n-grams and the vector size of all the features for all the languages of the shared task are given in two approaches used for labels aggregation to train ml classifiers are described below: the difference between the two approaches lies in aggregating the labels as shown in figure 1 . while the blue dotted part indicates the model's prediction using pre-aggregation approach, red dotted part indicates that of post-aggregation approach. both the approaches use the same feature engineering step. model construction part consists of soft voting ensemble of rf, svm, and lr classifiers. the classifiers are selected based on their success in fazlourrahman et al. (2021a,d) for code-mixed multiscript tc tasks. the classifiers are empowered with hyperparameter tuning using gridsearchcv module from sklearn library foot_1 . a set of random values are assigned for each parameter corresponding to a particular classifier and then gridsearchcv is used to determine the best value for each parameter. however, the limitation of hyper-parameter tuning is that it requires a lot of time to find the best value for each parameter. owing to the time constraints, hyper-parameter tuning is done only for mulilingual dataset and those parameter values are in turn used for all the datasets. however, hyper-parameter tuning for each dataset separately is expected to enhance the performance of the classifiers. the final values of parameters for each classifier are presented in ## conclusion and future work this paper describes the models submitted by the team mucic for comma@icon shared task at icon 2021 for gender biased and communal language identification. the shared task is a three level multi-label tc task for code-mixed multiscripts texts in meitei, bangla, hindi as well as in multilingual. our previous work on code-mixed multi-scripts tc tasks is extended for this shared task with stacked word and char n-grams combined with mse vectors as features using preaggregation and post-aggregation of labels. a soft ensemble of three ml classifiers empowered by hyper-parameter tuning using gridsearchcv are trained with the stacked features for the three level multi-label tc task. the results of the shared task provided by the organizers show the highest results using pre-aggregation approach for meitei, bangla, and multilingual texts with instance-f1 scores of 0.350, 0.412, and 0.380 respectively. this illustrates the efficiency of the proposed approaches.