MUCS @ D ravidian L ang T ech2023: Leveraging Learning Models to Identify Abusive Comments in Code-mixed D ravidian Languages (2023)

Abstract: Abusive language detection in user-generated online content has become a pressing concern due to its negative impact on users and challenges for policy makers. Online platforms are faced with the task of moderating abusive content to mitigate societal harm, adhere to legal requirements, and foster inclusivity. Despite numerous methods developed for automated detection of abusive language, the problem continues to persist. This ongoing challenge necessitates further research and development to enhance the effectiveness of abusive content detection systems and implement proactive measures to create safer and more respectful online spaces. To address the automatic detection of abusive languages in social media platforms, this paper describes the models submitted by our team - MUCS to the shared task “Abusive Comment Detection in Tamil and Telugu” at DravidianLangTech - in Recent Advances in Natural Language Processing (RANLP) 2023. This shared task addresses the abusive comment detection in code-mixed Tamil, Telugu, and romanized Tamil (Tamil-English) texts. Two distinct models: i) AbusiveML - a model implemented utilizing Linear Support Vector Classifier (LinearSVC) algorithm fed with n-grams of words and character sequences within word boundary (char_wb) features and ii) AbusiveTL - a Transfer Learning (TL ) model with three different Bidirectional Encoder Representations from Transformers (BERT) models along with random oversampling to deal with data imbalance, are submitted to the shared task for detecting abusive language in the given code-mixed texts. The AbusiveTL model fared well among these two models, with macro F1 scores of 0.46, 0.74, and 0.49 for code-mixed Tamil, Telugu, and Tamil-English texts respectively.

## introduction abusive language encompasses the use of words to insult, demean, or harm others, often through vulgar or profane language, and can include sexism, misogyny, and other forms of discrimination @xcite @xcite . it may include words that provokes or aggravates an individual or a group of people. the phrase "abusive language" is also used synonymously with phrases like "offensive language" and "hate speech" @xcite . over the past few years, the prevalence of offensive behavior targeting individuals, groups, or entire communities on social media platforms has significantly increased @xcite @xcite . this rise is creating negative impact such as, cyber-bullying, usage of offensive language, hate speech, and triggering content etc., on the well-being of online users. hence, such negative content should be removed from the social media to keep online platforms healthy @xcite @xcite . despite the efforts of social media companies to combat offensive/abusive language, the problem continues to escalate due to the limitations of existing algorithms used for detecting such content @xcite @xcite . these algorithms often fail to grasp the nuances of subjectivity and context that are crucial in accurately identifying abusive language. for instance, a single message might seem innocuous when taken out of context, but within a thread of previous conversations, it can reveal a pattern of abusive behavior. similarly, certain phrases or words may have dif-ferent meanings depending on the context, making it challenging for algorithms to accurately assess their intent. this complexity poses difficulties for human reviewers who have to navigate through vast amounts of content. hence, achieving an effective and comprehensive solution to detect abusive language on online platforms require advancements in natural language processing (nlp), machine learning (ml) techniques and the ability to capture context in a more nuanced manner @xcite . it is a complex problem that necessitates ongoing research, collaboration between experts, mechanisms to create safer online environments. one of the challenges in addressing abusive language on social media is the prevalence of code -mixed data where regional languages like tamil, kannada, malayalam, etc., are combined with english at various levels such as sub-word, word, or sentence (hegde and shashirekha, 2022a). this linguistic diversity makes it difficult for abusive comment detection algorithms to accurately identify and categorize the offensive content. further, the use of internet slangs, abbreviations, words in short forms, words from other languages, and emojis complicates the issue. lack of annotated datasets specifically in low-resource languages like tamil and telugu pose an additional hurdle in developing effective abusive content detection algorithms for these languages @xcite . bridging this gap requires efforts to gather and annotate data in these languages to train models that can better understand and detect abusive content in diverse linguistic settings. to address the challenges of this shared task, in 1 https://codalab.lisn.upsaclay.fr/competitions/11096 this paper, we -team mucs, describe the two classification models: i) abusiveml model utilizing linearsvc fed with tf-idf of n-grams of words and char wb both in the range (1, 3) and ii) abu-sivetl -a tl model trained using three different versions of bert (tamil bert, telugu bert, and distilled multilingual bert (distilmbert)). the rest of the paper is arranged as follows: a review of related work is included in section 2 and the methodology is discussed in section 3. experiments and results are described in section 4 followed by concluding the paper with future work in section 5. ## related work abusive comments are statements that offend a person or a group of people. these comments are directed at people who belong to certain nationality, gender, caste, race, sexuality, etc. the objective of abusive content detection is to find abusive speech on social media platforms, such as hate speech, derogatory language, misogyny, and racism. the description of some works that are carried out to perform a similar task is given below: in their study, they combined language-agnostic sentence embeddings with the tf-idf of word vectors to train svm classifier and obtained an accuracy of 0.520 and macro f1 score 0.54. @xcite described the ml models (svm, linear classifier, mlp) for binary classification and, multi-class classification to detect the type of offensive content in three languages (english, german, and hindi). for both binary and multi-class classification, svm classifier trained with tf-idf of word n-grams in the range (1, 2) exhibited macro f1 scores of 0.66, 0.75, 0.46 and 0.42, 0.47, 0.23 for english, hindi and german languages respectively. balouchzahi et al. (2021c) submitted two distinct models: cooliensemble -an ensemble model of mlp, extreme gradient boosting (xgboost) and logistic regression (lr) trained using term frequencies and cooli-keras -a deep learning (dl) classifier, to identify offensive language in code-mixed kannada-english, malayalam-english, and rtamil texts. out of the two models, cooli-ensemble model outperformed the other model with weighted f1 scores of 0.97, 0.75, and 0.69 for malayalam-english, rtamil, and kannada-english respectively. hegde and shashirekha (2022b) proposed dynamic meta embedding (dme) based long short term memory (lstm) classifier to perform sentiment analysis and homophobia detection as task a (malayalam and kannada) and task b (tamil, english, rtamil) respectively. their proposed methodology exhibited macro f1 scores of 0.61, and 0.44 for malayalam, and kannada respectively in task a and for task b their models obtained macro f1 scores of 0.74 and 0.58 for english and rtamil languages respectively. das et al. (2021) explored three learning models (xgboost, lgbm, mbert) for abusive and threatening content detection in urdu. they trained xgboost and lgbm classifiers using pre-trained urdu laser embeddings. further, they fine-tuned mbert and dehatebertmono-arabic pretrained models for abusive and threatening content detection in urdu. their finetuned mbert models outperformed the other models with macro f1 scores of 0.88 and 0.54 for abusive and threatening content detection respectively. balouchzahi and shashirekha (2020) proposed three distinct models to identify hate speech in english, german, and hindi languages. they implemented i) ensemble of ml classifiers (rfc, lr, and svm) trained with tf-idf of word n-gram in the range (1, 2) and character n-grams in the range (1, 5), ii) tl based classifier using universal lan-guage model fine-tuning (ulmfit) model, and iii) a hybrid model which is an ensemble of ml (i)) and tl (ii)) models. the ensembled ml classifier obtained macro f1 score of 0.5044 for german and hybrid model obtained macro f1 score of 0.5182 for hindi. from the above related work, it is found that among ml, dl, and tl models, tl models outperformed the other models indicating the efficiency of the tl models in detecting abusive content on social media. though there are several models to identify abusive content in social media text, there is still scope for developing models for lowresource languages like tamil and telugu as these languages are not much explored in the realm of code-mixed content. ## methodology the objective of this work is to identify abusive comments in code-mixed tamil, telugu, and rtamil texts. this is achieved by proposing two distinct models, abusiveml and abusivetl. detailed description of the models are given below: ## experiments and results train, development, and test sets are provided by the shared task organizers @xcite for abusive language detection in codemixed tamil and telugu along with rtamil texts. multiple experiments are carried out, incorporating different resampling techniques (synthetic minority over-sampling technique (smote), random oversampling, and downsampling), feature combinations (pretrained word vectors, character count, and word count), and classifiers (lr, linearsvc, nb, and mlp). the models that exhibited considerably good performances on the development set were subsequently evaluated on the test set. figures 3 (a), 3 (b), and 4 show the label distribution in code-mixed tamil, rtamil, and telugu datasets respectively. the predictions of the proposed models are evaluated by the organizers of the shared task based on macro f1 score and performance of the proposed models on test and development sets are shown in in spite of using data imbalance handling mechanisms, for tamil and rtamil texts the macro f1 scores are still less. this may be due to the overlapping feature distributions in the train set. further, adding class @xmath0. while it generally helps to improve the performance, the macro f1 scores might decrease even after using this technique in certain scenarios, such as, data complexity, loss information, and features used. further, as random oversampling technique increase the instances in minority classes by duplicating samples, this can lead the model to become overly focused on the minority class, potentially causing overfitting. this means the model might perform exceptionally well on the train set but fail to generalize to new, unseen data. ## conclusion this paper describes the models submitted by our team -mucs, to "abusive comment detection in tamil and telugu" shared task at dravidian-langtech@ranlp 2023, to identify abusive content in code-mixed tamil, telugu, and rtamil texts. two models: i) abusiveml model that utilizes lin-earsvc algorithm fed with tf-idf of n-grams of words and char wb both in the range (1, 3) and ii) abusivetl model fine-tuned on oversampled train set with three different bert models (for three different languages), are proposed to detect abusive comments in the input text. abusivetl models outperformed the other models with macro f1 scores of 0.74, 0.46, and 0.49 securing 1 st , 1 st , and 4 th rank in the shared task for telugu, tamil, and rtamil texts respectively. efficient resampling techniques for handling imbalanced data with effective feature extraction will be explored further.