Avalanche at D ravidian L ang T ech: Abusive Comment Detection in Code Mixed Data Using Machine Learning Techniques with Under Sampling (2023)

Abstract: In recent years, the growth of online platforms and social media has given rise to a concerning increase in the presence of abusive content. This poses significant challenges for maintaining a safe and inclusive digital environment. In order to resolve this issue, this paper experiments an approach for detecting abusive comments. We are using a combination of pipelining and vectorization techniques, along with algorithms such as the stochastic gradient descent (SGD) classifier and support vector machine (SVM) classifier. We conducted experiments on an Tamil-English code mixed dataset to evaluate the performance of this approach. Using the stochastic gradient descent classifier algorithm, we achieved a weighted F1 score of 0.76 and a macro score of 0.45 for development dataset. Furthermore, by using the support vector machine classifier algorithm, we obtained a weighted F1 score of 0.78 and a macro score of 0.42 for development dataset. With the test dataset, SGD approach secured 5th rank with 0.44 macro F1 score, while SVM scored 8th rank with 0.35 macro F1 score in the shared task. The top rank team secured 0.55 macro F1 score.

## introduction in recent times, social media has emerged as a prominent platform for discussions due to its wide reach and accessibility. it has granted individuals the power to express themselves, but unfortunately, it has also become a breeding ground for attacks based on characteristics such as race, gender, sexual orientation, or even threats of violence towards others. according to the recent survey conducted by economic times, india in 2023 1 , 8 out of 10 urban women are using the internet for various purposes. nearly 83% of the people surveyed said that the 1 https://economictimes.indiatimes.com/news/india safety measures for the usage of internet need be enhanced. it states that, "key concerns of urban indian women when using the internet include online sexual harassment, trolling, abuse, extortion and fraud". hence it is a pressing need to identify the abusive content in internet and take necessary actions for that. to address the issue of data imbalance, sampling techniques are employed, and feature extraction is performed using count vectorizer and tf-idf. various machine learning classifiers are applied in the process of classifying the text as abusive or not and find the category. the paper is organized as follows: section 2 provides an overview of the relevant research conducted in the field. section 3 examines the provided dataset, and section 4 outlines the methodology employed for the task. the results obtained are presented in section 5, and the paper concludes with a summary in the final section. ## related work nobata et al., published in 2016 @xcite specifically addressing the detection of abusive comments. this seminal work introduced a methodology for identifying abusive language in various online platforms and utilized machine learning techniques for classification. it is widely recognized as one of the pioneering contributions in the field of abusive comment detection. waseem and hovy, published in 2016 @xcite focused on identifying predictive features for detecting hate speech on the twitter platform. the objective was to understand the characteristics of hate speech and develop effective detection models. davidson et al. (2017) @xcite addressed the challenge of automated hate speech detection and offensive language. their work involved constructing a dataset of twitter posts annotated for hate speech and employing machine learning algorithms to classify offensive content. the research aimed to develop robust models capable of identifying hate speech in social media. in 2018, founta et al @xcite conducted large-scale crowdsourcing to characterize abusive behavior on twitter. by collecting and analyzing a substantial amount of data, they sought to understand the prevalence and nature of abusive content. this study significantly contributed to the understanding of abusive behavior patterns and provided valuable insights for the development of detection systems. badjatiya et al. (2020) @xcite explored the application of deep learning techniques for hate speech detection in tweets. their research employed deep neural network architectures, including convolutional neural networks (cnns) and long short-term memory (lstm) networks, to classify tweets into different categories of hate speech. the study aimed to harness the power of deep learning for accurate hate speech detection. earlier we have worked on offensive language and misogyny detection for english language. offensive content is recognized in english tweets using deep learning techniques and machine learning techniques in @xcite . misogyny detection from the multimodal data with english language is done in @xcite . now we are experimenting our work on the low resource languages and code mixed data. anusha gowda @xcite spreading positive vibes or hope content on social media may help many people to get motivated in their life. to address hope speech detection in youtube comments, this paper presents the description of the models submitted by our team-mucic, to the hope speech detection for equality, diversity, and inclusion (hopeedi) shared task at association for computational linguistics (acl) 2022. this shared task consists of texts in five languages,namely: english, spanish (in latin scripts), and tamil, malayalam, and kannada. ## dataset analysis and preprocessing the provided dataset consists of comments extracted from social media platforms, primarily youtube, and the train data-set contains dimensions of 5948 rows and 2 columns and test dataset contains dimensions of 1856 rows and 1 column, and is available in both tamil and english languages @xcite . these comments are categorized into different classes, namely misogyny, misandry, xenophobia, transphobia, homophobia, counter-speech, and not abusive. initially, the data-set contains unwanted special characters and emojis. most of the comments in the dataset are short, typically consisting of a single sentence, with an average sentence count close to 1. for reference, the number of classes in training and development dataset with their count have been listed in it is important to note that the dataset exhibits a significant class imbalance, with some categories being more dominant than others. this class imbalance can potentially lead to biased predictions favoring the majority class during model training. to address the above mentioned issues, it is necessary to preprocess the raw dataset. the preprocessing step involves cleaning the data by removing special characters, punctuation, and irrelevant words that do not contribute significantly to the overall category or meaning of each comment. ## methodology the methodology involves the steps of data preprocessing, class balancing, encoding, feature extracting, model building, evaluating and fine tuning the model. after extracting the necessary features from the cleaned dataset, we used classifier algorithms namely stochastic gradient descent (sgd) and support vector machine (svm) to train the model and to predict the results i.e type of comment from the comments given in the dataset. ## observation results for reference, the models under consideration for the tamil-english dataset have been listed in in the study conducted on the tamil-english dataset, we employed two different classifiers, namely stochastic gradient descent (sgd) and support vector machine (svm) along with a simple transformer that involved pipelining. to convert categorical data into numerical labels, label encoding was applied. count vectorizer and tf-idf vectorizer are used for extracting features from text data. we evaluated the performance of various models and selected the best ones to generate performance scores for the test dataset. using the sgd classifier with both count vectorizer and tf-idf features, our model achieved a f1 score of 0.45 and an accuracy of 0.73 for the development dataset. in the case of the svm classifier with tf-idf vectorizer, the model attained a a f1 score of 0.42 and an accuracy of 0.72 for the development dataset. for the test dataset sgd classifier achieved 0.44 f1 score and svm achieved 0.35 f1 score. our submission achieved the 5 th rank in the test evaluation for sgd and 8 th rank for svm. ## inferences based on the observation, it can be noted that the datasets used in the study are relatively small, resulting in a limited number of training samples. since the dataset is small, we identified that deep learning methods are not giving good results when ## conclusion and future work in this study, we have performed a comprehensive analysis of different models for the dravidian-langtech@ranlp 2023 shared task focused on detecting abusive comments. we investigated the effectiveness of multiple classifiers on the preprocessed data by extracting relevant features. our findings indicated that the sgd classifier produced comparable results using both vectorizers. furthermore, we observed that the svm classifier achieved a similar level of accuracy as the sgd classifier. in future, we have planned to increase the accuracy and f1-score by involving other feature extraction techniques and augmentation techniques. the potential challenges for further research in this field includes multilingual and multimodal settings, adversarial attacks and domain and cultural variations. the directions for further research includes user-adaptive models, continuous learning, explainable ai and real-time detection.