Quartet@ LT - EDI 2024: A Support Vector Machine Approach For Caste and Migration Hate Speech Detection (2024)

Abstract: Hate speech refers to the offensive remarks against a community or individual based on inherent characteristics. Hate speech against a community based on their caste and native are unfortunately prevalent in the society. Especially with social media platforms being a very popular tool for communication and sharing ideas, people post hate speech against caste or migrants on social medias. The Shared Task LT–EDI 2024: Caste and Migration Hate Speech Detection was created with the objective to create an automatic classification system that detects and classifies hate speech posted on social media targeting a community belonging to a particular caste and migrants. Datasets in Tamil language were provided along with the shared task. We experimented with several traditional models such as Naive Bayes, Support Vector Machine (SVM), Logistic Regression, Random Forest Classifier and Decision Tree Classifier out of which Support Vector Machine yielded the best results placing us 8th in the rank list released by the organizers.

## introduction hate is a very strong emotion or feeling of not liking someone or something. hate expresses intense hostility towards others. hate speech reflecting the same, refers to the offensive remarks or comments uttered by a person targeting a community or an individual person. hate speeches are often uttered based on the target's inherent characteristics such as race, ethnicity, religion or gender. in recent days, it can be said that social media platforms have enormously changed the way people communicate with one another (o' @xcite . social media can be seen as an immensely great tool for people to share their thoughts and ideas with the world. this has allowed the people to voice out their opinions broadening their freedom of speech. while this can benefit people a lot, it also comes with its own disadvantages. social media platforms are a place where individuals voice out their opinions, but there are also people who spread hate against a community or individual. we see hate speeches being posted in social media platforms very often @xcite . hate speeches targeting a particular community based on their caste and native place are prevalent in social media. hate speech inflicts immediate harm on its victims and also contributes to discrimination against the targeted community or individual. these sort of hate comments against a caste or migrants must be obliterated from the society. with social media platforms being an inevitable and popular tool in the modern society, it becomes imperative to moderate the hate speech posts. it is essential for a more positive and inclusive society. sentiment analysis, also known as opinion mining can be employed to detect and moderate the hate comments prevailing on social media platforms. sentiment analysis is the process of determining the emotional tone that a digital text manifests @xcite . textual data can be analyzed and determined if the text expresses a positive, negative or neutral sentiment. sentiment analysis is an immensely powerful tool to automate the process of detecting caste and migration hate speech on social media by analyzing the sentiment or emotion that the text manifests. sentiment analysis can be carried out by supervised learning in case of availability to a well labelled and quality training data. in situations where one has no access to training data, unsupervised learning can also be utilized to perform sentiment analysis @xcite . the shared task lt-edi 2024: caste and migration hate speech detection was created with the motive to build an automation system that detects and classifies the text in tamil language on social media platforms as caste and migration hate speech or not. datasets containing text in tamil language were provided along with the shared task. this paper is organized as follows: section 2 encompasses the related works as per the literature survey; section 3 is entails information about the task and data; section 4 pertains to the methodology used to build the classification system; section 5 shows the results and analysis; section 6 entails the conclusion; sections 7 and 8 pertains to the limitaions of the model and the ethics statement respectively. ## related works sentiment analysis is a field in which constant works and researches are being carried on. they have many applications on social and e-commerce platforms. @xcite proposed a hate speech detection classifier by replacing or integrating the word embeddings (fasttext(ft), glove(gv) or ft + gv) with static word bert embeddings. with extensive experimental traits it is observed that the performance of a neural network with static bert embeddings is better than that wit ft, gv or ft + gv. a large-scale analysis of multilingual hate speech in 9 languages from 16 different sources was conducted by @xcite . it was observed that in low resource setting, simple models such as laser embedding with logistic regression performs the best, while in high resource setting bert based models perform the best. hatebert, a re-trained bert model was proposed by @xcite for abusive language detection in english. the model was trained on ral-e, a large-scale dataset of offensive reddit comments in english. @xcite built an automatic hate speech detection system against women by generating three types of features from the text: sentence embeddings, tf-idf vectors and bow vectors. these features were then concatenated and fed into a logistic regression model. @xcite experiments several machine learning models to classify hate speech in tamil texts. several models including bert, xlm-roberta, indicbert, mbert, tamillion and muril were experimented. it was observed that the highest performance was achieved by a combination of stemming the text data, embedding it with muril and using a majority voting ensemble as the downstream classifier. alatawi et al. (2021) investigates the feasibility of leveraging domain-specific word embedding in bidirectional lstm based deep model to automatically detect/classify hate speech. furthermore, the use of the transfer learning language model (bert) on hate speech problem as a binary classification task was investigated. ## task and data description the objective of the shared task lt-edi 2024: caste and migration hate speech detection foot_0 (rajiakodi et al., 2024) is to create an automatic classification system that detects and classifies whether a text is caste and migration hate speech or not. training and development datasets were provided in tamil language. the dataset encompassed two fields: text and label. the training dataset had a total of 5,355 records out of which 2,052 were labelled 1 representing caste and migration hate speech and 3,303 were labelled 0 representing non caste and migration hate speech. figure 2: data distribution in development dataset the development dataset had a total of 945 records out of which 351 were labelled as hate speech and 594 were labelled as non-hate speech. ## result and analysis the performance of various traditional models including naive bayes, support vector machine (svm), random forest classifier, decision tree classifier and logistic regression were evaluated and compared. the receiver operating characteristic (roc) curve was plotted and the area under receiver operating characteristic (auroc) was calculated for all the models. the roc curve is defined as the curve that is plotted against the true positive rate and false positive rate of the predictions obtained from a model at varying threshold levels. the roc curve is a very useful visual representation to analyze and compare the performance of classification models. on evaluating the metrics of the models, it is found that support vector machine (svm) produced the best numbers on both training data and development data. it is to be noted that though random forest classifier performed very slightly better than svm on unseen development data, with the macro average score on training data being 1.00, the model is considered overfitted. on evaluating with the test data given by the organizers, the svm model yielded a macro average f1 score of 0.74. we were ranked 8th in the rank list released by the organizers. ## conclusion by means of this paper, we experimented several traditional machine learning models on the features extracted by the tf-idf vectorizer. the metrics and roc curves of each model were plotted and analysed to effectively compare the performance of the models. it was observed that out of all models, support vector machine (svm) gave the best metrics and roc curve. while this model has good performance over the other models, it is to be noted that better results can be obtained by utilizing neural networks and more complex embedding systems.