I ts A ll G ood M an@ LT - EDI -2025: Fusing TF - IDF and M u RIL Embeddings for Detecting Caste and Migration Hate Speech (2025)

Abstract: Caste and migration hate speech detection is a critical task in the context of increasingly multilingual and diverse online discourse. In this work, we address the problem of identifying hate speech targeting caste and migrant communities across a multilingual social media dataset containing Tamil, Tamil written in English script, and English. We explore and compare different feature representations, including TF-IDF vectors and embeddings from pretrained transformer-based models, to train various machine learning classifiers. Our experiments show that a Soft Voting Classifier that make use of both TF-IDF vectors and MuRIL embeddings performs best, achieving a macro F1 score of 0.802 on the test set. This approach was evaluated as part of the Shared Task on Caste and Migration Hate Speech Detection at LT-EDI@LDK 2025, where it ranked 6th overall.

## introduction in india, caste and migration-based hate speech is a pervasive problem that has long been the focus of political discussion and still can be observed a lot in online forums and digital spaces. this carries severe real-world consequences, including psychological harm, the increase in social divisions, and the potential for inciting offline violence, particularly in linguistically diverse regions @xcite . this online discrimination takes place in numerous forms, including direct hate speech, derogatory remarks targeting specific castes and migration groups, cyberbullying of individuals based on their identity, threats of violence or social ostracism, and exclusion from online communities and groups. with the boom of social media, such biases have found new avenues to spread, often under the disguise of free speech and anonymity. the spreading of hate speech targeting caste and migration groups not only normalizes discrimination but also reinforces harmful stereotypes, further marginalizing already vulnerable communities. given the vast volume of online discourse and the rapid spread of harmful content, there is a pressing need to develop a system that is capable of recognizing and addressing such biases in real time. existing research on detection of hateful speech has primarily focused on widely spoken languages such as english. the complexity of code-mixed and regional language discourse prevalent in multilingual societies like india is often overlooked. tamil, a widely spoken dravidian language, frequently appears in code-mixed forms with english and other regional languages, making hate speech detection in tamil code-mixed text a challenging task. the lack of sufficient annotated datasets with code-mixed text further complicate the problem. recent studies have explored a variety of transformer-based, machine learning, and data augmentation approaches for the detection of hate speech in tamil, particularly in code-mixed and multilingual contexts. using an ensemble of models such as xlm-roberta, multilingual-cased bert, and muril, one of the best-performing models in the lt-edi-eacl 2024 shared task achieved an f1-score of 0.82 @xcite . another submission to this shared task evaluated 12 pre-trained transformer models in indian multilingual language settings and found that muril-large was the most effective, with an f1score of 0.81, which was obtained by ensembling the top-performing models @xcite . another method experimented with transformers, fasttext, and tf-idf; mbert did the best with an f1-score of 0.80 @xcite . to counter the difficulty of detecting masked abusive language in regional languages, @xcite developed a system for tamil and malayalam by using supervised learning techniques on roberta text embeddings. researchers have also investigated multimodal hate speech detection that incorporates text, speech, and video data in addition to shared tasks. a study that compared several tamil language models, such as tamil-bert, labse, hate-muril, and muril-large-cased, concluded that tamil-bert was the most successful @xcite . another paper focused on detecting offensive language in tamil-english code-switching by highlighting the potential of a hybrid system using both kans and standard classifiers to improve detection accuracy @xcite ). an averaging ensemble approach resulted in an accuracy score of 90.67% in identifying hate speech with mixed tamil-english codes by using conventional machine learning approaches such as support vector machine, naive bayes and ensemble methods @xcite . different neural networks were explored, out of which a hybrid cnn-bilstm model adjusted for data imbalance, performed best for identifying offensive language in dravidian languages @xcite . despite the substantial social impact of hate speech related to caste and migration, little research has been done on identifying it. the majority of current research focuses on hate speech in general, which leaves a gap in addressing these particular and culturally relevant types of online abuse. in this work, we explore machine learning models to detect hate speech related to caste and migration in tamil code-mixed text using tf-idf and pretrained model embeddings to identify the most effective approaches for handling this task. ## data the dataset used for this task includes text samples from social media platforms, including posts that are general and those that are specifically related to caste or migration hate speech, along with the labels that correspond to these posts for the purpose of identifying hate speech @xcite . the dataset includes three different language representations: english, tamil, and tanglish (a codemixed tamil and english). the provided train and development datasets were merged into a single training dataset. an overview of this combined dataset's distribution across classification labels can be found in ## methodology this section describes the methodology followed which includes data pre-processing, feature extraction and model training used in this study. the codebase is available at our github repository foot_1 . ## results the results of several machine learning models on different text representations have been explored in this section. initially, machine learning models were trained using tf-idf vectors and embeddings from a number of pre-trained models, including mbert, tamil bert, labse, xlm-roberta, and muril (base and large). of these, tf-idf vectors and muril large embeddings outperformed the others on the validation split. following this, a combination tf-idf vectors and muril embeddings were used to to further improve accuracy and f1-scores. the validation of our models trained were done using the 20% of the dataset provided to us for training. using this validation set, random forest classifier outperformed the other models trained on tf-idf vectors, achieving a macro f1-score of 0.77 and an accuracy of 0.80, whereas stacking classifier performed the best for models trained on muril embeddings, with a macro f1-score of 0.75 and an accuracy of 0.78. voting classifier and xg-boost trained on muril embeddings gave similar results with an accuracy of 0.77 and f1-scores of 0.73 and 0.74, respectively. on combining muril representations with tf-idf vectors, an overall improvement in classification performance can be observed. overall classification performance was improved by combining muril based embeddings with tf-idf vectors and training it on soft voting classifier with an accuracy score of 0.79 and f1 score of 0.77. following closely behind, xgboost and lightgbm returned comparable results with an accuracy of 0.78 and f1 scores of 0.77 and 0.76 respectively. these findings imply that improving classification performance requires using both contextual embeddings and traditional statistical features. when combined with tf-idf, muril embeddings helped to improve performance, but they did not outperform tf-idf-based models on their own. overall, these results show how well hybrid feature representations work to produce reliable text classification outcomes. in comparison to the best individual featurebased models (tf-idf only and muril only), the combination of muril embeddings and tf-idf vectors performed better on the held-out test set, obtaining the highest macro f1-score of 0.802. this supports how well contextual and statistical text representations work together to classify hate speech. the term importance based on the frequency of explicit hateful words and keywords is useful in identifying whether a speech is hateful or not and is obtained by using statistical features such as tf-idf. however, this feature alone might fail in some cases since it does not understand the semantic meaning behind these words, especially in codemixed and multilingual contexts. this is where muril embeddings comes into play, which, when combined with tf-idf, has proven to be a good feature representation for the hate speech classification task. ## conclusion this paper presents our system for the lt-edi@ldk 2025 shared task on detecting caste and migration hate speech across tamil, tanglish, and english. we experimented with both tf-idf vectors and transformer embeddings (especially muril) as input features for a range of machine learning classifiers. our experiments clearly showed that combining traditional tf-idf vectors with the contextual understanding from muril embeddings produced the best outcome. specifically, a soft voting classifier using this hybrid tf-idf + muril feature set achieved the highest macro f1-score of 0.802 on the competition's test data. using both tf-idf and muril together produced a better score than using either one individually. this likely happened because the two methods capture different kinds of useful information. tf-idf finds key hate terms through frequency, while muril understands the context and nuance, essential for the code-mixed and multilingual text we analyzed. our system using this method placed 6th overall in the shared task. this work shows that blending statistical text features with modern contextual embeddings offers a solid path forward for effectively detecting hate speech in complex, real-world linguistic scenarios like those found in indian social media.