Wise@ LT - EDI -2025: Combining Classical and Neural Representations with Multi-scale Ensemble Learning for Code-mixed Hate Speech Detection (2025)

Abstract: Detecting hate speech targeting caste and migration communities in code-mixed Tamil-English social media content is challenging due to limited resources and socio-cultural complexities. This paper proposes a multi-scale hybrid architecture combining classical and neural representations with hierarchical ensemble learning. We employ advanced preprocessing including transliteration and character repetition removal, then extract features using classical TF-IDF vectors at multiple scales (512, 1024, 2048) processed through linear layers, alongside contextual embeddings from five transformer models-Google BERT, XLM-RoBERTa (Base and Large), SeanBenhur BERT, and IndicBERT. These concatenated representations encode both statistical and contextual information, which are input to multiple ML classification heads (Random Forest, SVM, etc). A three-level hierarchical ensemble strategy combines predictions across classifiers, transformer-TF-IDF combinations, and dimensional scales for enhanced robustness. Our method scored an F1-score of 0.818, ranking 3rd in the LT-EDI-2025 shared task, showing the efficacy of blending classical and neural methods with multi-level ensemble learning for hate speech detection in low-resource languages.

## introduction hate speech is any kind of communication that attacks a person or group based on attributes like caste, religion, race, or other identity factors. with the advancement of technology and the advent of social media, individuals can now share their thoughts with anyone in the world. while this increased connectivity has many benefits, the anonymity offered by online platforms has unfortunately facilitated the spread of hateful messages, particularly those targeting vulnerable groups such as migrants and specific caste communities. to make online communities inclusive, it is essential to identify caste and migration-based hate speech. despite significant progress in natural language processing(nlp) through transformer architectures @xcite ) revolutionizing text classification tasks, detecting hate speech in low-resource languages like tamil poses unique challenges stemming from dialectal diversity and regional variations. this challenge is further intensified when detecting specific forms of hate speech, such as those targeting caste and migration, as these are often embedded in local socio-cultural nuances and context. in social media platforms, the texts are often code-mixed, i.e., english + tamil, making detection even more challenging. to address these challenges, our work presents a multi-scale hybrid architecture that combines classical term frequency-inverse document frequency(tf-idf) @xcite features at multiple scales (512, 1024, and 2048) with contextual transformer embeddings. for final classification, a hierarchical ensemble using majority voting across models and feature scales was used to make the model robust and generalizable. ## related work early hate speech detection systems relied on rulebased methods and keyword matching @xcite . these methods are unsuitable for the vast amount of data present today as they lack contextual understanding. machine learning algorithms like support vector machines @xcite and logistic regression @xcite , which used hand-crafted features such as tf-idf, n-grams, and parts of speech (pos) tags, emerged later. although these models improved performance, they were ineffective for code-mixed or culture-specific hate speech @xcite . deep learning techniques such as rnns (elman, 1990) and lstms @xcite , when combined with word embeddings like word2vec @xcite and glove @xcite , improved context-aware modeling and reduced the need for manual feature engineering @xcite . nevertheless, they struggled with long-range dependencies and noisy, code-mixed text. with the emergence of pre-trained language models such as bert @xcite , roberta @xcite , and multilingual variants like xlm-r @xcite , the performance improved further. fine-tuning these pretrained models on hate speech datasets has consistently yielded better results @xcite . in their work, @xcite used an ensemble-based approach to detect hate speech in code-mixed tamil and malayalam texts, as the individual models had a high misclassification rate. two ensemble techniques were used: one based on the average of the outcomes and another using custom weights. their ensemble model outperformed the previously reported state-of-the-art models, achieving an f1 score of 0.933 on tamil and 0.802 for the malayalam dataset. in their work, @xcite detected hate speech and offensive language (hos) in low-resource dravidian codemix languages @xcite . various multilingual transformer-based embeddings (e.g., muril, bert, xlm-r) were combined with traditional ml classifiers for hos detection. to address class imbalance, a cost-sensitive learning approach was used. experiments on six datasets showed that muril + svm performed best overall. ## task and dataset description the goal of this shared task @xcite is to develop a system to detect hate speech targeting caste and migrant communities in codemixed social media data, focusing on tamil, a lowresource language. the dataset @xcite has three columns: "text", which has the comments from social media platforms; "id", containing the id of the comments; and "label", which is set to 1 for hate speech and 0 for non-hate speech. the dataset description is provided in ## methodology the approach uses a multi-scale hybrid framework to identify hate speech in code-mixed tamil social media posts. two preprocessing schemes (with and without transliteration) are used to handle intrinsic noise in the data, such as emojis, urls, inconsistent spacing, repeated characters in transliterated tamil words, and code-mixing between tamil and english languages. feature extraction unites tf-idf vectors at three dimensions @xcite @xcite with contextual embeddings of five transformer models. such features are then concatenated to create integrated feature vectors with both statistical and contextual information. a set of 22 traditional ml classifiers are trained for each feature set, and the top 3 models for each were chosen. a three-level hierarchical ensemble approach employs majority voting across classifiers, feature sets, and dimensions to ensure resilient classification by combining heterogeneous preprocessing schemes, representation types, and model architectures ## result and analysis the model's performance was assessed using the f1-score @xcite , which is defined in equation 2. the results of the best-performing models for different dimensions (512, 1024, and 2048) are summarized in ## limitations even though the proposed pipeline performed well, there are a few limitations, which are as follows: