Algorithm Alliance@ LT - EDI -2024: Caste and Migration Hate Speech Detection (2024)

Abstract: Caste and Migration speech refers to the use of language that distinguishes the offense, violence, and distress on their social, caste, and migration status. Here, caste hate speech targets the imbalance of an individual’s social status and focuses mainly on the degradation of their caste group. While the migration hate speech imposes the differences in nationality, culture, and individual status. These speeches are meant to affront the social status of these people. To detect this hate in the speech, our task on Caste and Migration Hate Speech Detection has been created which classifies human speech into genuine or stimulate categories. For this task, we used multiple classification models such as the train test split model to split the dataset into train and test data, Logistic regression, Support Vector Machine, MLP (multi-layer Perceptron) classifier, Random Forest classifier, KNN classifier, and Decision tree classification. Among these models, The SVM gave the highest macro average F1 score of 0.77 and the average accuracy for these models is around 0.75.

## introduction in the age of rapid globalization and digital interconnectedness, social media platforms have become powerful tools for communication and community engagement. however, this unprecedented accessibility has also given rise to a darker aspect of online discourse -the proliferation of hate speech. of particular concern is the manifestation of hate speech related to caste and migration issues, which not only perpetuates discrimination but also poses a significant threat to social harmony. as our world embraces the digital age, technology plays a pivotal role in connecting people through platforms like facebook and twitter @xcite . despite its positive aspects, social media harbors drawbacks, with users sometimes engaging in discouragement or targeted hate speech. detrimental speech on these platforms has a lasting psychological impact on victims @xcite . this study highlights the surge in hate speech on social media, fuelled by anonymity and the absence of stringent controls, particularly targeting religion, gender, and race. online communities offer insights into understanding and combating online hate speech, suggesting new dimensions for future research @xcite . social media platforms struggle to manage the constant flood of comments and posts, making it challenging to effectively monitor and control content due to the sheer volume. finding a balance between limiting excessive posts and preserving freedom of speech poses a significant predicament. additionally, the diverse user base, representing various backgrounds, cultures, and beliefs, further complicates the issue, contributing to the widespread problem of hate speech. @xcite . the paper's structure includes a literature review in section 2, task and data description in section 3, methodology in section 4, results and analysis in section 5, and a conclusion in section 6. ## related works numerous studies have explored hate speech detection, including those focused on caste and migration @xcite . davidson et al. emphasized the subjective biases in hate speech classification, highlighting the need for objective methodologies. in caste-based hate speech detection, malmasi and zampieri addressed challenges using lexical properties like n-grams, character n-grams, word embeddings, and paragraph embeddings @xcite . 2023). sanguinetti et al. conducted automatic hate speech detection research, creating datasets annotated with hate labels and related dimensions (jahan and oussalah, 2023). the overview of the hope speech detection task is given in @xcite . in sentiment analysis, @xcite used the transformer model albert for hope speech detection in multiple languages like english, tamil, kannada, etc. @xcite proposed a convolutional neural network (cnn) model outperforming traditional models for hopespeech detection. the authors of @xcite performed binary and multi-class hope-speech classification. the binary task involved only two labels whereas the multi-class task involved three labels. in the paper, @xcite used hasoc 2021 hindi and marathi hate speech datasets for algorithm comparison. marathi uses binary labels; hindi has both binary and detailed labels. transformer models excelled, and basic models with fasttext embeddings showed competitive performance. intriguingly, after standard hyper-parameter tuning, basic models outperformed bert-based models, especially on the fine-grained hindi dataset. the authors of (roy et al., 2022) examined codemixed language use on social media, focusing on hindi-english, tamil-english, malayalam-english, telugu-english, etc. they proposed a weighted ensemble model combining transformer-based bert models and a deep neural network for offensive and hate speech detection. experimental results showed the framework outperformed state-of-the-art models, achieving 0.802 and 0.933 weighted f1 scores for malayalam and tamil code-mixed datasets. the authors of @xcite used lstm, deep learning, and hybrid models on tamil and malayalam datasets. in the paper @xcite applied transformer models like m-bertcased and xlm-roberta for hope speech detection, with m-bert-cased achieving the highest f1-score. the top model for the english dataset was the 2-parallel cnn-lstm using glove and word2vec embeddings, while the 3-parallel bi-lstm excelled on the malayalam dataset. in recent years, there's been a rise in studies addressing hate speech targeting specific groups, like caste and migration status. in today's digital age, hate speech based on caste or migration has become a significant concern. these studies showcase versatile models for sentiment analysis on social media comments. to enhance text classification accuracy, we opted for traditional models alongside a basic transformer model based on the literature survey. ## task and data description the overview paper for this task is explained in @xcite . the shared task on caste and migration hate speech detection at lt-edi-eacl 2024 is intended to determine whether the speech text format was legitimate or imposed hate towards caste and migration. the dataset consists of two fields namely speech text and a label. here, the label indicates the abovementioned category, and it is represented in hate and non-hate speech. the training dataset consists of around 5,355 text-converted speeches out of which 3,303 instances were labelled as non-hate speech and 2,052 instances were labelled as hate speech. the development dataset consisted of 945 instances out of which 594 instances were labelled as non-hate speeches and 351 instances were labelled as hate speeches. here, we used 1576 test data instances for testing the model. ## methodology several machine learning approaches may be used to achieve this task, but we chose the most effective one for the classification problems, i.e., detection of hate speech related to caste and migration. as shown in figure 1 , the distribution of data in the datasets indicates that 38.3% of collected data contains hate speech. ## conclusion in conclusion, we applied supervised learning models such as random forest, svm, and logistic regression to investigate hate speech identification and migration speech, with a macro f1 score of 0.77, the svm model stood out and demonstrated its efficiency by classifying the hate speech in these specific contexts. the following research could investigate the integration of deep learning models to boost accuracy. while emphasizing the ongoing need for adaptive and more flexible classification to deal with the evolving dynamics of these conversations.