CUET _ NLP _ M anning@ LT - EDI 2024: Transformer-based Approach on Caste and Migration Hate Speech Detection (2024)

Abstract: The widespread use of online communication has caused a significant increase in the spread of hate speech on social media. However, there are also hate crimes based on caste and migration status. Despite several nations efforts to bring equality among their citizens, numerous crimes occur just based on caste. Migration-based hostility happens both in India and in developed countries. A shared task was arranged to address this issue in a low-resourced language such as Tamil. This paper aims to improve the detection of hate speech and hostility based on caste and migration status on social media. To achieve this, this work investigated several Machine Learning (ML), Deep Learning (DL), and transformer-based models, including M-BERT, XLM-R, and Tamil BERT. Experimental results revealed the highest macro f1-score of 0.80 using the M-BERT model, which enabled us to rank 3rd on the shared task.

## introduction the advent of social media has reshaped the contours of communication, enabling individuals to share their thoughts and interact with a global audience instantaneously. while this has led to the democratization of information exchange, it has also given rise to an insidious byproduct of hate speech and hostility @xcite . hate speech, mainly rooted in caste discrimination and migration bias, is a pervasive element in online discourse, highlighting societal prejudices and perpetuating exclusion and animosity. in several nations, caste discrimination remains a persistent issue despite the country's legal strides toward equality @xcite . the caste system, an ancient social hierarchy, continues to influence individual and collective identities and relationships, often manifesting in the form of hate speech that targets marginalized communities @xcite . the repercussions of such expressions are not confined to the digital realm; they spill over into the real world, reinforcing social divisions and impeding efforts to establish a more equitable society. the issue of migration discrimination is similarly problematic, affecting nations worldwide @xcite . as people migrate across borders in search of better opportunities or refuge, they often face hostile attitudes and vilification on social media, contributing to xenophobia and nationalism, fostering fear and suspicion, and leading to divisive policies. thus, addressing these forms of hate speech is crucial, and computational linguistics can help us identify them effectively @xcite . the goal of this study is to develop a system capable of discerning caste and migration hate speech from non-caste and migration hate speech. the primary accomplishments include: ## related work social media and blogging platforms offer a platform for individual expression, but they can also promote antisocial conduct, such as hate speech and cyberbullying @xcite . a shared task was conducted (basile et al., 2019) to detect multilingual hate speech against immigrants and women on twitter. almatarneh et al. (2019) used tf-idf and lexicon to identify hate speech against migrants and women in english and spanish tweets, achieving f 1 scores of 0.36 and 0.54, respectively. romero-vega et al. (2021) addressed xenophobic hate speech in spanish tweets about venezuelan migrants in ecuador, with the svm model showing the highest performance f 1 -score of 0.98. farooqi et al. (2021) addressed hate speech in social media, emphasizing the need to consider conversation context; their system achieved the highest macro f 1 -score of 0.7253 leveraging neural networks and the ensemble of indic-bert, xlm-roberta, and multilingual bert. a recent study bhimani et al. (2021) utilized nlp and ml techniques to analyze hate speech on social media, considering aspects such as caste and religion, and gained 96.29% accuracy using logistic regression (lr). sachdeva et al. (2021) addressed the issue of hate speech on social media, underscoring the pressing demand for automated approaches in light of the increasing spread of biased content. they achieved an f 1score of 0.84 by using the random forest (rf) classifier. dhanya and balakrishnan (2021) surveyed hate speech detection in asian languages, focusing on developing an automated system for malayalam, addressing negativity related to societal factors with varying dataset sizes. hossain et al. (2022) identified abusive comments from tamil texts using lr and achieved a f 1 -score score of 0.39. sharif and hoque (2021) addressed aggressive content on social media, especially in regional languages like bengali, proposing an ensemble classifier trained on 10,095 annotated texts. using cnn, bilstm, and gru with diverse embeddings and ensemble strategies, their framework achieved the highest coarse-grained f 1 -score of 0.89 and fine-grained weighted f 1 -score of 0.84 on the dataset. despite extensive research in natural language processing, there is a lack of studies on detecting hate speech related to caste and migration. ## task and dataset description due to the complexity of code-mixed data in social media texts, it is challenging for systems trained on monolingual data to classify. this task aims to implement a system to identify hate speech related to caste and migration. in order to detect caste and migration hate speech from text data, task organizers 1 developed a code-mixed (tamil-engilsh) corpus. to develop such a system, we analyzed the corpus given by the task organizers @xcite . the corpus is split into training (5,355 texts), validation (945 texts), and test (1,575 texts) sets. the task involves a binary classification problem to identify caste and migration hate speech from the corpus. the classes are caste and migration hate speech (cmhs), containing 4,870 texts, and not caste and migration hate speech (ncmhs), containing 3,005 texts. we analyzed the dataset in further detail concerning sentence length. figure 1 displays the dataset's length-frequency distribution. according to the length-frequency distribution study, a few text samples had text lengths of more than 100 words. as a result, the maximum sentence length for this work was 100 words. the average sentence length is 18, with one word as the minimum. 4 methodology various ml and dl techniques are used for the baseline evaluation with appropriate feature extraction techniques. moreover, a few transformer models, such as m-bert, xlm-r, and tamil-bert, are examined. ## results and analysis the results showed that the lr and svm models obtained a macro f 1 -score of 0.75. when trained on fasttext feature vectors, the bilstm approach yielded a macro f 1 -score of 0.74. deep learningbased models obtained comparatively worse results than the ml-based models. the small size of the training data maybe the reason behind this. transformer-based models outperformed all other models. m-bert obtained the best performance, macro f 1 -score of 0.80. ## conclusion this work explored several ml, dl, and transformer-based techniques and analyzed their performance in detecting caste and migration hate speech in tamil. experimental assessment of the test dataset revealed that the m-bert model is the best performing model for detecting hate speech in tamil and outperformed all models by obtaining the highest macro f 1 -score (0.80). surprisingly, the bilstm + attention model performed poorly compared other ml and transformer models. these inferior results might occur because of the prevalence of local words, which still need to be discovered in the model. the future work includes adding more data in the respective classes to make a balanced dataset and investigating more sophisticated techniques such as muril and gpt for improved performance.