Overview of Shared Task on Caste and Migration Hate Speech Detection (2024)

Abstract: We present an overview of the first shared task on “Caste and Migration Hate Speech Detection.” The shared task is organized as part of LTEDI@EACL 2024. The system must delineate between binary outcomes, ascertaining whether the text is categorized as a caste/migration hate speech or not. The dataset presented in this shared task is in Tamil, which is one of the under-resource languages. There are a total of 51 teams participated in this task. Among them, 15 teams submitted their research results for the task. To the best of our knowledge, this is the first time the shared task has been conducted on textual hate speech detection concerning caste and migration. In this study, we have conducted a systematic analysis and detailed presentation of all the contributions of the participants as well as the statistics of the dataset, which is the social media comments in Tamil language to detect hate speech. It also further goes into the details of a comprehensive analysis of the participants’ methodology and their findings.

## introduction social media platforms have become an integral part of the daily life of today's society. it gives new meaning to how we communicate, connect, and share information among people @xcite . this digital landscape allows users to share their opinions, how they live, and their work worldwide. it has a huge impact on society, which makes it possible to influence everything around us. it encompasses the way of communication, how we perceive the world, disaster response, education, and how it makes information accessible to everyone, as well as giving our support to the things or people that are overlooked by society @xcite . however, social media platforms also present significant challenges to people, like discrimination and bias against certain kinds of people, including those based on caste/migration. it replicates the societal fractures openly or anonymously, enabling harassment, bullying, and exclusion based on caste/migration. the caste system also significantly influences instances of homicide and violence targeting inter-caste marriages @xcite . caste systems are in a hierarchical order from high to low. here, low-level caste people have been subjected to many kinds of discrimination in many firms @xcite . racial and extractive capitalism has exploited the people into hard labor, their dehumanizing treatment, and their psychological trauma experiences @xcite . with the rapid rise of social media @xcite , the shadows of age-old social prejudice, such as caste-based discrimination, bias, and hate speech, stepped into the digital realm, which especially targeted lower-caste people, thus leading to their psychological trauma and trampling on the dignity of humans. there are some cases where caste/migration-based discrimination leads to violence. this leads to a way of effectively addressing and detecting hate speech regarding caste/migration. with the help of machine learning, deep learning, and natural language processing technology, one can detect caste/migration-based hate speech. however, the quality of the detection depends on the quality of the training data without any biases and critical concerns. the task involves the use of a newly created dataset for detecting caste or migration-based hate speech in the tamil language. in this overview paper, we have discussed the shared task of caste and migration hate speech detection in tamil at lt-edi@eacl 2024. in the subsequent sections, we have discussed the task description, dataset statistics, methodologies used by the participants to detect caste/migration hate speech in tamil, and their results and ranking. ## related work in recent years, social media has developed rapidly, leading to advantages and disadvantages in its use. the common one is hate speech towards a certain kind of race, religion, sexual orientation, gender, caste, and beyond. this issue leads many researchers to research to research a methodology that can detect hate speech related to specific targets(ibrohim and budi, 2023). in particular, @xcite presents xlm roberta as a base model to detect religion-based hate speech on english, russian, and hindi datasets. afterward, the base model was finetuned with the different datasets, and to improve the model, a text attack algorithm was applied. @xcite focuses on hate speech towards afghan immigrants in iran and also shows the subtle ways of spreading hate speech without directly using any hateful words. @xcite focuses on the hate speech comments on youtube regarding the roma community, and it also points out how the obvious and subtle way of hate comments and discrimination spread against the roma community. it also highlighted the term "entitlement racism," which is when individuals believe it's justifiable to propagate racial animosity. @xcite highlights how online platforms lack the detailed guidelines to successfully combat online hate speech when incorporating hrdd (human rights due diligence) into their terms of service (tos). in order to protect marginalized people, it advocates online platforms, including the tos, with european human rights norms. ## task description the primary goal of this task is to develop an automated classification system that can accurately determine the presence of hate speech on caste or migration within textual content on social media platforms. this shared task is held through the codalab competition 1 . the participants will pro-1 https://codalab.lisn.upsaclay.fr/ competitions/16089 vided with training, development, and test datasets in the tamil language. to access and contribute to the data, navigate to codalab and select the "participate" tab. to the best of our knowledge, this represents the first shared task dedicated to identifying hate speech concerning caste/migration. the annotation has been done in two categories. ## dataset description this dataset was methodically collected from a social media platform with a concentrated focus on the youtube platform by utilizing the youtube-comment-scraper tool to collect the youtube video comments in support of the shared task on 'caste/migration hate speech detection' at lt-edi@eacl 2024. it represents the first dataset focusing on caste and migration hate speech in low-resource languages, with a particular emphasis on tamil. a total of 7,875 comments were collected, and each of the comments was meticulously annotated for the presence of caste/migration hate speech (labeled as '1') and the absence of such hate speech (labeled as '0'). a few samples of the dataset have been discussed in the previous section. the samples contain some words that point at some caste or migrated people such as vadakan, marvadi, parayan, vaduka pallan, vaellalar, mukkulathor. the offensive texts that may come along with the above words or beyond may have targeted a certain community. since there are many castes, some tend to discriminate against other castes which leads to these kinds of offensive comments. the dataset was segmented into training, development, and test data subsets to help with the thorough analysis and to facilitate the model training. the accompanying ## results the hate speech detection shared task for caste/migrants was conducted for one of the lowresource languages that is tamil. as mentioned in the former part, many participants have contributed to the shared task. a total of 51 teams participated in the shared task of detecting hate speech on caste/migration in the tamil language. among them, 15 teams submitted their results. the ranking and evaluation of the shared tasks was based on the average macro f1 score. ## conclusion we presented the overview of the shared task on caste and migration hate speech in social media comments using a dataset in tamil. it represents an important step toward the creation of healthy online communities. we can mitigate hate speech on certain individuals, castes, and migration by exploiting advanced technologies, algorithms, and computational tools.