Findings of the Shared Task on Hate and Offensive Language Detection in T elugu Codemixed Text ( HOLD - T elugu)@ D ravidian L ang T ech 2024 (2024)

Abstract: This paper examines the submissions of various participating teams to the task on Hate and Offensive Language Detection in Telugu Codemixed Text (HOLD-Telugu) organized as part of DravidianLangTech 2024. In order to identify the contents containing harmful information in Telugu codemixed social media text, the shared task pushes researchers and academicians to build models. The dataset for the task was created by gathering YouTube comments and annotated manually. A total of 23 teams participated and submitted their results to the shared task. The rank list was created by assessing the submitted results using the macro F1-score.

## introduction in the present technological era, detecting hate comments on social media has become a crucial and challenging task @xcite @xcite . the growth of internet platforms made it easier for people to disseminate information, including offensive and violent postings and comments. consequently, it is now crucial to address and mitigate hazardous content in order to automatically maintain online platforms clean @xcite @xcite . this is a challenging endeavour because of the complexity of the languages and codemix nature of the contents. however, recently, sophisticated machine learning algorithms and methods were presented to automatically detect and flag offensive remarks, ranging from threats and harassment to hate speech and cyberbullying @xcite . these technologies analyze the post content and context for hate language. in a codemixed dravidian language, it is much harder to find hateful words because the text is codemixed and has linguistic properties like morphological richness and agglutinative characteristics @xcite . furthermore, sizable datasets of tagged offensive content are needed to train and optimize the ai-based models and make them capable of identifying trends and differentiating between benign and harmful texts @xcite . the shared task on hate and offensive language detection in telugu codemixed text intends to detect hate and offensive content in social media posts and comments written using codemixed telugu data. the shared task was conceived as a binary class problem, where the dataset has two labels for each data -hate and non-hate. this paper discusses the task in detail and the models submitted to task by the participants. ## task description we used the codalab platform to conduct the task 1 . the task aims to develop models to identify hate and offensive language content in telugu-english codemixed social media comments. the hateful remarks on youtube were gathered to create the dataset. finding the videos where the hate comments could be found was the first challenge. when generating the dataset, consideration was given to comments containing both telugu and english words written in their respective scripts and comments that wrote telugu characters using latin scripts. according to youtube's rules 2 , we annotated the comments into hate and non-hate categories. slang was taken into account when annotating the telugu remarks with additional care. furthermore, an additional obstacle was presented by the existence of spam content, which was extraneous to the dataset due to its lack of contextual information. those remarks were disregarded with respect to the intended dataset. the effective analysis and categorization of youtube comments may present a challenge due to the prevalence of incorrect syntax, typographical errors, and nonstandard language usage in social media posts and comments. before annotating the text, these remarks were reviewed to ensure that the annotators understood the context properly. the annotators were native telugu speakers with strong academic credentials and fluency in english. in conclusion, the dataset comprised 4,500 annotated comments, of which 4,000 were training data and 500 were test data. some statistics about the dataset is given in the test data contained 250 hate and non-hate data, while the training dataset contained 1,939 hate and 2,061 non-hate comments. there is not 1 https://codalab.lisn.upsaclay.fr/competitions/16095 2 http://tinyurl.com/ys56hrr5 sixty-nine teams signed up for the competition. only twenty-five teams, though, turned in their predictions for the test set. each team was allowed to submit up to three runs, and the run with the best performance score was considered for creating the rank list, which is displayed in ## system description this section discusses the models submitted to the shared task. ## conclusion this paper discussed the findings of the shared task on hate and offensive language detection in telugu codemixed text (hold-telugu) conducted as part of the fourth workshop on speech, vision, and language technologies for dravidian languages (dravidianlangtech-2024) at eacl 2024. the datasets used for the competition were collected from youtube comments and annotated with experts' help in compliance with youtube's regulations. there were twenty-five submissions for this task. most teams used multilingual bertbased pre-trained models to transform the input text into the feature vector. the other submissions consisted of models using tf-idf features and machine learning classifiers. we used macro f1-score to compute the classification performance and prepared the rank list accordingly.