Overview of Shared-task on Abusive Comment Detection in T amil and T elugu (2023)

Abstract: This paper discusses the submissions to the shared task on abusive comment detection in Tamil and Telugu codemixed social media text conducted as part of the third Workshop on Speech and Language Technologies for Dravidian Languages at RANLP 20239. The task encourages researchers to develop models to detect the contents containing abusive information in Tamil and Telugu codemixed social media text. The task has three subtasks - abusive comment detection in Tamil, Tamil-English and Telugu-English. The dataset for all the tasks was developed by collecting comments from YouTube. The submitted models were evaluated using macro F1-score, and prepared the rank list accordingly.

## introduction abusive comment detection from social media has become an essential and challenging task in the current age of technology @xcite @xcite . the proliferation of online platforms helped people to spread information, including harmful and violent comments and posts. therefore, addressing and mitigating harmful content to keep online platforms clean automatically has become very important @xcite @xcite . this task is challenging due to the complexities of the languages. however, advanced machine learning algorithms and techniques were proposed to automatically identify and flag abusive comments, ranging from hate speech and cyberbullying to threats and harassment, recently. these systems analyze the content of the posts and the context to determine the presence of abusive language and malicious intent. the complexity of detecting the abusive contents from a code-mixed dravidian language is even high due to the code-mixed nature of the text and the intricacies of the language, such as morphological richness and agglutinative property @xcite . in addition, large datasets of labelled abusive content are required to train and fine-tune the artificial intelligence (ai)-based models, enabling them to recognize patterns and distinguish between harmful and benign texts. a considerable amount of research has been conducted to detect abusive and similar harmful content from social media posts and comments @xcite @xcite . in addition, several shared tasks were organized to promote the research for automatically detecting social media comments containing abusive content. this shared task focuses on detecting abusive comments in two dravidian languages -tamil and telugu. tamil is predominantly spoken in tamil nadu, a state in india and nearby countries, whereas telugu is the official language of two states in india -andhra pradesh and telangana @xcite @xcite @xcite @xcite . this paper summarizes the findings of the research works submitted to the shared task on abusive language detection in tamil and telugu. besides, this paper details the dataset developed and used for conducting the experiments. ## literature review in an attempt by chakravarthy et. al. @xcite , a set of four datasets comprising abusive comments in tamil and code-mixed tamil-english extracted from youtube is presented. each dataset has undergone comment-level annotation, wherein polarities are assigned to the comments. to establish baselines for these datasets, the authors conducted experiments using various machine learning classifiers and presented the results in terms of f1-score, precision, and recall. prasanth et al. @xcite conducted a study focusing on the detection of abusive comments within given text. the authors employed tf-idf with char-wb analyzers and utilized the random kitchen sink (rks) algorithm to generate feature vectors. for classification purposes, they employed the support vector machine (svm) classifier with a polynomial kernel. the proposed method was applied to both tamil and tamil-english datasets, resulting in f1-scores of 0.32 and 0.25, respectively. priyadharsini et. al. @xcite provides a comprehensive review of a shared task focused on identifying abusive comments encompassing various categories such as homophobia, misandry, counter-speech, misogyny, xenophobia, transphobic, and hope speech. the participants were provided with a dataset extracted from social media, which was labeled with the aforementioned categories in both tamil and tamil-english code-mixed languages. the participants employed diverse machine learning and deep learning algorithms for their approaches. the paper presents an overview of this task, including detailed information about the dataset and the results achieved by the participants. the objective of the study by @xcite is to automate the identification and categorization of abusive comments into specific categories like misogynism, misandry, homophobia, and cyberbullying. the datasets utilized in this research were provided by the dravidianlangtech@acl2022 organizers and consisted of code-mixed tamil text. the authors trained these datasets using pre-trained transformer models such as bert, m-bert, and xlnet. remarkably, they achieved a weighted average f1 score of 0.96 for tamil-english codemixed text and 0.59 for tamil text. gupta et al. @xcite conducted a study where they introduced a model called abusexlmr, designed specifically for detecting abusive content. this model was pre-trained on a vast amount of social media comments in over 15 indic languages. notably, abusexlmr exhibited superior performance compared to xlm-r and muril when evaluated on multiple indic datasets. in addition to providing annotations, this study also released mappings between comment, post, and user ids, enabling the modeling of relationships among them. furthermore, competitive baselines for monolingual, cross-lingual, and few-shot scenarios were shared, intending to establish the collected dataset as a benchmark for future research. the primary goal of the study by marreddy et. al. @xcite is to address the challenges posed by limited resources in the telugu language. the authors make several valuable contributions to enrich resources for telugu. they have curated a large annotated dataset containing 35,142 sentences for various nlp tasks, including sentiment analysis, emotion identification, hate-speech detection, and sarcasm detection. to enhance model efficiency, the authors have developed separate lexicons for sentiment, emotion, and hate speech and utilized pretrained word and sentence embeddings. furthermore, the authors have created different pre-trained language models specifically for telugu, such as elmo-te, bert-te, roberta-te, albert-te, and distilbert-te, using a sizable telugu corpus comprising 8,015,588 sentences. notably, the authors demonstrate that these developed models significantly enhance the performance of the four nlp tasks and provide benchmark results for telugu. ## task description we used the codalab platform to conduct the task foot_0 . the task includes three subtasks -abusive language detection in • tamil: abusive language detection from tamil codemixed social media text ## system descriptions this section summarizes the systems submitted to the abusive comment detection in tamil and telugu tasks. ## result analysis and discussion this section discusses the submission by different teams in the shared task. ## conclusion this paper discussed the findings of the shared task conducted as part of the third workshop on speech and language technologies for dravidian languages at ranlp 2023 on abusive comment detection in tamil, tamil-english and telugu-english data. the datasets used for the competition were collected from youtube comments and annotated with experts' help in compliance with youtube's regulations. there were nine, eleven and eight submissions in tamil, tamil-english and telugu-english tasks, respectively. most teams used multilingual bert-based pre-trained models to transform the input text into the feature vector. the other submissions consisted of models using tf-idf features and machine learning classifiers. we used macro f1-score for computing the classification performance and prepared the rank list accordingly.