Tewodros@ D ravidian L ang T ech 2024: Hate Speech Recognition in T elugu Codemixed Text (2024)

Abstract: This study goes into our team’s active participation in the Hate and Offensive Language Detection in Telugu Codemixed Text (HOLDTelugu) shared task, which is an essential component of the DravidianLangTech@EACL 2024 workshop. The ultimate goal of this collaborative work is to push the bounds of hate speech recognition, especially tackling the issues given by codemixed text in Telugu, where English blends smoothly. Our inquiry offers a complete evaluation of the task’s aims, the technique used, and the precise achievements obtained by our team, providing a full insight into our contributions to this crucial linguistic and technical undertaking.

## introduction the prevalence of hate speech and provocative language in online interactions has raised serious concerns about their negative impact on people and society. with its unequal reach and simplicity of use, social media has become a breeding ground for the spread of such toxic information. while considerable strides have been made in identifying hate speech in english, the issue remains largely unexplored for dravidian languages such as telugu. telugu, which is spoken by almost 80 million people in india, is a sophisticated language that incorporates words and phrases from other languages, mostly english. this linguistic feature creates additional challenges for effectively detecting hate speech in telugu literature. recognizing the need to bridge this knowledge gap and support innovation in telugu hate speech detection, the dravidianlangtech@eacl 2024 workshop included the shared task on hate and offensive language detection in telugu codemixed text (hold-telugu).the collaborative effort aims to bring together diverse experts in order to create effective methods for detecting hate and abuse terms in telugu codemixed text. ## related work while hate speech identification in english has gained significant study interest, the topic for dravidian languages like telugu remains relatively new and unexplored. however, in the past year, tasks such as sentiment analysis have been done on the tulu-english code-mixing language dataset @xcite , and the following earlier research gives vital facts and insights linked to the hold-telugu joint task: in the work of @xcite , the authors attempt to present an overview of detecting abusive comments and hope speech involving homophobia, misandry, counter-speech, misogyny, xenophobia, and transphobia using data in tamil and tamil-english code-mixed languages. along with dataset details and participant findings. in @xcite , the authors provide the evaluation of two powerful baselines for offensive language identification (perspective) and categorization (bert). their experiment shows that perspective outperformed bert in detecting toxicity, whereas bert outperformed perspective in categorizing the offensive type. in the semeval-2019 offenseval competition, perspective ranked 12th in detecting an offensive post, while bert ranked 11th in categorizing it. in @xcite , the authors constructed a multilingual, manually annotated dataset and experimented with machine learning and deep learning algorithms. the dataset contains around 60,000 youtube comments, including approximately 44,000 comments in tamil-english, 7000 comments in kannada-english, and 20,000 comments in malayalam-english. they make it publicly available on github and zenodo. the authors in (ayele et al., 2022b) build a dataset of 5,267 tweets, and machine learning methods lr and svm achieve an f1 score of 0.49, whereas nb achieves an f1 value of 0.46. the deep learning algorithms (lstm, bilstm, and cnn) achieve an equal f1 score of 0.44, the lowest of all models. am-flair and am-roberta, two contextual embedding models, attain f1 scores of 0.48 and 0.50, respectively. the authors in (shahiki-tash et al., 2023) conducted experiments using transformer architectures and bert-based models, present hate speech detection toward the mexican spanish-speaking lgbt+ population, and achieve results with a macro f1 score of 0.73. the authors in @xcite presented a word-based tokenization approach to train a convolutional neural network (cnn). in @xcite , to examine language patterns, the authors use deep learning models such as long short-term memory (lstm) and recurrent neural networks (rnns). the lstm model, which is a form of rnn, is used to comprehend the context by capturing long-term relationships and detailed patterns in input sequences. the authors in (ayele et al., 2022a) examine the primary concerns associated with crowdsourcing annotation for the collection of amharic hate speech data using yandex toloka.an estimated 1,000 tweets are annotated annually, or 5,400 in total. classification models based on deep learning were developed utilizing lstm and bilstm. both models achieved an f1-score of 0.44. ## methodology various machine learning algorithms, including classical methods and deep learning techniques, can be used. neural networks and deep learning are among the most helpful ai approaches that have been developed @xcite . rnn is often used as a building block in modern neural networks to detect hate speech @xcite . our team deployed a task of automatic hate and offensive language detection in telugu codemixed text using a deep learning-based simple neural network (simple rnn) model. this design was chosen for its efficacy in capturing temporal dependencies within text data vital for understanding the sequential nature of language, and their ability to model short-term dependencies is useful for applications such as hate speech and offensive language detection. they are computationally simple compared to advanced architectures like long short term memory (lstm) or gated recurrent unit (gru), which allow faster training, especially on small datasets. the interpretability of simple rnns is advantageous in applications requiring a transparent decision-making process, such as hate speech detection. the model is trained on a training and validation dataset and evaluated on the test set. we perform pre-processing procedures, including tokenization and padding, removing punctuation marks, and filtering out stop words, to turn the text data into a format appropriate for neural network computation. the general methodology pipeline is shown in figure 2 3.1 dataset analysis the main point of our research resides in the hold-telugu dataset which is collected from social media @xcite , carefully curated with 4000 items of telugu codemixed text. each entry is extensively annotated for hate or nonhate content, capturing varied linguistic phrases and cultural nuances. code-mix is used in nearly all social media networks where individuals speak many languages. the use of code-mixed data in natural language processing (nlp) research is receiving a lot of interest right now @xcite . due to the expansion and significance of social media in communication, hate speech detection of social media code-mixed text has been an attractive subject of study in recent years @xcite . our investigation delves into the nature of the dataset, analyzing the distribution of hate vs. non-hate samples, the intricacies of codemixing patterns, and the possible issues these aspects represent for hate speech recognition programs. we show the distribution of data in figure 1 3.2 shared task description the hold-telugu shared task supplied users with a rich dataset of telugu code-mixed text @xcite , painstakingly annotated for hate and offensive content. this dataset includes different online sources, including social networking platforms, discussion forums, and news websites. participants were tasked with constructing models capable of reliably detecting whether a particular comment contained hate or derogatory language. the performance of the suggested models was evaluated using the macro-f1 score, a balanced metric that combines both precision and recall across both classes,ensuring a full and trustworthy assessment. ## discussion while our model achieved promising results, we acknowledge the limits inherent in the simple rnn design and the issues provided by codemixing. in future work, we will have a plan to address potential routes for development, including studying more complicated neural networks,adding additional variables like sentiment analysis, and leveraging larger and more diverse datasets. ## conclusion the hold-telugu collaborative initiative effectively addressed the critical problem of identifying hate speech in telugu code-mixed text. the shared employment enabled significant discoveries in this under-researched subject by bringing together divergent researchers and promoting teamwork. the large and diverse dataset, high-quality submissions, and intelligent analysis have paved the way for the continuing development of powerful hate speech detection algorithms for telugu code-mixed text. the successes of collaborative work go beyond technical advancements. the hold-telugu work helps to a more inclusive and healthier online environment for telugu communities by reducing the prevalence of toxic language. the shared task resources and results enable researchers and developers to continue this endeavor, resulting in better online interactions and protecting people from the negative consequences of hate speech. looking forward, the hold-telugu collaboration lays the groundwork for future research on hate speech detection in dravidian languages and code-mixed text. more research into sophisticated nlp methods, such as multilingual language models and fine-tuning procedures, has the potential to significantly improve the accuracy and general applicability of hate speech detection systems. furthermore, the data from the shared job may be used to create tools and resources that enable people and organizations to reject hate speech and promote online safety. the achievement of the hold-telugu joint endeavor demonstrates the enormous potential of collaborative research in addressing difficult social issues. research communities may positively contribute to the establishment of safer and more inclusive online environments for everybody by facilitating open data exchange, fostering diverse perspectives, and concentrating on practical applications.