Lidoma@ D ravidian L ang T ech 2024: Identifying Hate Speech in T elugu Code-Mixed: A BERT Multilingual (2024)

Abstract: Over the past few years, research on hate speech and offensive content identification on social media has been ongoing. Since most people in the world are not native English speakers, unapproved messages are typically sent in code-mixed language. We accomplished collaborative work to identify the language of code-mixed text on social media in order to address the difficulties associated with it in the Telugu language scenario. Specifically, we participated in the shared task on the provided dataset by the Dravidian- LangTech Organizer for the purpose of identifying hate and non-hate content. The assignment is to classify each sentence in the provided text into two predetermined groups: hate or non-hate. We developed a model in Python and selected a BERT multilingual to do the given task. Using a train-development data set, we developed a model, which we then tested on test data sets. An average macro F1 score metric was used to measure the model’s performance. For the task, the model reported an average macro F1 of 0.6151.

## introduction india is a multilingual nation with a diverse linguistic past in south asia. as the official and administrative language of the state of andhra pradesh in southern india, telugu is one of the most widely spoken languages in the country. having 96 million or more telugu speakers as native speakers @xcite . in addition to their native, local, or regional tongue, many in the region feel at ease utilizing english for daily communication. these multilingual people prefer to share their thoughts, opinions, and comments on social media sites in several scripts and/or languages, which makes code-mixing the norm on social media @xcite @xcite . the spread of hate speech @xcite and objectionable content has far-reaching effects, increasing tensions, encouraging discrimination, and widening societal divisions as social media and online platforms become an essential part of daily life in india. in light of the pressing need to address such content, this study attempts to manage the complexities of telugu codemixed @xcite language by utilizing sophisticated natural language processing (nlp) models, most notably bert (bidirectional encoder representations from transformers) @xcite . sentiment analysis @xcite @xcite , a powerful tool in natural language processing, often focuses on discerning emotions conveyed in the text. when applied to hate speech, it plays a crucial role in understanding the underlying sentiment behind abusive language, shedding light on the detrimental impact of hateful expressions within the digital sphere. the results of this work aim to establish a basic framework for continued attempts to prevent the spread of damaging content, provide platforms with efficient tools for moderation, and foster a more favorable and supportive online environment among the diverse range of languages found in india's digital space. ## related work the identification of hate speech has grown in importance in the social media and internet communication era. due to the increase in hate speech occurrences, academics are investigating many approaches to effectively address this problem, such as deep learning @xcite , transformer-based models, convolutional neural network (bade and seid, 2018; sha), and machine learning @xcite . traditional machine learning techniques played a major role in the early stages of hate speech identification, laying the groundwork for later studies in the area. davidson et al @xcite made a significant addition in 2017 by offering a data set and a number of features created especially for the detection of hate speech. this groundbreaking discovery launched a trajectory of developments in the field and signaled the beginning of systematic hate speech detection research. a crucial element of conventional machine learning methodologies was the feature engineering process. scholars employed attributes like n-grams, sentiment analysis, and lexical aspects to efficiently depict textual content. sentiment analysis assessed the text's emotional tone, whereas n-grams in particular demonstrated how language is sequential. lexical features, which include language patterns and vocabulary. hate speech detection research was first driven by traditional machine learning techniques, which were crucial in laying the groundwork for further advancements and offering insightful information. the advent of more sophisticated strategies was spurred by these systems' limits in addressing language complexity and context, despite their promising outcomes. zhang et al @xcite introduced a convolutional neural network (cnn) model presenting a novel method for detecting hate speech. this model outperformed other approaches in terms of performance. because hate speech frequently uses certain phrases, keywords, and linguistic clues, cnns are particularly good at identifying local patterns within the text. in 2018 @xcite presented a hierarchical attention-based model this strategy focused on attention mechanisms and hierarchical representations in order to address the need to record nuanced hate speech @xcite . a more thorough examination of the substance of hate speech was made possible by the use of hierarchical attention models, which made it possible to examine data at the word and sentence levels. a breakthrough in natural language processing, bert (bidirectional encoder representations from transformers) was introduced by @xcite in 2019 . the novel aspect of bert is its capacity to comprehend a word's context by taking the complete phrase into account. this contextual awareness is especially important in the complex and frequently subtle realm of hate speech. bert can more precisely and thoroughly identify hate speech by collecting the entire context @xcite . the promise of bert in the area of hate speech identification was immediately recognised by researchers. they efficiently used bert's potent language understanding capabilities to discern between hateful and non-hateful information by honing it on hate speech data sets @xcite . modern outcomes in hate speech identification can be attributed to this adaptation.the transformer family has grown ever since bert was introduced. ## data set and task description the task at hand focuses on hate speech classification within a data set encompassing 4000 sentences expressed in telugu, represented both in native script and romanized forms. within this data set, 2061 sentences are categorized as non-hate, while 1939 sentences are designated as hate in the training set foot_0 .moreover, an additional test data set containing 500 sentences is provided, lacking categorized labels. the primary goal of this task is to employ bert multilingual model to discern patterns from the labeled training data in order to predict whether the 500 test sentences fall into the categories of hate speech or non-hate @xcite . this classification task presents a significant challenge in analyzing and identifying hate speech within telugu text, considering the multilingual aspect involving both native script and romanized forms. with a substantial dataset comprising labeled examples of hate and non-hate speech, machine learning models can be trained to recognize intricate patterns, linguistic nuances, and context-specific features associated with hate speech @xcite . ## data preprocessing during the preprocessing phase of our data, a series of crucial steps were undertaken to enhance the quality and applicability of the text data for hate speech identification and other natural language processing (nlp) tasks. a key focus was placed on text preprocessing, involving the removal of special characters, numeric digits, and emoticons. emojis, despite often expressing emotions or nonverbal cues, were eliminated from the text due to their limited contribution to semantic meaning in text analysis. moreover, special characters and numerical digits were also omitted from the text corpus. these elements were considered as they could potentially complicate later processing stages and generally convey minimal linguistic information within nlp tasks. the primary objective behind this meticulous cleaning procedure was to streamline the text data, reducing noise and irrelevant features, while retaining a more refined and cohesive representation suitable for subsequent analysis or model development. ## evaluation metrics and results the f1-score, computed as the harmonic mean of precision and recall, provides a balanced assessment by considering both metrics. achieving a macro f1-score of 0.6151 in our task indicates a moderate level of overall performance, suggesting a reasonable balance between precision and recall across multiple classes. this metric signifies the model's effectiveness in correctly identifying hate and offensive content in a multi-class classification scenario, highlighting its general capability in accurately categorizing various classes within the data set. ## error analysis the bert multilingual model applied to telugu hate speech exhibits notable accuracy, particularly in correctly identifying true positives. however, a significant challenge arises with false positives, misclassified instances even in a balanced dataset. this pattern necessitates thorough analysis and adjustments in the model's discriminatory capabilities. comprehensive evaluations on validation and test sets are essential for assessing the model's adaptability. proposed strategic modifications involve fine-tuning parameters and scrutinizing false positive occurrences to enhance overall accuracy and efficacy. ## limitations challenges in a telugu language dataset include limited resources hindering preprocessing techniques. the dynamic nature of evolving fake news poses adaptability issues for models trained on historical data. identifying relevant features for effective training is challenging, particularly in a language with unique linguistic characteristics not well-captured by standard nlp techniques. ## conclusion utilizing the bert model, this study focuses on detecting hate and offensive language within telugu codemixed text, achieving a macro f1-score of 0.6151. it showcases the model's proficiency in identifying hate speech amidst the intricate linguistic composition of telugu codemixed text. despite this success, the research underscores the imperative for continual improvement in both model architecture and data set expansion to heighten the accuracy of hate speech detection. the study serves as a foundational milestone, laying the groundwork for future advancements. it sets a benchmark for the development of more sophisticated and sensitive systems crucial for accurately identifying and mitigating harmful information present in multilingual digital realms. this work not only validates the potential of the bert model but also emphasizes the ongoing need for refinement and innovation in combating hate speech in diverse linguistic contexts.