EM -26@ LT - EDI 2025: Detecting Racial Hoaxes in Code-Mixed Social Media Data (2025)

Abstract: Social media platforms and user-generated content, such as tweets, comments, and blog posts often contain offensive language, including racial hate speech, personal attacks, and sexual harassment. Detecting such inappropriate language is essential to ensure user safety and to prevent the spread of hateful behavior and online aggression. Approaches base on conventional machine learning and deep learning have shown robust results for high-resource languages like English and find it hard to deal with code-mixed text, which is common in bilingual communication. We participated in the shared task “LT-EDI@LDK 2025” organized by DravidianLangTech, applying the BERT-base multilingual cased model and achieving an F1 score of 0.63. These results demonstrate how our model effectively processes and interprets the unique linguistic features of code-mixed content. The source code is available on GitHub.1

## introduction in recent years, the rise of smartphones and the affordable internet has made social networks a central part of everyday life @xcite . platforms like twitter, instagram, and facebook have allowed users to communicate and share ideas widely. although these platforms offer improved communications and networking benefits, they also pose risks, especially with regard to privacy, misinformation, and hate speech. such issues have been significantly affected during crises @xcite . racial hoaxes, a form of information disorder @xcite , involve spreading false or misleading content that targets individuals based on ethnicity or nationality @xcite . although the relationship between disinformation and hate speech is complex, the two often overlap and can destabilize public opinion. researchers have categorized information disorders into three main types: misinformation, disinformation, and malinformation all of which disrupt trust and communication @xcite @xcite . these narratives can intensify hostility, polarize groups, and fuel stereotypes or threats against communities base on race, religion, or other attributes @xcite . such content may also cause lasting psychological harm, including anxiety and depression. during emergencies, it can mislead the public and result in harmful decisions @xcite . the covid-19 pandemic revealed the scale of racial hoaxes, where particular groups were unjustly blamed, often resulting in discrimination and violence @xcite . the prevalence of code-mixed language adds further complexity, as current nlp systems struggle to handle informal and linguistically diverse expressions. this underscores the need for improved hate speech detection techniques in multilingual contexts. researchers aim to address these issues by participating in shared tasks and contributing to safer, more inclusive digital spaces. ## related work many researchers carried out important early work on the detection of hate-or fake-generated content. disinformation, in particular, relies on identitybase controversies and adversarial narratives. it uses a variety of rhetorical techniques and forms of knowing, including truths, half-truths, and judgments laden with value, to exploit and amplifies identity-driven controversies @xcite . because it can use the truth or portions of the truth to misinform, the concept of disinformation extends much beyond what is true or not @xcite . the deliberate creation of deceptive or inaccurate content has led to the emergence of what is commonly referred to as fake news @xcite @xcite . the term fake news typically describes fully fabricated stories @xcite ) that are knowingly false yet often presented with enough realism to appear credible. while defining fake news precisely remains challenging, scholars generally agree that it involves the intentional misleading of large audiences by individuals or groups outside of traditional media, using sensationalist and seemingly trustworthy formats crafted to deceive @xcite . what makes fake news particularly damaging is its ability to imitate and exploit legitimate news sources, drawing on their credibility while simultaneously eroding it. one key feature that distinguishes fake news from conventional journalism is its emotional appeal it tends to use surprising and emotionally charged content to increase user engagement, sharing, and memory retention @xcite . hence, it is necessary to address hateful and fake narratives by considering both their targets and severity @xcite . @xcite made the first step for using supervised learning methods to identify harassment in online platforms. researchers used a supportvector machine (svm) to group social media posts base on local contextual and sentiment cues @xcite . researchers investigated the effectiveness of character n-grams, word n-grams, and skip-grams in detecting hoax speech in social media content. their system, trained on an english dataset with three class labels, achieved a classification accuracy of 78% @xcite . researchers introduced a convolutional neural network (cnn) model, which was a system to detect offensive tweets in hindi-english code-switched language @xcite . researchers curated hindi-english codemixed tweets to aid the development of methods to identify hate speech @xcite @xcite . the dataset consists exclusively of twitter data written in the roman script. the authors used character and word n-grams, punctuation, lexicon, and negation features for their classification method, using either svm or random forest classifiers. any combination of all features with svm achieved the best performance accuracy, up to 71.7% to detect hate speech @xcite . although the automatic detection of offensive language has been extensively studied in resourcerich languages such as english @xcite @xcite , research in the resourcepoor hindi language remains extremely limited. as a contribution to the initiative on online hate and societal harmony, this work advances the current state of research by addressing the detection of offensive content in code-mixed text using a bert-base multilingual cased model, demonstrating its effectiveness in the context of the "lt-edi@ldk 2025" task organized by dravidianlangtech. related efforts by the cic-nlp team has also shown the applicability of multilingual transformer models for detecting ai-generated and deceptive content across languages, including english and dravidian code-mixed text @xcite . these approaches collectively emphasize the growing potential of transformer-based models in handling complex, multilingual, and socially sensitive nlp tasks. ## methodology this study employed the bert-base-multilingualcased model from the hugging face transformers library. it was chosen for its strong contextual understanding across 100+ languages, crucial for handling code-mixed social media text. the model was fine-tuned for binary classification to distinguish racial hoaxes from non-hoax content. while it is well known that pre-trained transformers outperform shallow models, we included cnn and transformer-ffnn as baselines to quantify performance differences and highlight trade-offs in lowresource scenarios. pytorch was used for training and evaluation with gpu support. ## results we compared several models for detecting hoax speech in code-mixed content. as shown in ## discussion due to the linguistic complexity and social sensitivity involved, detecting racial hoaxes in codemixed hindi-english social media posts remains a challenging task. our results demonstrate that multilingual transformer models, particularly bert, perform well in this context. bert achieved an f1-score of 0.6700 and an accuracy of 0.8000, reflecting strong generalization capabilities and effective contextual understanding, even when handling informal and noisy data. xlm-roberta achieved the highest auc (0.7781), reflecting good class separation, but its lower f1-score shows an imbalance between precision and recall. cnn, though faster and more efficient, lacked the depth to capture the nuanced meaning in racial hoax texts. similarly, the transformer-ffnn model underperformed, suggesting that shallow architectures struggle with the ambiguity and language mixing common in such posts. these results highlight the importance of deep contextual modeling for identifying deceptive narratives in multilingual environments. while bert demonstrated the best overall performance, all models were challenged by codeswitching and subtle sarcasm, highlighting the need for more diverse and culturally annotated training data. ## error analysis the confusion matrices in figures 1 and 2 highlight a repeated pattern of misclassification, particularly for the minority class "racial hoax." out of 247 actual "racial hoax" instances, 197 were misclassified as "not racial hoax" indicating a strong bias toward the majority class. in contrast, the model performed well on the "not racial hoax" class, correctly classifying 755 out of 774 cases. this imbalance indicates that the model has difficulty identifying the small linguistic or contextual signals that identify racial hoaxes. the results highlight the need to consider methods such as class balancing, deep semantic understanding, and advanced feature engineering. increasing the sensitivity of the model about minority class characteristics would enhance the overall classification rate and reduce false negatives in critical categories such as racial hoaxes.