From Scarcity to Capability: Empowering Fake News Detection in Low-Resource Languages with LLM s (2025)

Abstract: The rapid spread of fake news presents a significant global challenge, particularly in low-resource languages like Bangla, which lack adequate datasets and detection tools. Although manual fact-checking is accurate, it is expensive and slow to prevent the dissemination of fake news. Addressing this gap, we introduce BanFakeNews-2.0, a robust dataset to enhance Bangla fake news detection. This version includes 11,700 additional, meticulously curated fake news articles validated from credible sources, creating a proportional dataset of 47,000 authentic and 13,000 fake news items across 13 categories. In addition, we created a manually curated independent test set of 460 fake and 540 authentic news items for rigorous evaluation. We invest efforts in collecting fake news from credible sources and manually verified while preserving the linguistic richness. We develop a benchmark system utilizing transformer-based architectures, including fine-tuned Bidirectional Encoder Representations from Transformers variants (F1-87%) and Large Language Models with Quantized Low-Rank Approximation (F1-89%), that significantly outperforms traditional methods. BanFakeNews-2.0 offers a valuable resource to advance research and application in fake news detection for low-resourced languages. We publicly release our dataset and model on GitHub to foster research in this direction.

## introduction the widespread dissemination of fake news, defined as intentionally misleading information, has become a critical issue in modern society with social consequences. fake news and misinformation circulate across media channels-from social networks to online news portals-often aiming to mislead and manipulate public opinion. the consequences of such disinformation can range from 1 github: https://github.com/shibu4064/indonlp † these authors have equal contributions. shaping public opinion on critical matters to catalyzing large-scale societal unrest. for example, during the covid-19 pandemic, misinformation regarding vaccine safety led to substantial vaccine reluctance @xcite . in bangladesh, the effects of such misinformation have been severe, including incidents of violence and communal discord spurred by false rumors online (shirina and prodhan, 2020; @xcite . moreover, the infodemic-defined as an overabundance of information, including false or misleading details-further complicated efforts to combat covid-19 globally, as highlighted in studies exploring misinformation trends and mitigation strategies @xcite @xcite . this challenge extends to various content forms, such as articles, images, videos, and memes, amplifying the difficulty of detection @xcite @xcite . detecting fake news in low-resource languages like bangla remains challenging due to limited datasets and resources. while english-language fake news detection has progressed, robust datasets for bangla remain scarce, hindering model development. although efforts like the banfakenews dataset @xcite and others (al-zaman and noman, 2023) have made initial strides, existing datasets remain limited in size and coverage, and manual fact-checking is impractical at scale. to address these limitations, we present banfakenews-2.0, a substantially extended dataset tailored for improved bangla fake news detection. building upon banfakenews, this new dataset includes 13,000 source-verified fake news articles, forming a balanced collection of 60,000 news items (47,000 authentic, 13,000 fake) across 13 diverse categories compared to the previous largest banfakenews dataset. manually curating an independent test set of 1,000 news articles further enables rigorous model evaluation. our benchmarks incorporate transformer-based models, such as bert, and fine-tuned large language models (llms) using quantized low-rank approximation (qlora). bloom is a state-of-the-art, open-access large language model that is collaboratively developed by hundreds of researchers and trained on the multilingual roots corpus. it supports 46 natural and 13 programming languages, enabling broad applications and competitive performance across benchmarks @xcite . we observe that our fine-tuned bloom 560m model achieves the highest performance, with a macro f1 score of 89. this dataset and benchmark represent a crucial step in advancing fake news detection for low-resource languages like bangla, providing a foundation for future research and practical applications. our main contributions include: we focused on data preparation to ensure linguistic richness and dataset diversity with two main objectives: (1) collect verified fake news from diverse sources and domains and (2) enhance dataset variety while minimizing redundancy. our newly curated dataset comprises approximately 13,000 fake and 47,000 authentic news articles from online news portals and mainstream media. we have collected the misleading or false context type of news mostly from www.jaachai.com and www.bdfactcheck.com . these two websites provide a logical and informative explanation of the authenticity of the news published on other sites. so, we have also collected the news mentioned on those two sites from the actual publishing sites and ensured that we avoid duplicates. we have used python's web-scraping method for automated and accurate collection of category-based news from different online news portals, such as politics, sports, entertainment, medical, religious, etc. the initial screening has been conducted by evaluating the credibility of sources and verifying claims through fact-checking platforms, authoritative references, or collaborative verification methods. relevant keywords such as "rumor," "hoax," "viral news," and bangla-specific terms linked to sensational topics have helped in categorizing the articles. employing automated web-scraping techniques alongside manual validation ensures data accuracy and quality. additionally, maintaining a balanced representation of topics, time-frames, and domains has been ensured to create this dataset. for authentic news, we selected the top 30 bangladeshi news portals, recognized for their credibility and high readership. for fake news, we gathered content from six major fact-checking platforms that frequently debunk misinformation in bangladesh, identifying and validating articles as probable fake news for inclusion. to ensure uniqueness, we filtered out duplicates and removed items with over 50% or 300 words of token overlap, aiming to expand vocabulary diversity and contextual variety. this broad range of content enhances the robustness of our classification system, supporting better generalization across various linguistic styles. each article was cross-checked by three annotators to confirm authenticity. five undergraduate students, guided by detailed source verification protocols, reviewed potentially misleading sources and excluded redundant entries. note that, we define a verified source of news when the source is at least a person or organization capable of verification of claimed news. when no specific source is available, the reporters or journalists themselves are considered the source of the news. we used majority voting to assign a final label of "fake" or "authentic," achieving a high inter-annotator agreement score of 0.93, indicating strong labelling consistency @xcite . during dataset analysis, we standardized categories to align with the classifications used in banfakenews @xcite , resulting in 13 distinct categories. categories were assigned based on the classification of the news at its source. if the source did not provide a category, the news was thoroughly read to understand its context and categorized accordingly. we focus on increasing the number of fake news articles to reduce the data imbalance, with 500 fake news articles per category. still, we face challenges in the lifestyle, medical, and religious categories. the final dataset, comprising 60k news articles, is distributed across 13 categories in ## methodologies here, we will outline the methods to create a benchmark model for detecting fake news in bangla. our methodologies include traditional linguistic attributes as well as neural networks and transformerbased models. ## result and analysis among word n-grams, unigrams achieved the highest f1 score of 84%, outperforming bigrams (83%) and trigrams (78%). combining these ngrams resulted in an f1 score of 85%, demonstrating that multi-gram approach enhances classification accuracy. character n-grams yielded similar performance; however, combinations of character n-grams did not provide substantial gains. across experiments, authentic news classification achieved over 90% in p, r, and f1. however, fake news classification showed greater variability. traditional svm models, employing linguistic features, outperformed llms and transformers-based models in identifying authentic news. conversely, llm-based models excelled in detecting fake news, yielding higher f1 scores. notably, the transformer models multilingual bert (m-bert-unc) and bloom achieved an f1 score of 81% in the fake news class, surpassing the 77% f1-score achieved by the c3-gram model. however, traditional models performed slightly better overall, reaching an f1 score of 98% in the authentic class, compared to the highest f1 score of 96% for transformers. this discrepancy may stem from the increased volume of fake news in the dataset, posing unique challenges for transformers in handling nuanced model train dataset test dataset mac. f1 svm (all) banfakenews test (internal) 74 svm (all) banfakenews-2.0 test (internal) 86 svm (all) banfakenews test (external) 39 svm (all) banfakenews-2.0 test (external) 91 bloom banfakenews test (internal) 78 bloom banfakenews-2.0 test (internal) 89 bloom banfakenews test (external) 29 bloom banfakenews-2.0 test (external) 67 to assess the generalisability of our models, we evaluated them using a manually curated external test set of 1,000 samples. we tested the top-performing models-the traditional linguistic feature-based svm and the llm-based bloom-both trained on the banfakenews-2.0 dataset, as shown in ## conclusion and future works the study presents banfakenews-2.0, a bangla fake news dataset with 13k manually annotated articles across 13 categories aimed at improving fake news detection in bangla. our evaluation demonstrated that bloom and m-bert-unc models outperformed other models, highlighting the importance of contextually diverse datasets over basic linguistic features for achieving high accu-racy. banfakenews-2.0 allowed transformer models and llms to excel, highlighting the need for diverse datasets and robust detection tools. future work will focus on enhancing dataset features, refining models, and exploring real-time monitoring. testing emerging llms like mistral, minitron, and gpt 4 in zero-shot settings may provide further insights. banfakenews-2.0 provides a strong foundation for advancing research in bangla fake news detection and mitigation.