LakshmiNarayanan-sugumar/fake-news-bert

Fake News Detector — BERT (Fine-tuned)

Deprecation Notice (July 2026) This model was found to rely on a shortcut during training: it learned to associate the word "Reuters" with the "real" class (present in 61.8% of real articles vs. 1.8% of fake ones in WELFake), rather than learning generalizable signals of fake news. This produced misleadingly high benchmark accuracy (99.53%) while failing badly on real-world out-of-distribution text — especially satire and hoaxes (53.8% OOD accuracy).

Use fake-news-bert-debiased instead, which fixes this issue and documents the full diagnosis, fix, and remaining known limitations.

This model is kept public as part of the project's documented history — see the debiased repo's README for the full before/after story.

---

Model Description

This model is a fine-tuned version of bert-base-uncased for binary fake news classification (real vs. fake), trained on the WELFake dataset. It is part of the Fake News Detector V2 project.

• Developed by: Lax (LakshmiNarayanan Sugumar)
• Model type: BERT-base, fine-tuned for sequence classification
• Language: English
• License: Apache 2.0 (inherited from base model)
• Finetuned from model: bert-base-uncased
• Repository: github.com/LakshmiNarayanan-Sugumar/fake_news_detector_v2
• Demo: HuggingFace Space — fake-news-detector

Uses

Direct Use

Given a news article's title and body text, the model predicts whether the article is real or fake. Intended for educational and portfolio demonstration purposes.

Out-of-Scope Use

Not intended for production moderation or fact-checking decisions without human review. Performance on news domains, time periods, or writing styles not represented in WELFake (e.g. non-English text, social media posts, satire) is unverified.

Bias, Risks, and Limitations

The model is trained on WELFake, a dataset merged from several existing fake/real news sources. Exact-duplicate articles were removed before training and evaluation, but near-duplicate or reworded articles covering the same story were not separately filtered, so reported metrics may carry minor residual optimism. The model reflects whatever stylistic or topical patterns exist in WELFake and may not generalize to other domains, languages, or adversarially written content.

How to Get Started

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("LakshmiNarayanan-sugumar/fake-news-bert")
model = AutoModelForSequenceClassification.from_pretrained("LakshmiNarayanan-sugumar/fake-news-bert")

text = "Your article title here. Your article body text here."
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
with torch.no_grad():
    logits = model(**inputs).logits
prediction = torch.argmax(logits, dim=-1).item()
print("Fake" if prediction == 1 else "Real")  # adjust label mapping to match your training

Training Details

Training Data

WELFake dataset (~72k articles before cleaning), a merge of four public fake/real news datasets. After removing rows with missing title/text and exact-duplicate articles, the cleaned dataset contained 62,200 rows, split 80/10/10 into train/validation/test (stratified by label).

Training Procedure

Title and body text were concatenated into a single input field, tokenized with the BERT tokenizer (max length 512, truncation enabled). Fine-tuned for 3 epochs with a per-device batch size of 16. The best checkpoint was selected based on validation loss (epoch 2 of 3), and the final reported metrics come from a single evaluation on a held-out test set never used during training or checkpoint selection.

Training Hyperparameters

• Epochs: 3
• Batch size: 16 (train and eval)
• Base model: bert-base-uncased
• Hardware: Kaggle T4 GPU(s)

Evaluation

Testing Data

10% held-out test split (6,220 articles), evaluated once after training completed.

Results

Metric	Score
Accuracy	99.53%
F1	99.47%
Precision	99.64%
Recall	99.31%

These metrics reflect a corrected training pipeline: the original version had a feature-engineering bug (duplicated text concatenation) and lacked a proper 3-way split, which risked inflating reported accuracy. This version fixes both issues; see repository README for details.

Technical Specifications

Model Architecture and Objective

BertForSequenceClassification (2 labels), built on bert-base-uncased.

Compute Infrastructure

Trained on Kaggle Notebooks using a T4 GPU.

Model Card Contact

LakshmiNarayanan Sugumar — GitHub