sarcasm-distilbert

A fine-tuned distilbert-base-uncased model for classifying news headlines as Sarcastic or Not Sarcastic.

• GitHub repo (training notebook, EDA, baseline comparison): https://github.com/krskumarsatyam777-glitch/news-headline-sarcasm-detector
• Live demo: https://huggingface.co/spaces/satyame639291/news_headline_scarcasm_detector

Model Description

Fine-tuned version of distilbert-base-uncased for binary sequence classification, trained on the News Headlines Dataset for Sarcasm Detection (Misra, 2019) — headlines from The Onion (satirical) paired with headlines from HuffPost (genuine).

Label	Meaning
0	Not Sarcastic (genuine, from HuffPost)
1	Sarcastic (satirical, from The Onion)

How to Get Started

from transformers import pipeline

classifier = pipeline("text-classification", model="satyame639291/sarcasm-distilbert")
classifier("Shocking, Usain bolt realisies he can use legs for walking")

Training Details

• Base model: distilbert-base-uncased
• Epochs: 3 | Learning rate: 2e-5 | Batch size: 32 | Weight decay: 0.01
• Model selection: Best checkpoint by validation F1 (load_best_model_at_end=True) — epoch 2 was selected as the final model.
• Hardware: Single T4 GPU (Google Colab).

Results

Evaluated on a held-out test set (4,276 headlines) not used in training or model selection:

Model	Accuracy	F1 (weighted)
TF-IDF + Logistic Regression (default)	77.34%	0.7729
TF-IDF + Logistic Regression (tuned)	77.39%	0.7736
DistilBERT (this model)	92.00%	0.9200

Full methodology, EDA, and error analysis: training notebook.

Error patterns: false positives tended to be plain, non-humorous headlines rather than obviously quirky ones; false negatives mostly required outside knowledge (e.g. a public figure's reputation) that isn't present in the text itself.

Limitations

• Labels come from source publication (Onion vs. HuffPost), not human sarcasm annotation — the model may partly learn outlet writing style rather than sarcasm itself.
• Not validated on social media, reviews, conversational text, or non-English text.
• Sarcasm requiring external/cultural knowledge is harder to detect than sarcasm signaled by wording alone.

Contact

Satyam — GitHub | Hugging Face