roberta-base-sarcasm-headlines

roberta-base fine-tuned to classify English news headlines as satirical or real.

Labels: LABEL_0 = real news, LABEL_1 = satire.

Training data

News Headlines Dataset For Sarcasm Detection (Misra & Arora): satirical headlines from The Onion and real headlines from HuffPost. After deduplication, 28,503 headlines (47.5% satirical), stratified 80/10/10 split.

Training

Full fine-tuning with a new classification head: learning rate 2e-5, batch size 16, fp16, on a single T4 GPU. The checkpoint with the lowest validation loss (after epoch 1 of 3) was kept, because validation loss increased in later epochs.

Results (test set, 2,851 headlines)

Model	Accuracy	F1 (satire)
Majority class baseline	0.524	0.000
This model	0.932	0.926

Precision for satire is 95.6% and recall is 89.7%.

Limitations

All satirical examples come from The Onion and all real ones from HuffPost, so the model partly learns the style and typical topics of these two outlets rather than sarcasm in general. It tends to flag neutral local news as satire (e.g. "city council approves budget for new public library") and misses satire that imitates plain news formats or requires world knowledge. Headlines in the dataset are lowercase, so lowercase your input.

Usage

from transformers import pipeline

classifier = pipeline("text-classification", model="NexLiR/roberta-base-sarcasm-headlines")
classifier("local man confident he can fix printer by staring at it harder")