arslanahmed3/emotion-bigbird

BigBird fine-tuned for emotion detection

This is google/bigbird-roberta-base fine-tuned to classify English text into six emotions: anger, fear, joy, love, sadness and surprise.

It is one of four models compared in the paper:

Comparative Analysis of DistilBERT, XLNet, RoBERTa & BigBird for Emotion Detection in Conversational Text Arslan Ahmed, Barkha, Tushar Kanjwani, Shahzad Nasim. ILMA Journal of Technology & Software Management (IJTSM), Vol. 6, Issue 1, pp. 1–12, 2025. 📄 Paper · 💻 Code, notebooks & web demo

Results (test set, 2,000 samples)

Accuracy	Macro F1	Weighted F1
93.00 %	0.88	0.93

Strengths: sadness; long inputs (up to 512 tokens). Like every model in the study, it is weakest on surprise, the rarest class.

Other models from the paper: DistilBERT · XLNet · RoBERTa · BigBird

Labels

id	0	1	2	3	4	5
label	anger	fear	joy	love	sadness	surprise

Usage

# pip install transformers torch sentencepiece
from transformers import pipeline

clf = pipeline("text-classification", model="arslanahmed3/emotion-bigbird", top_k=None)
print(clf("i feel so lucky to have friends like you"))

Training

Dataset	Emotions Dataset for NLP (16,000 train / 2,000 val / 2,000 test)
Framework	PyTorch
Optimizer / LR	2e-5 (AdamW, HF Trainer)
Batch size	8
Max sequence length	512
Epochs	6
Hardware	Google Colab GPU

The training notebook is in the GitHub repo.

Citation

@article{ahmed2025emotion,
  title   = {Comparative Analysis of DistilBERT, XLNet, RoBERTa \& BigBird for Emotion Detection in Conversational Text},
  author  = {Ahmed, Arslan and Barkha and Kanjwani, Tushar and Nasim, Shahzad},
  journal = {ILMA Journal of Technology \& Software Management (IJTSM)},
  volume  = {6},
  number  = {1},
  pages   = {1--12},
  year    = {2025},
  url     = {https://ijtsm.ilmauniversity.edu.pk/arc/Vol6/i1p1}
}