xxxier/distilbert-emotion-classifier

DistilBERT Emotion Classifier

A fine-tuned DistilBERT model for emotion classification on Twitter text data.

Model Description

This model is a fine-tuned version of distilbert-base-uncased on the emotion dataset. It classifies text into 6 emotion categories.

Emotion Labels

Label ID	Emotion
0	sadness
1	joy
2	love
3	anger
4	fear
5	surprise

Training Details

Training Configuration

• Base Model: distilbert-base-uncased
• Training Epochs: 3
• Learning Rate: 2e-5
• Batch Size: 16
• Weight Decay: 0.01
• FP16: Enabled

Performance

Metric	Value
Test Accuracy	92.8%
Test F1 (weighted)	0.9276
Validation Accuracy (best)	93.55%

Training Progress

Epoch	Validation Accuracy	Validation Loss
1	92.45%	0.215
2	93.55%	0.164
3	93.9%	0.156

Usage

from transformers import pipeline

classifier = pipeline("text-classification", model="YOUR_USERNAME/distilbert-emotion-classifier")

result = classifier("I am so happy today!")
print(result)
# [{'label': 'joy', 'score': 0.999}]

Or load the model directly:

from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("YOUR_USERNAME/distilbert-emotion-classifier")
model = AutoModelForSequenceClassification.from_pretrained("YOUR_USERNAME/distilbert-emotion-classifier")

Limitations

• The model may struggle with sarcasm and irony
• Typos can affect predictions
• The "surprise" category has lower accuracy compared to others

Citation

If you use this model, please cite the original emotion dataset:

@inproceedings{saravia-etal-2018-carer,
    title = "{CARER}: Contextualized Affect Representations for Emotion Recognition",
    author = "Saravia, Elvis  and Liu, Hsien-Chi Toby  and Huang, Yen-Hao  and Wu, Junlin  and Chen, Yi-Shin",
    booktitle = "Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing",
    year = "2018",
    publisher = "Association for Computational Linguistics",
}