|
--- |
|
language: en |
|
tags: |
|
- text-classification |
|
- pytorch |
|
- roberta |
|
datasets: |
|
- go_emotions |
|
license: mit |
|
widget: |
|
- text: "I am not feeling well today." |
|
--- |
|
|
|
## This model is trained for GoEmotions dataset which contains labeled 58k Reddit comments with 28 emotions |
|
- admiration, amusement, anger, annoyance, approval, caring, confusion, curiosity, desire, disappointment, disapproval, disgust, embarrassment, excitement, fear, gratitude, grief, joy, love, nervousness, optimism, pride, realization, relief, remorse, sadness, surprise + neutral |
|
|
|
|
|
## The training script is provided here: https://github.com/bsinghpratap/roberta_train_goEmotion |
|
- The model works well on most of the emotions except: 'desire', 'disgust', 'embarrassment', 'excitement', 'fear', 'grief', 'nervousness', 'pride', 'relief', 'remorse', 'surprise'] |
|
- I'll try to fine-tune the model further and update here if RoBERTa achieves a better performance. |
|
|
|
## Some Training details: |
|
- Each text datapoint can have more than 1 label. Most of the training set had 1 label: Counter({1: 36308, 2: 6541, 3: 532, 4: 28, 5: 1}) |
|
- So currently I just used the first label for each of the datapoint. Not ideal but it does a decent job. |
|
|
|
## Current Performance |
|
============================================================ |
|
Emotion: admiration |
|
============================================================ |
|
GoEmotions Paper: 0.65 |
|
RoBERTa: 0.62 |
|
Support: 504 |
|
============================================================ |
|
Emotion: amusement |
|
============================================================ |
|
GoEmotions Paper: 0.80 |
|
RoBERTa: 0.78 |
|
Support: 252 |
|
============================================================ |
|
Emotion: anger |
|
============================================================ |
|
GoEmotions Paper: 0.47 |
|
RoBERTa: 0.44 |
|
Support: 197 |
|
============================================================ |
|
Emotion: annoyance |
|
============================================================ |
|
GoEmotions Paper: 0.34 |
|
RoBERTa: 0.22 |
|
Support: 286 |
|
============================================================ |
|
Emotion: approval |
|
============================================================ |
|
GoEmotions Paper: 0.36 |
|
RoBERTa: 0.31 |
|
Support: 318 |
|
============================================================ |
|
Emotion: caring |
|
============================================================ |
|
GoEmotions Paper: 0.39 |
|
RoBERTa: 0.24 |
|
Support: 114 |
|
============================================================ |
|
Emotion: confusion |
|
============================================================ |
|
GoEmotions Paper: 0.37 |
|
RoBERTa: 0.29 |
|
Support: 139 |
|
============================================================ |
|
Emotion: curiosity |
|
============================================================ |
|
GoEmotions Paper: 0.54 |
|
RoBERTa: 0.48 |
|
Support: 233 |
|
============================================================ |
|
Emotion: disappointment |
|
============================================================ |
|
GoEmotions Paper: 0.28 |
|
RoBERTa: 0.18 |
|
Support: 127 |
|
============================================================ |
|
Emotion: disapproval |
|
============================================================ |
|
GoEmotions Paper: 0.39 |
|
RoBERTa: 0.26 |
|
Support: 220 |
|
============================================================ |
|
Emotion: gratitude |
|
============================================================ |
|
GoEmotions Paper: 0.86 |
|
RoBERTa: 0.84 |
|
Support: 288 |
|
============================================================ |
|
Emotion: joy |
|
============================================================ |
|
GoEmotions Paper: 0.51 |
|
RoBERTa: 0.47 |
|
Support: 116 |
|
============================================================ |
|
Emotion: love |
|
============================================================ |
|
GoEmotions Paper: 0.78 |
|
RoBERTa: 0.68 |
|
Support: 169 |
|
============================================================ |
|
Emotion: neutral |
|
============================================================ |
|
GoEmotions Paper: 0.68 |
|
RoBERTa: 0.61 |
|
Support: 1606 |
|
============================================================ |
|
Emotion: optimism |
|
============================================================ |
|
GoEmotions Paper: 0.51 |
|
RoBERTa: 0.52 |
|
Support: 120 |
|
============================================================ |
|
Emotion: realization |
|
============================================================ |
|
GoEmotions Paper: 0.21 |
|
RoBERTa: 0.15 |
|
Support: 109 |
|
============================================================ |
|
Emotion: sadness |
|
============================================================ |
|
GoEmotions Paper: 0.49 |
|
RoBERTa: 0.42 |
|
Support: 108 |
|
|