bsingh commited on
Commit
9a7d2ee
1 Parent(s): d6e362c

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +128 -0
README.md ADDED
@@ -0,0 +1,128 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: en
3
+ tags:
4
+ - text-classification
5
+ - pytorch
6
+ - roberta
7
+ datasets:
8
+ - go_emotions
9
+ license: mit
10
+ widget:
11
+ - text: "I am not feeling well today."
12
+ ---
13
+
14
+ ## This model is trained for GoEmotions dataset which contains labeled 58k Reddit comments with 28 emotions
15
+ - admiration, amusement, anger, annoyance, approval, caring, confusion, curiosity, desire, disappointment, disapproval, disgust, embarrassment, excitement, fear, gratitude, grief, joy, love, nervousness, optimism, pride, realization, relief, remorse, sadness, surprise + neutral
16
+
17
+
18
+ ## The training script is provided here: https://github.com/bsinghpratap/roberta_train_goEmotion
19
+ - The model works well on most of the emotions except: 'desire', 'disgust', 'embarrassment', 'excitement', 'fear', 'grief', 'nervousness', 'pride', 'relief', 'remorse', 'surprise']
20
+ - I'll try to fine-tune the model further and update here if RoBERTa achieves a better performance.
21
+
22
+ ## Some Training details:
23
+ - Each text datapoint can have more than 1 label. Most of the training set had 1 label: Counter({1: 36308, 2: 6541, 3: 532, 4: 28, 5: 1})
24
+ - So currently I just used the first label for each of the datapoint. Not ideal but it does a decent job.
25
+
26
+ ## Current Performance
27
+ ============================================================
28
+ Emotion: admiration
29
+ ============================================================
30
+ GoEmotions Paper: 0.65
31
+ RoBERTa: 0.62
32
+ Support: 504
33
+ ============================================================
34
+ Emotion: amusement
35
+ ============================================================
36
+ GoEmotions Paper: 0.80
37
+ RoBERTa: 0.78
38
+ Support: 252
39
+ ============================================================
40
+ Emotion: anger
41
+ ============================================================
42
+ GoEmotions Paper: 0.47
43
+ RoBERTa: 0.44
44
+ Support: 197
45
+ ============================================================
46
+ Emotion: annoyance
47
+ ============================================================
48
+ GoEmotions Paper: 0.34
49
+ RoBERTa: 0.22
50
+ Support: 286
51
+ ============================================================
52
+ Emotion: approval
53
+ ============================================================
54
+ GoEmotions Paper: 0.36
55
+ RoBERTa: 0.31
56
+ Support: 318
57
+ ============================================================
58
+ Emotion: caring
59
+ ============================================================
60
+ GoEmotions Paper: 0.39
61
+ RoBERTa: 0.24
62
+ Support: 114
63
+ ============================================================
64
+ Emotion: confusion
65
+ ============================================================
66
+ GoEmotions Paper: 0.37
67
+ RoBERTa: 0.29
68
+ Support: 139
69
+ ============================================================
70
+ Emotion: curiosity
71
+ ============================================================
72
+ GoEmotions Paper: 0.54
73
+ RoBERTa: 0.48
74
+ Support: 233
75
+ ============================================================
76
+ Emotion: disappointment
77
+ ============================================================
78
+ GoEmotions Paper: 0.28
79
+ RoBERTa: 0.18
80
+ Support: 127
81
+ ============================================================
82
+ Emotion: disapproval
83
+ ============================================================
84
+ GoEmotions Paper: 0.39
85
+ RoBERTa: 0.26
86
+ Support: 220
87
+ ============================================================
88
+ Emotion: gratitude
89
+ ============================================================
90
+ GoEmotions Paper: 0.86
91
+ RoBERTa: 0.84
92
+ Support: 288
93
+ ============================================================
94
+ Emotion: joy
95
+ ============================================================
96
+ GoEmotions Paper: 0.51
97
+ RoBERTa: 0.47
98
+ Support: 116
99
+ ============================================================
100
+ Emotion: love
101
+ ============================================================
102
+ GoEmotions Paper: 0.78
103
+ RoBERTa: 0.68
104
+ Support: 169
105
+ ============================================================
106
+ Emotion: neutral
107
+ ============================================================
108
+ GoEmotions Paper: 0.68
109
+ RoBERTa: 0.61
110
+ Support: 1606
111
+ ============================================================
112
+ Emotion: optimism
113
+ ============================================================
114
+ GoEmotions Paper: 0.51
115
+ RoBERTa: 0.52
116
+ Support: 120
117
+ ============================================================
118
+ Emotion: realization
119
+ ============================================================
120
+ GoEmotions Paper: 0.21
121
+ RoBERTa: 0.15
122
+ Support: 109
123
+ ============================================================
124
+ Emotion: sadness
125
+ ============================================================
126
+ GoEmotions Paper: 0.49
127
+ RoBERTa: 0.42
128
+ Support: 108