Dimensional Emotion Detection from Categorical Emotion (2021)

Abstract: We present a model to predict fine-grained emotions along the continuous dimensions of valence, arousal, and dominance (VAD) with a corpus with categorical emotion annotations. Our model is trained by minimizing the EMD (Earth Mover’s Distance) loss between the predicted VAD score distribution and the categorical emotion distributions sorted along VAD, and it can simultaneously classify the emotion categories and predict the VAD scores for a given sentence. We use pre-trained RoBERTa-Large and fine-tune on three different corpora with categorical labels and evaluate on EmoBank corpus with VAD scores. We show that our approach reaches comparable performance to that of the state-of-the-art classifiers in categorical emotion classification and shows significant positive correlations with the ground truth VAD scores. Also, further training with supervision of VAD labels leads to improved performance especially when dataset is small. We also present examples of predictions of appropriate emotion words that are not part of the original annotations.

## introduction in psychology literature, basic emotions are categorized as happy, sad, angry and so on @xcite , however, we can feel and express more subtle and complex emotions beyond them. they can be systematically represented with the valence-arousal-dominance (vad) model which maps emotional states to 3-dimensional continuous vad space. this space allows various emotions to be projected into the space with measurable distances from one another @xcite , covering a wider range of subtle emotions compared to the categorical models with a finite set of basic emotions. capturing such fine-grained emotions with dimensional vad models could benefit clinical natural language processing (nlp) @xcite , emotion regulation such as psychotherapy @xcite . for example, analyzing the client's utterance and acknowledging the negative emotion as 'neglected' rather than 'sad', which is known as 'affect labeling', would reduce negative physiological, behavioral, and psychological responses resulting from that emotional state. thus developing a dimensional emotion detection model would be very useful, but one problem is a lack of required annotated resources. there is a relatively small sentence-level corpus with full vad annotations @xcite , and a corpus annotated with v and a dimensions @xcite , and only with v @xcite . we could build additional resources by labeling vad scores by best-worst scaling @xcite . instead, we approach this problem with a novel and more efficient method to predict vad scores from existing corpora annotated with categorical emotions @xcite @xcite @xcite @xcite @xcite . in this paper, we propose a framework to learn the vad scores from sentences with categorical emotion labels by leveraging the vad scores of the label words obtained from the nrc-vad lexicon @xcite . we demonstrate our approach by fine-tuning a pre-trained language model roberta @xcite . our model learns conditional vad distributions through supervision of categorical labels and uses them to compute vad scores as well as to predict the emotion labels for a given sentence. our contributions are as follows. ## approach overview. we predict vad scores for a given text from a model trained on a dataset with categorical emotion annotations. the key idea is to train vad prediction model by using categorical emotion labels. it is possible because we find that those categorical labels can be mapped to word-level vad scores by using nrc-vad lexicon (mohammad, 2018). thus we conceptualize categorical emotion as a point in the vad space. then we sort the labels by each vad dimension to obtain (sparse) ground truth conditional vad distributions (fig. 1a , 1b ). then we train a model to predict the vad distributions, rather than an emotion category, by minimizing the distance between the predicted and the ground truth distributions. this allows the model to predict the vad scores (expectations of predicted distributions, fig. 1d ) and pick an emotion label within a given set of categorical labels (argmax of emotion labels, fig. 1c ). model architecture (fig 1a ). formally, an emotion detection model is p (e|x) where e is an emotion drawn from a set of pre-defined categorical emotions @xmath0, anger, sadness, ...}, and @xmath1, x 2 , ..., x n } is a sequence of symbols x i representing the input text. usually e is a one-hot vector in emotion classification. unlike classification models directly learning p (e|x), we learn each distribution of v, a, and d from a pair of input text x and categorical labels. to this end, we map the categorical emotion labels to the three-dimensional vad space, @xmath2, a, d), using the nrc-vad lexicon. each v, a and, d ranges from 0 to 1. for example, an emotion label "joy" is mapped to (0.980, 0.824, 0.794) and "sad" to (0.225, 0.333, 0.149) @xcite . using es, our model predicts the following distribution: furthermore, since the vad dimensions are nearly independent @xcite , we simply assume mutual independence: p (v, a, d|x) = p (v|x)p (a|x)p (d|x). for each decomposed conditional distribution, we can use any type of trainable function with sufficient complexity to capture the linguistic patterns from the given input. as a demonstration, we use pre-trained bidirectional language model roberta @xcite which shows high performances in natural language understanding tasks if fine-tuned over task-specific datasets. we stack a softmax or sigmoid activation layer over the hidden state corresponding to @xcite token in the model for each conditional distribution. ## experiments we mainly focus on demonstrating our approach can effectively predict continuous emotional dimensions only with categorical emotions throughout experiments. ## ablation study we further conduct ablation study to investigate our model's vad prediction performances. since we use pre-trained roberta and fine-tune them with different datasets, the effect of model architecture, ## qualitative examples in tweet categorical label nearest neighbors from vad scores gooood morning it is such a #blessing to see another day all that read this i hope have a great morning joy, optimism reaffirm, shimmer, brighten, affections, mythological not only was and responsible for the unnecessary outrage of this movie, but made the director look bad anger, disgust refusal, liar, falsified, disrespect, unsavory mentally suffered #iwanttodie #worthless #lifewithoutcolor #pain #suicidal disgust, pessimism, sadness orphaned, wasting, decomposed, hopelessness, dead given our model's predicted vad scores, we find the nearest neighbor words for those scores by using nrc-vad-lexicons @xcite . we first rescale our model's predicted vad scores from 0 to 1 for each vad dimension since the nrc-vad lexicons have values from 0 to 1. to do this, we first predict vad scores for every sentence in semeval test set and then we rescale the scores by following: (x -min(x))/(max(x) -min(x)) to make all dimension scores range from 0 to 1. next, we find the nearest neighbor words by using the rescaled vad values. euclidean distances between the values and all words in nrc-vad-lexicons are computed, and we pick the top five nearest words among them with the smallest distances. we present the words in the right column of ## related work categorical model of emotion assumes that the categories represented by emotion words compose the building blocks of human emotion. supporting evidence includes six basic emotions @xcite , and findings of universally adaptive emotions @xcite . an alternative to understand how people conceptualize emotional feelings is the dimensional model of emotion. @xcite suggested the initial idea of emotion coordinates. @xcite further constructed pleasure or valence-arousal-dominance (pad, vad) model, a semantic scale model to rate emotional state, representing an emotional state as a pair of orthogonal coordinates on vad dimensions. absolute values of the intercorrelations among the three scales show considerable independence among the scales @xcite , categorical emotion states can be represented in 3d (vad) emotion space. based on emotional dimensions, word-level vad annotation of english words has been created @xcite @xcite . also, there are few sentencelevel va or vad annotated corpora @xcite @xcite . by using these resources, recent work tried to predict vad scores from sentences based on variational autoencoders @xcite , adversarial learning @xcite , ensemble learning @xcite . however, sentence-level vad annotated corpus is scarce, we use more common resource which is sentences annotated with basic categorical emotions for vad score prediction @xcite @xcite @xcite @xcite @xcite . these datasets are commonly used for emotion classification, we use them to predict vad scores from sentences with word-level vad scores of categorical emotion labels. recently, a lot of dataset related to emotion has been released. especially, there are dataset in healthcare domain @xcite , relation between emoji and emotion (shoeb and de melo, 2020), and emotional text from social media @xcite . all of these are cateogrical annotations which again shows the lack of dimensional annotations thus the need for our model to capture fine-grained emotion detection. also, our work could be extended to a large domain: it could help better performance of multimodal emotion detection @xcite , emotion in conversation @xcite , and emotion change in a paragraph @xcite . there are multiple emotion datasets annotated with various types of label sets. to train model across the various shaped emotion dataset, several existing studies aggregate various format of emotion dataset into a common annotation schema, and show better performance using unified dataset @xcite . however, still the labels are mapped to other predefined emotions and the datasets are limited to categorical labels. in @xcite , they convert categorical emotions into vad representation using simple feed-forward neural networks. they train model with dataset labeled with both emotion categories and vad. however, in our paper, we convert categorical emotion knowledge to vad without any labeled pairs. ## discussion and conclusions we propose learning to predict vad scores from the text with categorical emotion annotations. our framework predicts vad score distributions by minimizing the emd distances between predicted vad distributions and sorted label distributions as a proxy of target vad distributions. even our model assumes vad emotion space and order between emotions, our model shows significant prediction performances in real-world datasets. robustness. our framework could be applied to multimodal datasets. if we apply our framework to iemocap @xcite , the zero-shot vad predictions are significantly correlated with ground truths (v: 0.396, a: 0.241, d: 0.197) as well. however, the performance is rather low since our model does not leverage other modalities such as audio or videos. once our framework is extended to integrate such information through image/speech encoders, performance would be improved. we use nrc-vad to estimate distance between emotions because it is constructed very carefully to locate words in vad space. if we use other word-level vad resources such as anew @xcite , we observe positive results as well (v: 0.682, a: 0.270, d: 0.296). ethical considerations. a model trained by our approach could be used to understand and regulate one's own emotional states and to save people from suicide. in addition, social bots capable of emo-tion recognition could help people in various ways. however, a model trained by our approach could be misused to detect or control others' emotional states against their will. it may reveal private information about mental or physical health or private feelings an individual does not wish to share. this concern is even more serious when we consider that machine learning models can be cost-effective and thus used at scale for pervasive monitoring of emotions @xcite . an example of a harmful use of the technology is manipulating the semantic emotive content of user news feeds which can affect the choices of both individuals and groups on the platform to engage and interact @xcite . from a different perspective, problems might occur from the inaccurate results of the model. mispredictions of the models could result in harmful outcomes even in systems designed to be helpful, and this is a serious problem in many languages with relatively low resources (i.e., languages other than english and a few others that are extensively studied within nlp), resulting in inequity with respect to the benefits gained by this technology. basically, resources to train emotion detection models are scarce in most languages, and their quality would degrade if translated to other languages from english since cultural nuances to defining emotions vary. therefore, one should follow guidelines for the ethical use of emotional ai technologies, which present a checklist for anyone engaged with data about human emotion. @xcite for example, mcstay and pavliscak's guidelines @xcite include a number of salutary suggestions for taking action as a practitioner. we hope our framework will be helpful in building an annotated sentence-level vad emotion dataset by providing machine-annotated vad scores as a start, or use it just as vad score prediction model. most of the languages except english would not have such corpus with vad annotations, so our model will be helpful to build resources using multilingual corpora with categorical emotion labels @xcite .