Guilt by Association: Emotion Intensities in Lexical Representations (2021)

Abstract: What do linguistic models reveal about the emotions associated with words? In this study, we consider the task of estimating word-level emotion intensity scores for specific emotions, exploring unsupervised, supervised, and finally a self-supervised method of extracting emotional associations from pretrained vectors and models. Overall, we find that linguistic models carry substantial potential for inducing fine-grained emotion intensity scores, showing a far higher correlation with human ground truth ratings than state-of-the-art emotion lexicons based on labeled data.

## introduction there has been substantial research on methods to label words with associated emotions. crowdsourcing approaches have been used to compile databases to study the nexus between them @xcite @xcite . another strategy, adopted by the well-known depechemood @xcite and depechemood++ @xcite lexicons is to apply simple statistical methods to emotionally tagged data crawled from specific online sources. in this paper, we consider the question: what do linguistic models reveal about the emotions associated with words? word vectors @xcite have often been evaluated on standard word relatedness benchmarks. in this paper, we instead explore to what extent they encode emotional associations. earlier methods @xcite used corpus statistics in tandem with dimensionality reduction techniques for emotion prediction. @xcite considered word embeddings trained on emojis for emotion analysis. @xcite proposed predicting sentiment polarity ratings from word vectors, while other studies @xcite predicted valence, arousal, and dominance using supervised deep neural networks. @xcite proposed incorporating valence, arousal, and dominance ratings as additional signals into dense word embeddings. @xcite showed that emotion intensity scores can be predicted based on a lexicon providing valence-arousal-dominance ratings. the task of emotional association of words has been studied on other languages as well. @xcite presents a dataset of spanish words labelled with ekman's six emotions, while others explored cross-lingual propagation from one language to another @xcite or to hundreds of other languages @xcite . we show that we can make use of linguistic models to obtain high correlations with emotion intensities without any need for manual ratings. while this work focuses on english, the methodology can be applied to data in any language. overview. we introduce the task and experimental setup in section 2. then, section 3 presents our techniques to address this task based on linguistic models. we provide detailed empirical evaluation results in section 4. finally, section 5 concludes the paper by discussing the relevance of our experiments. contribution. overall, our intriguing finding in this paper is that pretrained linguistic models allow us to predict much more accurate emotion associations than state-of-the-art emotion lexicons. we show that this holds even without any supervision, while different kinds of supervised setups yield even better results. ## emotion intensities as associations in our study, we seek to predict the emotions associated with individual words. such predictions can also be useful for sentence-level predictions, as considered in section 4.2, but that is not the primary focus of this paper. while many emotion lexicons only provide binary emotion labels, it is clear that words may exhibit different degrees of association with an emotion. for instance, the word party suggests a greater degree of joy than the word sea, although the latter may as well be associated with joy. emotion inventory and ground truth. the nrc emotion/affect intensity lexicon (mohammad, 2018) provides ground-truth human ratings of emotion intensity for english words. the latest release of the data covers the eight basic emotions proposed in plutchik's wheel of emotions @xcite , i.e., anger, anticipation, disgust, fear, joy, sadness, surprise, and trust. however, the techniques we consider in our paper apply broadly to any inventory of discrete emotions. the ratings in the dataset are scaled to [0, 1] such that a score of 1 signifies that a word "conveys the highest amount of" a particular emotion, while 0 corresponds to the lowest amount. these ratings were solicited via crowdsourcing and then standardized using best-worst scaling. benchmark data. in our work, instead of viewing the resource as a lexicon that provides emotion intensity scores, we propose to treat it as providing gold standard associations between pairs of words, one of the two words being an emotion name. thus, we derive a semantic association benchmark similar to widely used word relatedness benchmarks such as the rg-65 @xcite and ws353 @xcite sets. in order to more fairly evaluate resources that only provide unigrams, we disregard bigrams in the lexicon, resulting in a total of 9,706 word pairs. to facilitate an evaluation of unsupervised and supervised techniques, for each emotion, we split the corresponding data into training/validation/test portions with a ratio of 64%/16%/20%. this results in a total of 7,762 pairs in the training sets, and 1,944 pairs in the test sets. the remaining 1,552 instances serve as validation data. we adopt these data splits for all of the experiments in this paper and also make them available online for reproducibility. 1 ## estimating emotion intensity emotion lexicons are typically created using crowdsourcing @xcite or draw-1 http://emotion.nlproc.org/ ing on emotion-labeled text @xcite . we investigate deriving emotion scores from word embedding models without any emotion-specific signals. ## conclusion in this paper, we show that pretrained linguistic models readily provide higher-quality emotional intensity information than state-of-the-art emotion lexicons, despite not being trained on emotionlabeled data, particularly if the representation space is post-processed. we find that a regular supervised variant obtains very high correlations, while a self-supervised variant that does not require gold standard training data is able to outperform the unsupervised method and can as well be applied to arbitrary emotion labels. overall, our results confirm that linguistic models have remained underexplored for word-level emotion intensity assessment. our training/validation/test splits and newly induced lexicons created using our method are available from http://emotion.nlproc.org/ to promote further research in this area.