Appraisal Theories for Emotion Classification in Text (2020)

Abstract: Automatic emotion categorization has been predominantly formulated as text classification in which textual units are assigned to an emotion from a predefined inventory, for instance following the fundamental emotion classes proposed by Paul Ekman (fear, joy, anger, disgust, sadness, surprise) or Robert Plutchik (adding trust, anticipation). This approach ignores existing psychological theories to some degree, which provide explanations regarding the perception of events. For instance, the description that somebody discovers a snake is associated with fear, based on the appraisal as being an unpleasant and non-controllable situation. This emotion reconstruction is even possible without having access to explicit reports of a subjective feeling (for instance expressing this with the words “I am afraid.”). Automatic classification approaches therefore need to learn properties of events as latent variables (for instance that the uncertainty and the mental or physical effort associated with the encounter of a snake leads to fear). With this paper, we propose to make such interpretations of events explicit, following theories of cognitive appraisal of events, and show their potential for emotion classification when being encoded in classification models. Our results show that high quality appraisal dimension assignments in event descriptions lead to an improvement in the classification of discrete emotion categories. We make our corpus of appraisal-annotated emotion-associated event descriptions publicly available.

## introduction the task of emotion analysis is commonly formulated as classification or regression in which textual units (documents, paragraphs, sentences, words) are mapped to a predefined reference system, for instance the sets of fundamental emotions fear, anger, joy, surprise, disgust, and sadness proposed by @xcite , or by @xcite , which includes also trust and anticipation. machine learning-based models need to figure out which words point to a particular emotion experienced by a reader, by the author of a text, or a character in it. depending on the resource which has been annotated, the description of an emotion experience can vary. on twitter, for instance, other than direct reports of an emotion state ("i feel depressed"), hashtags are used as emotion labels to enrich the description of events and stances ("i just got my exam result #sad"). in news articles, emotional events are sometimes explicitly mentioned ("couple infuriate officials" @xcite ) and other times require world knowledge ("tom cruise and katie holmes set wedding date", labeled as surprise @xcite ). in literature, a sequence of events which forms the narrative leads to an emotion in the reader. in this paper, we focus on those texts which communicate emotions without an explicit emotion word, but rather describe events for which an emotion association is evident. such textual examples became popular in natural language processing research with the use of the data generated in the isear project @xcite . the project led to a dataset of descriptions of events triggering specific affective states, which was originally collected to study event interpretations with a psychological focus. in text analysis, to infer the emotion felt by the writers of those reports, an event interpretation needs to be accomplished. for instance, in the text "when a car is overtaking another and i am forced to drive off the road", the model needs to associate the event with fear. to date, nearly all computational approaches that associate text with emotions are agnostic to the way in which emotions are communicated, they do "not know" how to interpret events, but, presumably, they purely learn word associations instead of actual event interpretations. one might argue that approaches predicting fine-grained dimensions of affect, namely arousal and valence, actually tackle this problem @xcite . however, these typically do not infer downstream emotion categories. further, particularly regarding events, psychological theories offer more detailed information. as an example, the emotion component model @xcite advocates that cognitive appraisal dimensions underly discrete emotion classes @xcite . these appraisal dimensions foot_0 evaluate (1) how pleasant an event is (pleasantness, likely to be associated with joy, but unlikely to appear with disgust), (2) how much effort an event can be expected to cause (anticipated effort, likely to be high when anger or fear is experienced), (3) how certain the experiencer is in a specific situation (certainty, low, e.g., in the context of hope or surprise), (4) how much attention is devoted to the event (attention, likely to be low, e.g., in the case of boredom or disgust), (5) how much responsibility the experiencer of the emotion holds for what has happened (self-other responsibility/control, high for feeling challenged or pride), and (6) how much the experiencer has control over the situation (situational control, low in the case of anger). despite their richness, cognitive theories of appraisal and their empirical results have not been exploited for emotion prediction in text yet. we fill this gap with this paper and analyze the relation between appraisal dimensions and emotion categories in a text classification setting. we post-annotate an english emotion corpus of self-reports of emotion events @xcite , which already contains annotations related to the emotions of anger, disgust, fear, guilt, joy, sadness, and shame, and add the appraisal dimensions by @xcite mentioned above. further, we analyze if an automatic prediction of these dimensions from text is possible with standard neural methods, and if these predictions contribute to emotion classifications. our main contributions are: (1) the first event-centered text corpus annotated with appraisal dimensions; (2) the evaluation of how well text classification models can recognize these appraisal dimensions; (3) we show emotion classification benefits from the information of appraisal dimensions, when high quality predictions of these are available. further, (4), we replicate the study by @xcite from a cl/nlp perspective, based on textual event descriptions. ## corpus the main objective of this study is to understand the relation between appraisal dimensions and emotion categories. therefore, we build appraisal annotations on top of enisear, an existing corpus of 1001 english event descriptions which are already labeled with the discrete categories of anger, disgust, fear, guilt, joy, sadness, and shame @xcite . each instance has been generated by a crowdworker on the platform figureeight by completing the sentence "i feel [emotion name], when . . . ". this corpus has an advantage over the original isear resource because it has a german counterpart which can be used in further studies; moreover, its emotion labels have been intersubjectively validated. the corpus, annotations, and our implementations are available at http://www.ims.uni-stuttgart.de/ data/appraisalemotion . ## experiments we now move to our evaluation if automatic methods to recognize emotions can benefit from being informed about appraisal dimensions explicitly. we first describe our models and how we address our research questions and then turn to the results. ## conclusions and future work we investigated the hypothesis that informing an emotion classification model about the cognitive appraisal regarding a situation is beneficial for the model performance. we were able to show that emotion classification performs better than text-based classification, under the assumption that perfect appraisal predictions are possible and shows complementary correct predictions. yet, neither in a multi-task learning nor a pipeline, in which the appraisal was predicted as a basis, could we show an improvement in emotion classification. this provides evidence that, though our appraisal predictor is of reasonable performance, the model suffers from error propagation. this is still an encouraging result, suggesting that future work should further investigate the combination of appraisal information with emotion prediction, particularly in the light of our oracle ensemble that indicated a clear improvement. this first study on the topic raises a couple of research questions: would there be other neural architectures which are better suited for including the appraisal information? will more annotated data improve the prediction quality sufficiently? finally, it should be analyzed if giving the annotators access to the emotion label when making the appraisal annotation could have changed the results.