Experiencer-Specific Emotion and Appraisal Prediction (2022)

Abstract: Emotion classification in NLP assigns emotions to texts, such as sentences or paragraphs. With texts like “I felt guilty when he cried”, focusing on the sentence level disregards the standpoint of each participant in the situation: the writer (“I”) and the other entity (“he”) could in fact have different affective states. The emotions of different entities have been considered only partially in emotion semantic role labeling, a task that relates semantic roles to emotion cue words. Proposing a related task, we narrow the focus on the experiencers of events, and assign an emotion (if any holds) to each of them. To this end, we represent each emotion both categorically and with appraisal variables, as a psychological access to explaining why a person develops a particular emotion. On an event description corpus, our experiencer-aware models of emotions and appraisals outperform the experiencer-agnostic baselines, showing that disregarding event participants is an oversimplification for the emotion detection task.

## introduction computational emotion analysis from text includes various subtasks, with the most prominent one being emotion classification or regression. its goal is to assign an emotion representation to textual units, and the way this is done typically depends on the domain of the data, the practical application of the task, and the psychological theories of reference: emotions can be modelled as discrete labels, in line with theories of basic emotions @xcite , as valence-arousal value pairs that define an affect vector space where to situate emotion concepts (illustrated, e.g., by @xcite , or as appraisal spaces that correspond to the cognitive evaluative dimensions underlying emotions foot_0 @xcite . irrespective of the adopted representations, most work in the field detects emotions from a single perspective -either to recover the emotion that the writer of a text likely expressed (e.g., with respect to emotion categories and intensities @xcite , and cognitive categories @xcite ), or to predict the emotion that the text elicits in the readers (e.g., using news articles, @xcite . only a few approaches combine or compare the reader's with the writer's perspective @xcite . however, none of them looks at the perspectives of the participants in events (both mentioned or implicit) as described by a text. focusing on such perspectives separately is essential to develop an all-round account of the affective implications that events have. it would emphasize how the facts depicted in text are amenable to different "emotion narratives", by pushing one or the other perspective in the foreground. for instance, a possible interpretation for the sentence "as the waiter yelled at her, the expression on my mother's face made all the staff look repulsed", could be: "my mother"→sadness, "the waiter"→anger, and "the staff"→disgust. there, one entity is responsible for an event (screaming), one is influenced by it, and the third is affected by the emotion emerging in the other (the facial expression, which can be seen as an event in itself). our goal is close to emotion role labeling, a special case of semantic role labeling (srl) @xcite . srl addresses the question "who did what to whom, where, when, and how?" @xcite , emotion srl asks "who feels what, why, and towards whom?" @xcite , mainly to detect causes of emotion-eliciting events @xcite for certain entities. here, we tackle a variation of this question, namely, "who feels what and under which circumstances?". the circumstances refer to the explanation pro-vided by appraisal interpretations, another novelty that we contribute to the emotion srl panorama. appraisal-based emotion representations capture entity-specific aspects that lead to an emotion, as they describe the subjective qualities that an individual sees in events. we propose a method for experiencer-specific emotion and appraisal analysis that bridges emotion classification and semantic role labeling. given texts that describe events and that include annotations for all participants, we assign an emotion and an appraisal vector to each potential emoter. our proposal is computationally simpler than creating a full graph of relations between causes and entities, as is normally done in (emotion) srl. yet, its fine-grained focus on event participants is beneficial over traditional classification-and regressionbased approaches: by predicting an emotion and scoring multiple appraisals for each entity, our model strongly outperforms text-level baselines. thus, the results demonstrate that assigning one emotion to the entire instance, or multiple emotions without considering for whom they hold, is a simplification of the emotional import of the text. ## related work in natural language processing, emotions are usually represented as discrete names following theories of basic emotions @xcite , or as values of valence and arousal @xcite . computational models based on such representations have been applied to many text sources, including reddit comments @xcite and tales @xcite , but also to resources created as part of psychological research. an example is the isear corpus. it consists of short reports collected in lab @xcite , instructing participants to describe events that caused in them a certain emotion. a similar collection practice was adopted by @xcite . in their enisear, crowdworkers completed sentences like "i felt [emotion name] when . . . " for seven emotion names. the emotions of entities are considered in emotion srl, whose goals comprise the recognition of emotion cue words, emotion experiencers/emoters and descriptions of emotion causes and targets @xcite @xcite . yet, most work focused on detecting causes (i.e., emotion-triggering events), and less on other semantic roles @xcite @xcite . the gap between entity-specific emotion analysis and emotion srl was partially filled in by @xcite . they aimed at better understanding the readers' attempts to interpret the experience of the texts' authors. they post-annotated instances from enisear with emotions and 22 appraisal concepts, both for the writer and all other event participants mentioned in the text. the appraisal variables include evaluations of events, as they were likely conducted by the event experiencers, including if authors felt responsible, if they needed to pay attention to the environment, whether they found themselves in control of the situation, and its pleasantness (see finally, our work is related to structured sentiment analysis @xcite , in which opinion targets, their polarity, but also an opinionholding (or expressing) entity is to be detected. most studies focused on sentiment targets and aspects @xcite , but there are also some that aim at detecting the opinion holder @xcite @xcite . annotation model input instance emotion appraisal exp ⟨exp⟩writer⟨/exp⟩ i felt bad . . . for him {guilt} (5, 1, 1, . . .) writer i felt bad . . . for ⟨exp⟩him⟨/exp⟩ {sadness} (1, 3, 1, . . .) text writer i felt bad . . . for him {guilt, sadness} (3, 2, 1, . . .) ## methods and experimental setting model. we model the task of experiencerspecific emotion analysis as a classification of instances which consist of experiencers e in the context of a text t @xmath0, . . . , t n ). there can be multiple experiencers in one text, therefore t @xmath1. each experiencer consists of a corresponding token sequence (t i , . . . , t j ) (1 ≤ i, j, ≤ |t e |), a set of emotion labels e @xmath2, fear, joy, . . .}, and a 22-dimensional appraisal vector a @xmath3; 5] 22 . to predict a e and e e for each experiencer e with the help of t e , we use as input a positional indicatorencoding of the experiencers in context (inspired by @xcite . the writer is encoded with an additional special token t @xmath4. we refer to this experiencer-specific model as exp. baseline. we compare this model to a baseline in which we simplify the experiencer-specific classification as text-level classification. during training, we assign the text t the union of all emotion labels of all contained experiencers, namely e @xmath5,@xmath6. analogously, the aggregation of the appraisal vectors is the centroid of all experiencers in one text: a @xmath7,@xmath8. we refer to this baseline model as text(-based prediction). ## results quantitative evaluation. tables 3 and 4 show the results. for emotion classification, we report precision, recall, and f 1 measures for the baseline text and the experiencer-specific predictions by exp in analysis. we show some examples in we observe that the text model has a tendency to predict the union of the emotions for all experiencers, but sometimes predicts more additional categories. this is a consequence of the tendency towards high recall predictions of this model. in example 1, both exp and text correctly assign the emotions anger, disgust and no emotion, but only exp distributes them correctly between "writer" and "the owners" (sadness is wrongly detected by both models). in example 2, joy is not predicted by text, but correctly assigned to "a group of children" by exp. exp further distributes shame and sadness to the correct entities (with a mistake assigning anger and no emotion to "a group of children" as well as anger and fear to "another child"). in example 3, exp correctly assigns sadness and shame to "writer" and sadness and no emotion to "my sister", while text fails to detect no emotion. in example 4, exp's prediction of anger and fear (for "our children") could be accepted to be correct despite it not being in line with the gold annotation. exp further predicts the correct emotions for "writer" (but makes a mistake assigning joy to "my ex husband"). in example 5, the emotions of "writer" are correctly assigned; "my son" is wrongly assigned joy in addition to no emotion (text mistakenly predicts other as well). however, the correctness of this annotation is debatable. maximal values for the gold appraisal values for self/other control and self/other responsibility are, in nearly all cases, mutually exclusive across experiencers. the text model is not informed about that and distributes the values across all entities. the exp model does indeed recover the individual values for the appraisals, but to varying degrees. in examples 2, 3, and 4, nearly all experiencers receive appraisal values close to the gold annotations. example 2 appears to be challenging: the writer has a high gold annotation value for self responsibility which is not automatically detected. further, "a id text 1 i felt . . . working in the street seeing faeces of dogs. the owners should take care of them but are being so lazy and neglected, that is terrible. 2 i felt . . . when i remember being part of a group of children at school who verbally bullied another child. 3 i felt . . . when i lost my sister's necklace that i had borrowed. 4 i felt . . . when my ex husband was hateful towards our children. 5 i felt . . . when my son was born. (a) example texts gold text exp id experiencer text emotion appraisal emotion appraisal emotion appraisal 1 writer a d a d no sa a d sa the owners no a d no sa no 2 writer sh a no sa sh sh a group of children j sh a no sa sh a j no sh another child sa a no sa sh a f sa 3 writer sa sh sa sh sa sh my sister sa no sa sh sa no 4 writer a sa a f j no sa sh a sa my ex husband a sh a f j no sa sh a j sh our children sa a f j no sa sh a f sa 5 writer j j o no j my son no j o no j no (b) annotations ## discussion and conclusion we presented the first approach of experiencerspecific emotion classification and appraisal regression. our evaluation on event descriptions shows the need for such methods, and that a text-instance level annotation is a simplification. this work provides the foundation for future research focused on texts in which multiple emotion labels co-occur, including reader/writer combinations or turn-taking dialogues. we propose to integrate experiencer-specific emotion modeling within such settings, for instance in novels, or news articles. it can also enrich the work of emotion recognition in dialogues @xcite : chains of emotions have been modeled, but not considering mentioned entities. our work focused on a corpus that has been annotated specifically for writers' and entities' emotions. there exist, however, also other corpora with experiencer-specific emotion annotations, namely emotion role labeling resources @xcite @xcite . in addition to other information, they also provide experiencerspecific emotion labels, though not in such an eventfocused context. still, modeling them following our method needs to be compared to more traditional approaches that aim at recovering the full role labeling graph. our approach to encoding the experiencer position in the classifier has been a straightforward choice. other model architectures (including positional embeddings, wang and chen, 2020) might perform better. another interesting methodological avenue is to model the predictions of multiple experiencers jointly to exploit their relations. finally, an open question is how to incorporate information from existing resources that are not labeled with experiencer-specific information. for instance, @xcite provide appraisal and emotion annotations for many more instances that might be beneficial in a transfer-learning setup.