WASSA 2022 Shared Task: Predicting Empathy, Emotion and Personality in Reaction to News Stories (2022)

Abstract: This paper presents the results that were obtained from WASSA 2022 shared task on predicting empathy, emotion, and personality in reaction to news stories. Participants were given access to a dataset comprising empathic reactions to news stories where harm is done to a person, group, or other. These reactions consist of essays and Batson’s empathic concern and personal distress scores. The dataset was further extended in WASSA 2021 shared task to include news articles, person-level demographic information (e.g. age, gender), personality information, and Ekman’s six basic emotions at essay level Participation was encouraged in four tracks: predicting empathy and distress scores, predicting emotion categories, predicting personality and predicting interpersonal reactivity. In total, 14 teams participated in the shared task. We summarize the methods and resources used by the participating teams.

## introduction emotion and empathy prediction and analysis, in its broader perspective, has been an active research area in the last two decades, with growing volume of studies that provide insightful findings and resources. emotion classification in natural languages has been studied over two decades and many applications successfully used emotion as their major components. empathy utterances can be emotional, therefore, examining emotion in textbased empathy possibly has a major impact on predicting empathy. analyzing text-based empathy and emotion have different applications; empathy is a crucial component in applications such as empathic ai agents, effective gesturing of robots, and mental health, emotion has natural language applications such as commerce, public health, and disaster management. despite the progress, improvements can be made to develop or further enhance the prediction and detection of emotions and psychological constructs in natural texts including empathy, distress, and personality. in this paper, we present the wassa 2022 shared task: predicting empathy and emotion in reaction to news stories. we used the same dataset provided by @xcite which is an extension of @xcite 's dataset that includes news articles that express harm to an entity (e.g. individual, group of people, nature). each of these news articles is associated with essays in which authors expressed their empathy and distress in reactions to these news articles. each assay is annotated for empathy and distress, and supplemented with personality traits and demographic information of the authors (age, gender, ethnicity, income, and education level) (refer to section 3 for more details). given this dataset as input, the shared task consists of four tracks: in the remainder of this paper, we first review related work (section 2), after which we introduce the dataset used for both tracks (section 3). the shared task is presented in section 4 and the official results in section 5. a discussion of the different systems participating in both tracks is presented in section 6 and we conclude our work in section 7. ## related work we provide related work for each track: emotion predictions (section 2.1), empathy and distress (section 2.2), personality prediction, and interpersonal reactivity prediction (section 2.3). ## data collection and annotation we used the same dataset provided in wassa 2021 shared task @xcite . ## shared task we setup all four tracks in codalab ( https://competitions.codalab.org/ competitions/28713 ). we describe each task separately (objectives and metadata) in section 4.1 and then describe dataset, resources, and evaluation metrics in section 4.2. note that the first two tracks are the same as offered by wassa 2022 shared task while the last two tracks (per and iri) are new contributions of this shared task. ## overview of submitted systems a total of 14 teams participated in the shared tasks with 10 teams participating in both emp and emo and 2 participated in all tracks. in this section, we provide a summary of the machine learning models, features, resources, and lexicons that were used by the teams. ## conclusions in this paper we presented the shared task on empathy and emotion prediction of essays that were written in response to news stories to which five teams participated. based on the analysis of the systems we can conclude that fine-tuning a transformer language model or relying on features extracted from transformer models along with jointly learning related tasks can lead to a robust modeling of empathy, distress, and emotion. despite the strength of these strong contextualized features, we also observed that task-specific lexical features extracted from emotion and sentiment lexicons can still create a significant impact on empathy, distress, and emotion models. furthermore, the topperforming emotion models used external datasets to further fine-tune the language models, which indicates that data augmentation is important when modeling emotion, even if the text genre is different from the genre of the task at hand. finally, using demographic and personality information as features revealed a significant impact on empathy, distress, and emotion models. particularly, joint modeling of distress and empathy coupled with those features yielded the best results for most of the top-ranked systems that were developed as part of this shared task.