Transformer-based Architecture for Empathy Prediction and Emotion Classification (2022)

Abstract: This paper describes the contribution of team PHG to the WASSA 2022 shared task on Empathy Prediction and Emotion Classification. The broad goal of this task was to model an empathy score, a distress score and the type of emotion associated with the person who had reacted to the essay written in response to a newspaper article. We have used the RoBERTa model for training and top of which few layers are added to finetune the transformer. We also use few machine learning techniques to augment as well as upsample the data. Our system achieves a Pearson Correlation Coefficient of 0.488 on Task 1 (Empathy - 0.470 and Distress - 0.506) and Macro F1-score of 0.531 on Task 2.

## introduction empathy and distress are quite important regarding human health. emotion classification in natural languages has been studied for over two decades, and many applications successfully used emotion as their principal component. empathy utterances can be emotional. therefore, examining emotion in text-based empathy has a significant impact on predicting empathy. empathic concern and personal distress are empathic responses that may result when observing someone in discomfort @xcite . some news stories are also displayed in this task, and people have reacted to them. the news is disturbing or discomforting to some people. and hence, regarding that, their empathy and distress are noted. this paper presents the wassa 2022 shared task: predicting empathy and emotion in reaction to news stories. this shared task included four individual tasks where teams developed models to predict emotions, empathy, and personality in essays in which people expressed their empathy and distress in reaction to news articles in which an individual or a group of people were harmed. additionally, the dataset also included the demographic information of the authors of the essays, such as age, gender, ethnicity, income, education level, and personality information. the shared task consisted of four tracks (optional): track 1: empathy prediction (emp) task consists of predicting both the empathy concern and the personal distress. (evaluation based on an average of pearson correlation @xcite of empathy and distress). track 2: emotion classification (emo) consists of predicting the emotion (sadness, joy, disgust, surprise, anger, or fear, taken from the six basic emotions @xcite ) also including neutral) at the essay-level (evaluation based on the macro f1-score). track 3: personality prediction (per), which consists in predicting the personality of the essay writer, knowing all their essays and the news article from which they reacted (evaluation based on the average of pearson correlation over personality values @xcite -conscientiousness, openness, extraversion, agreeableness, and stability). track 4: interpersonal reactivity index prediction (iri) consists of predicting the personality of the essay writer. (evaluation based on an average of pearson correlation over iri values -fantasy, perspective taking, empathetic concern, personal distress). we participated in only the first two tasks. ## related work over the last few years, earnest endeavors have been made in the nlp community to analyze empathy and distress. for text-based empathy prediction, @xcite laid a firm foundation for predicting @xcite empathic concern and personal distress scores in reaction to news articles. they present the first publicly available gold-standard dataset for text-based empathy and distress prediction. tagsets, and most studies are focussed on basic emotion models that psychological emotion models offer. the most popular one is the ekman 6 basic emotions @xcite . the emotions presented in this dataset are the same six emotions by ekman plus one extra emotion (neutral). ## dataset the dataset is an extension to the one provided by @xcite . for all the tasks, a traindev-test split was provided. the dataset consists of essays collected from participants who had read news articles about a person, a group of people, or disturbing situations. the dataset had an essay (300-800 characters), empathy score, a distress score, emotion label, and other demographic information (age, gender, race, education, income) as well as personality information (conscientiousness, openness, extraversion, agreeableness, stability) and interpersonal reactivity index (iri) scores (fantasy, perspective taking, empathetic concern, personal distress). ## conclusion this paper describes our submission to the wassa 2022 shared task, where we have used the already trained roberta model on a large dataset and then used its power by just finetuning on the given dataset. by the approach we have used, it can also be deduced that text augmentation and upsampling helped in emotion classification and predicting the empathy and distress scores as most of the time, the larger amount of data helps improve the training process of a model.