PVG at WASSA 2021: A Multi-Input, Multi-Task, Transformer-Based Architecture for Empathy and Distress Prediction (2021)

Abstract: Active research pertaining to the affective phenomenon of empathy and distress is invaluable for improving human-machine interaction. Predicting intensities of such complex emotions from textual data is difficult, as these constructs are deeply rooted in the psychological theory. Consequently, for better prediction, it becomes imperative to take into account ancillary factors such as the psychological test scores, demographic features, underlying latent primitive emotions, along with the text’s undertone and its psychological complexity. This paper proffers team PVG’s solution to the WASSA 2021 Shared Task on Predicting Empathy and Emotion in Reaction to News Stories. Leveraging the textual data, demographic features, psychological test score, and the intrinsic interdependencies of primitive emotions and empathy, we propose a multi-input, multi-task framework for the task of empathy score prediction. Here, the empathy score prediction is considered the primary task, while emotion and empathy classification are considered secondary auxiliary tasks. For the distress score prediction task, the system is further boosted by the addition of lexical features. Our submission ranked 1st based on the average correlation (0.545) as well as the distress correlation (0.574), and 2nd for the empathy Pearson correlation (0.517).

## introduction in recent years, substantial progress has been made in the nlp domain, with sentiment analysis and emotion identification at its core. the advent of attention-based models and complex deep learning architectures has led to substantial headways in sentiment/ emotion classification and their intensity prediction. however, the studies addressing the prediction of affective phenomenons of empathy and distress have been relatively limited. factors such as lack of large-scale quality labeled datasets, the weak notion of the constructs themselves, and inter-disciplinary dependencies have hindered the progress. the wassa 2021 shared task on predicting empathy and emotion in reaction to news stories @xcite provides a quality, gold-standard dataset of the empathic reactions to news stories to predict batson's empathic concern and personal distress scores. empathy, as defined by @xcite , is considered as "reactions of one individual to the observed experiences of another." it is more succinctly summarized by @xcite in three key components, @xcite knowing what another person is feeling (cognitive), (b) feeling what another person is feeling (emotional), and (c) responding compassionately to another person's distress (behavioral)." distress, on the other hand, as delineated by @xcite , is "a strong aversive and self-oriented response to the suffering of others, accompanied by the desire to withdraw from a situation in order to protect oneself from excessive negative feelings." empathy and distress are multifaceted interactional processes that are not always self-evident and often depend on the text's undertone. moreover, along with the textual data, multiple psychological and demographic features also play a vital role in determining these complex emotions. evidence by @xcite suggests that empathy and distress are not independent of the basic emotions (happiness, sadness, disgust, fear, surprise, and anger) the subject feels during a given scenario. this appositeness of the primitive emotions with empathy and distress can be aptly exploited using a multi-task learning approach. multi-task learning has led to successes in many applications of nlp such as machine translation @xcite , speech recognition @xcite , representation learning @xcite , semantic parsing @xcite , and information retrieval @xcite to name a few. nlp literature @xcite suggests that under specific circumstances and with well-crafted tasks, multi-task learning frameworks can often aid models to achieve state-of-the-art performances. @xcite further asserts that seemingly related tasks can often have similar underlying dynamics. with the same intuition, deep et al. (2020) designed a multi-task learning model for sentiment classification and their corresponding intensity predictions. building on these findings, we propose a multi-input, multitask, transformer-based architecture for the prediction of empathy and distress scores. the multiinput nature of the framework aggregates information from textual, categorical, and numeric data to generate robust representations for the regression task at hand. exploiting the latent interdependencies between primitive emotions and empathy/ distress, we formulate the multi-task learning problem as a combination of classification and regression. the model simultaneously classifies the text into its correct basic emotion, detects if it exhibits high empathy/ distress, and accordingly predicts its appropriate empathy/ distress intensity score according to batson's scale @xcite . this multi-input, multi-task learning paradigm is further bolstered with the addition of nrc emotion intensity lexicons @xcite ; nrc valence, arousal, and dominance lexicons @xcite ; and relevant features from empath @xcite . moreover, our proposed models have less than 110k trainable parameters and are still able to achieve relatively high pearson's correlation of 0.517 and 0.574 for empathy and distress, respectively, and 0.545 for average correlation, outperforming other teams. ## related work over the last few years, earnest endeavours have been made in the nlp community to analyze empathy and distress. earlier work in empathy mostly addressed the presence or absence of empathy in spoken dialogue @xcite @xcite @xcite . for text-based empathy prediction, @xcite laid a firm foundation for predicting batson's empathic concern and personal distress scores in reaction to news articles. they present the first publicly available gold-standard dataset for text-based empathy and distress prediction. plated a computational approach for understanding empathy in text-based health support. they developed a multi-task roberta-based bi-encoder paradigm for identifying empathy in conversations and extracting rationales underlying its predictions. wagner (2020) analysed the linguistic undertones for empathy present in avid fiction readers. computational work done for predicting distress is relatively modest. @xcite analysed textual data to examine associations between linguistic features and clients distress during psychotherapy. they combined linguistic features like positive and negative emotion words with psychological measures like outcome questionnaire-45 @xcite and outcome rating scale @xcite . @xcite studied the affiliation between distress, condolence, and empathy in online support groups using nested regression models. ## data description the wassa 2021 shared task @xcite provides an extended dataset to the one compiled by @xcite . the dataset has a total of 14 features spanning textual, categorical, and numeric data types. essays represent the subject's empathic reactions to news stories he/she has read. the demographic features of gender, race, education, and the essay's gold-standard emotion label cover the categorical input features. the numeric features include the subject's age and income, followed by personality traits scores (conscientiousness, openness, extraversion, agreeableness, stability) and interpersonal reactivity index (iri) scores (fantasy, perspective taking, empathetic concern, personal distress). the train-development-test split of the dataset is illustrated in ## proposed methodology in this section, we posit two multi-task learning frameworks for empathy and distress score prediction. they are elucidated in detail as follows: ## results and discussion the performance comparison of the various models on the validation data is reported in ## conclusion and future work in this work, we propose a multi-input, multi-task, transformer-based architecture to predict batson's empathic concern and personal distress scores. leveraging the dependency between the basic emotions and empathy/ distress, as well as incorporating textual, categorical, and numeric data, our proposed model generates robust representations for the regression task at hand. the addition of certain lexical features further improves the model's performance for distress score prediction. our submission ranked 1 st based on average correlation (0.545) as well as distress correlation (0.574), and 2 nd for empathy pearson correlation (0.517) on the test data. as for the future work of this research, a weighted loss scheme could be employed to enhance the results. from a psychological and linguistic standpoint, features such as part-of-speech tags, syntax parse tree, and the text's subjectivity and polarity scores could also be exploited.