T eam_ H awk at WASSA 2023 Empathy, Emotion, and Personality Shared Task: Multi-tasking Multi-encoder based transformers for Empathy and Emotion Prediction in Conversations (2023)

Abstract: In this paper, we present Team Hawk’s participation in Track 1 of the WASSA 2023 shared task. The objective of the task is to understand the empathy that emerges between individuals during their conversations. In our study, we developed a multi-tasking framework that is capable of automatically assessing empathy, intensity of emotion, and polarity of emotion within participants’ conversations. Our proposed core model extends the transformer architecture, utilizing two separate RoBERTa-based encoders to encode both the articles and conversations. Subsequently, a sequence of self-attention, position-wise feed-forward, and dense layers are employed to predict the regression scores for the three sub-tasks: empathy, intensity of emotion, and polarity of emotion. Our best model achieved average Pearson’s correlation of 0.7710 (Empathy: 0.7843, Emotion Polarity: 0.7917, Emotion Intensity: 0.7381) on the released development set and 0.7250 (Empathy: 0.8090, Emotion Polarity: 0.7010, Emotion Intensity: 0.6650) on the released test set. These results earned us the 3rd position in the test set evaluation phase of Track 1.

## introduction empathy involves understanding and sharing others' feelings. in conversation, empathy is demonstrated through active listening, acknowledging emotions, and providing supportive responses. emotion polarity refers to the positive or negative nature of expressed emotions, while emotion intensity relates to the strength of those emotions. computing empathy is an emergent paradigm and become an important component in conversational ai @xcite . empathy is critical for clinical applications such as automated behavioral therapy @xcite . implementing complex emotionalmotivational states and effectively responding in an empathetic manner remains a significant challenge in human-machine interaction. the dataset shared by the organizers comprises news stories and corresponding brief essays written by participants during conversation sessions. participants engage in dialogues, assessing each other's conversation turns for empathy, emotion intensity, and polarity. a third-party annotator confirms the emotional dimensions of empathy, intensity, and polarity at the end of the session @xcite @xcite . the wassa 2023 shared task @xcite in this paper, we present our submission for track 1. the provided dataset contains conversations between two participants, along with scores for empathy, emotion polarity, and emotion intensity. the task revolves around two participants who read a news article and initiate a conversation based on that particular news. they are then required to assess each other's empathy, intensity of emotion, and polarity of emotion based on their discussion. the objective of the task is to develop a system capable of automatically assessing empathy, intensity of emotion, and polarity of emotion within participants' conversations. our model consists of two roberta-based encoders to encode article and conversation followed by a 3-layered transformer encoder. the representation is then passed to a sequence of layers which provides regression output of three tasks (cf. track 1) -empathy, emotion polarity, and emotion intensity with a multi-tasking framework (mlt). in track 1 participation, our best model achieved average pearson's correlation of 0.771 on the released development set and 0.725 on the released test set. the key findings of this research can be summarized as follows: (i) residual skip connections are effective in enhancing the conversation encoder, (ii) including the previous dialogues of both participants in a conversation along with the current dialogue helps in preserving the context of the conversation within the dialogue encoder for a particular session, and (iii) token interactions between articles and conversations utilizing multi-head selfattention yield significant and informative results. the rest of this paper is organized as follows. we introduce related work in §2. we discuss problem statement in §3. the proposed model is described in §4 and experiment and result in § 5. finally, we conclude our paper in §6. ## related work recently, transformer based @xcite pre-trained models such as bert @xcite , openai gpt @xcite , roberta @xcite etc. have been shown superior performance in various downstream tasks, including text classification task @xcite @xcite , generation task such as question answering @xcite ) and many more. recent works have shown that using such pre-trained methods can achieve state-of-the-art performance. towards that end, @xcite investigated a multitask roberta-based bi-encoder paradigm for comprehending empathy in text-based health support, zhou and jurgens (2020) investigated the link between distress, condolence, and empathy in online support groups using nested regression models. many research @xcite have given various strategies for emotion recognition. the effectiveness of using transformer encoders for emotion detection was investigated by @xcite . @xcite proposed a multi-task deep learning methods to address empathy detection, emotion classification and personality detection. inspired from @xcite and multi-encoder based architectures @xcite @xcite , we propose a multi-encoder based architecture followed by mlp and a linear layer output layer. our core architecture is similar to @xcite , the difference is we use two roberta @xcite encoders for inputs. ## problem statement our model is based on a multi-task learning based framework (mtl) to force the model to consider three different objectives i.e. three emotional dimensions of empathy (y 1 ), intensity (y 2 ), and polarity (y 3 ). given a set of conversation for a single session c for a corresponding article a c , the output probability @xmath0, y 2 , y 3 in the model setting is calculated as in equation 1 . the model acts as a regressor, the output head provides regression results. the network consists of a two-layered multi-layered perceptron foot_0 (mlp) with relu activation between layers and output head with a 'linear' activation. given the output of the regression head, the loss can be calculated as: our model is trained for three emotional dimensions of empathy (y 1 ), intensity (y 2 ), and polarity (y 3 ) with corresponding losses l 1 , l 2 , and l 3 respectively, in an end-to-end fashion that jointly optimizes the loss as in eq. 3, where α, β, and γ are learnable parameters. ## experiments and results in this section, we provide a summary of how our systems were trained, tuned, and combined to create the team hawk submissions for track 1 of the wassa 2023 shared task. we evaluated our system using pearson's correlation, which measures the relationship between the regression outputs of our model and the gold standards. all benchmark evaluation scores are reported based on the development set released by the organizers. ## conclusion and future work in this paper, we introduced our methodologies for investigating the emergence of empathy during conversations. the task is introduced as part of the wassa 2023 shared task track 1, which focuses on empathy and emotion prediction in conversations (conv). we developed a multi-tasking framework that leverages a core multi-encoder based architecture. this framework enables automatic assessment of empathy, intensity of emotion, and polarity of emotion in participants' conversations. our systems achieved average pearson's scores of 0.7710 on the released development set and 0.7250 on the released test set. our submission ranked 3 rd in the shared task. due to time constraints, we are unable to conduct an exhaustive number of experiments with various architecture variations before reaching a final conclusion. however, for future research, we will conduct a comprehensive analysis involving architecture variations and data preprocessing methods. additionally, we plan to investigate the influence of other features such as gender, age, etc., on the model's decision-making process.