Domain Transfer for Empathy, Distress, and Personality Prediction (2023)

Abstract: This research contributes to the task of predicting empathy and personality traits within dialogue, an important aspect of natural language processing, as part of our experimental work for the WASSA 2023 Empathy and Emotion Shared Task. For predicting empathy, emotion polarity, and emotion intensity on turns within a dialogue, we employ adapters trained on social media interactions labeled with empathy ratings in a stacked composition with the target task adapters. Furthermore, we embed demographic information to predict Interpersonal Reactivity Index (IRI) subscales and Big Five Personality Traits utilizing BERT-based models. The results from our study provide valuable insights, contributing to advancements in understanding human behavior and interaction through text. Our team ranked 2nd on the personality and empathy prediction tasks, 4th on the interpersonal reactivity index, and 6th on the conversational task.

## introduction empathy, a fundamental component of interpersonal communication, emerges in broad spectrum of conversational and discourse settings, ranging from informal dialogues to purpose-driven conversations. with an invaluable role for cooperative interactions, modeling empathetic language is a growing area in natural language processing (nlp) research, enabling the improvement of dialogue agent experiences, analysis of online supportive interactions, and the development of educational tools @xcite @xcite . despite its significance, predicting empathy poses an immense challenge due to the scarcity of resources and the complexities involved in establishing a gold standard for this nuanced phenomenon @xcite . a major hindrance to empathy prediction in the nlp field is the lack of accessible, high-quality datasets. many studies are conducted on sensitive data, which cannot be disclosed publicly @xcite . though some publicly available datasets exist, built on social media platforms or through specific data collection tasks, they are sparse and each comes with its inherent limitations due to the challenges in the data collection and annotation process @xcite . defining empathy in a concrete, measurable way for consistent and relevant gold standard annotations is another formidable challenge. empathy definitions vary greatly across psychological research, and nlp datasets are often annotated by third parties, rather than the individuals experiencing or receiving empathy. this approach captures specific language aspects identified by an external observer but fails to provide insight into how particular empathetic experiences influence language @xcite . in parallel to empathy, personality traits are fundamental to interpersonal relationships and social interactions. the big five personality model (per) @xcite , a widely studied framework, is instrumental in understanding human social behavior. a comprehensive understanding of human behavior can be achieved by combining empathy, gauged by the interpersonal reactivity index (iri) @xcite , and personality traits. this paper presents our experimental work on the wassa 2023 empathy and emotion shared task @xcite . we predict perceived empathy, emotion polarity, and emotion intensity at the turn-level in a conversation (the conv task), and empathy and distress at the essay level (the emp task). we explore the use of adapters, which provide greater efficiency compared to complete fine-tuning, and an adjusted approach derived by @xcite at the essay level. additionally, we embed demographic information to predict iri subscales (perspective taking, personal distress, fantasy, and empathic concern) and the big five personality traits (conscientiousness, openness, extraversion, agreeableness, and stability) using bert-based models @xcite . the paper is structured as follows: section 2 reviews the task and dataset, provides an overview of the iri, per constructs, and empathy in conversations, emphasizing their significance in understanding human behavior. section 3 describes our implementation. section 4 presents the results and discussion of our findings. finally, section 5 concludes the paper and proposes directions for future research. in this section, we describe the dataset and tasks employed in our research aimed at predicting empathy, distress, personality traits, and iri subscales using the dataset provided by @xcite . the dataset utilized in our experiments comprises empathetic reactions captured in essays and conversation responses to news articles involving harm to a person, group, or other entities. these reactions are documented in essays that range between 300 and 800 characters in length, as well as in conversations that consist of an average of 23 speech turns. the dataset also includes the original news articles and demographic information at the person-level, such as age, gender, ethnicity, income, and education level. each essay in the dataset is supplemented with @xcite 's empathic concern and personal distress scores, providing an insight into the individual's empathetic response after engaging with the news article. in addition, the dataset provides @xcite 's big five personality scores and inter-personal reactivity index (iri) scores for the respective user, further enhancing our understanding of their empathetic capacity. the big five of @xcite was shown to predict many traits about people, their behavior, and relationships. each dimension can be rated on a continuous scale, where a person has more or a lesser degree of the qualities associated with that dimension. the following facets are from john et al. ( 1999): 1. neuroticism -anxiety, angry hostility, depression, self-consciousness, impulsiveness, vulnerability 2. extraversion -warmth, gregariousness, assertiveness, activity, excitement seeking, posi-tive emotions 3. openness -fantasy, aesthetics, feelings, actions, ideas, values 4. agreeableness -trust, straightforwardness, altruism, compliance, modesty, tendermindedness 5. conscientiousness competence, order, dutifulness, achievement striving, self-discipline, deliberation the iri index is discussed in davis (1983), who constructed a 28-item survey to measure four aspects of empathy using a 5-point likert scale. the items (directly taken from the paper) are as follows: in the case of conversations, each speech turn has been annotated by a third person for perceived empathy, emotion polarity, and emotion intensity. this offers a comprehensive view of the interaction, enabling a detailed examination of the empathetic exchanges within the conversation. the tasks of our research involve predicting empathy and emotion polarity & intensity on conversational turns (conv), empathy and distress scores (emp), personality traits (per), and iri subscale values (iri). systems are evaluated based on the pearson's r correlation between the predicted and actual values in a test set, similar to the approach adopted in the previous edition of the shared task @xcite . ## results and discussion our submissions to the post-evaluation phase on the test dataset have yielded promising results, as showcased in in the domain of personality prediction, as dis-played in the top portion of turning to the interpersonal reactivity index (iri) prediction, as seen in the middle of at the essay level, our approaches have shown encouraging results for empathy and distress prediction, as evidenced by the second to last portion of lastly, as we move to the conversation level prediction in bottom of ## conclusion in this paper, we have presented our methodologies and findings from predicting a range of empathyrelated features in text, specifically in essays and conversation responses from the @xcite dataset. we developed and evaluated a series of models, each addressing unique aspects of the prediction tasks. at the essay level, we employed a domainadapted model based on the work of @xcite , modified to perform regression instead of classification, effectively predicting empathy and distress scores. our results across different measures are encouraging. the demographic-embedding approach performed quite well in predicting the conscientiousness, openness, and agreeableness aspects of the big five personality traits. in contrast, the performance on the interpersonal reactivity indices was less impressive. the domain-adapted model excelled in predicting empathy and distress at the essay level. at the conversation level, our adapter approach achieved satisfactory results in predicting emotional polarity, emotional intensity, and empathy.