E mpath BERT : A BERT -based Framework for Demographic-aware Empathy Prediction (2021)

Abstract: Affect preferences vary with user demographics, and tapping into demographic information provides important cues about the users’ language preferences. In this paper, we utilize the user demographics and propose EmpathBERT, a demographic-aware framework for empathy prediction based on BERT. Through several comparative experiments, we show that EmpathBERT surpasses traditional machine learning and deep learning models, and illustrate the importance of user demographics, for predicting empathy and distress in user responses to stimulative news articles. We also highlight the importance of affect information in the responses by developing affect-aware models to predict user demographic attributes.

## introduction modeling complex human reactions and affect from text has been a challenging research area with innovations focusing on sentiment and emotion understanding @xcite @xcite @xcite . the study of non-trivial human reactions has been limited. these methods, often rooted in psychological theories, have turned out to be more complex in terms of annotation and modeling @xcite . a critical affective phenomena, empathy, has received surprisingly less attention. empathy assesses feelings of sympathy towards others, and distress measures anxiety and discomfort oriented towards self @xcite . empathy has been positively associated to a number of wellbeing activities, such as volunteering @xcite , charity @xcite , and longevity @xcite , and in consumer marketing, advertising and customer interfaces @xcite . works on empathy in text have focused on spoken dialogue, addressing conversational agents, psychological interventions, or call center transcripts @xcite @xcite @xcite . @xcite collected an empathy-distress dataset by leveraging users' reactions to textual stimulus content. @xcite constructed an empathy lexicon by obtaining word ratings from documentlevel ratings from this dataset. @xcite ; @xcite ; @xcite presented predictive models for empathy in the healthcare domain. however, we believe none of the above works focus on (a) predicting empathy from textual reactions, and (b) studying the impact of demographics on the expression of empathy. language preferences vary with user demographics @xcite @xcite @xcite , and this has led to studies leveraging the user demographic information to obtain better language representations and classification models for various nlp tasks @xcite @xcite . owing to the recent success of large language models such as bert @xcite and gpt-2 @xcite in improving the performances of several downstream tasks, we propose a bert-based demographic-aware framework for empathy (distress) prediction, and through several comparative experiments, show that it surpasses existing baselines and demographic-agnostic approaches. this paper makes three main contributions. (1) we present empathbert, a demographic-aware empathy (distress) prediction framework, using bert-based models infused with demographic information. (2) through comparisons against several baseline and demographic-agnostic approaches, we illustrate the importance of user demographics in end-to-end modeling and predicting empathy (distress). (3) conversely, we show that empathy (distress) also contributes to demographic attribute prediction, by developing affectaware models for demographic attribute prediction, backed by empirical comparison with baselines and generic models. to the best of our knowledge, ours is the first computational effort addressing empathy (distress) through the lens of demographic biases, a phenomenon well-understood in psychology. ## dataset we use the empathy-distress dataset introduced by @xcite . it consists of 418 news articles from popular news platforms, and responses to them from 403 annotators (5 articles each), resulting in a total of 2,015 responses. filtering the responses that deviated from the task description led to 1,860 responses (empathy: 916, distress: 905) , with a total token count of 173,686 (min: 52, max: 198, median: 84). the number of responses per article ranges from 1 to 7, with an average of 4.46 responses per article. we report some example responses from the dataset in ## empathbert in this section, we describe our approach for demographic-aware empathy (distress) prediction from text. figure 1 shows the proposed architecture. our model takes as input a response (a sequence of words w 1 , w 2 , . . . , w n ) and demographic information of the corresponding annotator. we represent the response using bert, a bidirectional transformer-based @xcite language model. we use the final 768-dimensional hidden vector corresponding to the [cls] token as the aggregate sequence representation. we employ cross-domain pre-training @xcite , finetuning, and multi-task fine-tuning @xcite techniques to customize bert for our tasks. ## conclusion we proposed a novel demographic-aware empathy prediction framework based on fine-tuning and multi-tasking using bert, showed that it surpasses existing methods, and illustrated the impact of demography in modeling subjective phenomena such as empathy and distress. our framework is generalizable, and we extended it to empathy-aware demography prediction, and showed that empathy also improves demographic prediction. we believe this is a significant checkpoint towards developing models for empathy (distress), and tapping into demographic information while doing so.