A Critical Reflection and Forward Perspective on Empathy and Natural Language Processing (2022)

Abstract: We review the state of research on empathy in natural language processing and identify the following issues: (1) empathy definitions are absent or abstract, which (2) leads to low construct validity and reproducibility. Moreover, (3) emotional empathy is overemphasized, skewing our focus to a narrow subset of simplified tasks. We believe these issues hinder research progress and argue that current directions will benefit from a clear conceptualization that includes operationalizing cognitive empathy components. Our main objectives are to provide insight and guidance on empathy conceptualization for NLP research objectives and to encourage researchers to pursue the overlooked opportunities in this area, highly relevant, e.g., for clinical and educational sectors.

## introduction interest in empathetic language continues to grow in natural language processing research. empathy recognition and empathetic response generation tasks have become well-established research directions, especially since the introduction of now fairly mainstream benchmark datasets for each task, empathic reactions @xcite and empathetic dialogues @xcite , and an empathy detection shared task established on the former @xcite . these empathy-focused tasks are highly motivated by myriad benefits, including (i) improved experiences with conversational agents @xcite and satisfaction with customer care dialogue agents @xcite , (ii) computational social science analyses, e.g., supportive interactions in online forums @xcite @xcite , and (iii) tools to assist in training and evaluating health practitioners @xcite . with this increased interest in empathy-focused nlp, however, has come little clarity on what empathy is and how it is being operationalized. papers vary substantially in how they define, annotate, and evaluate empathy, leading to a fractured landscape that hinders progress. most nlp work has loosely defined empathy as the ability to understand another person's feelings and respond appropriately. as a result, these works have focused primarily on detecting sentiment and emotions in text as proxies for understanding feelings and consider empathetic responses to be those that demonstrate success in emotion recognition and express so with a tone consistent with the valence of the target's sentiment. under this view, systems that primarily recognize or perform emotionally coherent interactions supposedly fulfill the goal of having an empathetic system. however, established findings and theories about human empathy show that this objective is short-sightedor even misdirected-and misses a critical system known as cognitive empathy. we argue that nlp research has omitted key aspects of empathy through its narrow focus on emotion and, as a result, led us to neglect the cognitive components. our paper offers three key contributions. first, to show this gap in research on empathy, we provide a theoretical grounding for empathy from psychology ( §2). we then survey empathy literature in nlp focusing on acl * venues ( §3) and highlight three central problems: (1) computational work has overlooked much of empathy through vague definitions and a focus on emotion ( §4), (2) our current underspecification of empathy leads to issues in construct and data validity ( §5), and (3) the narrow empathy tasks we choose as a community limit our progress ( §6). finally, we propose a way forward ( §7) through a clear conceptualization of empathy that operationally considers processes of cognitive empathy and highlight overlooked research opportunities ( §8). ## what is empathy? a theoretical guide empathy is a multi-dimensional construct that contains both emotional and cognitive aspects which relate to how an observer reacts to a target @xcite . in practice, empathy is diversely defined in terms of these social, emotional, cognitive, and even neurological dimensions @xcite . indeed, while folk conceptions of empathy refer to a single construct, multiple studies have pointed to distinct neurological systems for emotional and cognitive empathy @xcite . in describing empathy in communication, we adopt the standard terminology of (i) a target as someone experiencing an emotion or situation and (ii) an observer as another person at the disposition for an empathic experience through perceiving the target's emotions and situation. in psychology, the most discussed aspects of empathy are its affective and cognitive components @xcite . the affective components, referred to as emotional empathy, relate to the observer's emotional reaction to the target @xcite . the cognitive components, referred to as cognitive empathy, relate to active processes used by the observer to infer the mental state of the target @xcite . emotional empathy represents automatic (bottom-up) processes whereas cognitive empathy represents controlled (top-down) processes @xcite , and they interact with each other. ## literature review: empathy in nlp client well, i know i need to stay active. "use it or lose it," they say. i want to get my strength back, and they say regular exercise is good for your brain, too. interviewer so that's a puzzle for you -how to be active enough to get your strength back and be healthy, but not so much that would put you in danger of another heart attack. client i think i'm probably being too careful. my last test results were good. it just scares me when i feel pain like that. interviewer it reminds you of your heart attack. client that doesn't make much sense, does itstaying away from activity so i won't have another one? interviewer like staying away from people so you won't be lonely. client right. i guess i just need to do it, figure out how to gradually do more so i can stick around for a while. we summarize our findings from reviewing a collection of computational linguistics and natural language processing papers. papers were identified by searching for the term "empath" in the acl anthology. then we narrowed the resulting sample from 90 to 69 papers whose investigation involves empathy, as opposed to mentioning "empathy", "empathetic", or "empathic" rhetorically. we also included a selection of relevant works outside the acl anthology from venues such as aaai. we focused on papers presenting systems for empathy prediction or empathetic response generation, evaluations of empathy dialogue models, empathy annotation schemes and datasets, and corpus analyses of empathetic language. we labeled a total of 48 papers (excluding 14 wassa foot_0 shared task papers) based on their descriptions of empathy and empathy evaluation approaches. ## the measurement problem given the overwhelmingly abstract portrayal of empathy, the effectiveness and validity of our approaches for measuring, annotating, and evaluating it are questionable. simply put, we do not know if we are investigating the same thing or doing so consistently. following, we outline where nlp needs to improve in its measurements to move forward. ## the narrow task problem while empathy is widely recognized as a construct nlp should be modeling, we argue that the field has focused on a narrow set of tasks that have held back progress. by bridging current work with a perspective of cognitive empathy, we motivate a series of new and reimagined tasks that would advance the field's ability to model empathy. empathy is more than emotions. the predominant empathy conceptualization requires the observer to understand the target's emotions and respond appropriately. we recommended that researchers disambiguate sympathy, compassion, and tenderness and view these as subareas of empathy research. however, we argue that this research would still suffer by ignoring cognitive empathy and its interaction with emotion understanding and empathetic responses. some perspectives from psychology emphasize that the distinction between emotional and cognitive empathy is less important than the interaction between the two components (cuff et al., 2016)-and we can draw inspiration from that. cognitive empathy processes are necessary considerations that can more effectively achieve goals for the emotional empathy subareas. cognitive empathy processes can improve methods for understanding emotions. in nlp, we often consider emotions first, assuming they can be inferred directly from what the target expresses. however, the target could minimize expressions, making the emotions harder to perceive (refer to the discussion on empathic accuracy §2.2). furthermore, most papers only operate on text (e.g., transcripts), which misses opportunities for empathetic cues in other modalities. cognitive processes help with better emotional understanding yielding higher empathic accuracy of inferences about the target's affective state. @xcite argue that the observer can infer emotionality through perspective-taking, imagination, and retrieval of relevant memories. in this way, a cognitive approach first leads to an understanding and more accurate perception of emotions. we argue that cognitive empathy can benefit from approaches such as common sense reasoning @xcite , external knowledge @xcite , and abductive nli @xcite . for example, @xcite integrated external knowledge to improve counselor reflection generation. @xcite presented an approach to understanding the target's emotional state using commonsense knowledge that improves over more emotion-focused models. their approach was inspired by @xcite 's survey of empathetic dialogue systems, which argued that future work should go beyond emotional empathy by pursuing personalization and knowledge (both contextual and general external knowledge. integrating these types of knowledge supports reasoning about emotion cause, as opposed to only recognizing emotions, which itself is insufficient @xcite . models that incorporate reasoning about the cause of emotion were shown to outperform emotion recognition and mirroring counterparts, notably including those that included external commonsense and emotional lexical knowledge, by human evaluated empathy scores, i.e. @xcite @xcite . studies may even be able to specify and investigate what type of information is necessary in order to understand the user's emotions and model empathically accurate responses. this can be done with a controlled selection of samples from knowledge bases. for instance, @xcite 's work uses particular aspects of common sense and domain-specific knowledge. researchers must become more intimate with data and understand the nature of dialogue to design empathetic systems. cognitive empathic processes can improve methods to select appropriate response strategies. approaches inspired by cognitive processes could thus result in not only valuable representations of emotion but also enhanced methods for response strategies. though we critiqued the generally-lacking specificity about response appropriateness earlier, the most salient idea we grasp is that observers should mirror the target's emotion or express a similar sentiment valence. contrary to this idea, @xcite find that empathetic listeners often respond to negative emotions such as sadness and anger with questions rather than expressing similar or opposite emotions. in addition, specific empathetic question intents of observers may play more significant roles in regulating the tar-get's emotions @xcite . approaches designed both on emotional and cognitive schemes also had effective results in assisting students write peer-reviews that are perceived more empathetic @xcite . ## refocusing our efforts we overlook research opportunities that better align with described motivations, and these problems necessitate cognitive empathy. most empathy research in nlp focuses on emotional chatbots or social media analysis. this focus probably drives the overemphasis on emotional empathy. neither of the domains, we argue, is helpful for the purposes where empathetic nlp systems would be needed the most; for clinical or educational praxis or other rather formal contexts. such scarce applications include research on motivational interviewing @xcite @xcite @xcite , assistive writing systems for peer-to-peer mental health support @xcite , and other counseling conversations @xcite @xcite . work in these areas make the need for cognitive empathy even more apparent. empathic capacities vary across individuals, as some experience more intense affective responses to certain target perceptions than others @xcite . without techniques to manage affective responses, people can experience significant distress. cognitive empathy skills strengthen observers' ability to regulate or manage their affective responses. thus, these skills are critical for individuals in caregiving roles (e.g., counselors and doctors) who interpersonally engage with others who are vulnerable, suffering, or in crisis daily. at the same time, the ability to empathize with the target is essential for their roles in providing effective treatment, making the role of cognitive empathy significant both for self-care and care for others. nlp research is highly needed for virtual standardized patient (vsp) systems. vsps provide an effective way for learners to develop cognitive empathy and clinical skills @xcite . the need for vsps has grown from the necessity for methods that allow learners to practice rare and realistic crisis scenarios and concern for the safety and expenses of standardized patients. needs in this area include but are not limited to conversational models for practice in intercultural commu-nication, end-of-life discussions, and breaking bad news. development of such systems naturally requires the skills of nlp researchers, and these research efforts would benefit the field. so far, the task of modeling and simulating the empathy target has received little attention in nlp, leaving a significant research gap that our community is positioned to fill, with exciting challenges that align with the motivations of research on empathy and that can have a broad impact within and beyond our field. one work, for instance, focused on simulating individuals in crisis for the use case of counselor training @xcite . similarly, modeling and simulating the target could support broader educational objectives, such as training students' emotionally and cognitively empathic feedback skills @xcite . this direction of simulating a mental health support seeker aligns with the goals of related work for developing tools to assist with evaluating and training counselors @xcite @xcite . the ethics section further discusses our perspective on this approach compared to developing empathetic response generation models for support-related motivations. ## a forward perspective here we summarize the main obstacles we identified and our recommendations for moving forward. the definition problem: define research goals and design methodologies based on empathy's more concrete and measurable aspects. the abstractness of empathy has negative implications for the construct validity of measurement approaches and, thereby, the scientific effectiveness and comparability. the measurement problem: draw inspiration from measurements established in psychology, and in addition, learn from investigations and critiques on their construct validity. as well as referencing psychology literature to aid the developments of measurement techniques (yalçın, 2019), we should be familiar with existing evaluations and discussions on their construct validity. future work is needed to investigate the validity of our current approaches methodologically. the narrow task problem: lessons from our own field. the interaction between cognitive and emotional processes is significant. cognitive processes can enable higher empathic accuracy. perspective-taking is a method of cognitive empathy that enables better empathic understanding, and the appraisal theory of empathy provides a framework specific to situational aspects an observer can consider. these processes relate clearly to reasoning processes explored by nlp tasks. we recommend future work that intersects with the newer area of abductive commonsense reasoning @xcite . refocusing our efforts: supporting clinical and educational domains aligns better with the motivations of the empathy research area. much of what motivates the empathy research area is a desire to support those in need (i.e., compassionate motivations, to employ the new terminology). our position is that current efforts in empathetic dialogue generation should be reallocated to support the needs of clinical domains, e.g., systems that support communicative skill training for people in care-providing roles and systems for supportive cognitive empathic skills development in-general. 3 the need for cognitive empathy is even more pressing for these applications, and therefore nlp research can best support these needs by emphasizing the cognitive aspects in our empathy conceptualizations going forward. ## conclusion language technology is deployed increasingly in interpersonal settings that require an empathetic understanding of the speaker. our field's construct represents a significant obstacle to advancing our ability to develop empathetic language technologies. while multiple nlp works have attempted to incorporate empathy into their models and applications, we argue that these works have underspecified empathy-focusing primarily on emotionand overlooked a major component: cognitive empathy. however, we argue that this gap represents a significant opportunity for developing new models that better reflect cognitive processes and theory of mind while supporting much-needed humancentered applications, such as those in clinical settings.