Enriching Deep Learning with Frame Semantics for Empathy Classification in Medical Narrative Essays (2022)

Abstract: Empathy is a vital component of health care and plays a key role in the training of future doctors. Paying attention to medical students’ self-reflective stories of their interactions with patients can encourage empathy and the formation of professional identities that embody desirable values such as integrity and respect. We present a computational approach and linguistic analysis of empathic language in a large corpus of 440 essays written by pre-med students as narrated simulated patient – doctor interactions. We analyze the discourse of three kinds of empathy: cognitive, affective, and prosocial as highlighted by expert annotators. We also present various experiments with state-of-the-art recurrent neural networks and transformer models for classifying these forms of empathy. To further improve over these results, we develop a novel system architecture that makes use of frame semantics to enrich our state-of-the-art models. We show that this novel framework leads to significant improvement on the empathy classification task for this dataset.

## introduction empathy is a complex phenomenon concerning how we seek to understand and experience, to some extent, the experiences of others @xcite -i.e., having a sense of the other's story and the context in which it takes place. one way to get to an appreciation of one's complex situation (i.e., the embodied actions and the contexts within which they act) is through narratives of lived experience @xcite . self-reflective (i.e., first person) narratives, for instance, offer a wide range of resources for empathy, as they bring together one's inner and outer worlds, thus giving meaning to experience @xcite . in this respect, narratives seem necessary for empathy, as our first-person experience is grounded in the contextualized content of the narrative. they also provide a form or structure that allows us to frame an understanding of others, together with a learned set of skills and practical knowledge that shapes our understanding of what we and others are experiencing. reflective writing is a dynamic process that allows for an active engagement with knowledge and experience, being widely used in clinical practice @xcite @xcite . putting into words the focused inspection of their thoughts, feelings, and events enables one to reprocess the experience, build new insights, and new ways to conceive reality @xcite . thus, narrative exercises like self-reflective stories can help medical students recognise and derive meaning from key experiences, which in turn can support critical thinking, self-consciousness, and the development of personal skills, communication and empathy skills, self-knowledge, professional identify development, and instill behavior change @xcite @xcite @xcite @xcite . such writing can lead to an increase in experience-taking skills @xcite and can decrease stereotyping, prejudice, and racial bias in healthcare @xcite . in this research, we take a narrative approach to empathy and explore the experiences of premed students at a large university by analysing their self-reflective writing portfolios (a large corpus of first-person essays written by premed students in narrated simulated patient-doctor interactions). specifically, we introduce an exploratory study of empathy in clinical encounters paying attention to the discourse of three types of empathy: cognitive (the drive and ability to identify and understand another's emotional or mental states), affective (the capacity to experience an appropriate emotion in response to another's emotional or mental state), and prosocial behavior (a response to having identified the perspective of another with the intention of acting upon the other's mental and/or emotional state), following established practices in psychology @xcite @xcite . we introduce a set of informative baseline experiments using state-of-the-art recurrent neural networks and transformer models for classifying the various forms of empathy. as initial experiments show relatively low scores, we explore a novel framenet-based system architecture where we use sentence frames to extract additional semantic features. we apply this framework to state-ofthe-art and representative neural network models and show significant improvement in the empathy classification task for this dataset. although previous research suggests that narrative-based interventions tend to be effective education-based methods, it is less clear what are some of the mechanisms through which narratives achieve such an effect, which is another contribution of this research. ## related work in spite of its increasing theoretical and practical interest, empathy research in computational linguistics has been relatively sparse and lacks cohesion. even more so, investigations of empathy as it relates to clinical practice have received even less attention mainly due to data and privacy concerns. most of the research on empathy detection has focused on conversations or interactions, as dialogue systems @xcite @xcite , or in online platforms (e.g. @xcite @xcite @xcite @xcite , a few on news stories and other narratives @xcite @xcite @xcite , and even less on empathy in clinical settings @xcite . @xcite used crowd-sourced workers to self-report their empathy and distress levels and to write empathic reactions to news stories. @xcite built a text corpus of student peer reviews collected from a german business innovation class annotated for cognitive and affective empathy levels. furthermore, using batson's empathic concern-personal distress scale @xcite , @xcite have focused only on negative empathy instances (i.e., pain and sadness "by witnessing another person's suffering"). this year, the wassa shared task focused on predicting empathy, emotion, and personality in reaction to news stories @xcite . the dataset is an extension of @xcite 's dataset -i.e., it includes news articles that express harm to an entity (e.g. individual, group of people, nature). each article comes with reaction essays in which authors expressed their empathy and distress toward these news articles. each essay is annotated for empathy and distress, and with authors' personality traits and demographic information (age, gender, ethnicity, income, and education level). here, we could not compare our models with the wassa results -our dataset does not capture the meta-data in wassa. moreover, our empathy instances are not always negative @xcite : a dataset reflecting empathetic language should ideally allow for expressions of empathy that encompass a variety of positive and negative emotions. we could not compare against its best performing system due to limited reproducibility @xcite . in multimodal research, r. m. frankel (2000) and @xcite identify sequential patterns of empathy frequently expressed in video-recorded exchanges by medical graduates interacting with a cancer patient. @xcite analyzed the discourse of conversations in online peer-to-peer support platforms. they successfully trained novice writers to improve low-empathy responses by giving the writers feedback with examples of sentences that are typical of recognition and interpretation of others' feelings or experiences. in a subsequent set of experiments @xcite , they suggested that empathic written discourse should be coherent, specific to the conversation at hand, and lexically diverse. to our knowledge, no self-reflective narrative text corpora have been developed for computational linguistics investigations of clinical student training. adding to the scarcity of empathy-dedicated resources, there is also a lack of understanding of which linguistic features might contribute to the various types of empathy, like cognitive, affective, and prosocial behavior. ## self-reflective narrative essays in medical training in this research, we focus on self-reflective narratives written by premed students given a simulated scenario. simulation is strongly set on our first-person experiences, relying on resources that are available to the simulator. in a simulation process, the writer puts themselves in the other's situation and asks what "i would do if i were in that situation." perspective taking (i.e., cognitive empathy) is crucial for fostering affective abilities, enabling writers to imagine and learn about the emotions of others and to share them, too. as empathy is other-directed @xcite , this means that we, as narrators, are open to the experience and the life of the other, in their context, as we can understand it. this study's intervention was designed as a written assignment in which premed students were asked to consider a hypothetical scenario where they took the role of a physician breaking the news of an unfavorable diagnosis of high blood cholesterol to a middle-aged patient 1 . they were instructed to recount (in first person voice) the hypothetical doctor-patient interaction where they explained the diagnosis and prescribed medical treatment using layman terms and language they believed would comfort as well as persuade the hypothetical patient to adhere to their prescription. with the students' consent, we collected a corpus of 774 essays over a period of one academic year @xcite . following a thorough annotation process, annotators (undergraduate and graduate students in psychology and social work) 2 labeled a subset of 440 randomly selected essays (henceforth, "the corpus"). using a rich color code schema, each sentence in every essay was labeled as either cognitive empathy (green; e.g., "she looked tired"), affective empathy (yellow; e.g.: "i felt the pain"), or prosocial behavior (cyan; e.g.: "i reassured her this was the best way") (everything else was "no empathy") @xcite @xcite . the six paid undergraduate students were trained on the task and instructed to annotate the data. two meta-annotators, paid graduate students with prior experience with the task, reviewed the work of the annotators and updated the annotation guidelines at regular intervals, in an iterative loop process after each batch of essays 3 . the meta-annotators reached a cohen's kappa of 0.82, a good level of agreement. disagreed cases were discussed and mitigated. at the end, all the essays were re-annotated per the most up-to-date 1 the patient was referred to as betty or john. 2 the students were hired based on previous experience with similar projects in social work and psychology. ## empathy classification task in this research, our goal is to explore machine learning models of empathy classification in narrative essays to better our understanding of the mechanisms through which empathy can be expressed. since we are interested in the linguistic expressions of empathy, we zoom in to the sentence level. given such a corpus of essay sentences, we first build a binary classifier which can be useful in applications requiring a general linguistic understanding of the presence of empathy. in some cases such as medical communication training of pre-med students, a more fine-grained understanding of different kinds of empathy is useful. thus, we also build a classifier that can identify each type of empathy: cognitive, affective, and prosocial. for both types of classifiers, we first experiment with several state-of-the-art statistical and machine learning models. as our research is focused on the subcategorization of empathy, we seek to improve our multi-label classifier. thus, we introduce a new and better performing system architecture by employing framenet @xcite , the research and development project which builds on the theory of frame semantics. using a state-ofthe-art framenet sentence parser @xcite , we extract semantic frames from each sentence in our corpus and use this resource to enhance our original (baseline) models with these additional knowledge. as we will show in subsection 4.4, incorporating framenet semantics into state-of-the-art deep learning models leads to an increase in empathy classification results. ## discussion and conclusions medical education should and can incorporate guided self-reflective practices that show how important it is for the students to develop an awareness of the emotional and relational aspects of the clinical encounter with their patients @xcite . the way people identify themselves and perform in particular roles and in relations to others brings together a specific set of values, attitudes, and competencies that can be supported through ongoing self-reflection. thus, students learn not only how to diagnose and treat patients' medical conditions, but also how to witness the patient's illness experience. in practice, they often switch between these positions: witnessing what it is like for the patient, as well as understanding what they need medically. often, clinical encounters can be highly charged emotionally especially for patients in case of serious illness. unfortunately, medicine lags behind other health professions (like nursing, social work, psychology) which learn from reflective practice and respect it from the beginning. yet, acknowledging the patient's situation, who they are and their experience can make a huge impact on the quality computational approaches to empathy can be very valuable, but it is clear that such ai initiatives must be multidisciplinary, using and developing a variety of core sets of requirements and expertise and engaging many participants, e.g. ai designers, developers, frontline clinical teams, ethicists, humanists, patients, caregivers @xcite . the research experiments and findings summarized in this paper are part of a larger interdisciplinary and highly collaborative project where we analyze both self-reflective narratives of simulated interactions, as well as multimodal patient-doctor encounters in real clinical settings @xcite . in this paper, we presented a computational approach and linguistic analysis of empathic language in a large corpus of premed student essays of narrated simulated patient-doctor interactions. specifically, we showed that semantic information at the sentence level can be very useful not only in empathy identification but provides details on the differences among the three main types of empathy: cognitive, affective, and prosocial. we presented novel and performant framenet-based transformer models for empathy classification. in future work, we will expand this analysis by considering discourse-level context. we will also integrate other resources like wordnet @xcite , verbnet @xcite , and take advantage of larger discourse.