HEART -felt Narratives: Tracing Empathy and Narrative Style in Personal Stories with LLM s (2024)

Abstract: Empathy serves as a cornerstone in enabling prosocial behaviors, and can be evoked through sharing of personal experiences in stories. While empathy is influenced by narrative content, intuitively, people respond to the way a story is told as well, through narrative style. Yet the relationship between empathy and narrative style is not fully understood. In this work, we empirically examine and quantify this relationship between style and empathy using LLMs and large-scale crowdsourcing studies. We introduce a novel, theory-based taxonomy, HEART (Human Empathy and Narrative Taxonomy) that delineates elements of narrative style that can lead to empathy with the narrator of a story. We establish the performance of LLMs in extracting narrative elements from HEART, showing that prompting with our taxonomy leads to reasonable, human-level annotations beyond what prior lexicon-based methods can do. To show empirical use of our taxonomy, we collect a dataset of empathy judgments of stories via a large-scale crowdsourcing study with N=2,624 participants. We show that narrative elements extracted via LLMs, in particular, vividness of emotions and plot volume, can elucidate the pathways by which narrative style cultivates empathy towards personal stories. Our work suggests that such models can be used for narrative analyses that lead to human-centered social and behavioral insights.

## introduction empathy, which is a foundational psychological process that drives many prosocial functions, @xcite , is often delivered through storytelling and sharing of personal experiences @xcite . empathetic responses evoked by stories are affected by factors figure 1 : narrative empathy can be evoked through the way a story is told (narrative style). this work introduces heart, a theory-driven taxonomy of narrative elements that contribute to empathy. in this work, we fill this gap by presenting the following contributions. (1) we introduce heart (human empathy and narrative taxonomy), a theory-driven taxonomy of narrative style elements that relate to empathy. (2) we use llms to quantify aspects of narrativity in our taxonomy and evaluate how well llms represent these elements in line with human judgments. for a subset of narrative elements with available lexica, we compare lexical measures with llm measures, finding that in most cases, gpt-4 and llama 3 outperform lexica. (3) through a human study of @xmath0, 624 participants, we introduce a new crowdsourced dataset (heartfelt stories dataset) of empathetic reactions to personal narratives, including annotated narrative style elements, reader characteristics and narrative reactions. (4) with our dataset, we conduct an analysis of pathways through narrative style and reader characteristics leading to empathy, demonstrating the value of heart in exploring empirical behavioral insights around narrative empathy. in particular, we find that narrative styles with heightened vividness of emotions, character development and action, and plot volume, are tied to narrative empathy. we additionally show that empathy is personalized, with high variability even for the same story, and that beyond narrative style, factors like a reader's trait empathy and similarity of experiences to the narrator also significantly impact empathy. ## related work computational linguistic methods can be used to analyze many aspects of narrativity across a large corpus of stories @xcite . prior works have used lexicon-based approaches to extract psychologically-grounded word categories and relate these to empathy @xcite . @xcite use linguistic style features such as degree of interdependent thinking and integrative complexity (the ability of a person to recognize multiple perspectives and connect them) to predict a viewer's empathy towards a specific situation. @xcite ap-ply narrative analysis techniques to birth stories online, and show patterns of affective and eventbased sequences over time. more recently, @xcite used linguistic features, such as word phrases and topics, and leveraged lda to analyze language that separates more empathetic people from more compassionate people, showing that compassionate people use more other-focused language than empathetic people. other works leverage recent natural language processing (nlp) methods to predict empathy and prosociality from text @xcite @xcite , but do not explore pathways via which readers feel empathy. a few works have explored the power of llms in characterizing aspects of narrative. in particular, @xcite show that llms serve as good approximations of human annotators in narrative event segmentation. other works show that llms achieve reasonable performance on character profiling tasks for fictional narratives, particularly in factual consistency and motivation understanding. however, @xcite indicate that llms fail to perform authentic summarization of stories in line with feedback from writers, apart from successfully drawing on thematic components of the stories. ultimately, llms demonstrate growing potential for narrative understanding tasks @xcite , but how well they perform, what types of tasks they succeed in, and how they can reveal human behavioral insights, is an active area of research @xcite . our work leverages llms to extract narrative style elements that may play a role in narrative empathy through our grounded taxonomy. we evaluate the performance of prompting llms to extract such elements against expert human raters. our empirical study using llm-extracted narrative elements focuses more on the scientific and behavioral question of how to untangle aspects of narrative style and reader characteristics to understand their contribution towards empathy, rather than improving performance on empathy prediction alone. ## background empathy in the context of narratives has been the subject of many studies in psychology and literary studies. we briefly summarize those below. ## heart taxonomy for empathy and narrative style based on the aforementioned theoretical and empirical research, we propose heart, a taxonomy of narrative style elements that can lead to empathy. in a theory of narrative empathy, keen posits that aspects of characterization, narrative situation, internal perspective, and techniques to represent character consciousness can contribute to narrative empathy. we use these concepts as precursors for developing heart. our theoretical model serves as a starting point for understanding what aspects of narrative characteristics might lead to empathy and how we can measure these factors using computational approaches. figure 2 shows our full taxonomy, which delineates narrative style as it relates to narrative empathy via four main categories: (1) character identification (2) plot (3) point of view and (4) setting. in the remainder of this section, we outline each element of our taxonomy and the theoretical and empirical roots of how each element may contribute to narrative empathy. ## heart-felt stories dataset annotation with our theory-grounded taxonomy, we next evaluate how well llms can approximate narrative style elements. in order to do so, we annotate the heart-felt stories dataset, a corpus of personal narratives with expert ratings on a subset of stories. ## llms for narrative style extraction our work explores how llm-extracted narrative features can be used to yield empirical social insights around empathy and storytelling. as such, we validate whether llms are capable of narrative style annotations in line with expert human judgments. to this end, we prompt gpt-4 foot_2 and the instruction-tuned variant of llama 3 8b foot_3 with the same instructions and codebook given to human annotators (appendix c). in we observe similar patterns in agreement between gpt-4 and human raters as we do in agreement between our two expert annotators. gpt-4 provides ratings with substantial agreement for narrative features such as character vulnerabil-ity, optimistic tone, and resolution. for most features, the gpt-4 ratings are more positively correlated with human annotations than are the llama 3 ratings. as such, we use gpt-4 to extract the narrative elements for all the remaining stories in our corpus and exclude features that have low agreement with human gold labels in our subsequent empirical study. ## human study for measuring empathy to demonstrate the empirical use of our taxonomy and how extracted narrative elements can be used to explore behavioral insights around narrative empathy, we conduct a large-scale user study presenting stories to different participants and asking them to rate their empathy towards the story. in this section, we discuss our study participants, the task procedure, and our data collection and measures used. ## empirical insights on narrative empathy next, we demonstrate the efficacy of our taxonomy in exploring empirical questions around empathy with a relevant subset of features from our dataset. narrative style affects empathy first, we aggregate empathy ratings for each story by taking 1032 the mean across the 3 raters. then, we split stories into high vs. low presence of each narrative feature and apply mann-whitney u-tests to the averaged state empathy for the stories. figure 5 shows that high aggregated empathy stories have more character development and plot volume. these results are statistically significant, after applying benjamini-hochberg correction to account for nine comparisons (@xmath1.03 for character development, @xmath2.03 for plot trajectory). our work is, to the best of our knowledge, the first to empirically test the effect of character development and plot volume on narrative empathy. while some prior works @xcite propose narrative features that relate to character identification, these are lower level than character development, such as the flatness/roundness or vulnerability of a character. our findings regarding plot volume are in line with prior works that discuss how salient plot events can mark important moments in narratives that influence the emotional impact of the story @xcite . prior works primarily from narrative studies use handcrafted features on smaller story sets (fernandez-quintanilla, 2020; @xcite , but do not find significant effects of narrative features such as viewpoint and foregrounding. these studies focus primarily on literary texts rather than narratives that are more common online, and do not take into account other aspects of narrative style and narrative traits that are a part of our theorized taxonomy. these findings suggest future focused works, for example looking at how narrative style relates to empathy across narrative forms (literary vs. personal stories, spoken vs. textual, etc.) narrative empathy is not "one size fits all" while our previous analysis captures aggregated empathy, different people can have diverse emotional reactions to the same story. in figure 6 (appendix b), we show the standard deviations in state empathy scores for the same story, finding that on average this std. dev. is significantly greater than zero (@xmath3.001), indicating that the same narrative can evoke different levels of empathy. to address within-subject variance, we fit mixed effects models of empathy ratings using demographic groups, grouping individuals of similar age, sex, trait empathy, and ethnicity and conditioning on multiple ratings for a single story. we find through a likelihood ratio test that empathy predicted by demographic group results in significantly better model fit (@xmath4.002). these two results indicate that there is high variance in empathy for the same story and that incorporating information regarding diverse demographic profiles can improve empathy model fit, aligning with prior works (august et al., 2020). our findings have implications in broader empathy prediction tasks within nlp @xcite , which often optimize for a single objective empathy score assigned to a piece of text, aggregating empathy which can overlook individual factors. ## conclusion in this work, we quantify narrative style as it relates to narrative empathy. we introduce heart, the first theory-driven taxonomy delineating elements of narrative style that can evoke empathy towards a story. we evaluate the performance of llms in extracting narrative elements from heart, showing that prompting gpt-4 with our taxonomy leads to reasonable, human-level annotations beyond what prior lexicon-based methods can do, but that llms struggle in specific tasks, such as gpt-4's limited ability to extract expressions of cognition and evaluations. through a crowdsourced study with over 2,000 participants, we demonstrate how heart can be used to empirically understand the empathic role of narrative style factors. we find that vividness of emotions expressed, character development and plot volume are related to narrative empathy, and contextual factors such as a person's baseline trait empathy or sharing an experience with the narrator contribute to these effects. additionally, we show that empathy responses are highly variable even in the same story, and that narrative style preferences are personalized to people with different demographic profiles (such as varying levels of trait empathy). our findings show the promise of using llms for annotating complex story features that can yield interesting social and behavioral insights.