Feel the Difference? A Comparative Analysis of Emotional Arcs in Real and LLM -Generated CBT Sessions (2025)

Abstract: Synthetic therapy dialogues generated by large language models (LLMs) are increasingly used in mental health NLP to simulate counseling scenarios, train models, and supplement limited real-world data. However, it remains unclear whether these synthetic conversations capture the nuanced emotional dynamics of real therapy. In this work, we introduce RealCBT, a dataset of authentic cognitive behavioral therapy (CBT) dialogues, and conduct the first comparative analysis of emotional arcs between real and LLM-generated CBT sessions. We adapt the Utterance Emotion Dynamics framework to analyze fine-grained affective trajectories across valence, arousal, and dominance dimensions. Our analysis spans both full dialogues and individual speaker roles (counselor and client), using real sessions from the RealCBT dataset and synthetic dialogues from the CACTUS dataset. We find that while synthetic dialogues are fluent and structurally coherent, they diverge from real conversations in key emotional properties: real sessions exhibit greater emotional variability, more emotion-laden language, and more authentic patterns of reactivity and regulation. Moreover, emotional arc similarity remains low across all pairings, with especially weak alignment between real and synthetic speakers. These findings underscore the limitations of current LLM-generated therapy data and highlight the importance of emotional fidelity in mental health applications. To support future research, our dataset RealCBT is released at https://gitlab.com/xiaoyi.wang/realcbt-dataset.

## introduction mental disorders pose a major global health challenge, affecting nearly one in eight individuals worldwide (who). as demand for mental health services continues to rise, a significant barrier remains: the shortage of trained counselors. to help * corresponding author bridge this gap, large language models (llms) such as chatgpt and gemini are increasingly explored as conversational agents capable of simulating therapeutic interactions and supporting clients in need. the effectiveness of these systems, however, hinges heavily on the quality and diversity of training data-particularly, counseling dialogues grounded in real-world counseling practice. unfortunately, access to real counseling dialogues remains extremely limited due to stringent privacy, ethical, and legal constraints. this scarcity of authentic data has led to the widespread adoption of synthetic therapy dialogues generated by llms. these synthetic interactions are now commonly used in mental health nlp applications for training, evaluation, and scenario simulation. for example, datasets like cactus @xcite incorporate psychological principles from cognitive behavioral therapy (cbt) to structure modelgenerated conversations. while these synthetic dialogues are often fluent and structured around known therapeutic techniques, it remains unclear whether they capture the nuanced emotional dynamics that characterize real counseling sessions. emotion plays a central role in both diagnosis and treatment. in cbt, client emotions reveal cognitive distortions, inform intervention strategies, and signal therapeutic progress. a meaningful session often involves shifts in emotional state, such as the alleviation of distress or the emergence of insight. these emotional trajectories, or emotional arcs, offer a powerful lens through which to assess the depth, authenticity, and therapeutic quality of a dialogue. it is thus important to understand the extent to which synthetic therapy conversations mirror the emotional dynamics observed in real sessions. prior work has demonstrated that emotional arc analysis can yield rich insights into narrative structure across various domains, including novels @xcite @xcite , social media posts @xcite , and movie scripts @xcite . however, emotional arcs remain largely unexplored in the context of psychotherapy, and no prior study has directly compared real and synthetic counseling dialogues from an emotional dynamics perspective. to address this gap, we curate realcbt, a dataset of authentic cognitive behavioral therapy dialogues transcribed from public video-sharing platforms (e.g., youtube and vimeo). we then conduct a comparative emotion trajectory analysis between these real-world sessions and synthetic dialogues from the cactus dataset @xcite . we examine emotional arcs from two perspectives: (1) the emotion dynamics of real vs. synthetic therapy dialogues: how does the emotion change in real and llm-generated cbt dialogues: for entire dialogues, for counselors, and for clients? (2) emotion arc alignment across speaker pairs: to what extent do emotional trajectories align within and across speaker types (specifically among real-real, synthetic-synthetic, and real-synthetic pairings) for both counselors and clients? to answer these questions, we conduct the first in-depth comparison of emotional dynamics between real and llm-generated cbt dialogues. we use the well-established utterance emotion dynamics (ued) framework @xcite , adapted specifically for cbt dialogues, to measure emotion trajectories over time within counselor-client interactions. our goal is both to compare real and llm-generated cbt dialogues and to establish an empirical benchmark for assessing whether synthetic sessions capture the nuanced emotional dynamics of real therapy. this benchmark provides an important foundation for future work exploring more contextualized and neural emotion models. our analysis reveals that although synthetic dialogues are structurally fluent, they diverge from real sessions across several key emotional dimensions. real therapy sessions display greater emotional variability, more emotion-laden language, and more authentic patterns of emotional reactivity and regulation. emotional arc correlations are low across all pair types, though synthetic-synthetic pairs show slightly higher alignment than real-real or real-synthetic pairs. these findings suggest that while current llm-generated dialogues may capture surface-level therapeutic cues, they fall short in reproducing the nuanced, co-constructed emotional flow that characterizes real-world therapy, especially in the client role. our contributions are threefold: ## realcbt: real-world cbt dialogue dataset in this section, we describe the process of curating the realcbt dataset. ## emotion dynamics computation we adapt emotion dynamics methods from prior work by mohammad and colleagues @xcite @xcite , who demonstrated robust approaches for capturing emotional arcs in narratives and movies. specifically, we adopt the utterance emotion dynamics (ued) framework to compute emotion metrics from sequences of utterances, treating each role (counselor or client) as a separate character trajectory. given that cbt sessions are considerably shorter than novels or long-form narratives, we adjust the emotion arc computation accordingly. following prior work @xcite , we apply a rolling window of 10 words, advancing one word at a time, to compute fine-grained emotion values across each dialogue. at each windowed step, we estimate the emotional state using version 2 of the nrc valence, arousal, and dominance (vad) lexicon (mohammad, 2025), which provides word-level emotion scores for the three affective dimensions: valence (v), arousal (a), and dominance (d). each v/a/d score ranges from -1 to 1, where 1 indicates a strong positive association and -1 indicates a strong negative association with the corresponding affec-tive dimension. from the resulting vad time series, we derive a set of ued metrics @xcite ) that capture the shape and character of emotion progression in therapy dialogues: (1) emotion mean: the average v/a/d score across a dialogue or speaker's arc; (2) emotion variability: the standard deviation of v/a/d scores, representing emotional fluctuation or richness; (3) average displacement length: measures the average number of emotion words spoken more or less than usual. longer displacements suggest stronger or more persistent emotional shifts; (4) emotion rise rate: captures how quickly and intensely a speaker shifts into an emotional state (emotional reactivity or sensitivity); (5)emotion recovery rate: indicates how quickly a speaker returns to their baseline emotional state after a shift (emotion regulation). based on the ued metrics and nrc vad lexicon, we compute three types of emotional trajectories per cbt session: (1) the overall dialogue-level arc, (2) the counselor's arc, and (3) the client's arc. these arcs are computed separately for both real and synthetic cbt sessions, allowing us to compare matched pairs across datasets (i.e., real vs. synthetic dialogues, real vs. synthetic counselors, and real vs. synthetic clients). ## experimental setup we conduct our study using two datasets. cac-tus consists of 31,564 dialogues with broadly balanced distributions across multiple attributes, making it well suited for large-scale statistical analysis. in contrast, realcbt comprises 76 dialogues collected from real counseling sessions. although smaller in scale, realcbt captures authentic therapeutic interactions and naturally reflects demographic and attitudinal patterns found in practice. to ensure a fair and meaningful comparison between the two datasets, we adopt a problemfocused sampling strategy. specifically, we focus on the three most frequent client concerns in realcbt, including anxiety and fear (25 dialogues), self-esteem and confidence issues (19), and relationship-related concerns (14). together, these account for 76.3% of the dataset (58 out of 76 dialogues). for detailed summary statistics and distributional information, see while cactus is broadly balanced, realcbt reflects natural demographic and attitudinal skews (e.g., a higher proportion of female clients and generally positive orientations toward therapy). such contextual characteristics make strict subgroup balancing less stable, further motivating our problemfocused design. to enhance the robustness of our analysis, we repeat the sampling process 10 times without replacement, generating 10 non-overlapping subsets of synthetic dialogues. all quantitative results are averaged across these runs to account for sampling variability. ## conclusion in this work, we conduct the first systematic comparison of emotional dynamics between real and llm-generated cbt dialogues. using the ued framework, we analyzed emotion mean and variability, reactivity, regulation, and arc similarity across counselors and clients. our findings show that while synthetic dialogues are fluent and structured, they diverge from real therapy in key emotional dimensions-particularly in variability and emotional arc alignment. these results highlight the limitations of current llms in replicating the nuanced emotional flow of real therapeutic interactions, especially on the client side. this work introduces the realcbt dataset alongside an indepth analysis for evaluating affective realism in synthetic therapy, paving the way for future research on more emotionally grounded and psychologically credible dialogue generation.