H eal M e: Harnessing Cognitive Reframing in Large Language Models for Psychotherapy (2024)

Abstract: Large Language Models (LLMs) can play a vital role in psychotherapy by adeptly handling the crucial task of cognitive reframing and overcoming challenges such as shame, distrust, therapist skill variability, and resource scarcity. Previous LLMs in cognitive reframing mainly converted negative emotions to positive ones, but these approaches have limited efficacy, often not promoting clients’ self-discovery of alternative perspectives. In this paper, we unveil the Helping and Empowering through Adaptive Language in Mental Enhancement (HealMe) model. This novel cognitive reframing therapy method effectively addresses deep-rooted negative thoughts and fosters rational, balanced perspectives. Diverging from traditional LLM methods, HealMe employs empathetic dialogue based on psychotherapeutic frameworks. It systematically guides clients through distinguishing circumstances from feelings, brainstorming alternative viewpoints, and developing empathetic, actionable suggestions. Moreover, we adopt the first comprehensive and expertly crafted psychological evaluation metrics, specifically designed to rigorously assess the performance of cognitive reframing, in both AI-simulated dialogues and real-world therapeutic conversations. Experimental results show that our model outperforms others in terms of empathy, guidance, and logical coherence, demonstrating its effectiveness and potential positive impact on psychotherapy.

## introduction cognitive reframing @xcite , a key part of cognitive-behavior therapy (cbt), helps individuals detach from their thoughts and situations, effectively addressing issues from mild negative thinking to severe depression and anxiety @xcite . due to the extensive dialogue and significant empathy required in psychotherapy, large language models (llms) hold immense potential whether as an adjunct to human-based mental health treatment or as a standalone therapeutic tool @xcite . llms can help overcome obstacles @xcite such as shame or distrust often associated with traditional therapy methods @xcite . additionally, they address issues like the limited availability of psychotherapeutic resources and the variability in therapists' skill levels @xcite . contrasting with previous methods that conceptualize cognitive reframing as a sentence rewriting task @xcite , where negative emotions are transformed into neutral or positive expressions emphasizing factors like specificity and actionability @xcite , our approach marks a significant shift. since cognitive reframing emphasizes the importance of clients undergoing cognitive changes themselves, rather than directly receiving guidance or suggestions from therapists @xcite @xcite , our method employs a conversational model that directly engages with and actively transforms the client's own negative thoughts. thus, despite the significant potential shown by llms in prior research, they encounter crucial obstacles when it comes to cognitive reframing. (1) if viewing cognitive reframing as a sentence rewriting task, clients might not spontaneously discover alternative perspectives and could perceive the reframing as preaching or imposition rather than selfrealization. (2) llms cannot consistently generate concrete and specific empathetic responses @xcite , which are crucial in psychotherapy @xcite . for instance, a specific empathetic response might be, i understand how upsetting it is that your friend forgot your birthday. in contrast, a more general response i feel extremely betrayed and angry. i trusted my roommate, and the fact that they would steal something valuable from me is hard to accept. i'm glad to see that you're considering proactive steps to address the situation and take control of it. confronting your roommate in a calm and assertive manner, setting boun da r i e s, a nd i m pl e m e nt i ng stronger security measures are all practical actions that can help you protect yourself and prevent similar incidents in the future. i am a psychotherapist prac-ticing to help clients. you will play the role of a client seeking therapy. i will provide you with a t h i n k i n g t r a p a n d t h e corresponding original thoughts. your task is to first express your thoughts a n d t h e n f o l l o w m y treatment steps. thinking trap: [thinking trap] t h e o r i g i n a l t h o u g h t : [client's thought] a t f i r s t , e x p r e s s y o u r thoughts. you are a psychotherapist practicing to use 'cognitive reframing' strategy to help clients reframe their negative emotions. i will play the role of the client. you should follow these three steps: 1. at first, the client will express their negative emotion. you need to guide them to separate their situation and thoughts. then the client will reply to you. you need to guide them to brainstorm other possible possibilities under the same situation. 3. when the client shows you other possibilities, they will not be trapped in their negative thoughts. that is what 'cognitive reframing' means. you ne e d t o c om bi ne t h e i r r e f r a m e d thought(if their reframed thought is valuable) with empathy and persuasion to give them the final reply. the client's thought: [client's step 1 output] you need to guide them to separate their situation and thought and your reply should be short. 1 addition prompt to the ai psychology therapist 2 3 a client (chatgpt) ai psychotherapy model (healme) seperate client's situation and thought 1 guide the client to brainstorm while considering his/her thinking trap 2 combine the brainstorm -ing with empathy and persuation for the final reply 3 follow instructions of the therapist 2 3 express thoughts to the therapist 1 perhaps... maybe i need to set boundaries for us... [client's step 3 output] would be, i understand your feelings. (3) while llms are commonly used for answering human queries, the role is reversed in psychotherapy: psychotherapists are required to guide humans @xcite . according to westerners effect, excessive external motivation can undermine internal motivation. as such, therapists giving direct suggestions may hinder clients' self-discovery and the development of self-efficacy. moreover, guidance fosters a more collaborative environment, allowing clients to explore and understand their thoughts and feelings, leading to more sustainable and selfdirected change. to tackle these challenges, we propose a specialized model helping and empowering through adaptive language in mental enhancement (healme 1 ), for cognitive reframing therapy. we emphasize the empowerment of the client rather than reliance on therapist-driven solutions. we leverage dialogue data imbued with empathy and guidance for instruction tuning, ensuring empathetic and directive responses. grounded in pro-1 our data and code are available at healme. fessional psychological literature @xcite , our domain-expert coauthors distill and organize a structured cognitive reframing therapy process, effectively emulating a complete psychotherapeutic procedure. healme operates in three main stages, as depicted in the blue section of figure 1 : 1) distinguishing between situations and thoughts for a rational outlook, 2) brainstorming for alternative perspectives to mitigate negative thinking, and 3) offering suggestions that acknowledge the client's effort and encourage positive action. this streamlined process aids clients in understanding their issues more clearly, accepting new interpretations, and moving toward constructive solutions. to build dialogue data for psychotherapy to train our model, we design prompts based on the (thinking trap, client's thought) pairs from @xcite , prompting chatgpt to simulate both client and psychology therapist roles. to test our model, we simulate interactions between the chatgpt client and our model along with baselines (including chatglm3-6b @xcite and llama2-7b-chat @xcite ). we also conduct experiments to evaluate the models' effectiveness in practical scenarios. we create a detailed psychological evaluation metric for our experiments, incorporating a three-dimensional scoring system to evaluate ai therapy models in ai client scenarios. for real-person client scenarios, we directly employ professional psychological metrics to evaluate ai therapy models. the results show that healme excels in both ai-to-ai conversations and real-world dialogues. in ai-to-ai dialogues, healme demonstrates superior empathy, guidance, and logical coherence compared to other models. during real-person testing, some clients using healme experienced notable decreases in negative emotional attributes, with negative scores dropping from 5/5 to 1/5, highlighting healme's efficacy in real-world scenarios. our contributions are as follows: (1) we introduce an ai psychotherapy model, healme, that effectively implements cognitive reframing therapy, overcoming the challenge of maintaining continuous high empathy and guidance with llms. (2) we propose a comprehensive set of professional ai psychotherapy evaluation metrics applicable to both public and non-public therapy dialogue scenarios. (3) we conduct extensive comparative analyses of our approach against other llms, both in aito-ai conversations and human interactions. these experimental results underscore the superiority of our method, paving the way for ai to develop more advanced and specialized psychotherapeutic strategies. ## problem definition and goals cognitive reframing therapy with large language models involves guiding clients out of cognitive traps during dialogues between llms and clients. in this process, llms utilize cognitive reframing strategies to alleviate negative emotions and provide concrete suggestions. for ai-simulated clients, the therapeutic performance of llms is evaluated based on the empathy, logical consistency, and guidance exhibited in the llms' responses. with real human clients or in other scenarios where therapy dialogues are not public, the effectiveness of llms is assessed by observing changes in the clients' emotional attributes before and after the therapy sessions. ## dataset construction in this study, we leverage an existing raw dataset focused on cognitive reframing and expand it to include multiple rounds of dialogue. specifically, we conduct a manual review of the selected raw dataset and select 1,000 well-composed pairs of (thinking trap, client's thought) from it. the raw dataset we utilize in this study is introduced by @xcite . our selection of this dataset is based on its comprehensive representation of common thinking traps, characterized by effectively articulated thoughts. the creation process of the raw dataset involves assigning specific thinking traps (identified as thinking patterns in the original study) and engaging crowd-sourced workers with a psychology background to manually generate these thoughts. this methodology ensures the dataset's relevance and quality, making it an ideal foundation for our research. to simulate the roles of a client and a psychotherapist, we employ chatgpt (gpt-3.5-turbo-0125) as the virtual client and psychotherapist, respectively. we choose chatgpt as the client because it can generate detailed narratives based on the provided (thinking trap, client's thought) pairs, thereby enriching the client's personality and the distressed story. to maintain the immersion of chatgpt as a client and prevent it from deviating from its role, we conducted human intervention and manual inspection for each round of dialogue. our dataset aims to mimic the simplified process of using cognitive reframing strategies in psychotherapy. we prompt both the ai client and ai psychotherapy model to generate the expected output. the prompts are shown in figure 1 , and our constructed dataset statistics are shown in cases rounds case sources train 900 3 @xcite al., 2023) valid 100 3 (maddela et al., 2023) test 300 3 (sharma et al., 2023b) ## training we partition the initial 100 multi-round conversations from the training set to form the validation set. utilizing the training data, we construct healme by conducting a 3-epoch training (costing 2h 12m 44s) of llama2-7b-chat @xcite . we select the best-performing model based on validation results from the designated validation set. the model undergoes training using the adamw optimizer @xcite , where we set a maximum learning rate of 3e-4 with a warmup ratio of 1%. all model training processes are executed on 4 nvidia geforce rtx 3090 gpus, each equipped with 24gb of memory. ## experiments this chapter evaluates the therapy capabilities of our model and baselines. ## related work the empathic capabilities of language models have been a subject of widespread interest in recent years. recently, with large language models (llms) demonstrating potential in empathy, @xcite conduct an experiment to compare the responses of chatgpt and physicians to patients expressing negative emotions on social media. the study find that 78.6% of evaluators preferred the responses of chatgpt, rating them as significantly higher in quality and empathy. additionally, chen et al. (2023a) explores the feasibility of using chatgpt in psychiatry, paving the way for the application of llms in psychological counseling. further, several initiatives utilize llms' apis to develop empathic psychological counseling platforms @xcite . while these works position llms as powerful tools in the mental health domain, empathy alone is insufficient for psychotherapy, which requires a more directive approach. llms without fine-tuning, including chatgpt or gpt-4, may struggle to consistently maintain the role of a psychotherapist and ensure high levels of empathy and guidance. our model takes a different approach by selecting the open-source llm, llama2-7b-chat, as its base and fine-tuning it to ensure the model consistently maintains the role of a psychotherapist with high empathy and guidance capabilities. in terms of therapeutic strategy, cognitive reframing has been a focal point due to its efficiency and wide applicability. while cognitive reframing is a psychotherapeutic strategy, previous approaches integrating it with llms primarily focus on rewriting negative emotions @xcite @xcite . our model, however, takes this further by implementing the process of cognitive reframing for psychotherapy, demonstrating a more holistic application of this technique in mental health care. mental enhancement (healme) model, a novel approach in the realm of llms for psychotherapy. this model effectively employs cognitive reframing to tackle deep-rooted negative thoughts, promoting balanced perspectives through empathetic dialogue grounded in psychotherapeutic principles. distinguished from traditional llms, healme emphasizes not just converting negative emotions but fostering self-discovery and rational thought processes in clients. our comprehensive psychological evaluation metrics, a first in this field, confirm healme's superiority over existing models in empathy, guidance, and coherence, signifying its potential to foster psychotherapy through ai-enhanced methodologies. ## limitations although our model can alleviate negative emotions in clients and achieve a certain level of therapeutic effectiveness, it becomes apparent in humanmachine dialogues that when clients face multifaceted issues (refer to client 5 for an example), our model addresses only some of these concerns. this limitation stems from our model supporting only three rounds of dialogue, potentially leaving clients with unresolved feelings after the conversation. our model's step-by-step guided approach, while enhancing specificity, restricts its flexibility due to the structured prompts used. in future work, we plan to incorporate a broader range of psychotherapeutic strategies and generate data for dialogues with flexible rounds. it will be helpful for the model to handle complex psychological issues more adaptively and effectively. ## ethical considerations in our study involving real-person clients, we adhere to the right to @xcite , ensuring that participants can withdraw at any time if they experience any discomfort. in such cases, we promptly delete all related data to protect their privacy. fortunately, no participants withdrew during our experiments. our participants are genuinely interested in our project and willingly share their panas scores and their feelings after interacting with a model. it is crucial to emphasize that all dialogues between the participants and the model are treated with strict confidentiality. once a participant exits the dialogue, the model ceases to record any conversation content, comprehensively safeguarding the participants' privacy.