CATCH : A Novel Data Synthesis Framework for High Therapy Fidelity and Memory-Driven Planning Chain of Thought in AI Counseling (2025)

Abstract: Recently, advancements in AI counseling based on large language models have shown significant progress. However, existing studies employ a one-time generation approach to synthesize multi-turn dialogue samples, resulting in low therapy fidelity and failing to capture the decision-making rationale behind each response. In this work, we propose CATCH, a novel data synthesis framework designed to address these challenges. Specifically, to improve therapy fidelity, we introduce the Progressive Dialogue Synthesis strategy, which extracts goals, resources, and solutions from a client’s self-report, organizes them into structured outlines, and then incrementally generates stage-aligned counseling dialogues. To capture decision-making rationale behind each response, we propose the Memory-Driven Dynamic Planning thinking pattern that integrates memory enhancement, global planning, and strategy reasoning; a collaborative multi-agent optimizer then leverages MDP to attach explicit chain-of-thought to each dialogue turn. Extensive experiments and human evaluations demonstrate that CATCH significantly enhances fidelity and logical coherence in AI counseling.

## introduction mental health issues are becoming increasingly prominent worldwide @xcite , highlighting counseling as a crucial intervention method. however, due to the scarcity and high costs associated with professional counselors, a significant gap persists between service availability and demand. recent advancements in large language models (llms) @xcite @xcite offer promising opportunities to bridge this gap, providing novel tools for mental health counseling @xcite @xcite @xcite . to effectively leverage llms in mental health counseling, large-scale, high-quality datasets of multi-turn counseling dialogues are essential. although recent methods have enabled the synthesis of large-scale multi-turn counseling dialogue datasets, they typically adopt a one-time generation approach @xcite @xcite , where a large language model (llm) produces the entire dialogue in a single inference step. this approach offers limited control over the therapeutic process, making it difficult to ensure that the dialogue adheres to the structured stages of a given therapy, resulting in low therapy fidelity @xcite @xcite -the degree to which an intervention aligns with established therapeutic guidelines. furthermore, these datasets generally lack explicit chain-of-thought (cot) @xcite that expose the decision-making rationale behind each counselor response. consequently, models trained on such data tend to imitate surfacelevel conversational patterns rather than grasping the underlying therapeutic logic. these challenges cause ai counseling models to suffer from therapy drift @xcite , where the dialogue progressively deviates from the intended therapeutic trajectory. in this work, we introduce catch, a collaborative multi-a-gent-based framework for turn-level c-hain-of-thought and h-igh-fidelity counseling dialogues synthesis, thereby mitigating therapy drift. to enhance therapy fidelity, catch adopts a progressive dialogue synthesis (pds) strategy, which incrementally generates dialogues stage by stage, aligning with structured therapeutic intervention protocols. starting from client self-reports, pds identifies key elements-counseling goals, resources, and solutions-and organizes them into outlines. these outlines, combined with the client's personality traits, guide llms to synthesize the dialogue for each therapeutic stage. to explicitly capture the decision-making rationale behind each counselor response, we introduce a memory-driven dynamic planning (mdp) thinking pattern that incorporates memory enhancement, global planning, and strategy reasoning. built upon mdp, we develop a collaborative multi-agent approach to generate mdp cot. to evaluate catch's effectiveness, we developed 8b and 14b scale models using dialogue datasets enriched with mdp cot. extensive experiments and human evaluations demonstrate that catch improves the fidelity and logical coherence of ai counseling, effectively mitigating therapy drift. the contributions of this work are as follows: ## methodology in this work, we present a comprehensive introduction to the proposed data synthesis framework, catch, designed for structured psychotherapy. this work uses sst as a specific therapy. in section 3.1, we describe the progressive dialogue synthesis (pds) to generate a counseling dialogue dataset with high therapy fidelity. section 3.2 introduces the generation of the memory-driven dynamic planning (mdp) chain-of-thought(cot). section 3.3 focuses on developing a llms-based ai counselor utilizing the synthesized dataset enriched with the mdp cot. ## experiments 4.1 implementations we employ gpt-4o foot_1 to implement the pds strategy for the dialogue dataset, and utilize doubao-1.5-pro-256k foot_2 to synthesize mdp cot, result- general counseling skill metrics. same as counselingeval @xcite , we selected three metrics from the ctrs @xcite to evaluate general counseling skills: understanding, interpersonal effectiveness, and collaboration. ## conclusion in this work, we introduced a novel data synthesis framework, catch, designed to enhance the quality of multi-turn counseling dialogues in ai counseling. by employing the pds strategy, we generated a counseling dialogue dataset that aligns with therapeutic intervention principles, thereby improving therapy fidelity. additionally, the mdp cot clarified decision-making motivations for each dialogue turn, facilitating a deeper understanding of therapeutic logic. our extensive experiments demonstrate that catch not only enhances the fidelity and logical coherence of ai counseling but also effectively mitigates therapy drift. this framework represents a significant advancement in the field of ai counseling, providing a robust foundation for future research and applications.