CP sy C oun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for C hinese Psychological Counseling (2024)

Abstract: Using large language models (LLMs) to assist psychological counseling is a significant but challenging task at present. Attempts have been made on improving empathetic conversations or acting as effective assistants in the treatment with LLMs. However, the existing datasets lack consulting knowledge, resulting in LLMs lacking professional consulting competence. Moreover, how to automatically evaluate multi-turn dialogues within the counseling process remains an understudied area. To bridge the gap, we propose CPsyCoun, a report-based multi-turn dialogue reconstruction and evaluation framework for Chinese psychological counseling. To fully exploit psychological counseling reports, a two-phase approach is devised to construct high-quality dialogues while a comprehensive evaluation benchmark is developed for the effective automatic evaluation of multi-turn psychological consultations. Competitive experimental results demonstrate the effectiveness of our proposed framework in psychological counseling. We open-source the datasets and model for future research.

## conclusion in this paper, we introduce cpsycoun, an innovative framework for report-based multi-turn dialogue reconstruction and evaluation in chinese psychological counseling. our research encompasses data collection, effective data construction methods, and domain evaluation benchmarks. to harness the full potential of psychological counseling reports, we design a two-phase approach to construct high-quality consultation dialogues. concurrently, we propose a comprehensive evaluation benchmark for multi-turn consultation dialogue, inclusive of metrics, datasets and methods. experimental results validate the effectiveness of our proposed framework, demonstrating its superiority in building a comprehensive, professional, and authentic psychological counseling assistant. all datasets and model weights developed in this paper are publicly available. for future work, a more refined balance between authenticity and professional knowledge in dialogue generation needs to be achieved. we aspire this work will furnish fresh perspectives and references for the development of llms in the field of psychological counseling.