Multi-Level Feedback Generation with Large Language Models for Empowering Novice Peer Counselors (2024)

Abstract: Realistic practice and tailored feedback are key processes for training peer counselors with clinical skills. However, existing mechanisms of providing feedback largely rely on human supervision. Peer counselors often lack mechanisms to receive detailed feedback from experienced mentors, making it difficult for them to support the large number of people with mental health issues who use peer counseling. Our work aims to leverage large language models to provide contextualized and multi-level feedback to empower peer counselors, especially novices, at scale. To achieve this, we co-design with a group of senior psychotherapy supervisors to develop a multi-level feedback taxonomy, and then construct a publicly available dataset with comprehensive feedback annotations of 400 emotional support conversations. We further design a self-improvement method on top of large language models to enhance the automatic generation of feedback. Via qualitative and quantitative evaluation with domain experts, we demonstrate that our method minimizes the risk of potentially harmful and low-quality feedback generation which is desirable in such high-stakes scenarios.

## introduction realistic practice and tailored feedback are key processes for training peer counselors with clinical skills. providing feedback could significantly enhance peer counselor skills, thereby improving support quality and benefiting many seeking help online @xcite . however, it is often timeconsuming and costly for counseling supervisors to provide detailed feedback @xcite to beginner peer counselors. without appropriate guidance, peer counselors might develop biased or @xcite annotated using our feedback taxonomy. feedback components (appropriateness, goal definition and alignment, areas for improvement, alternative goal-aligned response) are demonstrated on one utterance of the peer counselor's response (in blue). optionally, one can also provide positive reinforcement by highlighting areas in categories peer counselors excelled at. numerous recent studies have explored the feasibility of applying computational techniques to differentiate between low and high-quality counseling automatically @xcite @xcite @xcite @xcite @xcite @xcite . in doing so, prior work mostly provides numeric feedback to counselors about how well a particular skill is used. some recent studies provide utterance-level suggestions of responses to use according to appropriate counseling helping skills @xcite , or alternatives for more empathetic responses @xcite . yet, little attention is given to developing automatic feedback that closely mirrors how clinical supervisors provide feedback to novice counselors. to this end, we co-designed a feedback framework with senior psychotherapy supervisors to reflect the content and delivery of feedback they give to novice counselors. concretely, we conducted a contextual inquiry @xcite with supervisors engaging in a representative task of providing feedback on a transcript of an emotional support conversation @xcite as if they were communicating the feedback to a novice counselor. we then developed a multi-level feedback framework by modeling the common patterns at different granularity observed in interviews and important feedback dimensions highlighted in textbooks and training for foundational active listening skills @xcite . with this multilevel feedback framework presented in figure 1 , we introduce a publicly available dataset of conversations enriched with comprehensive feedback annotations, building upon an existing public emotional support conversations dataset esconv @xcite . specifically, we leverage a modelin-the-loop annotation paradigm where gpt-4 and counseling domain experts work together to produce the annotations for 400 conversations. to enable transparent model development, especially for a high-stakes domain like counseling, we fine-tuned the open-source llama-2 model to generate multi-level feedback. we further introduce a simple but effective self-improvement method to forecast how specific feedback might improve subsequent interaction and use this forecast information to supervise feedback generation. unlike general natural language generation tasks, we aim at optimizing feedback generation for worstcase performance since failures (e.g., generating poor advice) matter more in this high-stakes scenario. using both quantitative evaluation and quali-tative evaluation with domain experts, we demonstrate that our approach generates high-quality feedback and significantly boosts the worst-case performance on multi-level feedback generation compared to baselines. in summary, this paper makes the following contributions: ## related work there have been different approaches to building automated methods that help peer counselors improve their skills, ranging from scoring-based methods (e.g., measures of empathy; the use of counseling-specific dialogue acts) to automatically generated suggestions for alternative responses. scoring for assessment @xcite train a classifier to distinguish between highquality and low-quality youtube and vimeo motivational interviewing videos. @xcite introduce a system that detects reflections and open questions used by a trainee in a simulated chat and scores the whole conversation by proving the percentage frequency of those. @xcite combine technical methods of @xcite @xcite ) and develop a system for numerical scoring of motivational interviewing skills, amount of empathy, and number of reflections and open questions. @xcite design a model to identify and quantify the level of empathy in emotional support conversations and @xcite develop a system to assess the quality of real therapy sessions by automatically rating transcripts using the cognitive therapy rating scale. @xcite focus on motivational interviewing reflection skills and train a model that assigns a score in the range (0,1) to evaluate the reflection quality. in concurrent work to ours @xcite use gpt-4 with in-context learning to solve the task of multi-classification of conversational behaviors of clients and therapists. generation of suggestions @xcite create a model to automatically generate good reflections. @xcite introduce a system that rewrites non-reflective counseling responses to transform them into reflective ones. @xcite build a model that rephrases counseling responses that contain advice without permission into ones that adhere to motivational interviewing guidelines. @xcite develop hailey ai-agent that provides just-in-time suggestions on how to respond more empathetically. @xcite ) introduce the care system that generates response suggestions based on predicted appropriate motivational interviewing strategies. rather than taking a technical perspective focusing on the feedback systems that can be built with scoring or response generation methods, we posit that one can design better-automated feed-back methods for peer counseling training by understanding and mirroring the existing ways supervisors deliver feedback to novices. our co-design reveals that post-session feedback for peer counseling encompasses and extends beyond scoring and suggestions for improving individual response quality. crucially, it emphasizes that each response should be based on the counseling goals it should serve at the specific point in the session. incorporating contextualized goals into the feedback structure provides a purpose-led orientation (see ## feedback framework given the crucial role of human supervision and tailored contextual feedback in the peer counselors training process @xcite @xcite , we collaborated with senior psychotherapy supervisors (each with over 20 years of experience) to develop an automated feedback system that is aligned with best peer counseling practices. together, we codesigned a multi-level feedback framework for peer counselor training. four one-hour co-design sessions with these senior supervisors revealed that initial training of novice therapists emphasizes foundational active listening skills and that these are generic skills common to all therapy approaches, including peer counseling @xcite @xcite . details about the co-design process including research questions, key themes, and the outcomes are given in appendix b. via our co-design, we found that the structure of the supervisors' feedback spans different levels: it often starts with positive reinforcement, followed by a line-by-line analysis of session transcripts; for any utterances needing improvement, supervisors clarified the session goals, identified categories of skills that could be improved, and suggested alternative responses. ## feedbackesconv dataset in order to develop an automatic model that provides contextualized feedback at multiple levels, we use the feedback taxonomy to annotate peer counseling conversations. given the sensitive nature of peer counseling data and the involved ethical implications, we chose a publicly available counseling dataset esconv @xcite as our starting point, which contains a large number of emotional support conversations. esconv was collected on a crowd-sourcing platform, thus requiring quality control. we performed a manual review to filter out conversations that were either low quality or irrelevant to peer counseling (refer to appendix c for the comprehensive filtering criteria). we divided the obtained dataset into three parts: a dataset with 400 conversations for further annotation by domain experts; a dataset of 150 conversations (preferences qesconv) used for obtaining self-scored preference pairs as described in section 5; and a test dataset of 67 conversations. ## model we leverage the resulting feedbackesconv dataset to develop models that can generate contextualized feedback at different levels for peer counseling. to enable transparent model development, we build upon the open-source llama-2 model and introduce a simple but effective self-improvement method to generate multi-level feedback. the open-source approach is crucial in our domain application due to data sensitivity concerns, as it allows the model to be hosted in-house by data owners, ensuring compliance with data storage and analysis regulations. moreover, an open-source model provides a cost-effective solution that scales well to large mental health communities, circumventing the significant expenses associated with relying on third-party api calls, such as those required by gpt-4. ## evaluation and results in section 6.1, we compare the quality of feedback generated with m self-imp vs. those generated with baseline models via automatic scores and domainexpert ratings. after validating the improved feedback quality of m self-imp over baselines, in section 6.2, we compare its feedback to the feedback coannotated by gpt-4+experts (approach described in section 4.2) to understand if the m self-imp model matches in quality. ## conclusions we introduced a multi-level feedback framework for training counseling skills by co-designing with senior psychotherapy supervisors, constructed a public dataset of counseling conversations with feedback annotations, and proposed a simple but effective self-improvement method for feedback generation. we demonstrate through extensive qualitative and quantitative evaluation that our method minimizes the risk of low-quality feedback generation and generates feedback that domain experts find useful. this work holds the potential to improve the quality and effectiveness of counseling skill training through llms.