Understanding Client Reactions in Online Mental Health Counseling (2023)

Abstract: Communication success relies heavily on reading participants’ reactions. Such feedback is especially important for mental health counselors, who must carefully consider the client’s progress and adjust their approach accordingly. However, previous NLP research on counseling has mainly focused on studying counselors’ intervention strategies rather than their clients’ reactions to the intervention. This work aims to fill this gap by developing a theoretically grounded annotation framework that encompasses counselors’ strategies and client reaction behaviors. The framework has been tested against a large-scale, high-quality text-based counseling dataset we collected over the past two years from an online welfare counseling platform. Our study show how clients react to counselors’ strategies, how such reactions affect the final counseling outcomes, and how counselors can adjust their strategies in response to these reactions. We also demonstrate that this study can help counselors automatically predict their clients’ states.

## introduction there can be no human relations without communication, yet the road to successful communication is paved with obstacles @xcite . given the individuality and separateness of human consciousness, it is hard to guarantee one can receive the message sent by another. even if the message is fully understood, there can be no assurance of its acceptance. by getting feedback from their partners, communicators can better understand their communicative states. this allows communicators to adjust their communication strategies to fit better their communication environment, which is crucial for successful communication. however, most work on improving the success rates in communication, such as persuasion @xcite and mental health support @xcite @xcite , focuses on speakers' strategies. but little research is on how listeners' reactions shape trajectories and outcomes of conversations. in this work, we address the gap by examining how to use the reactions of clients to predict and improve the quality of psychological counseling, a field that has profound societal and research impact. psychological counseling is one of the most challenging and skillful forms of communication @xcite . counselors take clients through their mental health concerns while balancing the stress they are experiencing @xcite . to do it well, counselors rely on training and on continuing experience with clients to acquire the consultative skills. however, it is difficult for counselors to get direct feedback on their interventions from clients in practice @xcite . besides, due to the lack of accurate assessments of general counseling interventions @xcite , the prior studies found no noticeable improvement or effectiveness of counselors' interventions after training or counselings @xcite @xcite . as a result, some have even argued that psychological counseling is" a profession with-out any expertise" @xcite . in this regard, one solution to facilitate counselors noticing the effectiveness of interventions is to know clients' feedback during counseling conversations. however, researchers in the field mainly study counselors' skills and language patterns to provide feedback on interventions @xcite @xcite . they first separate counselings into two groups, highquality and low-quality counselings. then, features of counselors' interventions, such as language diversity, ability to handle ambiguity and make progress, are analyzed. in the end, the general patterns of the features of good counseling are reported. nonetheless, apart from the counselors' interventions, the counseling, as a process of interactive communication, also includes clients' reactions @xcite . importantly, the clients' reactions towards counselors' intervention reflect the feedback on the effectiveness of the interventions @xcite . thus, to complete the assessment of counselors' interventions from the client's perspective and to provide feedback for counselors, we are motivated to categorize the clients' reactions although identifying their reactions in the psychological counseling is difficult, even more so than categorizing counselors @xcite . in this paper, we introduce a theoretically grounded annotation framework to map each turn of the conversation into counselors' intentions and their clients' reactions. the framework is applied to label a large-scale text-based chinese counseling dataset collected from an online welfare counseling platform over the last two years. using the annotation, we analyze the associations between clients' reactions and behaviors in the counselling conversation and their assessment of conversation effectiveness. we demonstrate that the counselors' different intentions and strategies elicit different follow-up reactions and behaviors from the clients. following this analysis, we examine how counselors should adjust their strategies to encourage clients' positive behaviors based on different conversation stages and historical interaction patterns. we also analyze how the counselors address the clients' behaviors that negatively impact the conversation effectiveness. along with the automatic annotation classifiers we built, the findings of above analyses would help develop user-centered mental health support dialog systems. ## related work we mostly draw inspiration from conversational analysis in nlp and psychotherapy. despite the abundance of nlp research relating to emotional chat @xcite , emotional support @xcite , and psychocounseling @xcite , in most cases, these studies are still in their infancy. humanhuman interaction patterns are rarely studied due to the lack of large-scale conversational datasets @xcite . meanwhile, the main research focus is either on proposing new datasets or studying consultation skills. dataset for mental health support. because of the sensitive nature of mental health data, most of the available mental health support conversation corpora are collected from public general social networking sites or crowdsourcing @xcite @xcite . the potential for understanding human-human interaction patterns is limited with these single-turned or crowd-sourced datasets. @xcite propose a multiturn mental health counseling conversation corpus collected from a text-based crisis intervention platform, which is the best-related dataset up to now. however, the length of conversation in @xcite is shorter than ours (42 vs. 78 utterances), and the analysis mostly focuses on the counselors' utterances. in contrast, we emphasize the understanding and recognition of client reactions, which could facilitate counselors to understand the clients' feedback of their interventions as the psychological counselings proceed. understanding mental health support conversations using nlp. many researchers have endeavored to employ machine learning and nlp techniques to analyze mental health support conversations automatically, including modeling social factors in language that are important in the counseling setting @xcite @xcite , behavioral codes @xcite @xcite , predicting session-or utterance-level quality @xcite @xcite , and detecting mental health problems @xcite . however, these studies again mostly focus on studying consultation skills. there are methods @xcite that try to classify clients' responses but only limit to a particular mental health support genre called motivational interviewing, which has an existing coding scheme with three classes for clients. our annotation scheme is not genre specific and has more fine-grained analysis, and is more related to research in psychotherapy. ## annotation framework to understand interaction patterns between counselors and clients in text-based counseling conversations, we develop a novel framework to categorize the reactions and behaviors of clients as well as the intentions and conversational strategies of counselors (figure 2 ). in collaboration with experts in counseling psychology, we adapt and synthesize the existing face-to-face counselingfocused taxonomies, including client behavior system @xcite , therapeutic collaboration coding scheme @xcite , helping skills @xcite , and client resistance coding scheme @xcite , to the online text-only counseling conversation settings. we 2 working alliance: "the alliance represents interactive, collaborative elements of the relationship (i.e., therapist and client abilities to engage in the tasks of therapy and to agree on the targets of therapy) in the context of an affective bond or positive attachment" @xcite . have three developers foot_0 to carefully build the framework, following the consensual qualitative research method @xcite @xcite . the details of the framework development process are shown in appendix a.1. we also compare our framework with existing annotation frameworks in appendix a.2. ## data collection to validate the feasibility of our proposed framework in the psychological counseling conversation, we collect a large-scale counseling corpus and carefully annotate a subset of these conversations according to the framework. our dataset will be made available for researchers who agree to follow ethical guidelines. ## application to online counseling to illustrate how the proposed framework can be used to monitor and improve the effectiveness of conversations, we conduct the following analyses: first, we demonstrate clients' positive and negative reactions and behaviors affect the final counseling effectiveness (section 5.1). we then show how clients react to counselors' intentions and strategies (section 5.2). based on these findings, we investigate how counselors can adjust their strategies accordingly to make entire conversations more effective (section 5.3). finally, we build a baseline model for automatically labeling each counseling strategy and client behavior (section 5.4). ## conclusion we develop a theoretical-grounded annotation framework to understand counselors' strategies and clients' behaviors in counseling conversations. based on a large-scale and high-quality text-based counseling dataset we collected over the past two years, we validate the plausibility of our framework. with the labeled data, we also find that clients' positive reactions boost their ratings of counseling effectiveness, but negative reactions undermine them. meanwhile, clients are more likely to extend after counselor challenge their beliefs. moreover, our automatic annotation models indicate that clients' reactions and behaviors are more difficult to predict than counselors' intentions and strategies. due to the complexity of the data and the lack of labeled data for rare cases, our analysis is relatively shallow. we analyze the weakness of our work in section 7 and will dig deeper into each interaction pattern once we have more data.