How Can Client Motivational Language Inform Psychotherapy Agents? (2024)

Abstract: Within Motivational Interviewing (MI), client utterances are coded as for or against a certain behaviour change, along with commitment strength; this is essential to ensure therapists soften rather than persisting goal-related actions in the face of resistance. Prior works in MI agents have been scripted or semi-scripted, limiting users’ natural language expressions. With the aim of automating the MI interactions, we propose and explore the task of automated identification of client motivational language. Employing Large Language Models (LLMs), we compare in-context learning (ICL) and instruction fine-tuning (IFT) with varying training sizes for this identification task. Our experiments show that both approaches can learn under low-resourced settings. Our results demonstrate that IFT, though cheaper, is more stable to prompt choice, and yields better performance with more data. Given the detected motivation, we further present an approach to the analysis of therapists’ strategies for balancing building rapport with clients with advancing the treatment plan. A framework of MI agents is developed using insights from the data and the psychotherapy literature.

## introduction prior studies in psychotherapy in nlp have focused on understanding conversational strategies for better counselling outcomes @xcite @xcite . however, few works utilise client modelling to inform the counselling strategies @xcite . resistance to social influence is a well-known phenomenon in psychology. in therapies, resistance proves to be a serious issue, limiting its effectiveness @xcite . understanding client motivational language during therapy helps explain up to 35% in variance of treatment outcomes in psychotherapy @xcite . @xcite propose a data-driven annotation framework of clients' negative and positive reactions in therapies. their results suggest the complexities of the task. for example, negative reactions can be expressed via showing confusions, shifting topics, and giving sarcastic answers. each category can be further considered a separate task, and thus, learning them all jointly in one model is challenging. our work instead adopts the coding scheme from motivational interviewing (mi). mi tailors the therapeutic interventions based on the individuals' motivational level using the transtheoretical model of stages of changes @xcite . mi is an evidence-based client-centred approach to strengthen one's motivations for behaviour change @xcite . observably, in the context of cognitive behavioral therapy (cbt), if the client language shows ambivalence, the therapists are advised to adopt mi instead of persisting and thus risking alliance ruptures, which eventually leads to treatment dropout @xcite . similarly, @xcite find mi is likely to backfire if the client already shows willingness to change early in the session, suggesting personalised interventions at different levels of motivation. the task of predicting client motivational language can be divided into two subtasks. the first one, called the type task, is to detect the direction of motivation: whether the client is willing to change or not. the other one, called the strength task, is to detect the commitment level: if the client is willing to change or still shows resistance, how strong do they hold such belief? recently, large language models (llms) have demonstrated impressive capabilities on learning with limited data @xcite @xcite . two popular paradigms of llm usage are via in-context learning (icl) @xcite and instruction fine-tuning (ift) @xcite . using icl, the models' weights are kept frozen: no training stage takes place. inference is performed given an instruction with a few or no examples. in contrast, ift refers to fine-tuning the base models using instruction data and adapts the weights to the downstream tasks. in this paper, we first detect the types and strength of client motivational language. our experiments utilise the annomi @xcite dataset, consisting of mi dialogues annotated with the types of client language, but not the strength. using mi skill code @xcite , we obtain 178 examples with strength annotations, making the second task a low-resourced one. with varying training samples, we compare icl and ift, showing that both can perform under low-resourced setting. due to the difficulties in optimising the prompts, ift is arguably a better and cheaper paradigm and has proven its capabilities over icl in few-shot learning @xcite @xcite . our analysis further reveals that icl is, however, preferable to ift when the training data is heavily imbalanced as icl can exploit the massive underlying knowledge of llms to solve the task. after obtaining the labels, we calculate the motivational levels for client utterances in annomi as well as the distribution of next-turn therapist behaviours given the current clients' motivation. 1 our contributions are two-fold. first, we propose the task of detecting client motivational language. previous works in classifying mi codes @xcite focus on the type task only (i.e., the direction of motivation). instead, we combine it with the strength task (i.e., the commitment level) to give us a better estimate of the client motivational level. to the best of our knowledge, we are the first in nlp to adopt verbal commitment expressions to understand speakers' motivation in psychotherapy. second, we demonstrate how the detected motivation can be utilised to automate the conversational flow of mi agents. mi agents have been implemented in hri and health informatics @xcite but are either semi-or fully scripted. our proposed framework illustrates the potential usage of the motivational level to create more proactive agents for targeted therapeutic interactions. 1 the code for our experiments can be found at https: //github.com/vanhoang85/client_motivational_lang . detecting mi behaviour codes: automatic detection of mi behaviour codes is a popular research topic. as manual annotation is costly and time-consuming, automated methods are expected to assist with training by helping therapists quickly understand the therapy sessions and thus give effective feedback @xcite . mi behaviour codes have been utilised to assess the quality of not only mi but also cbt sessions @xcite . linguistic features are the most popular approach @xcite @xcite ), yet researchers have employed speech @xcite @xcite and facial expressions @xcite in multimodal systems. acoustic features, however, are found to contribute little to the prediction. in contrast, integrating both linguistic and facial information is effective in detecting client behaviour codes. ## client language in psychotherapy mi is an evidence-based therapeutic approach to strengthen ones' motivations for behaviour change. in mi, commitment to change is viewed as a lead-figure 1 : two sample dialogues from the annomi @xcite dataset. the upper one shows a strong resistance from the client (i.e., labelled as "sustain" for type and "high" for strength in our tasks). in the other dialogue, the client is ready to change though still reluctant (i.e., labelled as "change" and "low" respectively). motivational language varies in commitment strength @xcite , and can be expressed via linguistic markers of certainty @xcite . certainty is defined as the subjective degree of confidence one holds about their behaviour @xcite . for example, high certainty markers include phrases such as 'without doubt", and "for sure" while low certainty is indicated via phrases like "i guess" and "i think". two linguistic terms "boosters" and "hedges" are commonly used to refer to high and low certainty markers respectively. figure 1 illustrates one example of the client showing a strong resistance and another of having reluctance to change. broader research in psychotherapy also shows a positive correlation between strength and behavioural outcomes: the more one is motivated towards a goal, the stronger the intention-behaviour relationship @xcite , thus the more one should act upon their intention @xcite . moreover, recognising the client's motivational language helps determine the intervention treatment, i.e., whether the therapist should focus on addressing client's resistance or move to discuss action plans @xcite . despite the popularity of self-reported (i.e., questionnaires) measures, observational codes are found to correlate better with treatment processes and outcomes in mi @xcite . moreover, the commitment strength (i.e., the degree of certainty one holds for their utterance), rather than the frequency (i.e., counting each type), of the motivational language is a better predictor of change @xcite @xcite . @xcite argue that strength, not frequency, is related to positive outcomes as frequency fails to capture the correct commitment. for example, compare a highly motivated utterance "i want to get off drugs for good" with a low one "i sort of wish i could get off drugs". one client utters two times the former while another utters four times the latter. using frequency measure, the second client is assigned a higher commitment level than the first one while it should be the reverse. ## methodology our experiments are performed on (1) gpt-3.5 @xcite with icl, and (2) flan-t5 @xcite with both icl and ift. the base t5 models @xcite are developed using an encoder-decoder transformer-based architecture, framing all the tasks as a text generation problem and exploiting the benefits of transfer learning to improve models' performance. finetuned on 1800+ nlp tasks, flan-t5-xxl is shown to outperform the base t5-xxl model by 26.6% on average when evaluated on 4 different benchmark suites (98 tasks in total) @xcite . additionally, instruction-tuned flan-t5 as a starting checkpoint for single-task fine-tuning converges faster and yields better performance compared to non-instruction-tuned models @xcite . no fine-tuning is needed for icl as it performs inference using the default weights of the models. in contrast, ift requires further training to adapt the weights to the downstream tasks. as fine-tuning the entire llms proves to be too costly, parameterefficient fine-tuning (peft) aims to tackle this issue by training the downstream tasks only on a small number of parameters which can either be a subset of parameters of the existing models or newly added parameters @xcite . we employ lora @xcite , which performs parameter update of the weight matrix by decomposing the weight update into lower-rank matrices and then training them separately. when instruction-tuned models are employed for classification, the tasks are formulated as a text generation problem where the models should learn to generate the correct label for a given instruction. therefore, label-related information is critical to help identify the output space @xcite . we consider a set of dialogues where each consists of one therapist turn and one client turn. the former serves as dialogue history and the models learn to make predictions for the latter depending on the task. one turn can be comprised of multiple sentences but the output label is associated with the turn, not with the sentences. figure 1 shows two example dialogues. figure 2 illustrates our training and inference processes for ift and inference only for icl. the models are prompted to produce a type and/or strength classification by concatenating the dialogue with the corresponding instruction template. our goal is to automatically detect of the types and the strength of client motivational language during therapies. ## experimental results hallucinated output labels: framed as a generation problem, instruction-tuned models still can produce ill-formed candidates despite being trained on desirable labels: flan-t5 trained on 50 and 100 samples generates such outputs. in contrast, icl even with zero shot does not suffer from this issue. after the hallucinated labels are replaced with "neutral" foot_6 , f1 scores for flan-t5 with 50 and 100 training data jump from 0.36 and 0.47 to 0.59 and 0.62 respectively. consequently, the new score obtained on 100 samples completely outperforms other icl variants. unexpected results: observably, both icl and ift obtain little performance gain as the training data size increases. the reason could be because our training samples are not randomly selected. as explained in section 5.1, the fast vote-k algorithm by @xcite is employed to pick the most diverse samples for both training and validation sets. their paper shows that icl performance with this approach is quite stable once we have enough highquality data. hypothetically, the llms might have already obtained the most important features from the diverse dataset unless the models are trained on a full dataset with thousands of examples. ablation with output space labels: with ift, specifying output space labels proves crucial for classification tasks @xcite . in addition to the label list, one can add the label description to give extra information about the meaning of the labels. figure 3 illustrates two conditions all and simplified of our ablation studies. in contrast to @xcite who find that two conditions exhibit similar effect, we observe that all condition (i.e., having both label list and label description) outperforms simplified with varying data size. our results are similar to those of @xcite : the authors hypothesise that label description might be used to disambiguate labels with the same name but used in different tasks. error analysis: classification reports on individual labels reveal that both ift and icl struggle on "sustain": f1 scores are below 0.4 and 0.5 respectively. additionally, ift outperforms icl due to its capabilities in predicting "neutral" labels: more than half of the labels belong to this class. icl, though, still predicts more than twice "sustain" labels compared to ift. the mi type labels indicate the direction of motivation towards a certain behaviour change. they are, however, unable to capture (1) complete refusals from the clients to talk about their problems, and (2) strategies employed to avoid discussing difficult topics @xcite . in the misc guidelines, the latter can be coded as "change" because the clients tend to agree just to end the conversations. in contrast, the former behaviours are coded as "neutral". an inspection of the model predictions reveals that several instances of refusal and resistance to an undefined target behaviour change are predicted as "sustain". this explains models' poor performance on the "sustain" class, especially icl. we leave it for future works on how it might influence the design of the mi agents. ## application to psychotherapist agents an mi session consists of 4 stages: engaging, focusing, evoking, and planning @xcite . to control the conversational flow, @xcite define a fixed sequence of behaviours for each stage. we hypothesise that mi sessions can be automated using the detected motivational language. we demonstrate how it can inform the psychotherapist agents' next moves using annomi data and mi literature. therapists' strategies: using the best model from our experiments, we obtain strength labels and calculate the motivational levels for all client utterances in the annomi dataset @xcite . we employ a scale from -3 to +3, similar to @xcite . all "neutral" type equals to 0. strength labels "high", "medium", and "low" are given levels of 3, 2, and 1 respectively while type labels "change" and "sustain" indicate the positive and negative directions. for example, "changehigh" is mapped to +3 while "sustain-low" is -1. figure 4 illustrates the distribution of the nextturn therapist behaviour codes given the current clients' motivational level. we count all the possible codes in one utterance. since the "other" behaviour consists mostly of facilitating languages (e.g., mm-hmm, uh-uh, yeah) and greetings, we only compute the percentages for "question", "reflection", and "input". observably, "reflection" is employed frequently throughout the sessions, nearly 50% of the times when the clients show resistance (i.e. levels of -3, -2, and -1). more "input" and "question" behaviours are displayed when the clients are more ready to change. balancing objectives in therapies: in psychotherapy, the therapists need to balance two conflicting goals: building therapeutic rapport with the clients and pushing them towards task completion. @xcite argue that each therapist utterance aims to move backward from or forwards towards the goal. our hypothesis is that the mi behaviour codes can also be classified into rapport-building (i.e., reflections, focusing questions) and goal-pursuing strategies (i.e., evoking questions, inputs). reflections are restatements of the clients' thoughts and feelings, expressing the therapists' understandings of their inner worlds. inputs include a wide range of subbehaviours such as providing information, giving advice, offering options, and setting goals. focusing questions explore their perspectives, goals, and values while evoking questions aim to elicit their motivation to change. though no distinction between focusing and evoking questions is made in annomi, our belief is that this distinction would be beneficial to the mi agents. framework of the mi agents: with insights from the literature and the data, we would like to propose a computational framework of the mi agents. in an attempt to investigate who might benefit from mi and who not, @xcite measure the differences in clients' language early in the session and discover that those whose language reflects ambivalence (i.e. low motivated), benefit more from mi. in contrast, mi appears to be counterproductive for those who already show a readiness to change, suggesting that mi strategies should be adapted appropriately to the clients' presenting levels of motivation. the transtheoretical model of stages of health behaviour change @xcite hypothesises the first 3 stages are precontemplation ("not ready", or resistance), contemplation ("getting ready", or ambivalence) and preparation ("ready", or motivation). we hypothesise that after several interactions, if the detected motivational levels are mainly -3 and -2, the client is in precontemplation stage. if they are -1 or +1, it is contemplation. and if the levels are +2 or +3, the stage is preparation. once in the preparation stage, the mi agents should employ mainly goal-pursuing behaviours or switch to another more goal-oriented technique such as cbt (westra and norouzian, 2018), while occasionally utilising reflections and focusing questions when the users display low motivation to maintain the therapeutic alliance. besides rapportbuilding behaviours, the agents can be programmed to emphasising the users' autonomy, coded as "input" in the annomi, when the users display signs of the precontemplation stage. for example, "that is your choice. i can't make those choices for you, it is something that you decide to do." and "you're the boss. it's up to you what you want to do with you about your own health.". the stage should help inform the agents' strategies if the detected level is 0. our belief is that this information can be leveraged to design the instructions to train the agents to exhibit more mi-adherence interactions. clinical implications and potential applications: training using therapy data only might be insufficient to create psychotherapy agents as revealed by @xcite : their agent shows a lack of therapeutic behaviours and merely gives general advice. we believe that by monitoring the clients' motivational levels, the agents can act in a more proactive manner following the mi spirit. for example, giving advice and setting goals when the clients are ready enough and supporting them when resistance arises. as mi is a well-regarded, evidence-based, and widely used approach for behaviour change, the mi-aware agents can reduce the system burden and facilitate treatment delivery with lower costs to reach a wider range of users. ## conclusion and future works works in psychology suggest that monitoring client motivational language is an essential skill to deliver successful therapies. our belief is that a motivationaware system would have implications for the development of personalised healthcare agents. our experiments employ llms, and compare icl with ift on varying training data sizes. our findings indicate that both can perform in few-shot settings and be sensitive to the instructions. still, we observe that with icl, the predictions can change when adding something totally unrelated to the task itself (i.e., requesting a certain format of the output). ift is more stable; however, it suffers from generating ill-formed outputs when trained with a small number of samples. with the obtained labels, we devise a computational framework for mi agents based on the users' motivation at stage and utterance levels. insights from annomi data and mi literature suggest that the agents should exhibit mainly rapport-building behaviours when facing resistance and ambivalence. once the users indicate a strong willingness to change, goal-pursuing strategies are preferred. rapport-building behaviours are employed occasionally, when appropriate. in future works, we would like to investigate how to incorporate such information into the design of instructions to generate therapeutic interactions. 9 limitations mi practice: our paper is inspired by the mi approach to behaviour change. we try to give general readers a brief overview of the mi spirit, enough to understand the rationales behind the proposed framework. a comprehensive review of mi and/or cbt and their validity is, however, out of scope of the paper. we acknowledge that there exist different applications for mi and thus, the language should be contextualised for different clinical situations. nevertheless, the paper aims to show whether and how the motivational language can be utilised in general to direct the behaviours of the agents theoretically and experimentally without focusing on a particular clinical situation. dataset: as the conversations in the annomi dataset @xcite are role-play mi videos used for educational purposes, they might not reflect the real therapies in which the clients can behave in a more unexpected manner, especially the way they show their resistance. the language in use is english, and thus, might be unsuitable for other languages. furthermore, in practice, the therapists might use a mixture of different approaches, not just mi. all these limitations can effect generalisation to real-world applications. however, real mi therapies are scarce. the annomi demonstrations have been judged by mi experts to reflect real mi sessions. as our main purpose is to create an mi agent, we would argue that high-quality mi demonstrations should help create agents faithful to mi practice more than real therapies with mixed approaches. annotation labels: the misc guidelines suggest a fine-grained annotation based on sentences or phrases. however, the labels are assigned to turns, not sentences. a turn can consist of multiple sentences but can also be unfinished sentences or words (e.g., "-forms."). therefore, these samples contain no information to help the models make predictions. even though classifying turns might be desirable for speech systems, it might potentially teach the models inappropriate features for classification tasks. additionally, we observe many samples consisting of multiple sentences whose direction and strength of motivation can move from one end to another as the clients speak. this explains partly the low inter-annotator agreement on annomi. similarly, in the strength task, many utterances consist of multiple sentences whose certainty levels can go from one extreme to the other. this poses as a huge challenge for the annotation process. choice of models and prompts: as for the model choice, we experimented with several models before settling on flan-t5. despite not being the sota model in all tasks, the flan-t5 family is suitable for classification tasks. similarly, other parameter-efficient fine-tuning techniques are investigated but due to their poor performance, we did not include all the results except for lora. as for the prompt formats, we could have tested and compared different prompts. however, since the focus is to demonstrate how the detected motivation can be utilised to automate the mi agents' behaviours, we simply took the insights from @xcite in designing the optimal prompts. multimodal systems: we only utilise textual features to make predictions. prior works suggest incorporating visual features (i.e., facial expressions) for the type task @xcite as the client might hint their resistance by keeping silent and/or looking away. as for the strength task, experiments in linguistics show that acoustic features (e.g., pitch accents) convey speaker's commitment @xcite . when annotating the test set, we do observe that whether the speaker is fluent or hesitates about their actions can be a signal for their certainty level.