Generating Unexpected yet Relevant User Dialog Acts (2024)

Abstract: The demand for mental health services has risen substantially in recent years, leading to challenges in meeting patient needs promptly. Virtual agents capable of emulating motivational interviews (MI) have emerged as a potential solution to address this issue, offering immediate support that is especially beneficial for therapy modalities requiring multiple sessions. However, developing effective patient simulation methods for training MI dialog systems poses challenges, particularly in generating syntactically and contextually correct, and diversified dialog acts while respecting existing patterns and trends in therapy data. This paper investigates data-driven approaches to simulate patients for training MI dialog systems. We propose a novel method that leverages time series models to generate diverse and contextually appropriate patient dialog acts, which are then transformed into utterances by a conditioned large language model. Additionally, we introduce evaluation measures tailored to assess the quality and coherence of simulated patient dialog. Our findings highlight the effectiveness of dialog act-conditioned approaches in improving patient simulation for MI, offering insights for developing virtual agents to support mental health therapy.

## introduction the demand for mental health services has surged in recent years, resulting in a significant gap between demand and available resources @xcite . consequently, patients often face prolonged wait times before accessing therapy @xcite . to mitigate this challenge, virtual agents capable of emulating motivational interviews (mi) have emerged as a potential solution, offering immediate support, especially in therapy modalities requiring multiple sessions @xcite . these agents are not meant to replace therapists but rather supplement therapy. designing such agents can follow either a rule-based or data-driven approach. rulebased systems entail complex development and the creation of intricate rule sets. conversely, datadriven methods leverage large datasets to train models, potentially yielding optimal performance but requiring substantial data. given the difficulty in obtaining therapy data, patient simulation emerges as a viable alternative for generating large quantities of synthetic data, traditionally generated at the dialog act level. however, patient simulation relies on a high-quality simulation capable of generating dialog acts that differ enough from the existing dataset to create novel data and be contextually and syntactically correct. such a simulation should also explore all the possible dialog acts and produce diversified ones. however, the new data should also respect the structure of a real dialog. the objective is not merely to copy the observed behaviors in the dataset but to generate new ones with the following properties: be diversified, syntactically correct, and coherent in the context of the dialog. evaluating such a simulation poses challenges because traditional accuracy metrics for supervised models may not suffice, as they measure only how accurately the original data is reproduced. indeed, a generated dialog act may be different from the ones observed in the data but still be syntactically and contextually correct. this is particularly true in open dialog settings, such as mi, where the user's goal is unclear, unlike in task-based scenarios like booking systems. this paper investigates modeling methods to generate such patient dialog acts and explores evaluation methods for open-ended dialog user simulations. our contributions include: ## background and related works motivationnal interviewing (mi) is a collaborative communication style employed by therapists and educators to foster change. the goal of mi is to drive the patient towards wanting to change one of their unhealthy behaviors without giving them any solutions @xcite . the patient realizes what and how to change through a series of dialog strategies characteristic of mi, such as reflection, where the therapist reformulates what the patient just said to help them take a new perspective. in mi, therapists also create relationships with patients through social behaviors such as empathic reactions @xcite . virtual agents in healthcare is a developing area of research due to their proven effectiveness and acceptance as support tools @xcite @xcite . recently, mi conversational agents have been created in the form of chatbots @xcite and embodied conversational agents @xcite . these agents have shown promise in providing social support alongside therapy @xcite . some studies have also investigated adding empathetic behavior @xcite and humor @xcite to these agents. adaptability in such agents is important, as each patient requires a tailored approach @xcite . one way of managing dialogs is by using a rule-based dialog manager, which necessitates expert knowledge and a complicated set of rules @xcite . on the contrary, a data-driven dialog manager learns from data to anticipate the best therapist dialog acts based on context @xcite . however, this approach requires a significant amount of data that is difficult to obtain due to the private nature of therapy. simulating users has emerged as a viable approach to generate simulated data for training conversational systems @xcite . traditionally, users are simulated through a dialog manager utilizing statistical inference @xcite , inverse reinforcement learning @xcite , or transformers @xcite to select the next dialog act, enabling controllability and integration of expert or taskspecific knowledge. recently, social aspects have been incorporated into such user simulations, featuring different user types @xcite and engagement simulations @xcite . however, these techniques mainly focus on limited task domains and rely on template-based utterance generation. this approach is impractical for open application domains such as mi, where patients' responses can vary. the emergence of large language models (llms) has led to a new approach to simulated patients that addresses this challenge. this method uses llms as black boxes for user simulation, with the model generating the next patient utterance based on the dialog context @xcite . however, this technique lacks controllability and may significantly diverge from actual data without being coherent. we propose a hybrid approach that utilizes conditioned llms to overcome these issues. evaluating simulated users poses challenges as simulated users are intended to create novel data with our desired properties (i.e., syntactically correct, coherent in the dialog context, and diversified). existing works mainly evaluate their simulated users using accuracy metrics such as the f1 score @xcite that measures only the similarity with ground truth leaving aside novelty. another commonly used evaluation method involves computing the task success rate of systems trained with simulated users @xcite . while this method works well for task-based dialog, it is more complicated to apply to open-domain dialogs such as mi where social acts matter also. another evaluation method is to compare the distribution of the characteristics of generated dialogs with those of the ground truth, such as dialog length @xcite or dialog act distribution @xcite . however, these metrics do not capture the quality of the generated data. therefore, we propose metrics measuring how well user simulators fit the data and their capabilities to generate novel, syntactically and contextually correct data. to this aim, we adapt the serendipity measure to the dialog system domain. in the subsequent sections, we provide the context of our study (section 3), introduce our proposed method (section 4), present our proposed measures (section 5), and evaluate objectively and subjectively the method (section 6.2). ## context motivational interviewing (mi) is a therapeutic approach that prioritizes collaboration and fosters behavioral change. within mi sessions, therapists employ various strategies to facilitate patients' ex-pression of motivation for change @xcite . consequently, the study of mi focuses on the language of change. the language of change is defined in the motivational interviewing skill code (misc) @xcite that classifies patient behaviors into three categories: change talk (ct): reflecting actions toward behavior change, sustain talk (st): reflecting actions away from behavior change, follow/neutral (f/n): unrelated to the target behavior. this classification of the client's multimodal behavior is interesting as it predicts the therapy outcome. indeed, st is associated with poorer treatment results @xcite . furthermore, ct is linked to risk behavior reduction during follow-up assessments @xcite . these results make misc a promising tool for studying the efficacy of mi. ## dialog manager in this section, we discuss the architecture of our dialog manager that selects the next dialog act given the context (see fig. 2 ). ## definition of evaluation metrics assessing the performance of simulated users presents a challenge, as the objective is to generate behavior that aligns with real patient behavior while also introducing novel interactions. the aim is not to precisely replicate patient behavior but to produce novel data. consequently, a comprehensive analysis should involve multiple measures to evaluate the effectiveness of simulated users. these problems are similar to those encountered in rec-2 https://github.com/l-galland/ unexpectedrelevantusersimulation ommender systems evaluation, where the goal is to recommend diverse, novel, and relevant items to a particular user. metrics such as diversity, unexpectedness, relevance, or serendipity are commonly used to address these challenges @xcite . here, we propose translating these measures to the realm of user simulation. ## evaluation 6.1 baseline as a baseline for our evaluation, we employ a nonconditioned large language model (llm) tasked with responding as the patient. the llm, mistral 7b instruct, is prompted to act as a patient in an mi session and to produce the next utterance given the context. the associated prompt is visible in the appendix and the related code on github foot_2 . the resulting utterances are then classified into dialog acts using the classifier presented in @xcite and section 3.1.1. ## conclusion in this paper, we propose a dialog manager architecture and introduce comprehensive evaluation metrics tailored to open-ended dialog user simulation to address the simulation of mi patients and their evaluation. our contributions include the development of a dialog manager capable of simulating natural, coherent, and diverse patient behaviors, leveraging a combination of text and dialog act inputs. we have also proposed a set of evaluation metrics-accuracy, diversity, unexpectedness, naturalness and coherence, and serendipity-that provide a more complete assessment of simulated user performance than traditional accuracy measures. these measures have demonstrated the effectiveness of our approach in generating diverse, unexpected, natural, and coherent patient behaviors compared to a baseline llm model. our model's ability to capture and generalize from therapy data while generating novel interactions highlights its potential for training dialog models in mental health therapy settings. our findings underscore the significance of looking beyond conventional metrics and adopting a more comprehensive approach to evaluating simulated users. by focusing on diversity, unexpectedness, naturalness, and coherence, we can ensure that simulated users replicate existing behaviors and generate novel and meaningful interactions, enhancing their effectiveness as tools for supporting mental health therapy. one limitation of our study is the absence of evaluation through interactive sessions, which necessitates the development of a therapist mi dialog model. additionally, the naturalness and coherence metrics rely on the generated utterances, potentially susceptible to the methodology employed for utterance generation. nevertheless, the consistent use of the same large language model (llm) as both the baseline and the generation method mitigates this concern, utilizing highly similar prompts. a stronger baseline, such as gpt-4, would also strengthen these results. finally, conducting subjective naturalness and coherence measurements on a larger number of utterances and participants would further validate our findings, enabling the detection of significant differences. context original utterance baseline (llm) text (ours) da + text + type (ours) turn 71/79 therapist: you're quiet down. patient: yeah therapist: are you keeping interest in anythings, is there anything that that you find enjoyment in? [sustaining unhealthy behavior] i am not doing the thing there is nothing for me to do [sharing personal information] i like to watch movies and play video games. it really makes me happy when i score a goal. it really makes me happy when i score a goal. turn 21/80 therapist: have you criticized other coworkers? patient: yeah.therapist: so do you think that her criticism was within normal limits like what you've seen there regularly. [realization/understanding] i mean, now that you pointed out i mean, i guess i can view it in that way that it wasn't. [realization/understanding]well, i guess it depends on who it is and what they did. sometimes i think people deserve criticism. [laughs] but other times, maybe it's not warranted. it's hard to say for sure. it feels good to be able to express my thoughts openly and honestly without fear of judgment. turn 23/67 therapist: for the position. are you eligible for unemployment? patient: i'm not sure i'm still looking into that. therapist: because if you, i mean, i'm not a labor law expert. but i think if you're, if you're laid off, there's probably decent probability you could get on employment. how much for educational ? [sharing personal information] i just have my bachelor's. [sharing personal information] don't know yet. i haven't checked into that. i was just focusing on getting my unemployment situation figured out first. but i might look into educational assistance as well. it depends on the cost and what's available. [sharing personal information] don't know yet. i haven't checked into that. i was just focusing on getting my unemployment situation figured out first. but i might look into educational assistance as well. it depends on the cost and what's available.