<figure><img src="image_1.png" /></figure>

HAL
open science

# Virtual Agent for Real-Time Motivational Interviewing by Integrating Adaptive Nonverbal Behavior and Language Models

Lucie Galland, Nezih Younsi, Carla Baudonne, Laurence Chaby, Agnès Helme-Guizon, Florian Pecune, Catherine Pelachaud

► To cite this version:

Lucie Galland, Nezih Younsi, Carla Baudonne, Laurence Chaby, Agnès Helme-Guizon, et al. *Virtual Agent for Real-Time Motivational Interviewing by Integrating Adaptive Nonverbal Behavior and Language Models*. 2025. hal-05387255

HAL Id: hal-05387255

https://hal.science/hal-05387255v1

Preprint submitted on 1 Dec 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
1 **TITLE:**

2 Virtual Agent for Real-Time Motivational Interviewing by Integrating Adaptive Nonverbal

3 Behavior and Language Models

4

5 **AUTHORS AND AFFILIATIONS:**

6 Lucie Galland*¹, Nezih Younsi¹, Carla Baudonne², Laurence Chaby³, Agnes Helme-Guizon⁴, Florian

7 Pecune⁵, Catherine Pelachaud⁶

8

9 ¹ISIR - Sorbonne University, Place Jussieu, Paris, France

10 ²University of Bordeaux, cours de la Libération, Talence, CEDEX

11 ³Vision Action Cognition, Université Paris Cité, Paris, France

12 ⁴University of Grenoble Alpes, Grenoble INP, CERAG, Grenoble, France

13 ⁵CNRS - SANPSY, University of Bordeaux, Bordeaux, France

14 ⁶CNRS - ISIR, Sorbonne University, Place Jussieu, Paris, France

15

16 Email addresses of co-authors:

17 Nezih Younsi (younsi@isir.upmc.fr)

18 Carla Baudonne (carlabdne@hotmail.com)

19 Laurence Chaby (laurence.chaby@parisdescartes.fr)

20 Agnes Helme-Guizon (agnes.helme-guizon@univ-grenoble-alpes.fr)

21 Florian Pécune (florian.pecune@u-bordeaux.fr)

22 Catherine Pelachaud (catherine.pelachaud@isir.upmc.fr)

23

24 Corresponding author:

25 Lucie Galland (lucie.galland@isir.upmc.fr)

26

27 **SUMMARY:**

28 We present an open-source virtual agent platform for conducting real-time motivational
29 interviews, combining state-of-the-art language and diffusion models to adapt to users' behavior
30 and profile. Utilizing the Greta 2.0 platform, it supports various topics, including nutrition and
31 sport-focused interventions, and offers a flexible, validated tool for enhancing digital therapeutic
32 interactions.

33

34 **ABSTRACT:**

35 The growing demand for therapeutic support increasingly exceeds the capacity of available
36 professionals. A virtual agent capable of performing motivational interviewing (MI) offers a
37 promising solution to assist patients in reaching their goal of behavior change between sessions
38 with human therapists. MI is inherently a cooperative and adaptive form of communication.
39 Therefore, developing an agent capable of adapting its conversational strategies to the context
40 could significantly enhance the effectiveness of therapy. During MI sessions, human therapists
41 adjust both their verbal and nonverbal behaviors based on the human patients' responses, as
42 well as their profiles. Depending on the patient's level of motivation, the therapist will modify
43 their approach accordingly. Thus, personalization and adaptability are essential for developing
44 effective MI virtual agents. In this paper, we present a virtual agent capable of conducting MI
sessions by dynamically adapting verbally and nonverbally to users in real time. Leveraging state-of-the-art models, this system enables MI interactions. The virtual agent is embodied using the Greta 2.0 platform. Its nonverbal behavior is generated through a diffusion model called MODIFF, which adapts to the user's facial expressions and their readiness to change. These facial expressions were learned on an MI corpus and validated through a dedicated user study. The dialogue is generated using a state-of-the-art large language model (LLM), enhanced by a dialogue manager specifically designed for MI, with a reinforcement learning approach, and validated through user testing. Furthermore, the dialogue manager is able to adapt to different user profiles. The resulting platform is open-source and facilitates the generation of real-time, multimodal MI dialogues, providing new tools for digitally mediated therapeutic interactions.

## INTRODUCTION:

Motivational interviewing (MI) is a collaborative therapeutic approach that encourages behavioral change. During MI sessions, MI practitioners help patients articulate and foster their motivation for change¹. To this aim, they use dialogue strategies, such as reflections or questions, reinforced by non-verbal behaviors, such as smiles or specific head and body postures.

As the frequency of mental health issues has increased in recent years, a gap has emerged between the demand for mental health services and the available resources². This has led to longer waiting times for patients before they can access therapy²,³. One proposed solution to alleviate this issue involves virtual agents capable of replicating MI for patients awaiting appointments. By offering immediate support and intervention, these virtual agents can help reduce the impact of waiting periods, especially in therapeutic contexts that require multiple sessions⁴. Several studies have demonstrated the effectiveness of such agents or chatbots⁵⁻⁹ whose goal is not to replace MI experts; they are intended to serve as a complementary tool within the therapeutic process. For instance, they could provide MI intervention in between sessions.

Although previous MI agents often exhibit limited adaptive behaviors, MI is inherently a cooperative and adaptive form of communication. Therefore, developing an agent capable of adapting its dialogue to the context could significantly enhance the effectiveness of therapy¹⁰. During MI sessions, therapists adjust both their verbal (conversational strategies) and nonverbal behaviors based on the patient's responses, motivation, or engagement level¹¹,¹², as well as their individual profiles¹³. Depending on the patient's level of motivation, the therapist will modify their approach accordingly. Thus, personalization and adaptability are essential for developing effective MI virtual agents.

The Greta 2.0 platform¹⁴ is a modular virtual agent framework that supports real-time interactions. It has recently been extended to incorporate capabilities that enable the dynamic adaptation of generative AI models during interaction.

The platform includes a model called MODIFF-8¹⁵, which generates real-time facial expressions that dynamically adapt to the user's own facial expressions. This diffusion-based model was trained on multimodal data collected in the context of MI, namely the AnnoMI corpus,
constituted of counselling session videos between therapist and client pairs, collected on streaming platforms, such as YouTube and Viméo, and annotated in dialogue acts (a classification of utterances that denotes the actions performed e.g., Question, Greeting...) and action units (a taxonomy of human facial expression). This data conditions the generation not only on the user's current facial expression but also on MI-specific dialogue acts identified during the interaction, linked to the conversational strategies adopted. These include MI-relevant users' dialogue acts, such as change talk, as well as the agent's own dialogue acts, such as reflection or providing information. By integrating these multimodal signals in real time, the system allows the virtual agent to express socially and contextually appropriate facial expressions. By aligning its non-verbal behavior (NVB) to the context of the interaction, this virtual agent is designed to foster social rapport and support the initiation of intrinsic motivation throughout the interview process¹⁵.

Additionally, the platform integrates a dialogue system specifically designed for MI, called DREAM¹⁶. DREAM comprises a dialogue manager that selects the agent's next dialogue act based on the current dialogue state and the user's profile. Indeed, our analyses of the AnnoMI corpus revealed three profiles of MI patients: Open to Change (when the patient is motivated to change and seeks support), Receptive (when the patient has not yet considered change), and Resistant (when the patient does not intend to change)¹³. Before interacting with the virtual agent, users are attributed one of these three profiles according to their answers to the Decision Balance Scale (DBS) questionnaire¹⁷. This dialogue system is complemented by a large language model (LLM) that generates the agent's verbal output, corresponding to the dialogue act selected by the dialogue manager. This component enables contextual adaptation of the dialogue, marking an improvement over more rigid, rule-based approaches⁶. While recent studies have explored the use of LLMs to generate MI-consistent dialogue, they do not account for user profiles¹⁸,¹⁹. In contrast, this module dynamically adapts to the user's profile, which enhances rapport compared to these baseline systems.

The integration of these models into the Greta 2.0 platform enables dynamic MI interventions. Its modular implementation allows for the independent evaluation of each model's impact. In this paper, we present the method used to conduct experiments that assess the contributions of the MODIFF-8 and DREAM models to the MI dialogue delivered by the platform used. This method can also be used to deliver tailored MI interventions between sessions with MI practitioners. It requires a Windows computer with a 16 GB GPU and the installation of open-source software. Using this method, we demonstrate that our adaptation models improve the perceived quality of the MI delivery agent and enhance rapport building. We also observe improvement in the motivation of patients to change their behaviors when the agent uses adapted behaviors at the verbal and nonverbal levels, and conversational strategies levels. The results presented in this paper demonstrate the positive impact of tailoring the agent's MI intervention to each patient.

To support the design of this system, an in-depth literature review was conducted to explore the theoretical and practical foundations of nutritional coaching, including motivational interviewing, behavior change models, and user typologies. This review led to the development of a qualitative
knowledge base structured around the intersection of two key dimensions: 1) users' readiness for change following the Transtheoretical Model¹⁹ that identifies 5 stages: pre-contemplation, contemplation, preparation, action, and maintenance, and 2) their eating behavior profiles²⁰ based on diet and psychologically related factors. Three profiles were identified: the intuitive eater (i.e., listens to their hunger and fullness cues, maintains a varied and predominantly plant-based diet, prioritizes high-quality, often organic foods), the emotional eater (i.e., copes with stress, loneliness, and joy through frequent snacking on sweets or fatty foods, often feeling guilty afterward) and, the restrictive eater (i.e., obsessed with food control, they count calories, follow frequent diets with yo-yo effects, and may have an eating disorder).

For each combination of stage and profile, context-specific dialogue strategies were proposed, including tailored SMART goals, motivational formulations, and concrete behavioral suggestions. The aim was to enable the virtual agent to dynamically adapt its interventions based on both psychological readiness and personal tendencies. This groundwork ensures a higher degree of personalization and empathy in interaction, which are crucial to sustaining user engagement and long-term behavioral change.

**PROTOCOL:**
The method received ethical IRB approval from the INSEAD institute (Institut européen d'administration des affaires) with acceptance numbers: INSEAD 2024-78 and INSEAD 2025-23. All participants provided informed consent and were compensated for their experiment time.

1. **Recruitment of participants**

1.1 Call for participants using the institute's mailing lists. No constraints on participants, except familiarity with digital interfaces and fluency in French. Three Participants were excluded due to internet resource drops, which significantly increased the global system response latency.

2. **Pre-installation requirements**

2.1 Use a Windows computer. We use an Intel i7 CPU, 20 cores, 28 threads (50% usage), RAM 32GB (20% usage), 32GB Nvidia RTX 5000 GPU (20% usage).

2.2 Install Java SE Development Kit 8 (https://www.oracle.com/java/technologiesjavase/javase8-archive-downloads.html). Install Visual C++ Redistributable for Visual Studio 2013 (https://www.microsoft.com/en-us/download/details.aspx?id=40784).

2.3 Request a Cereproc license (https://app.cereproc.com/). Install OpenFace (https://github.com/TadasBaltrusaitis/OpenFace). Get a webcam.

3. **Greta installation**

3.1 Download the software release from https://github.com/isir/greta/releases. Compile
177 using NetBeans (detailed instructions at: https://github.com/isir/greta/wiki/Quick-start).

178

179 4. Mistral API key configuration

180

181 4.1 Get a Mistral API key from https://console.mistral.ai/api-keys.

182

183 4.2 Create the file: greta\bin\Common\Data\LLM\MICounselorRL\api_key.txt. Paste the API
184 key into the created file.

185

186 5. DeepGram API key configuration

187

188 5.1 Get a DeepGram API key from https://console.deepgram.com/.

189

190 5.2 Create the file: greta\bin\Common\Data\DeepASR\DeepGram\api_key.txt. Paste the API
191 key into the created file.

192

193 6. MODIFF model importation

194

195 6.1 Download the model weights from
196 https://drive.google.com/file/d/1n3udJf5pSNnFmxs_kt_zcFcF0m7zoLLr/view?usp=sharing and
197 place the file into greta\bin\Common\Data\MODIFF.

198

199 6.2 Download and uncompress dlib face detection models from
200 https://dlib.net/files/mmod_humanFace detector.dat.bz2 and
201 https://dlib.net/files/shape5FaceLandmarks.dat.bz2, then place them into
202 bin/Common/Data/TurnManagement/dlib_models.

203

204 6.3 Download all VAP models in model/VAP from
205 https://drive.google.com/drive/folders/10_MTPkRyJCoFLDt4LO_M9kRgoreh6gEF?usp=sharing
206 and place them into bin/Common/Data/TurnManagement/models/.

207

208 7. Launching Greta 2.0

209

210 7.1 Launch Modular.jar. Click File > Open > Greta > Advanced > 20250128 Greta - Expé
211 Lucie_full.xml.

212

213 7.2 Ensure that the configuration matches **Figure 1**.

214

215 8. Launching OpenFace

216

217 8.1 Start OpenFace Offline ZeroMQ.exe. A specific version of OpenFace allows direct
218 communication with the Greta platform using the ZeroMQ protocol through port 5000.

219

220 8.2 In the Record tab, uncheck everything except Broadcast with ZeroMQ. It allows the
221 software to send the extracted signals (AUs, Gaze, Head movements ...) to the Greta platform.

222
223 8.3 In the File tab, click **Open Webcam**. Authorize live feature extraction using the available
224 webcam.

225
226 8.4 Select the webcam to use and wait until the webcam is loaded (see **Figure 2**). Once the
227 webcam is loaded, the tool will automatically start the live feature extraction and send the data
228 to the Greta platform.

229
230 9. **Launching MODIFF-8**
231

232
233 9.1 In OpenFace2 Output Stream Reader, click the **Connect** button and wait for the list of
available features, sent from OpenFace, to populate (see **Figure 3**).

234
235 9.2 Click **Select All > Set**, to validate the set of features you want the platform to continuously
236 receive from OpenFace.

237
238 9.3 Under Modiff, click **Launch** and wait for the message: Connected to server at localhost
239 5560.

240
241 9.4 Click **Connect**. This step activates the connection between the facial expression
242 generation module Modiff and the Greta OpenFace data receiver.

243
244 9.5 Under Filter, click **Perform**, allowing the virtual agent to perform the data generated by
245 the Modiff Module.

246
247 9.6 In the MODIFF window, click **Enable**. Wait 90 s for the model to stabilize.

248
249 10. **Launching ASR**

250
251 10.1 In the DeepGram window, press **Enable**.

252
253 11. **Launching DREAM**

254
255 11.1 Select the Condition **RL** to use DREAM and Baseline to use a plain LLM.

256
257 11.2 In the MI Counsellor RL window, click **Enable**. Wait 30 s for the model to start.

258
259 12. **Set up the room**

260
261 12.1 Use a large monitor to put on a table with a chair in front. Put a webcam on top of the
262 monitor facing the chair. Put a directional microphone on the table in front of the chair.

263
264 13. **Welcoming the participant**
265 13.1 Welcome the participant to a private room (see **Figure 4**). Brief the participant as per the speech in **Supplementary File 1**.

268 13.2 Have the participant fill out the DBS questionnaire on a 5-step Likert scale (see items in **Table 1**).

271 13.3 Ask the participant to talk into the microphone. Ensure the webcam captures the participant's head and upper body

274 **14. Identify the participant profile**

276 14.1 If the DBS score is negative, classify the participant as Resistant to Change. If the DBS score is between 0 and 2, classify the participant as Receptive. Otherwise, classify the participant as Open to Change.

280 **15. Set the theme and profile**

282 15.1 In the MI Counsellor RL window, set the theme to be discussed as the participant chose. Set the profile as identified in step 14.

285 **16. Start the dialogue**

287 16.1 In the MI Counsellor RL window, click **Start** to have the agent initiate the discussion with the participant. Clicking this button makes the dialogue DREAM, and multimodal behaviors MODIFF models automatically drive the agent's behaviors.

291 **17. During the dialogue**

293 17.1 Monitor the Answer paragraph of the MI Counsellor window to ensure the LLM output is not harmful.

296 17.2 In the DeepGram window, click **Listen** after the agent's turn is finished.

298 17.3 If the speech is not recognized, fill the Request paragraph of the MI Counsellor RL window with the participant's speech and click **Send**. This automatically creates the agent's response.

301 17.4 Wait for the participant's answer.

303 17.5 Repeat 17.1 to 17.4 until the dialogue ends. The dialogue stops on its own.

305 **18. Post-intervention**

307 18.1 Ask the participant to complete the post-intervention questionnaires (see items 1-9 in
**Table 1).** Debrief the participant using the speech in **Supplementary File 1**.

**RESULTS:**

In the presented protocol, we collected interaction transcripts and user feedback using questionnaires adapted to the behaviors we want to measure. We compared the results of the interaction with or without our models.

**Using the protocol to evaluate the impact of the MODIFF-8 adaptive expressions generation**

The MODIFF-8 module was designed to enhance the social responsiveness of the virtual agent by enabling it to generate facial expressions that are both contextually appropriate and behaviorally synchronized with the user¹⁵. The core hypothesis behind this design is that adaptive nonverbal behavior, especially facial expressions aligned with the interactional content of the exchange, plays a role in building rapport and promoting self-disclosure during MI¹⁹⁻²¹.

To evaluate the impact of this real-time adaptive capability, we conducted a user study. We recruited 60 participants (N = 63; Aged 18-45 years; 30 males, 33 females) through the INSEAD recruitment platform at the Sorbonne University Behavior lab. The study manipulated the agent's expressivity and the behavioral alignment of its facial expressions across three experimental conditions following a between-subject design (with 20 participants per condition). The MODIFF-8-based condition, where the facial expressions were adaptive and aligned temporally with the context and the user. The Mismatched condition, where the expressivity was conserved according to the model-based condition, but the timings of the facial expressions were desynchronized, leading to temporally incoherent behavior. In the third condition, the agent was completely inexpressive, exhibiting lip synchronization with speech only. This design enabled us to evaluate the influence of adaptive nonverbal behavior on participants' perception of the agent and the outcomes associated with MI.

The evaluation relied on both objective and subjective measures. For an objective assessment, we focused on self-disclosure, operationalized as a combination of the number of words, sentiment valence, and sentimentality expressed by participants during the interaction²². Subjective measures were collected through questionnaires assessing perceived attitude and alignment of behaviour²³, social rapport²⁴, and MI quality, the latter being measured using the CEMI and MITI scales²⁴,²⁵. CEMI allows one to measure the MI quality from a patient-centered perspective, and the MITI is more about the collaboration between the therapist and the patient. The detailed questionnaires are provided in **Table 2**, **Table 3**, **Table 4**, **Table 5**, **Table 6**, and **Table 7**.

The MODIFF-8 adaptive condition yielded greater self-disclosures from the user than both the inexpressive and mismatched expression conditions²⁶. Users also tended to express more positive sentimentality with the adaptive agent (see **Figure 5**). This suggests that behaviorally aligned facial expressions encourage users to share more openly and positively about themselves. Although no statistically significant differences were found in the subjective questionnaire scores across the three expression conditions, a trend emerged: the mismatched condition was generally rated lower than both the inexpressive and MODIFF-8 conditions (see
**Figure 6**). Interestingly, participants rated the inexpressive and adaptive agents similarly, indicating that the absence of facial expressions may be perceived as less disruptive than expressions that are temporally or contextually incongruent. This raises important questions about expectancy violation and user tolerance toward nonverbal behavior in human-agent interaction and highlights the need for further exploration²⁷.

To better understand how facial expressions influence the perceived MI quality, we conducted a mediation analysis to examine whether the effect of perceived attitude on MI quality ratings was mediated by other social factors as social rapport. The results revealed a strong direct effect of attitude perception on MI quality (see **Figure 7**). Interestingly, this relationship was partially mediated (42%) by the perception of social rapport, indicating that participants who perceived the agent as having an aligned attitude were more likely to experience a sense of rapport, which in turn enhanced their evaluation of the MI session's quality. This observation supports psychological theories that highlight the role of relational factors, such as relatedness and rapport, in effective motivational counseling²⁴,²⁸,²⁹. It further underscores that adaptive nonverbal behaviors contribute not only individually but also through a cascading effect on the broader MI interactional dynamics.

Overall, these findings emphasize the value of synchrony and adaptation in enhancing user experience. They demonstrate that while adaptive nonverbal behavior can promote more disclosure, necessary in the MI context, mismatched or poorly timed expressions may backfire, undermining perceived attitude and alignment. Adapting facial expression thus represents a promising direction for designing socially intelligent agents in MI dialogues.

## Using the protocol to evaluate the impact of the DREAM Dialogue Manager

The impact of the DREAM dialogue manager was evaluated through another user study following the protocol presented in this paper³⁰. A total of 60 participants interacted with the Greta 2.0 agent, with the dialogue being governed by either a baseline plain LLM or the DREAM dialogue manager. For both conditions, the LLM used is a Mistral large 2.1 version of November 2024 with a temperature of 0.7. For both conditions, the prompt contains information about MI and the discussed subject (smoking habits, exercise, or drinking habits). For the DREAM condition, in addition, the prompt contains the next dialogue act to perform, as well as examples on how to perform this dialogue act.

The participants then filled out questionnaires to assess the quality of the interaction. The protocol was similar to one of the MODIFF experiments, but diverged in the questionnaire used, as we did not aim to evaluate the same behaviors (impact of facial expressions versus impact of dialogue strategies). This study uses 3 questionnaires to evaluate the quality of the intervention. First, the rapport scale³¹ (see **Table 8**) measures the rapport built between the agent and the participant, then the DBS¹⁷,³²,³³ (see **Table 1**) measures the motivation of the patient to continue the behavior targeted by the intervention and finally, the Client Evaluation of MI²⁶ (CEMI, see **Table 9**) score measures the quality of the MI intervention from the patient's perspective. The study compares these scores in two conditions: with a plain LLM or with the DREAM dialogue manager. As the architecture of Greta 2.0 is modular, the DREAM dialogue manager can easily
397 be removed by selecting the condition Baseline in Step 11.1.

398
399 The study shows that participants interacting with the DREAM dialogue manager build
400 significantly more rapport with the agent than those interacting with the plain LLM (see **Figure 8**). These results demonstrate that users have a better perception of the agent when the dialogue
401 is adapted to their particular profile. The results of this study also show that the DREAM dialogue
402 manager tends to adapt better to different user profiles than plain LLMs in terms of motivation
403 scores (DBS, see **Figure 9**). They also show that adapting to different user profiles helps foster
404 behavior change and further validates the use of the adapted DREAM dialogue manager.
405
406

407 Both conditions also produce CEMI scores (Baseline: 1.07, DREAM: 1.02; see Figure 10) above the
408 therapeutic threshold (CEMI = 0, as defined in Galland et al. 19), indicating that the protocol
409 generates relevant MI dialogues and can be used to test additional modules.
410

411 **FIGURE AND TABLE LEGENDS:**

412 **Figure 1: Configuration used in Greta 2.0.** The configuration used in this protocol with all the
413 necessary modules and how they are connected.

414
415 **Figure 2: OpenFace offline Window.** This Figure shows the OpenFace window as it is at the end
416 of step 9

417
418 **Figure 3: OpenFace2 output stream reader Window.** This Figure shows the OpenFace output
419 stream reader Window as it is at the end of step 9.

420
421 **Figure 4: Participant interacting with Greta 2.0.** (Left) This figure shows a participant interacting
422 with the Greta 2.0 agent; (Right) Image of the Greta 2.0 agent.

423
424 **Figure 5: Self-disclosure components over time.** This figure shows the evolution of compound
425 sentiment and sentimentality during the interaction, across the three expression conditions
426 (MODEL, MISMATCH, INEXPRESSIVE). Scores are averaged across three-time segments of the
427 session.

428
429 **Figure 6: Questionnaire ratings per facial expression condition.** This figure shows mean scores
430 for Attitude and Alignment, Social Rapport, and MI Quality across the Mismatch, MODIFF-8, and
431 Inexpressive conditions.

432
433 **Figure 7: Mediation analysis of attitude perception and MI quality.** This figure illustrates the
434 mediation analysis showing that the effect of attitude on MI quality is partially mediated by
435 perceived Social Rapport (43% mediation; 95% C.I. [0.23, 0.65]).

436
437 **Figure 8: Rapport SCORE post-intervention with the DREAM experiment.** This Figure shows the
438 Rapport score obtained with the DREAM condition and the baseline condition (t-test (56) = 2.15*
439 : p < 0.05, the error bars show the first and third quartiles).

440
441 **Figure 9: DBS SCORE post-intervention with the DREAM experiment.** This Figure shows the DBS Score obtained with the DREAM condition and the Baseline condition, separated by profiles (ANCOVA, F(2, 49) = 2.02, p = 0.1; error bars represent standard error).

444
445 **Figure 10: CEMI SCORE post-intervention with the DREAM experiment.** This Figure shows the CEMI Score obtained with the DREAM condition and the Baseline condition (Baseline; t-test (29) = 5.89, DREAM: t-test (26) = 9.71 ****p<0.0001). The error bars show the first and third quartiles.

448
449 **Table 1: DBS Questionnaire and post-intervention with the DREAM experiment.** DBS Survey statements, their polarity, mean scores, and 95% confidence interval for baseline and model conditions after intervention. We only report the values for exercise, as there are not enough participants in the other themes. Translation verified by bilingual expert review

453
454 **Table 2: Perceived attitude of the counselor questionnaire.**

455
456 **Table 3: Behavioral alignment questionnaire.**

457
458 **Table 4: CEMI questionnaire.**

459
460 **Table 5: MITI questionnaire.**

461
462 **Table 6: Social rapport questionnaire.**

463
464 **Table 7: Social presence questionnaire.**

465
466 **Table 8: Rapport questionnaire and post-intervention with the DREAM experiment.** Rapport Scale survey statements, mean scores, and 95% confidence interval for baseline and model conditions. Translation verified by bilingual expert review

469
470 **Table 9: CEMI questionnaire and post-intervention with the DREAM experiment.** CEMI Survey statements, their polarity, and mean scores and 95% confidence interval for baseline and model conditions. Translation verified by bilingual expert review

473
474 **Supplementary File 1: Brief and debrief speech.** Speech delivered to the participants by the experimenter before and after the interaction.

476
477 **DISCUSSION:**

478 In this paper, we present a protocol for delivering tailored Motivational Interviewing interventions and for testing individual behavior generation modules within a controlled and reproducible interaction framework. This method is built upon the modular architecture of the Greta 2.0 platform, which offers control over the components involved in agent behavior, allowing researchers to easily toggle specific modules on or off. This flexibility makes Greta 2.0 well-suited for both experimental manipulations and targeted evaluations of new features in human-agent multimodal interaction. This protocol can be used for two purposes: Testing the
validity of a new module in the Greta Platform (for instance, a module generating automatic gestures) or delivering tailored Motivational Interviews of quality above the therapeutic threshold. As demonstrated in the results, the use of these adaptation models enhances the perception of the agent and the motivation for behavioral change among users.

One key application of the platform is the delivery of MI conversational support between sessions with a human expert, especially in contexts where MI-trained personnel are less available. The platform enables non-expert supervision, increasing the accessibility of MI interventions. At the same time, it provides a research environment for systematically testing new behavioral modules such as full-body gesture generation, gaze, and turn-taking strategies. As it uses pre-validated dialogue structures and an integrated expression generation module (DREAM and MODIFF-8), there is no need to design and validate new interaction content from scratch. This time-consuming process can affect the validity of experimental results.

By providing real-time, ecologically valid interactions with virtual agents, the platform offers a shared experimental environment for researchers in AI and cognitive science to test computational models, such as reinforcement learning, generative models, and affective computing, and to examine their effects on human behavior through live interactions. This cross-disciplinary potential enables the investigation of open questions in cognitive psychology, such as how synchrony builds rapport, how nonverbal cues shape empathy, and how social presence affects motivation and cognitive effort.

Despite its strengths, the current protocol does present limitations. In particular, steps 13–15³⁴ remain operator-dependent, notably because of the use of external APIs for LLM and ASR management: the operator must monitor the interaction, manage turn-taking, which is currently handled by hand by the operator (Step 17.2), address automatic speech recognition (ASR) errors, and verify the appropriateness of the LLM responses in real time. ASR errors can be corrected on the fly through the ASR interface, before being sent to the LLM module. While this ensures a smooth interaction flow, it limits the system's full autonomy. Future work could focus on automating these critical components, for instance, by integrating real-time turn-taking prediction models, robust ASR error recovery mechanisms, and LLM safety filters. Such automation would not only reduce operator burden but also pave the way for fully autonomous MI delivery systems, suitable for deployment in real-world contexts, although safety filters should be carefully considered given the critical context of MI.

Additionally, although Greta 2.0 supports the testing of sophisticated modules such as MODIFF-8, these modules are still dependent on training data that may be domain-specific (e.g., MI dialogues). Generalizing behavior generation across broader interaction types or populations requires further research on domain adaptation and transfer learning for facial expression and dialogue act generation.

Another limitation is the limited number of user profiles currently considered, which may not fully capture the diversity and complexity of real-world eating behaviors. The current typology includes eight distinct profiles; each linked to a set of stage-specific recommendations. While this
provides a structured starting point, it remains based on expert-driven categorization and may overlook nuanced or evolving behavioral patterns.

The integration of more behavioral profiles and readiness-to-change stages into the MI-delivering agent could significantly enhance the system's ability to deliver personalized and context-aware advice. Based on the typology presented in the introduction, each user is associated with a profile (e.g., emotional eater, restrictive eater, utilitarian eater) and a corresponding stage of change (e.g., contemplation, preparation, action). This dual mapping can be added to the prompts to generate suggestions that resonate more effectively with users' cognitive, emotional, and motivational states. For instance, for an emotional eater, in the preparation stage, the therapist can propose: try three comforting plant-based meals this week and reflect on how they affect your mood, while for a utilitarian eater in the contemplation stage, the therapist can invite them to: test a quick and affordable plant-based dish and evaluate its impact on your energy. These micro-goals are embedded in motivational language and grounded in evidence-based coaching principles.

Preliminary user testing showed that this level of personalization increased perceived relevance, user engagement, and satisfaction. Participants reported that the proposed suggestions felt realistic and motivating, and the system's tone was described as supportive rather than prescriptive. This confirms the added value of combining psychological stage detection with eater profiles when designing digital health interventions.

Beyond evaluations with on-off modules, the platform offers potential for longitudinal or multi-session studies. Having participants interact with the agent for multiple months each week would allow us to investigate the impact of adaptation in long-term interaction and to observe whether the agent's insight leads to long-term behavior change in participants. This is particularly relevant in domains such as behavioral change, where therapeutic or developmental outcomes evolve over time. The Greta 2.0 platform can support repeated interactions with consistent agent behavior, which allows researchers to investigate how factors such as rapport, trust, or user engagement develop across multiple sessions. Future improvements should include expanding the profile typology using real-world user data, introducing probabilistic or adaptive models for stage detection, and enabling longitudinal tracking to capture behavioral evolution over time. Longitudinal studies are particularly valuable in the context of motivational interviewing, where change can be gradual over multiple sessions³⁵. Tracking users across time allows researchers to observe how rapport with the agent develops, how motivation fluctuates in response to different strategies, and whether adaptive behaviors, such as aligned facial expressions or tailored prompts, have cumulative effects on the MI outcomes. Such studies can reveal long-term patterns of engagement and resistance that are not detectable in single-session experiments¹.

Moreover, the stage-profile matching, currently static, does not yet reflect the fluidity of human behavior. Users may simultaneously occupy different stages of change across dietary dimensions (e.g., being in the preparation stage for reducing meat while still in contemplation for sugar intake) or change profile during the course of a longitudinal study, which the current system cannot dynamically detect or adjust to. Additionally, they provide a framework for testing the
sustainability of agent-driven interventions, helping to evaluate whether virtual agents can support users in maintaining progress over time. Additionally, although the proposed SMART goals and motivational prompts were grounded in literature, empirical validation across diverse populations remains necessary to ensure effectiveness and inclusivity.

This work was partially funded by the ANR-DFG-JST Panorama, ANR-JST-CREST TAPAS (19-JSTS-0001-01), Enhancer (ANR-22-CE17-0036), and PEPR-Ensemble PC3 (ANR-22-EXEN-0004) projects. The evaluation presented in this paper was funded by the Idex Sorbonne Université as part of the State support for the Investments for the Future programs. This work was also supported by a French government grant managed by the Agence Nationale de la Recherche as part of the France 2030 program, reference ANR-22-EXEN-0004 (PEPR eNSEMBLE / MATCHING).

The authors have nothing to disclose.

1. Miller, W. R., Rollnick, S. *Motivational Interviewing: Helping People Change*. Guilford Press (2012).

2. Cameron, G. et al. Towards a chatbot for digital counselling. *Proc 31st Br Comp Soc Human Comp Interact Conf.* 1–7 (2017).

3. Denecke, K., Vaaheesan, S., Arulnathan, A. A Mental Health Chatbot for Regulating Emotions (SERMO) - Concept and Usability Test. *IEEE Transact Emerging Topics Comput.* 9 (3), 1170–1182 (2021).

4. Fiske, A., Henningsen, P., Buyx, A. Your Robot Therapist Will See You Now: Ethical Implications of Embodied Artificial Intelligence in Psychiatry, Psychology, and Psychotherapy. *J Med Internet Res.* 21 (5), e13216 (2019).

5. Bickmore, T. W., Pfeifer, L. M., Jack, B. W. Taking the time to care: empowering low health literacy hospital patients with virtual nurse agents. *Proc SIGCHI Conf Human Factors Comp Syst.* 1265–1274 (2009).

6. Fitzpatrick, K. K., Darcy, A., Vierhile, M. Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial. *JMIR Mental Health.* 4 (2), e19 (2017).

7. Ring, L., Bickmore, T., Pedrelli, P. An Affectively Aware Virtual Therapist for Depression Counseling. *ACM SIGCHI Conf Human Factors Comput Syst (CHI) Workshop Comput Mental Health* 1-4 (2016).

8. Bickmore, T. W. et al. Managing Chronic Conditions with a Smartphone-based Conversational Virtual Agent. *Proc 18th Int Conf Intelligent Virtual Agents*. 119–124 (2018).

9. Mercado, J., Espinosa-Curiel, I. E., Martínez-Miranda, J. Embodied Conversational Agents Providing Motivational Interviewing to Improve Health-Related Behaviors: Scoping Review. *J Med Internet Res.* 25, e52097 (2023).

10. Masterson Creber, R. et al. Motivational interviewing to improve self-care for patients with chronic heart failure: MITI-HF randomized controlled trial. *Patient Educ Couns.* 99 (2), 256–64 (2016).
11. Younsi, N., Pelachaud, C., Chaby, L. Beyond Words: Decoding Facial Expression Dynamics in Motivational Interviewing. Proc 2024 Joint Int Conf Comp Linguistics Lang Resources Evaluation. 2365-2374 (2024).

12. Galland, L., Pecune, F., Pelachaud, C. EMMI - Empathic Multimodal Motivational Interviews: Automatic Multimodal Features Extraction And Patient Type Definition. Lang Resources Evaluation 10.1007/s10579-025-09856-x (2025).

13. Galland, L., Pelachaud, C., Pecune, F. EMMI -- Empathic Multimodal Motivational Interviews Dataset: Analyses and Annotations. arXiv. 10.48550/arXiv.2406.16478 (2024).

14. Saga, T., Galland, L., Younsi, N., Pelachaud, C. Greta 2.0: Social Interactive Agent system, optimized for neural network integration. Proc ACM Int Conf Intelligent Virtual Agents. 1-10 (2025).

15. Younsi, N., Pelachaud, C., Chaby, L. Modiff-8 to Better Motivate: Live Adaptive Human-Socially Interactive Agent Interaction. SSRN. 10.2139/ssrn.5125133 (2025).

16. Galland, L., Pelachaud, C., Pecune, F. Tailored Conversations beyond LLMs: A RL-Based Dialogue Manager. ACL ARR. 10.48550/arXiv.2506.19652 (2025).

17. Velicer, W. F., DiClemente, C. C., Prochaska, J. O., Brandenburg, N. Decisional balance measure for assessing and predicting smoking status. J Personality Social Psychol. 48 (5), 1279-1289 (1985).

18. Steenstra, I., Nouraei, F., Arjmand, M., Bickmore, T. Virtual Agents for Alcohol Use Counseling: Exploring LLM-Powered Motivational Interviewing. Proc ACM Int Conf Intelligent Virtual Agents. 1-10 (2024).

19. Galvão Gomes da Silva, J. et al. Experiences of a Motivational Interview Delivered by a Robot: Qualitative Study. J Med Internet Res. 20 (5), e116 (2018).

20. Torre, I., Goslin, J., White, L. If your device could smile: People trust happy-sounding artificial agents more. Comp Human Behav. 105, 106215 (2020).

21. Wang, N., Gratch, J. Rapport and facial expression. 2009 3rd Int Conf Affective Comp Intelligent Interact Workshops. 1-6 (2009).

22. Guy, L., Jean-Noël, G., Val, M., Cross, E. S. Tell me more! Assessing interactions with social robots from speech. Paladyn, J Behav Robotics. 12 (1), 136-159 (2020).

23. Krämer, N., Kopp, S., Becker-Asano, C., Sommer, N. Smile and the world will smile with you—The effects of a virtual agent's smile on users' evaluation and behavior. Int J Human-Comp Studies. 71 (3), 335-349 (2013).

24. Huang, L., Morency, L. P., Gratch, J. Virtual Rapport 2.0. Intell Virtual Agents. 68-79 (2011).

25. Kramer Schmidt, L., Andersen, K., Nielsen, A. S., Moyers, T. B. Lessons learned from measuring fidelity with the Motivational Interviewing Treatment Integrity code (MITI 4). J Subst Abuse Treatment. 97, 59-67 (2019).

26. Madson, M. B., Mohn, R. S., Schumacher, J. A., Landry, A. S. Measuring Client Experiences of Motivational Interviewing during a Lifestyle Intervention. Meas Evaluat Counseling Dev. 48 (2), 140-151 (2015).

27. Burgoon, J. K. Expectancy Violations Theory. Int Encycl Interpersonal Comm. 1-9 (2015).

28. Baker, Z. G., Watlington, E. M., Knee, C. R. The role of rapport in satisfying one's basic psychological needs. Motivat Emot. 44 (2), 329-343 (2020).

29. Ryan, R. M., Deci, E. L. Self-Determination Theory. Encycl Quality Life Well-Being Res.
661 6229–6235 (2023).

662 30. Galland, L., Pelachaud, C., Pecune, F. SMART-DREAM: To Condition or Not to Condition A
663 Study on the Impact of LLM Conditioning on Motivational Interview Dialog Virtual Agent. Proc
664 ACM Int Conf Intelligent Virtual Agents. 1–9 (2025).

665 31. Wang, N., Gratch, J. Can Virtual Human Build Rapport and Promote Learning? *Proc 2009
666 Conf Artif Intelligence Educat.* 737–739 (2009).

667 32. Eeckhout, C., Francaux, M., Heeren, A., Philippot, P. Mesure de la balance décisionnelle
668 en vue de pratiquer une activité physique régulière (BDAP) : adaptation et validation
669 francophone de l’échelle Decisional Balance for Exercise. *Eur Rev Appl Psychol.* **63** (3), 185–191
670 (2013).

671 33. Carey, K. B., Maisto, S. A., Carey, M. P., Purnine, D. M. Measuring Readiness-to-Change
672 Substance Misuse Among Psychiatric Outpatients I. Reliability and Validity of Self-Report
673 Measures. *J Studies Alcohol.* **62** (1), 79–88 (2001).

674 34. Skantze, G. Turn-taking in Conversational Systems and Human-Robot Interaction: A
675 Review. *Comp Speech Lang.* **67**, 101178 (2021).

676 35. Cross, R., Greaves, C., Withall, J., Kritz, M., Stathi, A. A qualitative longitudinal study of
677 motivation in the REtirement in ACTion (REACT) physical activity intervention for older adults
678 with mobility limitations. *Int J Behav Nutrit Phys Activity.* **20** (1), 50 (2023).

679