KMI : A Dataset of K orean Motivational Interviewing Dialogues for Psychotherapy (2025)

Abstract: The increasing demand for mental health services has led to the rise of AI-driven mental health chatbots, though challenges related to privacy, data collection, and expertise persist. Motivational Interviewing (MI) is gaining attention as a theoretical basis for boosting expertise in the development of these chatbots. However, existing datasets are showing limitations for training chatbots, leading to a substantial demand for publicly available resources in the field of MI and psychotherapy. These challenges are even more pronounced in non-English languages, where they receive less attention. In this paper, we propose a novel framework that simulates MI sessions enriched with the expertise of professional therapists. We train an MI forecaster model that mimics the behavioral choices of professional therapists and employ Large Language Models (LLMs) to generate utterances through prompt engineering. Then, we present KMI, the first synthetic dataset theoretically grounded in MI, containing 1,000 high-quality Korean Motivational Interviewing dialogues. Through an extensive expert evaluation of the generated dataset and the dialogue model trained on it, we demonstrate the quality, expertise, and practicality of KMI. We also introduce novel metrics derived from MI theory in order to evaluate dialogues from the perspective of MI.

## introduction in modern society, the issue of mental health is emerging as a critical problem, with an increasing demand for mental health services. this has led to a shortage of mental health workers available to meet the growing demand @xcite . thus, there has been considerable expansion in the investigation of ai-driven chatbots providing mental health assistance @xcite figure 1 : comparison between kmi and existing resources. kmi is the only dataset that is a non-english resource, based on mi theory, and not derived from transcripts. other datasets fall short in some of these aspects, limiting their suitability for chatbot applications. @xcite . however, there are many challenges in training such chatbots. it suffers from privacy issues, requires a considerable amount of cost and time for collecting data, and crucially, needs expertise in mental health. @xcite pointed out that many mental health chatbots are designed without any underlying theory for psychotherapy, casting doubt on their utility for mental health support. as a theoretical basis for enhancing expertise, motivational interviewing (mi) has been attracting increasing attention in developing mental health support chatbots @xcite @xcite . mi is a client-centered counseling technique to elicit behavior change by helping clients explore and resolve ambivalence @xcite . mi has been an active area of research intersecting the disciplines of psychotherapy and natural language processing (nlp), due to its effectiveness and well-structured behavioral coding systems such as miti (motivational interviewing treatment integrity) @xcite . few works have proposed mi datasets with annotations from mi behavioral coding systems. @xcite developed an mi dataset using automatic captioning of youtube and vimeo videos. however, the dataset is currently not publicly available due to ethical reasons. @xcite proposed an alternative of scraping data from online peer support forums. although these dialogues are available in abundance, they have shown significant differences from those collected from professional counselors in terms of mi, thus lacking expertise. in line with this problem, @xcite presented annomi, a dataset of 133 mi conversations that were professionally transcribed from mi demonstration videos and further annotated by experienced mi practitioners. despite its decent transcription and annotation quality, it significantly lacks in quantity due to the limited video sources and the intensive human labor required. in short, the mentioned datasets contain limitations to be used for training mi-boosted chatbots, such as lack of expertise or insufficient quantity. also, transcripts often include many onomatopoeic words and instances of stammering, which are undesirable in chatbots. furthermore, previous works have predominantly focused on english. while increasing language coverage is crucial in addressing the challenges of mental health support @xcite , research on other languages and cultures has been relatively understudied. to address these issues, we propose a novel framework to generate synthetic dialogues that simulate mi sessions using large language models (llms). we carefully design two agents, the therapist simulator and the client simulator, which alternately take turns to generate utterances based on few-shot in-context learning. to incorporate the expertise of professional therapists, we train an mi forecaster model, which predicts the next-turn therapist behavior. also, we focus on a non-english language, korean in our case foot_0 , and ground the dialogues in real-world contexts that reflect actual korean circumstances. consequently, we present kmi, a dataset of 1,000 korean motivational interviewing dialogues. as illustrated in figure 1 , kmi is the only dataset that overcomes the previously mentioned limitations of existing resources. kmi covers a wide range of concerns and anxieties common among koreans, while successfully integrating the mi strategy of professional therapists simultaneously. each therapist utterance within the dataset is annotated with a therapist behavior label. to the best of our knowledge, our work is the first attempt to construct an mi dataset using the generative capability of llms. we evaluate kmi through a comprehensive evaluation process by engaging professional counselors. we first validate the dataset's quality from the perspective of both mi and general dialogue systems. notably, for the mi-driven evaluation, we introduce novel metrics derived from mi theory to directly measure how well the conversation complies with the spirit of mi. then, we train a dialogue model using our dataset to assess its utility as a resource for training a chatbot for mental health support. evaluation results demonstrate that our dataset not only effectively captures the essence of mi, but also offers practical utility for chatbot development. our contributions are as follows. dialogue generation using llms recent research has increasingly focused on using llms to generate dialogues for various applications. kim et al. (2022) proposed a human-machine collaborative framework to build a large-scale dialogue dataset for training conversational agents to handle problematic content appropriately. chen et al. (2023a) used a small number of expert-written conversations as in-context examples to create synthetic multi-party conversations. chen et al. nlp applications in mi mi was developed as a technique to assist individuals in resolving ambivalence and committing to change @xcite , representing an evolution of client-centered therapy. along with the advancement in nlp, considerable research efforts have been underway to apply nlp techniques in the field of mi. several studies have been proposed to automatically classify a given utterance into one of the mi behavioral codes. while earlier approaches utilize linguistic features @xcite and recurrent neural network architectures @xcite @xcite , recent approaches make use of pre-trained language models such as roberta @xcite @xcite . some works adopt a multimodal approach, leveraging additional information such as speech features @xcite or facial features @xcite . @xcite demonstrated the potential of llms in boosting dialogues using the mi strategy. they observed that among the mi dataset curated from online platforms @xcite , 92.86% of the advice given by peers falls into the advise without permission category, which is mi non-adherent. to make the dataset more mi-consistent, they fine-tuned blenderbot @xcite and gpt-3 @xcite to rephrase these responses into more mi-adherent advise with permission responses. although this work demonstrated the possibility of leveraging llms in mi, the impact of rephrasing remains marginal, considering that its capability is restricted to revising the manner of speech and has limitations in modifying the content itself. in our work, we further exploit the generative ability of llms to generate the entire dataset from scratch. ## kmi: a dataset of korean motivational interviewing dialogues this section introduces a novel framework to generate realistic motivational interviewing dialogues. the overall framework is illustrated in figure 2 . first, we collect context data (section 3.1) that reflects the actual concerns and anxieties experienced by koreans. each context serves as the topic of each dialogue. next, we simulate an mi session (section 3.2) using a therapist simulator and client simulator. the resulting kmi dataset is presented in section 3.3. ## evaluation we evaluate the quality of the generated dataset (section 4.2) and the dialogue model fine-tuned with it (section 4.3). we compare our dataset with counselgpt 7 , the only korean counseling dataset to date that is created using openai api, and an-nomi. we translate annomi into korean using up-stage solar @xcite ) translation api 8 , in order to evaluate it in the context of korean. ## discussion and conclusion in this paper, we introduced a novel framework to generate synthetic motivational interviewing dialogues, along with kmi, the resulting dataset of 1,000 korean mi dialogues. through comprehensive evaluations, we demonstrated its quality and utility for chatbot development. for meaningful evaluation, we also proposed novel metrics derived from mi theory. our dataset has three main applications. first, it could be used for developing mental health chatbots grounded in mi theory. we trained a korean chatbot in this paper, where expert evaluation results and usage examples (see second, it could function as a labeled dataset of (utterance, mi label) pairs for classification and forecasting tasks. mi label accuracy of 96.0% demonstrates the reliability of the labels. third, it could serve as a reference for mi practitioners. full dialogues of therapy sessions are usually private, while mi textbooks provide only segments of dialogues. despite being synthetic, kmi can offer high-quality, full dialogues covering diverse topics. in addition, our generation method is potentially generalizable to other languages. korean-specific resources required in our framework are: (1) context data, (2) 3~4 high-quality examples for each mi label for few-shot prompting, and (3) a koreanto-english translation model. if these resources are available, our framework could be further used to boost non-english resources for psychotherapy.