Generation, Distillation and Evaluation of Motivational Interviewing-Style Reflections with a Foundational Language Model (2024)

Abstract: Large Foundational Language Models are capable of performing many tasks at a high level but are difficult to deploy in many applications because of their size and proprietary ownership. Many will be motivated to distill specific capabilities of foundational models into smaller models that can be owned and controlled. In the development of a therapeutic chatbot, we wish to distill a capability known as reflective listening, in which a therapist produces reflections of client speech. These reflections either restate what a client has said, or connect what was said to a relevant observation, idea or guess that encourages and guides the client to continue contemplation. In this paper, we present a method for distilling the generation of reflections from a Foundational Language Model (GPT-4) into smaller models. We first show that GPT-4, using zero-shot prompting, can generate reflections at near 100% success rate, superior to all previous methods. Using reflections generated by GPT-4, we fine-tune different sizes of the GPT-2 family. The GPT-2-small model achieves 83% success on a hold-out test set and the GPT-2 XL achieves 90% success. We also show that GPT-4 can help in the labor-intensive task of evaluating the quality of the distilled models, using it as a zero-shot classifier. Using triple-human review as a guide, the classifier achieves a Cohen-Kappa of 0.66, a substantial inter-rater reliability figure.

## introduction motivational interviewing (mi) is a counselling technique that is used to guide people towards behaviour change @xcite . mi has seen success in smoking cessation @xcite and alcohol consumption reduction @xcite , among other behaviours. our long-term goal is to automate mibased therapeutic conversations in smoking cessation @xcite . a key technique in mi (and many other talk conversation mi clinician: what are some things you don't like about your smoking addiction? client: i don't like making other people uncomfortable with my smoking. mi clinician (simple reflection): you don't enjoy making people feel uncomfortable with your smoking. mi clinician (complex reflection): you might be feeling self-conscious about your smoking. there has been recent work to automate the generation and classification of mi reflections using gpt-2 @xcite and gpt-3 @xcite . @xcite showed that a few-shot prompted gpt-3 generates mi reflections scoring over 89% success rate from human annotation and @xcite demonstrated a fine-tuned gpt-2 generates reflections which are scored by human reviewers as nearly identical to clinician curated reflections. furthermore, @xcite showed that a fine-tuned bert @xcite can classify reflections as acceptable at 80% success rate. in this work we explore the use of zero-shot prompting of @xcite to both generate and classify mi reflections. we use the high-quality reflections from the generator to fine-tune smaller, proprietary models. the latter provides greater privacy for sensitive health communications since the information pathways can be fully controlled when a model is owned by the operator. in collaboration with mi experts, we designed prompts to generate both simple and complex reflections and classify them with gpt-4. we present a method to distill the reflection generation capability from gpt-4. a dataset was created consisting of questions (that were presented to clients), their answers and generated gpt-4 reflections. these are used to fine-tune smaller language models, and we sought to determine the trade-off between size of the smaller model and its performance. in the larger context of a smoking-cessation chatbot that would use the generated reflections, there are situations when a simple reflection is called for, and other times when complex reflection is appropriate @xcite . for this reason we will distill two models, one for each type of reflection. figure 1 illustrates the overall approach used in this work. the fine-tuning datasets are created based on portions of transcripts from a previous chatbot created by the authors @xcite and simple or complex reflections generated by gpt-4. next, as a form of knowledge distillation, we fine-tune the gpt-2 @xcite family of models on the simple reflection or complex reflection dataset. to evaluate the student models we employ both human reviewers and use the gpt-4 model itself as a zero-shot classifier. that classification is done in two stages, the first to check for adherency to the principles of mi @xcite , and then to classify mi-adherent reflections as simple or complex. the idea of using a large foundational model as an zero-shot evaluator has just begun to appear in the literature @xcite and is not yet well studied. if it can be shown to be successful, it will reduce the costly human effort in determining the effectiveness of distilled and other models. previous works in mi reflection generation such as @xcite and @xcite have used human curated datasets to train classifiers. the contributions of this paper are: (1) stateof-the-art success rate in generation of reflections; (2) an example of end-to-end task-specific distillation from a foundational language model; and (3) demonstration of the effectiveness of using a foundational language models to evaluate reflections, which has the potential to reduce the amount of human labour in generative model work. ## method the goals of this paper are to generate high-success rate reflections using gpt-4, to distill that capability into smaller models and measure their success rate, and to determine how well a zero-shot promptbased gpt-4 model can evaluate the quality of reflections. this section describes the methods for each of these steps. ## results and analysis in this section we report the quality (using human review) of the reflections generated by the gpt-4 reflection generator. then, we compare the quality of the automatic evaluation using gpt-4 (with the evaluation prompt, described in section 3.5) with human review. finally, we present and discuss the performance of distilled gpt-2 models. the generation and evaluation results for all of these models are given in one large table , ## conclusion we have presented a method for generating simple and complex mi reflections using gpt-4, and shown that it is capable of near-perfect success, beyond the previous state of the art. we showed how to distill those capabilities into to smaller, gpt-2based student models, and that the range of sizes results in success rates ranging from 76% to 93%. one issue in distillation work is the labour to determine the success of the distilled models; we have shown that a classification prompt with gpt-4 as an evaluator is reliable. this paper provides a case study of distillation of a specific task from an expensive, privacy-challenged large foundational model into an owned, smaller pre-trained language model. the following is an interaction between a therapist and a client. act as the therapist and give a reflection to the client's response. the reflection must be a statement and not a question. the reflection must be a rephrasing of the client's response.