Predicting Client Emotions and Therapist Interventions in Psychotherapy Dialogues (2024)

Abstract: Natural Language Processing (NLP) can advance psychotherapy research by scaling up therapy dialogue analysis as well as by allowing researchers to examine client-therapist interactions in detail. Previous studies have mainly either explored the clients’ behavior or the therapists’ intervention in dialogues. Yet, modelling conversations from both dialogue participants is crucial to understanding the therapeutic interaction. This study explores speaker contribution-based dialogue acts at the utterance-level; i.e, the therapist - Intervention Prediction (IP) and the client - Emotion Recognition (ER) in psychotherapy using a pan-theoretical schema. We perform experiments with fine-tuned language models and light-weight adapter solutions on a Hebrew dataset. We deploy the results from our ER model predictions in investigating the coherence between client self-reports on emotion and the utterance-level emotions. Our best adapters achieved on-par performance with fully fine-tuned models, at 0.64 and 0.66 micro F1 for IP and ER, respectively. In addition, our analysis identifies ambiguities within categorical clinical coding, which can be used to fine-tune the coding schema. Finally, our results indicate a positive correlation between client self-reports and utterance-level emotions.

## introduction understanding the therapists' intervention and the clients' emotional response is crucial to developing more effective treatments in psychotherapy @xcite . psychotherapy studies have emphasized the central role of client emotions and therapist interventions in predicting treatment outcomes from psychotherapy dialogues @xcite . however, since these studies are mainly based on client self-reports or human coding, they have been limited in scale and specificity @xcite . this has led to a greater push for research in natural language processing (nlp) for psychotherapy @xcite . recent studies have demonstrated the usefulness of nlp in automatically identifying key processes in psychotherapy, such as emotional processes, by modelling therapy dialogues @xcite . psychotherapy dialogues like any conversation data can provide meaningful information about the speaker actions when explained for the shortest sentences within a dialogue; i.e, utterance-level. dialogue act (da) classification is commonly used to attribute meaning or intention behind the utterances in a conversation @xcite . as noted by @xcite the task and content related distinctions are important in da labeling for conversational speech. speaker roles define dialogue contributions in psychotherapy domain @xcite ; e.g, clients often express their emotions during conversation and the therapists offer various interventions such as helping clients to process and regulate their emotions. a sample excerpt of such a dialogue is shown in nlp studies of therapy dialogues tend to focus on identifying either the therapists' interventions @xcite or the clients' emotions @xcite . however, for psychotherapy researchers it is important to provide both, so the interdependence between them can be observed and analysed. therefore, we design two application-oriented da classification tasks based on speaker roles; i.e, da classification for therapist utterances -intervention prediction (ip) and da classification for client utterances -emotion recognition (er). ## experimental setup for our experiments, we selected four pre-trained models which are capable of handling the hebrew language 1) xlm-roberta-base model @xcite , a multilingual lm based on the roberta architecture @xcite 2) hebert @xcite , a monolingual bert model trained on hebrew data, and 3) alephbert @xcite , another monolingual bert-based model trained on a larger hebrew vocabulary, and 4) a multilingual t5 model, i.e., mt5 @xcite . we also experiment with light-weight adapter solutions on the aforementioned models where only a small number of task specific parameters are trained. we use bottleneck adapters @xcite and mix-and-match (mam) adapters @xcite for training. as the pre-trained sentence transformer for setfit, we use the paraphrasexlm-r-multilingual-v1 with 2 epochs for the contrastive fine-tuning. all models are implemented in pytorch using the transformers v4.18.0 library @xcite and its adapter-transformers v3.0.0 extension @xcite . for the experiments, 872_gold is split into training (70%), development (10%) and test (20%) sets. learning rates (lr) and epochs are determined via hyper-parameter tuning on the development set. the learning rate is set in both dc i and dc b setups to 1e-4 for adapters, 1e-3 for setfit, 2e-6 for xlm, and 3e-5 for the remaining lms. the maximum token size per utterance j is set to 128. to account for potential variability in the results, we run each setup as a 10-fold stratified cross validation (cv). we also conducted approximate randomization tests @xcite to test for significance (α = 0.05) between the dc b and dc i versions of a model. we further implement partial class balancing to counter the skewed class distribution @xcite , see appendix a.2 figure 2 . we follow previous works and report model performance with micro f1 as well as cohen's kappa @xcite . evaluating our results on cohen's kappa (upper bound of human annotations) and f1 (ground truth by clinical experts) helps contextualize results (human vs model) and characterize the difficulty of the tasks. to calculate the correlation (equation 1 ) and its corresponding significance values, we use the pearson implementation from the scipy library @xcite @xcite . ## conclusion we evaluated various transformer models for predicting clients' emotions and therapists' interventions, where for the latter we follow current trends in psychotherapy by employing an established pan-theoretical schema. our results indicate that adapter solutions offer a lightweight alternative to fine-tuning. also, adapters for smaller language models can achieve competitive predictive performance compared to fully fine-tuned models of significantly bigger size, like mt5 (580 million parameters), while having only a fraction of computational cost, see appendix a.2 our models and pan-theoretical approach paves the way for researchers from multiple psychotherapy schools to auto-annotate session data, enabling the examination of pivotal treatment processes on a significantly broader scale. in the future, the conclusions from our data-driven approach can be used to a) advance clinical coding schemes b) study language behavior with treatment-level outcomes to monitor and improve clients' well being; e.g, using emotion coherence analysis. furthermore, our study lays the groundwork for developing support tools for therapists to provide feedback in higher temporal resolution and guidance on the interventions that lead to positive responses in clients. as next steps, we plan to extend the model to a multimodal level, integrating speech data as well. furthermore, we will have a closer look at dialogue models and investigate how to efficiently include the grounded dialogue history.