Automatic Identification of Ruptures in Transcribed Psychotherapy Sessions (2021)

Abstract: We present the first work on automatically capturing alliance rupture in transcribed therapy sessions, trained on the text and self-reported rupture scores from both therapists and clients. Our NLP baseline outperforms a strong majority baseline by a large margin and captures client reported ruptures unidentified by therapists in 40% of such cases.

## introduction the client-therapist relationship within a psychotherapy treatment ('therapeutic alliance') is considered a powerful predictor of therapy success across treatment modalities and disorders @xcite . conversely, when a tension or a breakdown (rupture) occurs in the therapeutic alliance, it can often lead to unilateral termination of the treatment by the client or to poor psychotherapy outcomes @xcite . however, when alliance ruptures are recognised they can become meaningful therapeutic events @xcite . indeed, alliance ruptures have been found to be beneficial to the therapeutic process and outcome when they are recognized and followed by repair of the rupture @xcite and to hinder the process or outcome of therapy when they go unrecognized @xcite . challenges in capturing alliance rupture: most studies have explored alliance ruptures using selfreports at relatively low time resolution (once each session, typically weekly). however, ruptures may occur at higher time resolutions within a session @xcite . in addition, standardized subjective measures have critical shortcomings, including the extent of participants' self-insights, willingness to complete questionnaires, and the restricted choice of responses @xcite . recent studies have used within-session coding tools to detect ruptures moment-by-moment during a session, yielding important insights into the withinsession processes that lead to ruptures (e.g., @xcite . these insights have been used to train therapists to recognize ruptures when they happen @xcite . however, since observational human-coding is very labor intensive and expensive, these studies have focused on a small number of therapeutic components in a small sample of clients and at limited time points. benefits of capturing alliance rupture from text originating from the transcribed dialogue between therapist and client during therapy sessions include: contributions: to the best of our knowledge there is no work on capturing alliance rupture automatically from transcribed therapist or client utterances. recently goldberg et al. (2020) used 1,235 transcribed recorded sessions with client reported alliance to automatically predict per session alliance using the text from both therapist and client. they used four variants of a linear regression model with linguistic features from either the therapist or client. their best performing model was only 0.02 more accurate than a baseline predicting the average alliance rating. they also provided a list of unigrams which correlate most with high and low alliance scores respectively. here we make the following contributions: therapists and therapy: clients were treated by 52 therapists at various stages of their clinical training. clients were assigned to therapists in an ecologically valid manner based on therapist availability and caseload. 42 therapists treated one client each; eight treated two clients. each therapist received one hour of individual supervision biweekly and four hours of group supervision on a weekly basis. all therapy sessions were audiotaped for supervision by senior clinicians. supervision focused heavily on reviewing audiotaped case material and technical interventions designed to facilitate the appropriate use of therapist interventions. individual psychotherapy consisted of once-or twice-weekly sessions. the language of therapy was modern hebrew (mh). the dominant approach in the clinic includes a short-term psychodynamic psychotherapy treatment model (e.g., @xcite @xcite . on average, treatment length was 37 sessions (@xmath0.99, @xmath1. treatment was open-ended in length, but given that psychotherapy was provided by clinical trainees at a university-based outpatient community clinic, treatment duration was often restricted to 9 months. instruments and procedure: clients and/or therapists responded to several scales during the treatment, including the outcome rating scale @xcite and the post-session questionnaire (psq). in this work, we focus specifically on the alliance ruptures. alliance ruptures were assessed after each session with one question to the therapist and client: "did you experience any tension, misunderstanding, conflict or disagreement in the relationship with your client/therapist?". this item is answered subjectively on a 5-point likert scale from 1 ('not at all') to 5 ('constantly') by the two involved entities separately. following @xcite , a rupture was defined as any rating higher than 1 on the scale. the psq has been widely used in psychotherapy research and demonstrates sound psychometric properties, including predictive validity with a variety of process indices such as the working alliance inventory @xcite . here the psq mean score was 2.06 (@xmath2.43). transcription: due to the high associated cost manual transcriptions were conducted alternately (sessions 2, 4, 6, etc.). in cases where material was incomplete (e.g., questionnaire or poor recording quality), the following session was transcribed in-stead. the transcriber team was composed of seven graduate students in the university's psychology department. the transcribers went through a one day training workshop which included how to handle private/sensitive information; monthly meetings were held throughout the transcription process to supervise the quality of their work. the transcription protocol followed general guidelines, as described in @xcite . the word forms, the form of commentaries and the use of punctuation were kept as close as possible to the speech presentation. everything was transcribed, including word fragments as well as syllables or fillers (e.g., "ums", "ahs", "you know"). the transcripts included elisions, mispronunciations, slang, grammatical errors, non-verbal sounds (e.g., laughs, cry, sighs), and background noises. the rules were limited in number and simple and the format used several symbols to indicate comments (e.g. '[...]' to indicate the correct form when the actual utterance was mispronounced). there were 873 transcripts in total (the mean transcribed sessions per client was 12.56; @xmath3.93). the transcriptions totaled about four million words over 150,000 talk turns (i.e., switching between speakers). on average, there were 5800 words in a session, of which 4538 (78%; @xmath4.62; range 416-8176) were client utterances and 1266 (22%; @xmath5.99; range 160-6048) were therapist utterances. ## conclusion and future work in this work we focused on the task of automatically predicting alliance rupture between a therapist and a client from the language of therapy sessions. we collected and transcribed sessions between clients and their therapists, conducted in hebrew. we also obtained self-reported rupture labels for sessions by clients and therapists, used in clinical psychotherapy research. we tested baseline models leveraging the language used within a session to predict the occurrence of alliance rupture based on the perception of both the therapist and the client. we yield good performance and showcase the potential for using nlp for aiding therapists in identifying rupture during psychotherapy sessions. in the future we plan to build on our initial findings by incorporating contextual language models @xcite and i had to pick up my kind from his music lessons and i was busy and i asked my husband if he could take the child and he said he was busy and that i was the one who should give up. why do you think this is happening? i always have to run from one thing to another. he's busy with his own affairs. but what did you ask? i'm not in focus. no, no, it's okay, please continue. i feel like i was unlucky in life. yesterday i needed his help, but he is never there to help or hug me. i just don't have anyone who can do that for me. it's hard. i need someone who can support me. i never had such support in my life. i tried to get closer to him, but i feel that i am the only one whose needs are dismissed. he never gives up his needs. i feel so tired of all that. i have no desire to do anything. we talked in the last session about your difficulties to bring your needs. but last time you also said that you felt closer to him, didn't you? yes, i should try to get closer to him, i don't know, maybe i am wrong. how is it for you with other people? i don't know. (a) example of part of a session that was labelled by the client as 'rupture', but not from the therapist. logistic regression trained for the tr task predicted that there is a 'rupture'. i think i should be an employee instead of a boss. that pressure... i can't stand it. i'm not good at it. when a client comes i'm at the height of my enthusiasm, i have a lot of ideas on what to do, and i make plans & invest a lot of thought, i want it to be perfect, but something stops me, i cannot do it the way i want. you are afraid of disappointing. yes, exactly. i invest too much time in planning and then something stops me from doing it. i want it to be perfect and i'm working on the planning and i'm getting exhausted. i feel so much pressure to implement the plan & then i just become lazy and unable to actually do it. maybe if i was an employee then i would have cared less & the job would have been easier. sounds like there is a lot of pressure, also around the thought of finding another job. no, it's not about finding another job. but you also said -i feel that .. i have lots of strength and lots of motivation and i have many ideas, and suddenly when it comes to execution i can not find them. there is some kind of fatigue, laziness, i feel i do not have the strength, not the physical strength, the mental strength. something stops you. lets try to understand what it is. i tend to postpone everything. what do you postpone here? everything. what do you postpone here, in treatment? nothing specific. i just tend to postpone everything. (b) withdrawal rupture: a translated snippet of a session where the client reported a 'rupture', but not the therapist. logistic regression trained for the tr task predicted that there is a 'rupture', agreeing with the client. it's cold in here. cold? um .. this is, i'm coming here and the feelings are really .. confused, turbulent. i had a really completely confused week, i had a very very hard time at the end of the previous session. mmm.. it made me tense, and i was thinking if this form of treatment is good for me or if it's doing me any harm. i was looking for answers. i don't know if going deeper into things is good for me or if the right way for me is the opposite -to let go. mmmm and i met again that person i have worked with last summer. he is helping me to raise my self-confidence. sometimes that's what i need when i feel confused and unstable. i hear you. i also thought a lot about the hard things you talked about in the previous meeting. i felt overwhelmed and confused after the session. let's try to talk about what was it that you needed from me last time and that you felt that i did not provide. (c) confrontational rupture: this snippet of a translated transcribed session that was labelled by both client and therapist as a 'rupture'.