Working Alliance Transformer for Psychotherapy Dialogue Classification (2024)

Abstract: As a predictive measure of the treatment outcome in psychotherapy, the working alliance measures the agreement of the patient and the therapist in terms of their bond, task and goal. Long been a clinical quantity estimated by the patients’ and therapists’ self-evaluative reports, we believe that the working alliance can be better characterized using natural language processing technique directly in the dialogue transcribed in each therapy session. In this work, we propose the Working Alliance Transformer (WAT), a Transformer-based classification model that has a psychological state encoder which infers the working alliance scores by projecting the embedding of the dialogues turns onto the embedding space of the clinical inventory for working alliance. We evaluate our method in a real-world dataset with over 950 therapy sessions with anxiety, depression, schizophrenia and suicidal patients and demonstrate an empirical advantage of using information about therapeutic states in the sequence classification task of psychotherapy dialogues.

## introduction the working alliance between the therapist and the patient is an important measure of the clinical outcome and a qualitative predictor of therapeutic effectiveness in psychotherapy @xcite . the alliance entails a number of cognitive and emotional aspects of the interaction between these two agents, such as their shared understanding of the objectives to be attained and the tasks to be completed, as well as the bond, trust, and respect that will develop during the course of the therapy. while traditional methods to quantify the alliance depend on self-evaluative reports with point-scales valuation by patients and therapists about whole sessions @xcite , the digital era of mental health can enable new research fronts utilizing real-time transcripts of the dialogues between the patients and therapists. by analyzing the psychotherapy dialogues, we are interested in studying the usage of natural language processing technique to extract out turn-level features of the working alliance and see if it can help better inform us of the clinical condition of the patient. here we present working alliance transformer (wat), a transformer-based classification model to classify the psychotherapy sessions into different psychiatric conditions. our methods consists of a psychological state encoder that quantifies the degree of patient-therapist alliance by projecting each turn in a therapeutic session onto the representation of clinically established working alliance inventories, using language modeling to encode both turns and inventories, which was originally proposed in @xcite as an analytical tool. this allows us not only to quantify the overall degree of alliance but also to identify granular patterns its dynamics over shorter and longer time scales. we collated and preprocessed the alex street counseling and psychotherapy transcripts dataset @xcite , which consists of transcribed recordings of over 950 therapy sessions between multiple anonymized therapists and patients that belong to four types of psychiatric conditions: anxiety, depression, schizophrenia and suicidal. (the data publisher mentions that they have more clinical conditions other than the analyzed 4 classes, but due to the licensing and access limitations, we can only obtain the 4 classes we presented.) this multi-part collection includes speech-translated transcripts of the recordings from real therapy sessions, 40,000 pages of client narratives, and 25,000 pages of reference works. as open science and data sharing initiatives in the psychiatry domains become more prominent, we believe our methodologies can be adapted in a responsible way to a broader spectrum of clinical conditions. on this dataset, we evaluate quantitatively the effectiveness of this inference method in improving the classification / diagnosis capability of deep learning models to linguistically predict psychiatric conditions from therapy transcripts. lastly, we discuss how our approach may be used as a companion tool to provide feedback to the therapist and to augment learning opportunities for training therapists. ## methods we describe our pipeline in figure 1 . given the transcripts of a therapy session and the medical records of the patient. the dialogue are separated into pairs of turns as the timestamps. we can either choose to only use the turns by the patients, or by the therapists, or use both, as a paired input. empirically, the patients' turns are usually more narrative, as they are describing themselves, while the therapists' turns are usually more declarative, as they are usually confirming the patients, or leading conversations to certain topic. each patient response turn s p i followed by a therapist response turn s t i is treated as a dialogue pair. in total, these materials include over 200,000 turns together for the patient and therapist and provide access to the broadest range of clients for our linguistic analysis of the therapeutic process of psychotherapy. on the other hand, we have access to the working alliance inventory (wai), the clinical instrument. the modern wai consists of 36 statements in a self-report questionnaire which measures the therapeutic bond, task agreement, and goal agreement @xcite @xcite , where the since the original @xcite , the inventory has used parallel versions for clients and therapist with good psychometric properties and helped establish the importance of therapeutic alliance in predicting treatment outcomes. the modern version of the inventory consists of 36 questions, where the rater (i.e. the patient or the therapist) is asked to rate each statement on a 7-point scale (1=never, 7=always) @xcite . this inventory is disorder-agnoistic, meaning that it measures the alliance factors across all types of therapies, and provides a record of the mapping from the alliance measurement and the corresponding cognitive constructs underlying the measurement under a unified theory of therapeutic change @xcite . the inference goal is to compute a score that characterizes the working alliance given the clinical inventory, with for instance, a feature vector of 36 dimension that correspond to the 36 alliance algorithm 1 working alliance transformer (wat) for (i p j , i t j ) ∈ inventories (i p , i t ) do 6: aggregated feature x.append(x) 13: end for 14: obtain prediction @xmath0. operationally, the goal is to derive from these 36 items three alliance scales: the task scale, the bond scale and the goal scale. they measures the three major themes of psychotherapy outcomes: (1) the collaborative nature of the patient-therapist relationship; (2) the affective bond between therapist and patient, and (3) the therapist's and patient's capabilities to agree on treatment-related short-term tasks and long-term goals. the score corresponding to the three scales comes from a key table which specifies the positivity or the sign weight to be applied on the questionnaire answer when summing in the end. the full scale is simply the sum of the scores of the three scales. the key table is like a weighting matrix that specifies the directionalities of the scales. after computing the information regarding the predicted clinical outcome with our inferred working alliance scores, this feature vector highlights a bias towards what the clinicians would care about in the psychotherapy given the metrics provided by the working alliance inventory. we would then able to further use this information to potentially inform us of the psychiatric condition of a given patient. as such, we propose the working alliance transformer (wat), a classification model that utilizes an inference module that informs the downstream classifier where the current state is with respect to the therapeutic trajectory or landscape in the psychotherapy treatment of this patient. is this patients approaching a breakthrough? or is he or she susceptible to a rupture of trust? these therapeutic information about alliance can vary across clinical conditions, and thus, potentially beneficial to the diagnosis and monitoring of psychiatric disorders. algorithm 1 outlines the classification process. during the session, the dialogue between the patient and therapist are transcribed into pairs of turns. we denote the patient turn as s p i followed by the therapist turn s t i , as a dialogue pair. similarly, the inventories of working alliance questionnaires come in pairs (i p for the patient, and i t for the therapist, each with 36 statements). we compute the distributed representations of both the dialogue turns and the inventories with the sentence embeddings. the working alliance scores can then be computed as the cosine similarity between the embedding vectors of the turn and its corresponding inventory vectors. following @xcite , we use sentencebert @xcite and doc2vec embedding @xcite as our sentence embeddings for the working alliance inference. with that, for each turn (either by patient or by therapist), we obtain a 36dimension working alliance score. for the classification, we concatenate the 36-dimension working alliance scores computed from the current turn in the dialogue, along with the sentence embedding of the current turn, as our feature vector to fed into our transformer sequence classifier. the analytical features enabled by the working alliance inference are not only useful for the classification we investigate in this study but also other downstream tasks, such as predictive modeling and real-time analytics. in our case, the turns in a dialogue or monologue are fed into the sentence embedding sequentially as individual entries. and then, given the sentence embedding, we feed them each into the psychological state encoder that in-fer the psychological or therapeutic state of the dialogue at this turn. the encoder will generate a vector that characterizes the state, such as the 36dimension working alliance scores, corresponding to the 36 working alliance inventory items. then, the model aggregate both the sentence embedding feature vector and the psychological state vector. in this case, we concatenate them together as a first step. since we feed our input sentence by sentence (or turn by turn), we have a sequence of combined feature vector, which is then fed into a sequence classifier. we use the transformer @xcite as our classifier for its effectiveness in various sequence-based learning tasks, and potential interpretability from its attention weights. the output of this classification model is the predicted clinical condition of this sequence. the sequence of turns we feed to generate a label is a trimmed segment of the session of psychotherapy transcript. ## results here we present the transcript classification results. experimental setting. the psychotherapy dataset we evaluate is highly imbalanced across the four clinical conditions (495 anxiety sessions, 373 depression sessions, 71 schizophrenia sessions, and 12 suicidal sessions). if we directly train our models on this dataset, the classifier is likely to be highly biased towards the majority class. to correct for this imbalanceness issue, we are using the sampling technique. instead of going through the entire training data in epochs, we train the models in sampling iterations. in each iteration we randomly choose a class and then randomly sample one session from the class pool. before we sample the sessions, we split the dataset into 20/80 as our test set and training set. then during the training or the test phase, we perform the sampling technique for each iteration only in the fully separated training and test sets. then, for each sampled session, we feed into the classification model the first 50 dialogue turns of our transcript, turn by turn, and the sequence classifier will output a label predicting which psychiatric condition this session belongs to. model architecture. we evaluate two classifier backbones. the first one is the classical transformer model. for the multi-head attention module, we set the number of heads to be 4 and the dimension of the hidden layer to be 64. the dropout rates for the positional encoding layer and the transformer blocks are both set to be 0.5. the second backbone is a 64-neuron long short-term memory (lstm) network @xcite . ablation and baseline models. for each of the two classifiers, we compare three types of features as the input we feed into the sequence classifier component. the first one, the working alliance embedding, is the concatenated feature vector of both the sentence embedding vector and the psychological state vector (which in our case, is the 36-dimension inferred working alliance scores). the second type of feature, the working alliance score, is an ablation model which only uses the state vector (the working alliance score vector). the third type of feature, the embedding, is the baseline which only uses the sentence embedding vector directly. in other words, the working alliance score introduces the bias for wai. the sentence embedding doesn't. the working alliance embedding is the feature that combines both with concatenation. and since we have two sentence embeddings to choose from (the sentence bert and doc2vec), they each have 9 models in the evaluation pool. other than the classifier types (transformer or lstm), the embedding types (sentence-bert or doc2vec) and the feature types (working alliance embedding, working alliance scores, or simply sentence embedding), we also compared using only the dialogue turns from the patients, from the therapists, and from both the patients and the therapists. in the case where we use the turns from both the patients and the therapists, we consider them as a pair, and concatenate them together as a combined feature. this is as opposed to treating them as subsequent sequences, because we believe that the therapist's response are loosely semantic labels for the patient's statements, and thus, serve different semantic contexts that should be considered side by side, instead of sequentially, which would assume a homogeneity between time steps. training procedure. for all 12 models, we train them for over 50,000 iterations using the stochastic gradient descent with a learning rate of 0.001 and a momentum of 0.9. since the training set is relatively small for our neural network models, we observe some of the models exhibit overfitting at early stages before we finish the training. as a result, we report the performance of their checkpoints where they converge and have a plateau performance. then in the testing phase, we randomly sample class-balanced 1,000 samples. ## discussion our analytic approach reveals insightful features of therapeutic relationship and their usefulness in terms of clinical diagnosis merely based on the patient-doctor conversations. in our prior work, we observe systematic differences in the mean inferred alliance scores between patients and therapists, and also across disorders @xcite . however the in-session evolution of the inferred scores provide a much more interesting perspective, as shown in our dialogue sequence classification results. in particular, while all conditions show a systematic misalignment of scores between patients and therapists, this is significantly starker for suicidality, something observable in the mean as well as in the time trace for full and sub-scales, which can be useful for early detection of suicidal thoughts. as more and more successful applications of ai are deployed in clinical domains, there are many ethical considerations we practitioners of machine learning should be aware of and take into considerations. when dealing with patient data, the privacy and security is a top priority. following the suggestion of best practices from @xcite , all examples in this paper as well as the dataset we analyzed are properly anonymized with pre-and post-processing techniques. in addition, the dataset itself was sourced with proper license from pro-quest's alexander street platform. we remove all personally identifiable information (meta data, user name, identifiers, doctors' name) from the dataset. since the clinical domain of this work is men-tal health and psychological well-being, there are additional ethical considerations. emerging techniques in wearable devices, digital health records, brain imaging measurements, smartphone applications and social media are gradually transforming the landscape of the monitoring and treatment of mental health illness. however, most of these attempts are proof of concept as identified by this review @xcite , and requires extensive caution to prevent from the pitfall of overinterpreting preliminary results. the limitations of these prior studies, including our work here, reside in the difficulty of a systematic clinical validation and a uncertain future expectation of the technological readiness for patient care and therapeutic decision making approved by authorities. for instance, it was recently shown that despite the high predictability of statistical learning-based methods in analyzing large datasets in support of clinical decisions in psychiatry, existing machine learning solutions is highly susceptible to overfitting in realistic tasks which has usually a small sample sizes in the data, missing data points for some subjects, and highly correlated variables @xcite . these properties in real-world applications limits the out-of-sample generalizability of the results. ## conclusions in this work, we present a transformer-based classification model that characterizes the sequence of therapeutic states as beneficial feature to improve the classification of psychological dialogues into different psychiatric conditions. it combines the domain expertise from clinically validated psychiatry inventories with the distributed deep representations of language modeling provide a turnlevel encoding of working alliance at a turn-level resolution. we demonstrate on a real-world psychotherapy dialogue dataset that using this additional granular representation of the interaction dynamics between patients and therapists is beneficial both for interpretable post-session insights and linguistically diagnosing the patients. our results suggest that the inferred scores of therapeutic or psychological states of patientdoctor alignment can be useful in downstream tasks, such as diagnosis. although not a main focus in this work, future work would include a more systematic investigation of such downstream tasks, and exploiting the attention mechanism of the transformer blocks for interpretations.