Linguistic Complexity Loss in Text-Based Therapy (2021)

Abstract: The complexity loss paradox, which posits that individuals suffering from disease exhibit surprisingly predictable behavioral dynamics, has been observed in a variety of both human and animal physiological systems. The recent advent of online text-based therapy presents a new opportunity to analyze the complexity loss paradox in a novel operationalization: linguistic complexity loss in text-based therapy conversations. In this paper, we analyze linguistic complexity correlates of mental health in the online therapy messages sent between therapists and 7,170 clients who provided 30,437 corresponding survey responses on their anxiety. We found that when clients reported more anxiety, they showed reduced lexical diversity as estimated by the moving average type-token ratio. Therapists, on the other hand, used language of higher reading difficulty, syntactic complexity, and age of acquisition when clients were more anxious. Finally, we found that clients, and to an even greater extent, therapists, exhibited consistent levels of many linguistic complexity measures. These results demonstrate how linguistic analysis of text-based communication can be leveraged as a marker for anxiety, an exciting prospect in a time of both increased online communication and increased mental health issues.

## introduction the complexity loss paradox @xcite posits that individuals suffering from a wide range of illnesses tend to exhibit surprisingly periodic and predictable dynamics in their behavior, even though the diseases themselves are often called dis-orders. the paradox exists in patterns of behavior from diving in penguins @xcite 2000). in humans, the paradox has been observed in physiological systems from the indistinguishable tremors of parkinsonian patients @xcite to the cyclic oscillations of white blood cell counts in leukemia patients @xcite , but how the paradox manifests in one of our most important behavioral outputs-languagehas not been well-studied. in what form could the complexity loss paradox manifest in language? a line of psycholinguistics research, starting from the 1970s, has shown that the words people use can reveal important aspects of their mental health @xcite . for instance, vague and qualified speech can predict depression @xcite , diversity of word usage can indicate stress in interviews @xcite , and other work has found that lexical choices correlate with aphasia @xcite and suicide @xcite . in today's digital era, people suffering from mental illness have increasingly sought therapy services online, which can be more accessible than traditional clinicians' offices @xcite . many online platforms serve a large number of clients through text-based therapy, and so these conversations (when anonymized and used with consent) are well-suited for computational analysis. prior work has already used computational methods to predict symptom severity @xcite , measure counseling quality @xcite , and used topic models to support counselors during conversations @xcite . in this paper, we explore the complexity loss paradox in online therapy conversations of patients with anxiety. whereas much recent work using nlp to find linguistic indicators of mental health has turned to social media data @xcite , which is collected in a non-clinical context and may be unreliable, here we analyze a large-scale dataset of therapy conversations comprising 7,170 clients who sent more than three-million messages and answered 30,437 surveys about their mental health. moreover, therapy is a dynamic activity between clients and therapists, and so compared with related work that focuses solely on linguistic patterns of counselors @xcite @xcite , we investigate linguistic complexity in both clients and therapists. what linguistic complexity patterns in the language of clients and therapists during therapy reflect client mental health? ## dataset talkspace. in this work, we study text-based messages from talkspace, an online therapy platform with thousands of licensed therapists serving more than one-million users @xcite . anyone seeking therapy, henceforth clients, can sign up for a talkspace plan and get matched with a licensed therapist who will respond 5ˆa week through a chat room accessible by clients 24-7. to assess client mental health, counselors send surveys to clients at periodic intervals (on average, every three weeks). clients with different mental health conditions receive different surveys, with the most frequent surveys gauging anxiety and depression. in this work, we focus on anxiety, which clients self-reported using the generalized anxiety disorder 7-item scale @xcite ). clients answer how often in the last two weeks they were bothered by certain problems (e.g., trouble relaxing or feeling afraid as if something awful might happen) on a scale from 0-3 (0: not at all sure, 3: nearly every day). answers for the seven questions summed to a total score from 0-21, with 0 as the least anxious and 21 as the most anxious. dataset. our dataset (summarized by we take several precautions to reduce the probability of type i errors. upon receiving the dataset, we first followed @xcite and split the dataset by client into an exploratory dataset (80%) and a confirmatory dataset (20%). we used the exploratory dataset for running analyses and making design decisions, and then preregistered our analyses and expected results before accessing the confirmatory dataset to perform a full replication of experiments. as such, throughout the paper, we report numbers from the exploratory dataset, but only indicate statistical significance that holds on both the exploratory and confirmatory datasets. to further reduce potential false positives, because we run @xmath0, we apply the bonferroni correction @xcite and divide the traditional α = 0.05 by k so that we only consider statistical significance when p ă 0.001. ## linguistic complexity measures linguistic complexity is a multi-faceted topic for which there is no single agreed-upon measure for indexing complexity; instead, a toolbox of measures should be used to assess various linguistic features @xcite . in this work, we consider twelve well-known linguistic complexity measures, compiled from the work of @xcite , @xcite , and popular readability formulas. we group these twelve complexity measures into four broad categories: lexical diversity ( ), syntactic simplicity ( ), readability ( ), and prototypicality ( ). we list these complexity measures below, and direct the involved reader to the appendix for details. ## complexity correlates of anxiety we investigate how measures of linguistic complexity varied with reported client anxiety. for the 5,736 clients in the exploratory dataset, we retrieve all messages sent within one week prior to an anxiety survey response-henceforth conversations-totaling 24,287 conversation-survey pairs. for all conversation-survey pairs, we compute a value c m for each complexity measure m and both clients and therapist messages in that conversation. we then observe how each complexity measure changes with client anxiety (normalized for demographic variables) using a linear mixed model @xcite , which models random effects (variables that account for differences across individuals) as well as fixed effects in a general linear model. we predict anxiety using c m as a fixed effect, and, to control for demographic variables and individual differences, we also model time in therapy, gender, education, and age as fixed effects, and include therapist id and client id as random effects, with time as a random slope on client id. figure 1 (first and second panels) shows these results for client linguistic complexity c c and therapist linguistic complexity c t . for clients, most linguistic complexity measures had non-significant or slightly negative correlations with anxiety. moving average type-token ratio (mattr), which measures the ratio of unique words while accounting for sequence length, was the only significant predictor of anxiety. this correlation was negative, suggesting that clients showed less lexical diversity when they were stressed and providing some evidence that the complexity loss paradox might manifest in language-higher anxiety co-occured with less diverse word choice, a form of linguistic complexity loss. hd-d and mtld, the two other estimation techniques for lexical diversity, not decrease significantly with higher anxiety. hd-d samples words randomly and is thus unaffected by word order whereas mattr does account for word order, suggesting that the relationship between decreased word diversity and anxiety might exist in local linguistic structure rather than global word usage; mtld uses a previously established threshold based on books, whereas mattr does not use thresholding. these measures, which take varying approaches to estimating lexical diversity, relate differentially to anxiety; we leave investigating this phenomenon's underpinnings as future work. therapist language, on the other hand, showed higher reading difficulty, syntactic complexity, and age of acquisition when clients were more anxious, potentially reflecting a therapist's responsiveness to their client's current states. therapists listen closely to what clients say, and through reviewing survey results, build intuitions on clients' mental states. they also undergo extensive training before being licensed on talkspace, and so we speculate that when clients are more anxious, therapists are more likely to have detailed and involved discussions with clients, which can involve more complex language due to the sensitive nature of the conversation topics. in addition, both clients and therapists were more verbose (higher talkativeness) when clients were more anxious. in addition to c c and c t , we also investigate how difference in client and therapist language c t ´cc and similarity between client and therapist language |c t ´cc | correlate with anxiety (figure 1 , third and fourth panels). for c t ´cc , therapist language had higher measures of coleman-liau, flesch-kincaid, parse tree depth, and age of acquisition than client language when clients were more anxious. for |c t ´cc |, smaller differences in hd-d and mtld predicted lower client anxiety, suggesting that therapist and client lexical diversity was more similar when clients were less stressed. standard deviation (σ) range (∆) z σ c ‰ z σ t ? z ∆ c ‰ z ∆ t ? z σ c z σ t t p z ∆ c z ∆ t t p mattr -0.36 -0.33 -0.87 0.3842 -0.35 -0.29 -2.17 0.0298 hd-d -0.3 -0.32 0.53 0.5936 -0.3 -0.28 -0.83 0.4082 mtld -0.36 -0.35 -0.06 0.9561 -0.35 -0.34 -0.17 0.8665 dale-chall -0.46 -0.65 6.43* ă0.0001 -0.45 -0.51 1.91 0.0563 coleman-liau -0.68 -0.74 1.91 0.0558 -0.66 -0.61 -1.84 0.0664 flesch-kincaid -0.46 -0.93 15.68* ă0.0001 -0.47 -0.76 11.33* ă0.0001 parse tree depth -0.77 -0.89 4.12* ă0.0001 -0.74 -0.73 -0.48 0.6324 sentence length -0.44 -0.97 17.69* ă0.0001 -0.45 -0.79 13.54* ă0.0001 concreteness -0.49 -0.36 -4.64* ă0.0001 -0.48 -0.32 -6.57* ă0.0001 age of acquisition -0.48 -0.69 6.15* ă0.0001 -0.47 -0.51 1.44 0.1494 syllable count -0.44 -0.44 0.01 0.9913 -0.43 -0.36 -2.21 0.0273 talkativeness -0.31 -0.66 13.48* ă0.0001 -0.3 -0.58 11.88* ă0.0001 ## individual variation in linguistic complexity measures in addition to assessing whether linguistic complexity measures reflect mental health, we explore the extent to which individuals produce consistent values of complexity measures. was the complexity profile of a given client or therapist stable across their messages, or did it vary over time? because our dataset has a large number of individuals and a varying number of samples per individual, traditional analyses for exploring betweenindividual and within-individual variation (e.g., anova) were inadequate. therefore, we take an approach that compares within-individual variation with the expected variation from a random sample in the population, while accounting for the varying numbers of conversations per individual. for a given individual and complexity measure, we first compute that individual's standard deviation σ among their n conversations. then, we use σ to generate a z-score z σ by comparing σ with the distribution of standard deviations given by 1,000 random samples of the same size (same n conversations) from the entire population. if the distribution of z σ for all individuals did not significantly differ from n p0, 1q-the expected distribution of z-scores if there were no individual differencesthen individuals did not have consistent levels of that complexity measure. if the distribution of individual z-scores was significantly more negative than n p0, 1q, however, then individuals had more consistent values of that measure than expected and therefore had unique voices. we compute z σ , as well as z ∆ for ranges ∆, for both clients and therapists. ## conclusions we have studied linguistic complexity in online therapy conversations as it relates to mental health. we found that clients used less lexically diverse language as estimated by mattr when they were more anxious, supporting prior work that complexity loss due to anxiety may manifest in word diversity @xcite . in addition, we found that language of therapists also correlated with client anxiety and was generally more consistent than that of clients. our work shows that analyzing linguistic complexity can identify meaningful patterns in mental health, an important prospect in an era of both increased online communication and mental health illness @xcite .