Discourse-Level Representations can Improve Prediction of Degree of Anxiety (2023)

Abstract: Anxiety disorders are the most common of mental illnesses, but relatively little is known about how to detect them from language. The primary clinical manifestation of anxiety is worry associated cognitive distortions, which are likely expressed at the discourse-level of semantics. Here, we investigate the development of a modern linguistic assessment for degree of anxiety, specifically evaluating the utility of discourse-level information in addition to lexical-level large language model embeddings. We find that a combined lexico-discourse model outperforms models based solely on state-of-the-art contextual embeddings (RoBERTa), with discourse-level representations derived from Sentence-BERT and DiscRE both providing additional predictive power not captured by lexical-level representations. Interpreting the model, we find that discourse patterns of causal explanations, among others, were used significantly more by those scoring high in anxiety, dovetailing with psychological literature.

## introduction anxiety disorders are one of the most prevalent mental health conditions, affecting an estimated 284 million people worldwide @xcite and with an estimated financial burden of $46.6 billion annually in the u.s. alone @xcite . this puts the impact of anxiety on par with depression @xcite , yet much less work in the nlp community has focused on detecting anxiety disorders as has been done for depressive disorders. one of the key characteristics of anxiety disorders is cognitive distortion @xcite , or an illogical reasoning in dealing with life events @xcite . the primary window into such distortions is language, including one's own explanatory style -the way they reason about the occurrence of events @xcite . explanatory style may not be well represented by single words or words in context (i.e., lexicallevel features). for example, consider the catastrophizing statement (i.e., worrying that a bad event will lead to an extreme outcome) "i'm sick. now i'm going to miss my classes and fail them all." @xcite . to see that "fail them all" is catastrophizing the event "i'm sick" requires understanding that the latter is a causal explanation for the expected falling behind. this is discourse-level information -semantics at the level of complete clausal statements or relating statements to each other (discourse relations) @xcite . here, we propose a language-based assessment of anxiety utilizing both lexical-level and discourselevel representations. we first compare models that leverage discourse-level representations alone. we then propose a dual lexical-and discourse-level (lexico-discourse) approach and evaluate whether the combination of both types of representations leads to improved performance. finally, we explore specific types of discourse relations that are thought to be associated with cognitive distortions, and look at their association with anxiety in order to illuminate what our lexico-discourse approach can pick up on at the discourse semantics level. our contributions include: (1) proposal of a novel user-level language assessment model that integrates both discourse-level and lexical-level representations; (2) empirical exploration of different discourse and lexical-level contextual embeddings and their value towards predicting the degree of anxiety as continuous values; (3) examination of the association between a person's anxiety and their discourse relation usage, finding that causal explanations are the most insightful for prediction; and (4) finding that to the best of our knowledge, this is the first model of anxiety from language specifically fit against a screening survey (rather than users self-declaring having experienced anxiety symptoms, or annotators perceiving the presence of the condition). ## related work anxiety is characterized by disruptive feelings of uncertainty, dread, and fearfulness, and is generally defined as anticipation of future threats @xcite . researchers have recently been turning to social media language as a potential alternative source for mental health assessment, investigating, e.g., depression @xcite @xcite , ptsd @xcite @xcite , and suicide risk @xcite @xcite . such an approach was also utilized in analyzing anxiety @xcite @xcite @xcite . work towards this goal include shen and rudzicz (2017) who attempted to classify reddit posts into binary levels of anxiety by lexical features and guntuku et al. (2019) who explored ngram associations with anxiety in twitter users. few have attempted to capture discourse-level information in such systems. while some have focused on cognitive distortions in patient-therapist interactions @xcite @xcite , none have attempted to combine discourselevel information with more standard lexical-level embeddings in studying ecological (i.e., everyday, happening in the course of life) online language patterns. for mental health tasks, state-of-the-art systems have primarily relied on contextual word-level information from transformers like bert @xcite and roberta @xcite @xcite . furthermore, @xcite improved mental health task performance by reducing the dimensions of contextual embeddings to approximately 1 12 of the original. here, we seek to establish the role of the contextual embeddings as well as propose and evaluate a model that integrates discourselevel modeling with contextual embeddings, motivated by the ability of discourse relations to capture cognitive distortions. lexical embeddings. amongst potential options for state-of-the-art auto-encoder language models, we consider bert @xcite and roberta @xcite . such selection is supported by empirical evidence; these two models have previously been found to result in top performance in related mental health assessment tasks @xcite . beyond the fact that these models have lead to state-of-theart performance in language understanding tasks, they are also known to capture some discourse information @xcite . thus, they form a very high benchmark to try to out-predict with discourse-level embeddings. overall model. the architecture of our prediction models is laid out in figure 1 . each model consists of a discourse submodel and lexical submodel, and the two following equations demonstrate the aggregation of representations in each submodel. d, m, u each denotes discourse unit, message, and user. the discourse submodel takes discourse units parsed from a message foot_0 to derive discourse-level embeddings, denoted as e d u (eq. 1), which are aggregated into message-level and then into a userlevel embedding, e u (eq. 2) : the lexical submodel takes the embeddings derived from the word-based transformer models as message-level representations and aggregates them to user-level. compose is the embeddings aggregation function at each step, which can be mean, min, or max. here we follow the practice from ganesan et al. (2021) and matero et al. ( 2021 ) and use the mean. 2 finally, the concatenation of the representations acts as input to our feed-forward network (ffn) that predicts the degree of anxiety. 3 theoretically relevant discourse dimensions. previous work has suggested open vocabulary (latent) embeddings of discourse relations (i.e., dis-cre, sentence-bert) are more powerful than explicitly defined relations @xcite , thus we utilize models that score specific type of relations (e.g., causal explanation) as a means to explain what the embeddings and models are able to capture. we evaluate four discourse relations relevant to anxiety. causal explanations are a statement of why an event happened. using the model of @xcite with f1 of approximately .87 over social media, we computed the percentage of the messages written by a user that contain causal explanation. counterfactuals imagine what could have happened as an alternative to actual events. using the model of @xcite , we calcu-late the proportion of the messages from each user that communicates counterfactual thoughts. finally, dissonance refers to situations in which one's stated behavior or belief contradicts a prior belief; consonance is its opposite concept. we use the roberta-based topic-independent classifier that evaluates whether a pair of messages composes dissonance @xcite . instead of assessing all pairs, we take two temporally adjacent messages (maximum distance of 2) to reduce computation time. ## dataset our primary dataset comprises 12,489 facebook users who took a personality questionnaire, including assessment of anxiety, and consented to share their status updates for academic research @xcite . the anxiety assessment consists of the anxiety facet of the neuroticism factor (johnson, 2014), which has shown to correlate with other measures of anxiety such as gad-7 @xcite and stai @xcite as well as have high convergence with anxiety disorders themselves @xcite . each user was asked the following five questions: get stressed out easily, am not easily bothered by things (inverse coded), am relaxed most of the time (inverse coded), fear for the worst, worry about things. on 1-5 likert scales ("very inaccurate." to "very accurate."). the responses to these questions are averaged together to form a continuous variable which determines the degree of anxiety. ## results and discussion we evaluate our models by disattenuated pearson correlation coefficient r dis @xcite between the model predictions and anxiety scores derived from the survey as our main metric, but include mean squared error as well. while phrase-bert performs well in isolation, in explaining discourse improvement. we shine light on what the model is able to capture in terms of discourse-level information by finding whether theoretically-related dimensions of cognitive distortions are associated with the models'. shows the cohen's d which was computed using the following equation, we see that all discourse dimensions were related to the score, but causal explanations, often related to overgeneralization, had the highest difference (e.g., "you know life is going to be permanently complicated when your in-laws start turning their backs on you like a domino effect."). this suggests that the causal explanation discourse relation may account for unique information to improve the overall results. ## conclusion anxiety is one of the most prevalent mental health disorders, and the ability to more accurately assess it in a way that can capture cognitive distortions (i.e., via discourse-level features) could lead to improved diagnostics and treatment of the condition. we analyzed the effects of using both discourseand lexical-level information within a single model for the assessment of degree of anxiety from facebook status updates. we found benefit from the discourse-level information beyond lexical-level contextual embeddings (i.e., transformer language models) that have been found to produce state-ofthe-art results for other mental health assessment tasks, motivating the idea that anxiety-based models can benefit from capturing not only contextual lexical information but also higher-level semantics at the level of thought patterns. lastly, we examined the effect of theoretically relevant discourse relations in assessing anxiety, discovering that causal explanation is the most informative.