WWBP - SQT -lite: Multi-level Models and Difference Embeddings for Moments of Change Identification in Mental Health Forums (2022)

Abstract: Psychological states unfold dynamically; to understand and measure mental health at scale we need to detect and measure these changes from sequences of online posts. We evaluate two approaches to capturing psychological changes in text: the first relies on computing the difference between the embedding of a message with the one that precedes it, the second relies on a “human-aware” multi-level recurrent transformer (HaRT). The mood changes of timeline posts of users were annotated into three classes, ‘ordinary,’ ‘switching’ (positive to negative or vice versa) and ‘escalations’ (increasing in intensity). For classifying these mood changes, the difference-between-embeddings technique – applied to RoBERTa embeddings – showed the highest overall F1 score (0.61) across the three different classes on the test set. The technique particularly outperformed the HaRT transformer (and other baselines) in the detection of switches (F1 = .33) and escalations (F1 = .61).Consistent with the literature, the language use patterns associated with mental-health related constructs in prior work (including depression, stress, anger and anxiety) predicted both mood switches and escalations.

## introduction detecting shifts in mental health from language use could assist in identifying episodes of mental ill health and providing in-time treatment for conditions such as depression or anxiety. the accessibility and abundant usage of social media @xcite in comparison to traditional healthcare data (e.g. hospital visits) is enabling first steps toward unprecedented assessment and understanding of mental health, including detection of elevated risks @xcite @xcite . however, most language datasets for mental health classification are annotated statically such that a person has just one label across all of their language @xcite . longitudinal language datasets can help analyze the mental state of a person over time @xcite @xcite , but also open the door for many sequential, differencing, and time-series modeling techniques. here, we explore two types of modeling techniques that can capture changes over time: humanaware recurrent transformers @xcite and difference embeddings. these techniques were used as part of the wwbp-sqt-lite foot_0 system for the clpsych 2022 shared tasks @xcite : (task a) modeling user state changes over time @xcite , and (task b) the suicide risk associated with the user @xcite , our contributions are as follows: (a) evaluation of human-aware recurrent transformers (hart) and difference embeddings for task a (b) exploring sota methods for predicting state escalations and switches, and (c) exploring theoretically related linguistic assessments. internal train & validation sets. 119 (80%) randomly chosen users were sampled to form an internal train set, and the remaining 30 users were used for validation set. this resulted in 3, 974 posts (78%) over 164 timelines in train set and 1, 169 posts over 40 timelines in the validation set. ## conclusion we presented two approaches to detecting mental state changes in users through (a) a recurrent transformer model (hart) that encodes messages within context of previous ones and (b) a logistic regression model that relies on roberta difference embeddings along with previous and current text representations to capture change in language over time. compared to using other representation types, such as theoretically motivated (hyplex) or traditional open vocabulary features (n grams, topics), both approaches saw improved model performance when predicting changes over time. further, we found that theoretically relevant lexical scores had large associations with the change patterns. it showed emotional stability correlating with no change, and loneliness, depression, stress, anxiety and anger being associated with the mood change.