Language and Mental Health: Measures of Emotion Dynamics from Text as Linguistic Biosocial Markers (2023)

Abstract: Research in psychopathology has shown that, at an aggregate level, the patterns of emotional change over time—emotion dynamics—are indicators of one’s mental health. One’s patterns of emotion change have traditionally been determined through self-reports of emotions; however, there are known issues with accuracy, bias, and convenience. Recent approaches to determining emotion dynamics from one’s everyday utterances, addresses many of these concerns, but it is not yet known whether these measures of utterance emotion dynamics (UED) correlate with mental health diagnoses. Here, for the first time, we study the relationship between tweet emotion dynamics and mental health disorders. We find that each of the UED metrics studied varied by the user’s self-disclosed diagnosis. For example: average valence was significantly higher (i.e., more positive text) in the control group compared to users with ADHD, MDD, and PTSD. Valence variability was significantly lower in the control group compared to ADHD, depression, bipolar disorder, MDD, PTSD, and OCD but not PPD. Rise and recovery rates of valence also exhibited significant differences from the control. This work provides important early evidence for how linguistic cues pertaining to emotion dynamics can play a crucial role as biosocial markers for mental illnesses and aid in the understanding, diagnosis, and management of mental health disorders.

## introduction language is inherently social-from the way in which we say things, the expressions we use and the things we choose to share, being impacted by our social environment and lived experiences. as our social environments have evolved over time, language has evolved to better support our communication needs and collaborative societies. therefore, language is also variable, as the way in which * work done while at the university of alberta. given the prominent role of language in human evolution from hunters-gathers to collaborative societies, and the large extent to which we rely on language today, it is not surprising that our mental health impacts our language usage. quantitative features in language (e.g., aspects which can be measured) have already been shown to indicate and help clinicians monitor the progression of mental health conditions (mhcs), acting as biomarkers. a linguistic biomarker is a language-based measure that is associated with a disease outcome or biology in general @xcite @xcite . some wellknown linguistic biomarkers include: the proportion of pronouns (indicator of depression, @xcite ), syntax reduction (anorexia nervosa, @xcite ), certain lexical and syntactic features (mild cognitive impairment and dementia, @xcite ; @xcite ), and semantic connectedness (schizophrenia, @xcite . also, the emotions expressed in text have been shown to correlate with mental health diagnosis. for example, more negative sentiment in text by individuals with depression @xcite @xcite . other work has shown that suicide watch, anxiety, and self-harm subreddits had noticeably lower negative sentiment compared to other mental health subreddits such as autism and asperger's @xcite . while language can be a biomarker for mental health, the substantial social nature of language has implications. notably, the tremendous variability in language use-especially across social groups-means that we should be skeptical about universal biomarkers; and instead realize that linguistic biomarkers alone are not capable of predicting mhcs. a vast amount of contextual and clinical information (often only available to an individual's physician) helps determine well-being, and sometimes linguistic markers can aid the process. further, linguistic biomarkers are more likely to be a stronger indicator among groups with commonalities; for example, when applied to people from the same region, culture, or medium of expression (e.g., social media platform). for example, social factors such as parental socioeconomic status, neighbourhood, and institutionalization (e.g., group foster care by government) influence speech markers such as lexical diversity; and social class influences syntactic complexity @xcite . therefore, it is more appropriate to consider language as a biosocial marker for health as it is influenced by both social and biological factors @xcite . as language is increasingly being studied as a biosocial marker for mental health -accelerated by the ease and availability of nlp tools and language data online -there are important ethical implications. we must consider the sociolinguistic factors of such markers to ensure less biased and more accessible tools in clinics @xcite . if social factors are not considered, then this limits the utility of systems and their ability to predict well-being as they may be capturing confounding variables. in that context, our goal is to understand the extent to which patterns of emotion change act as biosocial markers for mental health? emotion dynamics studies the patterns with which emotions change across time @xcite . emotion dynamics have been shown to correlate with overall well-being, mental health, and psychopathology (the scientific study of mental illness or disorders) @xcite @xcite . further, studying emotion dynamics allows us to better understand emotions, and has been shown to have ties with academic success @xcite , and social interactions (e.g., shyness) in children @xcite . emotion dynamics have been measured in psychology through self-report surveys over a period of time (e.g., 3 times a day for 5 days). using these self-reports of emotion over time, various metrics can quantify how emotions change over time (e.g., the average intensity, variability, etc.). however, there are several drawbacks of using self-report surveys (which we discuss at the end of section 2.1). another window through which emotion dynamics can be inferred is through one's everyday utterances. @xcite proposed the utterance emotion dynamics (ued) framework which determines emotion dynamics from the emotion expressed in text. there are several metrics in the ued framework inspired by those in psychology (e.g., average emotion, emotion variability, rise rate, and recovery rate). ties between emotion dynamics metrics and mental health have been shown in psychology, however it is not known whether this relationship similarly exists between emotion dynamics in one's utterances/language and mental health. in this paper, we examine whether ued act as biosocial markers for mental health. x (formerly twitter) provides an abundant amount of textual data from the public. foot_0 by considering tweets from users who have chosen to self-disclosed as having an mhc @xcite , we can analyse the differences in ued metrics across a diagnosis and the control population. we describe how utterance emotion dynamics (ued) metrics compare between different each of the seven diagnoses (adhd, bipolar, depression, mdd, ocd, ppd, ptsd) and a control group; and for each mhc we explore the following research questions: ## background how people feel on average and the patterns of emotional change over time are well supported as a unique indicator of psychological well-being and psychopathology @xcite . below we review related work on emotion dynamics and overall well-being in psychology and related fields. ## datasets we use a recently compiled dataset-twitter-stmhd @xcite . it comprises of tweets from 27,003 users who have self-reported as having a mental health diagnosis on twitter. the diagnoses include: depression, major depressive disorder (mdd), post-partum depression (ppd), post-traumatic stress disorder (ptsd), attentiondeficit/hyperactivity disorders (adhd), anxiety, bipolar, and obsessive-compulsive disorder (ocd). we describe the dataset in section 3.1, and our preprocessing steps in section 3.3. while our focus is on the relationship between emotion dynamics in tweets and mhcs, as a supplementary experiment, we also briefly explore the relationship between emotion dynamics in reddit posts and depression. 2 textual datasets associated with mhcs are not common, but it is beneficial to contextualize findings on tweets in comparison to findings on datasets from other modalities of communication. due to the inherent differences in domains, dataset creation process, sample size, etc., we expect that there will be differences, however there may also be potential similarities in how ued relate to mhcs. suhavi et al. (2022) identified tweeters who selfdisclosed an mhc diagnosis using carefully constructed regular expression patterns and manual verification. we summarize key details in the appendix (section a). the control group consists of users identified from a random sample of tweets (posted during roughly the same time period as the mhc tweets). these tweeters did not post any tweets that satisfied the mhc regex described above. additionally, users who had any posts about mental health discourse were removed from the control group. note that this does not guarantee that these users did not have an mhc diagnosis, but rather the set as a whole may have very few mhc tweeters. the number of users in the control group was selected to match the size of the depression dataset, which has the largest number of users. ## experiments to determine whether ued metrics from tweets can act as biosocial markers for psychopathology, we compare ued metrics for each mhc to the control group to see if they are statistically different. we compute the following ued metrics per user in each condition, following the speaker ued approach as described in @xcite . in this approach, ued metrics are computed per speaker by placing all their utterances in temporal order and computing ued on these ordered utterances. these metrics often rely on the characterization of the steady state or home base for emotions (regions of high probability). @xcite define the region pertaining to one standard deviation on either side of the mean as the home base. previous work computed speaker ued metrics for characters in movie dialogues @xcite , and for users on twitter during the pandemic @xcite . we summarize the ued metrics below (also see figure 1 ): for each user, we order their tweets by timestamp and used the emotion dynamics toolkit (vishnubhotla and mohammad, 2022b; hipson and mohammad, 2021) foot_3 to compute ued metrics (average emotion, emotion variability, rise rate, and recovery rate). we performed analyses for valence, arousal, and dominance. for word-emotion association scores we use the nrc valence, arousal, and dominance (vad) lexicon @xcite . 5 it includes ∼20,000 common english words with scores ranging from -1 (lowest v/a/d) to 1 (highest v/a/d). afterwards, we performed an anova to test for significant differences between groups in ued metrics, and post-hoc analyses to determine which groups specifically had significant differences from the control group. ## results to analyse potential differences across groups and the control group, an anova statistical test can be conducted, however several assumptions must be met. the three primary assumptions are: the data for each independent variable are approximately normally distributed, the data are independent of each other, and the distributions have roughly the same variance (homoscedasticity). we can assume the mean is normally distributed according to the central limit theorem, due to the large sample size (law of large numbers). since there are different tweeters in each mhc group we can assume the data are independent of each other. however, we note that people are largely not independent of each other e.g., having friends across groups, interacting with content from various mental health groups, etc. in our case, levene's test indicated that the assumption for homogeneity of variance was violated for all metrics and emotions (results in appendix b.1). as such, we used welch's anova test which is an alternative to anova when the equal variance assumption is not met. we conducted a 1-way welch's anova for each of the ued metrics (emotional average, emotional variability, rise-rate, and recovery rate) and emotions (valence, arousal, and dominance). we examined a total of @xmath0, see the first part of our analyses are omnibus ftests for significant differences between groups. this test cannot tell us which groups are different from each other, just rather that there is a difference among groups. for each combination of ued metrics and emotions (e.g., emotion variability for valence) there was a significant main effect which means we can conclude that at least one of the mental health diagnoses significantly differed. we show the degrees of freedom, f-statistic, p-value, and corrected effect size in next, we would like to know exactly which groups differed from the control group. in order to do this, we performed post hoc analyses for pairwise comparisons between groups for each metric across the three dimensions of emotions. we applied a games-howell correction since the assumption for homogeneity of variance was violated. in the following sections we detail how the ued metrics compare across mhcs compared to the control group. in we contextualize our results with previous findings in psychology and studies in nlp. we note that the relationship between patterns of emotion change and well-being for the dimensional emotions arousal and dominance are under-explored -our findings provide important benchmarks for these emotions and ued metrics more generally. ## results: biosocial aspects we now explore whether the ued metrics that were found to be significant indicators of certain mhcs, continue to remain discriminating even when accounting for certain social factors such as how popular a user's posts tend to be. metadata for the number of likes per post is available in the dataset. so, we simply extracted this information and computed the average number of likes per tweeter. most users had 0-4 likes per post in both the mhcs and the control group, up to an average of 8 likes per post. we compare the average ued per emotion dimension across the various bins of average likes. if a popularity measure does not impact ued, then we would expect to see consistent differences between the control group and an mhc across popularity bins. however, if a popularity measure does influence ued then the differences across mhcs and the control may disappear when controlling for popularity. in figure 2 , we show the two ued metrics (e.g., average emotion intensity and emotional variability) for valence, across various bins of average likes per tweeter for each group. in the appendix (section d, e, and f), we show the remaining results for arousal and dominance. if a bin had less than ten users in it (e.g., average of 6-8 likes per post), then we did not plot a value for it. in each figure the control group is the blue line (no shapes). for mhcs which significantly differed from the control for an emotion and ued pair, we show the mean difference in appendix g. if this difference between an mhc and the control remains fairly consistent across ranges of average likes, this ued is most likely not influenced by this social aspect. average valence: in the previous section, we showed that average valence is markedly lower for adhd, mdd, and ptsd. from figure 2 (a), we observe that the average valence for these mhcs is markedly lower than control for various average-likes bins. (we also observe markedly greater differences when average likes is in the [6,8) range.) valence variability: in the previous section, we showed that valence variability is markedly higher for all mhcs except ppd. from figure 2 (b), we observe that the valence variability for these mhcs is markedly higher than control for various average-likes bins. (we also observe markedly lower variability for control when average likes is in the [6,8) range.) valence rise rate: in the previous section, we showed that valence rise rate is markedly higher for mdd, ptsd, and depression. from figure 3 (a) we observe that the valence rise rate for these mhcs is markedly higher than control for various average-likes bins. (we also observe markedly lower valence rise rate for control when average user likes is in the [6,8) range.) valence recovery rate: in the previous section, we showed that valence recovery rate is markedly higher for bipolar, mdd, ptsd, and depression. from figure 3 (b) we observe that the valence recovery rate for these mhcs is markedly higher than control for various average-likes bins. (we also observe markedly lower recovery rate for control when average user likes is in the [6,8) range.) thus, overall, we observe that for the valence ueds which showed significant differences across mhcs and control, these differences do not disappear when accounting for popularity measures such as the average number of likes a user's posts get. similar trends overall were found for arousal and dominance (appendix e and f). ued for all three emotion dimensions appear to be robust indicators for various mental health conditions, even when users have some varying social characteristics. ## conclusion we showed for the first time that there are significant relationships between patterns of emotion change in text written by individuals with a self-disclosed mhc compared to a control group. specifically, we found significant differences in four utterance emotion dynamics (ued) metrics (average emotion, emotion variability, rise rate, and recovery rate) across three emotion dimensions (valence, arousal, and dominance) for 7 mhcs. our findings provide important contextual information of overall well-being and supporting indicators (in addition to other assessments) to clinicians for diagnosis detection and management. looking ahead, we plan to explore ued metrics across different textual genres, regions, languages, and demographics (such as socioeconomic status), in collaboration with clinicians. through such a partnership, ued metrics could be studied in the context of clinical information as well. lastly, exploring new ued metrics that correlate with psychopathology is a promising direction for future work.