Language and Mental Health: Measures of Emotion Dynamics from Text as Linguistic Biosocial Markers
Authors: Daniela Teodorescu
Tiffany Cheng
Alona Fyshe
Saif M. Mohammad
Venue: EMNLP 2023 Main (2023)
Keywords: Utterance Emotion Dynamics
Mental Health
Social Media
Sentiment Analysis
Emotion Arcs
Emotional Reactivity
Lexicons
TL;DR: We study the relationship between patterns of emotion change in tweets (through metrics e.g., average emotion, variability, rise and recovery rate) and mental health disorders, finding significant differences by the user's self-reported diagnosis.

Abstract: Research in psychopathology has shown that, at an aggregate level, the patterns of emotional change over time---emotion dynamics---are indicators of one's mental health. One's patterns of emotion change have traditionally been determined through self-reports of emotions; however, there are known issues with accuracy, bias, and convenience. Recent approaches to determining emotion dynamics from one's everyday utterances, addresses many of these concerns, but it is not yet known whether these measures of utterance emotion dynamics (UED) correlate with mental health diagnoses. Here, for the first time, we study the relationship between tweet emotion dynamics and mental health disorders. We find that each of the UED metrics studied varied by the user's self-disclosed diagnosis. For example: average valence was significantly higher (i.e., more positive text) in the control group compared to users with ADHD, MDD, and PTSD. Valence variability was significantly lower in the control group compared to ADHD, depression, bipolar disorder, MDD, PTSD, and OCD but not PPD. Rise and recovery rates of valence also exhibited significant differences from the control. This work provides important early evidence for how linguistic cues pertaining to emotion dynamics can play a crucial role as biosocial markers for mental illnesses and aid in the understanding, diagnosis, and management of mental health disorders.

Language and Mental Health:
Measures of Emotion Dynamics from Text as Linguistic Biosocial Markers
Daniela Teodoreseu 1:2; Tiffany Cheng', Alona Fyshe Saif M. Mohammad
DDppt. Computing Science, Alberta Machine Intelligence Institute (Amii), University of Alberta "MaiNLP, Center for Information and Language Processing, LMU Munich, Germany
CCrleeon University
Dept.. Psychology, University of Alberta
National Research Council Canada
wttp:irre.oooaaaaabbreet.. ca,, tiffany. waa.nn..ceeg@crrettoooo ca, saif, mohammadenro-cnr...... cc
Abstract
use it has adapted to cultures and communi- ties around the world, and it is influenced by an individual's experiences.
Research in psychopathology has shown that, aggregate level, the patterns emotional change over time-emotion dynamics- are in- dicators of one's s mental health. One's 's pat- terns of emotion change have traditionally been determined through self-reports of emotions; however, there are known issues with accu- racy, bias, and ease of data collection. Recent approaches to determining emotion dynamics from one's everyday utterances addresses many of these concerns, but it is not yet known whether these measures of ulterance emotion dynamics (UED) correlate with mental health diagnoses. Here, for the first time, we study the relationship between tweet emotion dynam- ics and mental health disorders. We find that each of the UED metrics studied varied by the user's self-disclosed diagnosis. For example: average valence was significantly higher (i.e.. more positive text) in the control group com- pared to users with ADHD, MDD, and PTSD. Valence variability
Given the prominent role of language in human evolution from cunters-gathed to collaborative so- cieties, and the large extent to which we rely on language today, it is not surprising that our mental health impacts our language usage. Quantitative features in language (c.g., aspects which can be measured) have already been shown indicate and help clinicians monitor the progression of men- tal health conditions (MHCs), acting as biomark- ers. A linguistic biomarker is a language- measure that associated with a disease outcome or biology in general (Ballman, 2015; Gagliardi and Tamburini, 2022; Lena, 2021). Some well- known linguistic biomarkers include: the propor- tion of pronouns (indicator of depression, Koops et al. (2023)), syntax reduction (Anorexia Nervosa, Cuteri et al. (2022)), certain lexical and syntactic features (mild cognitive impairment and demen- tia, Calzà et al. (2021); Gagliardi and Tamburini (2021)), and semantic connectedness (schizophre- nia, Corcoran et al. (2020). Also, the emotions expressed in text have been shown to correlate with mental health diagnosis. For example, more neg- ative sentiment in text by individuals with depres- sion (De Choudhury al., 2013; Seabrook et al., 2018; De Choudhury et al., 2021). Other work has shown that suicide watch, anxicty, and self-harm subreddits had noticeably lower negative sentiment compared to other mental health subreddits such as Autism and Asperger's (Gkotsis et al., 2016).
significantly lower in the control group compared to ADHD, depres- sion, bipolar disorder, MDD, PTSD, and OCD but not PPD. Rise and recovery rates of valence also exhibited significant differences from the control. This work provides important early evidence for how linguistic cues pertaining to emotion dynamics can play crucial role as biosocial markers for mental illnesses and aid in the understanding. diagnosis, and manage- ment of mental health disorders.
1 Introduction
Language inherently social from the way in which we say things, the expressions we use and the things we choose to share, being impacted by our social environment and lived experiences. As our social environments have evolved over time, language has evolved to better support commu- nication needs and collaborative societies. There- fore, language also variable, as the way in which Work done while at the University of Alherta.
While language can be biomarker for mental health, the substantial social nature of language has implications. Notably, the tremendous vari- ability in language use-esspcially across social groups- that we should be skeptical about universal biomarkers; and instead realize that lin- guistic biomarkers alone are not capable of pre- dieting MHCs. vast amount of contextual and

---

clinical information (often only available to an indi- vidual's physician) helps determine well-being, and sometimes linguistic markers can aid the process, Further, linguistic biomarkers are more likely to be a stronger indicator among groups with commonal- ities; for example, when applied to people from the same region, culture, or medium of expression ge.g.. social media platform). For example, social factors such parental socioeconomic status, neighbour- hood, and institutionalization (e.g., group foster care by govermment) influence speech markers such as lexical diversity; and social class influences syn- tactic complexity (Lena, 2021). Therefore, it is more appropriate to consider language as a bioso- cial marker for health as it is influenced by both social and biological factors (Lena, 2021).
can be inferred is through onc's everyday utter- ances. Hipson and Mohammad (2021) proposed the Utterance Emotion Dynamics (UED) frame- work which determines emotion dynamics from the emotion expressed in text. There are several metrics in the UED framework inspired by those in psychology (e.g., average emotion, emotion vari- ability, rise rate, and recovery rate). Ties between emotion dynamics metries and mental health have been shown in psychology, however not known whether this relationship similarly exists between emotion dynamics in onc's utterances/anguagee and mental health.
In this paper, we examine whether UED act as biosocial markers for mental health. X (formerly Twitter) provides an abundant amount of textual data from the public.' By considering tweets from users who have chosen to self-disclosed as having an MHC (Suhavi et al., 2022), we can analyse the differences in UED metrics across a diagnosis and the control population. We describe how utterance emotion dynamics (UED) metrics compare between different each of the seven diagnoses (ADHD, bipolar, depression, MDD, OCD, PPD, PTSD) and a control group; and for each MHC we explore the following research questions:
As language is increasingly being studied a biosocial marker for mental health accelerated by the ease and availability of NLP tools and language data online- - there are important ethical implica- tions. We must consider the sociolinguistic factors of such markers to ensure less biased and more accessible tools in clinics (Lena, 2021). If social factors are not considered, then this limits the util- ity of systems and their ability to predict well-being as they may be capturing confounding variables.
In that context, our goal is to understand the ex- tent to which patterns of emotion change act as biosocial markers for mental health? Emotion dy- namics studies the patterns with which emotions change across time (Kuppens and Verduyn, 2015, 2017). Emotion dynamics have been shown cor- relate with overall well-being, mental health, and psychopathology (the scientific study of mental ill- ness or disorders) (Kuppens and Verduyn, 2017; Houben et al., 2015; Silk et al., 2011; Sperry et al.. 2020). Further, studying emotion dynamics al- lows us to better understand emotions, and has been shown to have ties with academic success (Graziano et al., 2007; Phillips et al., 2002), and social interactions (e.g., shyness) in children (Sosa- Hernandez et al., 2022).
1. Does the average emotional state differ between the MHC and the control group?
2. Does emotional variability differ between the MHC and the control group?
3. Does the rate at which emotions reach peak emotional state (i.c., rise rate) differ between the MHC and the control group?
4. Does the rate at which emotions recover from peak emotional state back to steady state (i.e., recovery rate) differ between the MHC and the control group?
We explore each of the above research questions for three dimensions of emotion valence, arousal, and dominance - further building on the findings in psychology which have traditionally focused only on valence. Our work provides baseline mea- sures for UEDs across MHCs and insights into new linguistic biosocial markers for mental health. These findings are important for clinicians because they provide a broader context for overall well- being and can help contribute to the carly detection, diagnosis, and management of MHCs.
Emotion dynamics have been measured in psy- chology through self-report surveys over period of time (e.g.. 3 times day for 5 days). Using these self-reports of emotion over time, various metrics can quantify how emotions change over time (e.g.. the average intensity, variability, etc.). However, there are several drawbacks of using self-report sur- veys (which we discuss at the end of Section 1). Another window through which emotion dynamics
Sinne this work began hefore the name change, we use the terms Twitter and tweets in this paper.

---

2 Background
Emotion Reactivity & Psychopatholoy: Research has found that individuals with psychopathologies tend to take longer to recover from differing emo- tional states (i.e., emotional resetting or recov- ery rate) than healthy individuals (Kragel et al., 2022). That is, difficulty moving between emo- tional states is associated with lower psychological well-being. Houben et al. (2015) also proposed that high emotional reactivity and slow recovery baseline states is maladaptive emotional pattern indicative of poor psychological well-being and psychopathology. In other words, people with poor psychological health may be highly reactive, emo- tionally volatile, and take a longer time to return to baseline state.
How people feel on average and the patterns of emotional change over time are well supported as a unique indicator of psychological well-being and psychopathology (Houben et al., 2015). Below we review related work on emotion dynamics and overall well-being in psychology and related fields.
2.1 Emotion Dynamies
Affective chronometry is a growing field of re- search that examines the temporal properties of emotions (i.e., emotion dynamics) and increasingly its relation to mental health and well-being have been studied (Kragel et al., 2022). Within this field, emotion reactivity is the threshold at which a response is elicited and the responses' amplitude and duration can be further deconstructed into the rise time to the peak response (i.e., rise rate), the maintenance time of the peak response, and the recovery time to baseline (i.e., recovery rate) (Davidson, 1998). Emotion variability is how vari- able an emotion is in terms of its intensity over time and in response to different stimuli. This variation can occur over multiple timescales (e.g., seconds, hours, days, weeks; Kragel et al., 2022). These emotion dynamic metrics are proposed to be pre- dictive of affective trajectories over time and in distinguishing between affective disorders (Kragel et al., 2022). Also, emotion dynamics contribute to maladaptive emotion regulation patterns and poor psychological health (Houben et al., 2015). The meta-analysis by Houben et al. (2015) has indi- cated that the timescale of emotion dynamics does not moderate the relation between emotion dynam- ics and psychological well-being. Therefore, the relationship between psychological well-being and emotion dynamics occurs whether it is examined over seconds, days, months, and so on.
Emotion Variability & Psychopathology: The Houben et al. (2015) meta-analysis findings also indicate that higher emotional variability is related to lower psychological well-being. In particular, variability was positively correlated with depres- sion, anxiety, and other psychopathologies (e.g.. bipolar, borderline personality disorder, etc.). This is supported by Heller et al. (2018) who found that higher positive and negative affect variability was associated with higher levels of depression and anx- iety, these effects persisted for anxiety even after controlling for average positive affect. In contrast, variability was no longer associated with depres- sion after controlling for average affect and the rate of recovery to baseline. This effect was attributed to anhedonia (the reduced ability to feel plcasure) which is a common symptom of depression that leads to reduced emotionality.
Overall, emotion dynamics research suggests that one's average emotional state, emotional vari- ability, rise rate, and recovery rate may vary by their mental health. Preliminary research suggests that these metries may also vary across different mental illnesses or psychopathologies. However, research on emotion dynamics within psychology and related fields has heavily relied on self-report measures and ecological momentary assessments (EMA). Affective self-report measures are subject to biases (Kragel et al., 2022) and thus carry certain limitations (i.c., social pressures to be perceived as happy). Additionally, data collection with these methods is time-intensive thus, sample size and study length are limited. Another window through which emotion dynamics can be inferred is through one's 's utterances (Hipson and Mohammad, 2021).
Average Emotional State & Psychopatholoyy: Av- erage or baseline emotional states are related to well-being and mental illnesses. Due to the maladaptive (i.e., dysfunctional) nature of psy- chopathology, those with mental illnesses tend to have more negative emotional baselines. For exam- ple, Heller et al. (2018) found that a higher average positive affeec is associated with lower levels of depression but not anxiety, and higher average negative affect was related to increased depression and anxiety. As well, those with post-traumatic stress disorder (PTSD) have reported lower aver- age positive affect (Pugach et al., 2023).

---

2.2 Utterance Emotion Dynamies
While our focus is on the relationship between emotion dynamics in tweets and MHCs, as a sup- plementary experiment, we also briefly explore the relationship between emotion dynamics in Reddit posts and depression." 2 Textual datasets associated with MHCs are not common, but it is beneficial to contextualize findings on tweets in comparison to findings on datasets from other modalities of communication. Due to the inherent differences in domains, dataset creation process, sample size, etc., we expect that there will be differences, how- ever there may also be potential similarities in how UED relate to MHCs.
Early computational work on emotion change sim- ply created emotion ares using word-emotion asso- ciation lexicons (Mohammad, 2011; Reagan et al., 2016). Teodorescu and Mohammad (2023) pro- posed mechanism to evaluate automatically gen- erated emotion ares and showed that lexicon-based methods obtain rear-perfect correlations with the true emotion ares. The Utterance Emotion Dy- namics (UED) framework uses various metrics in- spired by psychology research to quantify patterns of emotion change from the emotion expressed in text (from the emotion arcs). Using a person's utter- ances allows researchers analyse emotion dynam- ies since one's utterances can reasonably reflect one's thought process. UED allows for broader scale analyses across mediums (e.g., narratives, so- cial media, etc.) and regions (e.g., cities, countries, etc.). UED metrics have been used to study the emotional trajectories of movie characters (Hipson and Mohammad, 2021) and to analyse emotional patterns across geographic regions through Twit- ter data (Vishnubhotla and Mohammad, 2022a). Seabrook et al. (2018) studied the association be- tween depression severity and the emotion dynam- ics metric variability on Facebook and Twitter. The UED framework has also been applied to study de- velopmental patterns of emotional change in poems written by children (Téodorescu et al., 2023).
3.1 Twitter Dataset: STMHD
Suhavi et al. (2022) identified tweeters who self- disclosed an MHC diagnosis using carefully con- structed regular expression patterns and manual verification. We summarize key details in the Ap- pendix (Section A). The control group consists of users identified from random sample of tweets (posted during roughly the same time period as the MHC tweets). These tweeters did not post any tweets that satisfied the MHC regex described above. Additionally, users who had any posts about mental health discourse were removed from the control group. Note that this does not guarantee that these users did not have an MHC diagnosis, but rather the set as a whole may have very few MHC tweeters, The number of users in the control group was selected to match the size of the depression dataset, which has the largest number users.
This work explores the relationship between UEDs and mental health conditions. Also, unlike past work in emotion dynamics that has focused on or eysitiv--egaatin dimension), this work also explores the arousal slugive-l and dominance (in control-out of control, powerful-went valence, arousal, and dominance are considered the core dimensions of emotion (Russell, 2003).
valence beeasurr-dissleasee
For the finalized set of users, four years of tweets were collected for each user: two years before self- reporting mental health diagnosis and two years after. For the control group, tweets were randomly sampled from between January 2017 and May 2021 (same date range as other MHC classes).
dimensions. Together,
Users with less than 50 tweets collected were removed so as to allow for more generalizable con- clusions to be drawn. Similarly, users with more than 5000 followers were removed so as in- clude celebrities, or other organizations that use Twitter to discuss well-being.
3 Datasets
We use recently compiled dataset-Twitter- STMHD (Suhavi et al., 2022). It comprises of tweets from 27,003 users who have self-reported as having a mental health diagnosis on Twitter. The diagnoses include: depression, major depressive disorder (MDD), post-partum depression (PPD), post-traumatic stress disorder (PTSD), attention- defici/hhyperactivity disorders (ADHD), anxiety, bipolar, and obsessive-compulsive disorder (OCD). We describe the dataset in Section 3.1, and our preprocessing steps in Section 3.3.
3.2 Reddit Dataset: eRisk 2018
To further add to our findings, we also include the eRisk 2018 dataset (Losada et al., 2017, 2018) 18) in our experiments. It consists of users who self-
"The available Reddit data only included information about depression; we hope future work will explore other MHCs.

---

Dataset
Group
#People
Avg. #Posts/User
We computed the number of posts per user, and only considered users whose number of posts was within the interquartile range (be- tween 25th and 75th percentile) for the diag- nosis group they disclosed. This was to ensure that we are not including users that post very infrequently or very frequently.
Twitter
MHC
10,069
2,177.4
ADHD
3,866
2,122.2
Bipolar
721
3,193.3
Depression
3,017
2,084.0
MDD
133
2,402.9
OCD
605
1,822.9
PPD
105
1,67144
PTSD
1,622
1,944.9
Control
4,097
1,613.6
We removed punctuation and stop words. Table shows key details of the filtered Twitter-
Reddit
Depression
106
233.79
STMHD and Reddit c-risk datasets. We make our code for preprocessing the data publicly available."
Control
749
359.74
Table I:
The number of users in each mental health condition and the number of tweets per user the pre- processed version of the TwitterSTTMHD and Reddit eRisk datasets we use for experiments.
Experiments
To determine whether UED metrics from tweets can act as biosocial markers for psychopathology, we compare UED metrics for each MHC to the control group see if they are statistically different. We compute the following UED metrics per user in each condition, following the speaker UED approach as described Teodorescu et al. (2023). this approach, UED metrics are com- puted per speaker by placing all their utterances in temporal order and computing UED on these ordered utterances. These metrics often rely on the characterization of the steady state or home base for emotions (regions of high probability). Hipson and Mohammad (2021) define the region pertaining to one standard deviation on either side of the mean as the home base. Previous work computed speaker UED metries for characters movie dialogues (Hipson and Mohammad, 2021), and for users on Twitter during the pandemic (Vishnubhotla and Mohammad, 2022a). We summarize the UED metrics below (also Figure l):
disclosed as having depression on Reddit (expres- sions were manually checked), and a control group (individuals were randomly sampled). Contrary to the TWitter-SMHHD dataset where users in the control group were removed for discussing well- being topics, Losada et al. (2018) also considered users who discuss depression in the control group. The dataset includes several hundred posts per user, over approximately a 500-day period. We consider both the training set (which is from the eRisk 2017 task (Losada et al., 2017)) and the test set (from the eRisk 2018 8 task (Losada et al., 2018)).
3.3 Our Preprocessing
We further preprocessed both the Twitter-TMHHD dataset and the eRisk dataset for experiments (Section 4), we are specifically interested in the unique patterns of UED for each disorder. Sev- eral users self-reported as being diagnosed with more than one disorder, referred to as comorbidity. We found a high comorbidity rate between users who self-reported as having anxiety and depres- sion, as is also supported in the literature (Pollack, 2005; Gorman, 1996; Hirschfeld, 2001; Cummings al., 2014). Therefore, we removed the anxiety class and only considered the depression class as it was the larger class between the two. We also performed the following preprocessing steps:
Average Emotional Intensity: One's average emotion over time.
Emotional Variability: How much and how often onc's emotional state changes over time.
Rise Rate: The rate which one reaches peak emotional intensity from the home base (i.e., emotional reactivity).
We only consider users who self-repone relfrrepot as having one disorder. We removed I 1272 users who had disclosed more than diagnosis.
Recovery Rate: The rate at which one recovers from peak emotional intensity to home base, (i.e., emotional regulation).
We only consider tweets in English, removing other languages.
For each user, we order their tweets by times- tamp and used the Emotion Dynamics toolkit ( (Vish- nubhotla and Mohammad, 2022b; Hipson and Mo-
We filtered out posts that contained URLs.
We removed retweets (identified through tweets containing 'RT', 'rt').
hhttps://gitte https/tttb.cmmttteeeree/meiiiioo

---

Avg. Emotion Intensity
Peak
for all metries and emotions (results in Appendix B. As such, we used Welch's ANOVA test which is an alternative to ANOVA when the equal vari- ance assumption is not met. We conducted 1-way Welch's s ANOVA for each of the UED metrics (emo- tional average, emotional variability, rise-rate, and recovery rate) and emotions (valence, arousal, and dominance). We examined total of N 14166 users, see Table 1 for descriptives.
Home base
Emotion Variability
Rise
Recovery
Emotion Intensity
Peak
The first part of our analyses are Omnibus F- tests for significant differences between groups. This test cannot tell us which groups different from each other, just rather that there is differ- ence among groups. For each combination of UED metrics and emotions (e.g., emotion variability for valence) there was a significant main effect which means we can conclude that at least one of the mental health diagnoses significantly differed. We show the degrees of freedom, F-statistic, p-value, and corrected effect size in Table 4 in the Appendix for valence, arousal, and dominance. The effect size tells us how meaningful the difference between groups for each metric. For example, 3.31 % of the total variance in the emotional variability for valence is accounted for by diagnosis (small effect).
Time
Figure I: Utterance emotion dynamies metries quantify patterns of emotional change over time.
hammad, 2021)* to compute UED metries (aver- age emotion, emotion variability, rise rate, and re- covery rate). We performed analyses for valence, arousal, and dominance. For word-emotion associ- ation scores we use the NRC Valence, Arousal, and Dominance (VAD) lexicon (Mohammad, 2018). It includes ~20,000 common English words with scores ranging from (lowest V/A/D) (high- est V/A/D). Afterwards, we performed an ANOVA to test for significant differences between groups in UED metrics, and post-hoc analyses to deter- mine which groups specifically had significant dif- ferences from the control group.
Next, we would like to know exactly which groups differed from the control group. order to do this, we performed post hoc analyses for pair- wise comparisons between groups for each metric across the three dimensions of emotions. We ap- plied Games-Howell correction since the assump- tion for homogeneity of variance was violated. In the following Sections detail how the UED met- rics compare across MHCs compared to the control group. In Table 2 we show the pairwise compar- ison results for which UED metries and emotion combination significantly differed from the control group across diagnoses, and the direction of the dif- ference. We also show results on the eRisk dataset in Table 2, and contrast our findings for depression between the Twitter and Reddit datasets.
5 Results
To analyse potential differences across groups and the control group, an ANOVA statistical test can be conducted, however several assumptions must be met. The three primary assumptions are: the data for each independent variable are approximately normally distributed, the data are independent of each other, and the distributions have roughly the same variance (homoscedasticity). We can assume the mean is normally distributed according to the central limit theorem, due to the large sample size (law of large numbers). Since there are different tweeters in each MHC group we can assume the data are independent of each other. However, we note that people are largely not independent of each other e.g., having friends across groups, interacting with content from various mental health groups, etc. In our case, Levene's test indicated that the as- sumption for homogeneity of variance was violated
We contextualize our results with previous find- ings in psychology and studies in NLP. We note that the relationship between patterns of emotion change and well-being for the dimensional emo- tions arousal and dominance are under-explored our findings provide important benchmarks for these emotions and UED metrics more generally.
'ht wwtp:///riyyy22//mmooonnnmaaeee httt:://sai ficchamadd hww////baaaaaa///n/ccaaaaaaa html
"An effect size <0.01 is very small, between 0.01 to 0.06 is small, between 0.06 to 0. 14 is medium, and greater than or equal to 0.14 is large.

---

Average Emotion
Emotion Variability
Rise Rate
Recovery
Rate
Dataset
MHCC-Contro
D
A
V
Twitter-STMBR
ADHD-control control
T
Bipolar-control
1
f
1
MDD-contro
1
1
4
OCD-control
1
f
1
PPD-control
4
PTSD-control
f
f
1
f
1
Depression-control
4
1
1
1
Reddit cRisk
Depression-contri
Table 2: The difference UED metrics between each MHC group and the control. significant difference is indicated by arrow; arrow direction indicates the direction of the difference. E.g., + for ADHD-contrr and average emotion 'V" means that the ADHD group has significantly lower average valence than the control group.
5.1 How does the average emotion for an MHC compare to the control?
5.2 How does emotion variability for an MHC compare the control?
Valence: Variability for valence was significantly higher for the ADHD, depression, bipolar, MDD, PTSD, and OCD groups compared to the control. PPD did not show differences from the control. The depression group in the eRisk dataset also showed significantly higher valence variability compared to the control group.
Valence:
The average valence was significantly lower for the ADHD, MDD, and PTSD groups compared to the control group.
Arousal: The ADHD, depression, bipolar, PPD, and OCD groups showed significantly lower arousal compared to the control group.
Dominance: All MHC groups (ADHD, depres- sion, bipolar, MDD, PPD, PTSD, OCD) showed significantly lower dominance compared to the con- trol group. Additionally, the depression group in the eRisk dataset also had significantly lower dom- inance compared to the control group.
Arousal: All MHC groups (ADHD, depression, bipolar, MDD, PPD, PTSD, and OCD) showed significantly higher arousal variability.
Dominance: All MHC groups (ADHD, depres- sion, bipolar, MDD, PPD, PTSD, OCD) had sig- nificantly higher dominance variability than the control group. The depression group in the eRisk dataset also had significantly higher dominance variability compared to the control group.
Discussion: Our findings align with results in psy- chology, and NLP studies looking at the average emotion intensity expressed in text. Valence was found to be lower in individuals with depression (of which MDD is a type of depression) through self-reports questionnaires (Heller et al.. 2018; Silk et al., 2011) 1) and social media (Seabrook et al., 2018; De Choudhury al., 2013, 2021). Fur- ther, work in psychology has found individuals with PTSD and ADHD have lower valence (Pu- gach et al., 2023; Stickley al., 2018). has also been shown that lower arousal and dominance in speech is associated with depression (Stasak et al., 2016; Osatuke et al., 2007: Gumus et al., 2023). While average emotion intensity is one of the more commonly explored measures, there is still relatively few works studying the relationships between arousal and dominance with mental health, compared valence. Interestingly, dominance ap- pears differ for many MHCs (all studied here) from the control group, pointing towards an impor- tant indicator of well-being.
Discussion: In several studies in psychology, it has been shown that higher valence variability occurred for individuals with depression, PTSD (Houben et al., 2015; Heller et al., 2018) and is neg- atively correlated with overall well-being (Houben et al., 2015). Interestingly, Seabrook et al. (2018) found higher valence variability on Twitter indi- cated lower depression severity which contradicted their findings on Facebook. Kuppens et al. (2007) report that valence variability was negatively re- lated to self-esteem and was positively related to neuroticism and depression. Overall, our results align with emotional variability having strong ties with well-being. Arousal and dominance variabil- ity appear be biosocial markers across several MHCs, although minimally explored in the litera- ture (Ranney al. (2020) found higher affective arousal variability was associated with generalized anxiety disorder).

---

5.3
How does emotional rise rate for an MHC compare to the control?
whether rise and recovery rates act as indicators as well, perhaps by varying parameters involved in computing UED, such as the time-step granularity. Overall, we notice that there are quite a few UED metrics across emotion dimensions which are significantly different for MHCs from the control group. Often also, that the direction of difference is similar for valence, arousal, and dominance. Work in psychology supports that often valence and dom- inance move in similar directions. For example, (Graziotin et al., 2015) found software engineers were more productive when in a good mood asso- ciated with higher valence (i.e., more positive) and higher dominance (i.c., more in control of one's feelings). Likewise, There may be several factors which influence valence and dominance move in similar directions in the control group and MHCs.
Valence: Rise-rate for valence was significantly higher for the depression, MDD, and PTSD groups compared the control group.
Arousal: PTSD was the only group which had sta- tistically higher arousal rise rates than the control group.
Dominance: The ADHD, depression, and OCD groups had significantly higher rise rates than the control group.
Discussion: Rise-rate is analogous to emotional re- activity in psychology, and quickly moving to peak emotional states has been shown in individuals with maladaptive emotion patterns and lower psycholog- ical well-heing (Houben et al., 2015). It is interest- ing note that valence and dominance rise rates differed across MHC the control, whereas not to the same extent for arousal.
Results: Biosocial Aspects
We now explore whether the UED metrics that were found to be significant indicators of certain MHCs, continue to remain discriminating even when ac- counting for certain social factors such as how pop- ular user's posts tend to be. Metadata for the num- ber of likes per post is available in the dataset. So, we simply extracted this information and computed the average number of likes per tweeter. Most users had (-4 likes per post in both the MHCs and the control group, up to an average of 8 likes per post. We compare the average UED per emotion dimen- sion across the various bins of average likes. If a popularity measure does not impact UED, then we would expect to see consistent differences between the control group and an MHC across popularity bins. However, if a popularity measure does in- fluence UED then the differences across MHCs and the control may disappear when controlling for popularity.
5.4 How does emotional recovery rate for an MHC compare to the control?
Recovery rate for valence was signifi- cantly higher for the depression, bipolar, MDD, and PTSD groups compared to the control group. Arousal: The ADHD, depression, MDD, PTSD, and OCD groups showed significantly higher arousal recovery rates than the control group. Dominance: The ADHD, depression, MDD, and and OCD groups showed significantly higher dominance recovery rates than the control group.
Valence:
Discussion: Recovery rates can be thought of as a proxy of emotion regulation, and slower recov- ery from emotional events associated with psy- chopathology and poor psychological well-being (Houben et al., 2015; Kragel et al., 2022). Our re- sults, while pointing to higher recovery rates, indi- cate significant differences from the control group. This is interesting result that can be further plored if found in other mediums such as Reddit,
In Figure 2, we show the two UED metrics (e.g.. average emotion intensity and emotional variabil- ity) for valence, across various bins of average likes per tweeter for each group. In the Appendix (Sec- tion D, E, and F), we show the remaining results for arousal and dominance. If bin had less than ten users in it (e.g., average of 6-8 likes per post), then did not plot value for it. In each figure the control group is the blue line (no shapes). For MHCs which significantly differed from the con- trol for an emotion and UED pair, we show the mean difference in Appendix G. If this difference between an MHC and the control remains fairly
We expected to see differences in which UED significantly different from the control between the Twitter and Reddit datasets due to the different mediums, collection process, etc. as mentioned in Section 3. However, we also saw commonalities in that average dominance was significantly lower, and valence and dominance variability were signif- icantly higher for the depression group compared to the control. appears that despite dataset differ- ences, these biosocial markers are strong indicators for depression. Future work can explore further

---

0.310
0.30
Valence Average
Valence Varability,
0.300
0.295
0,785
0.275
0.770
46
58
02
24
46
688
Average Likes
Average Likes
Control ADI
MDU
Bipnler
MID
Depression
OCD
F5D
4DMD
Depressioo
OCD
MI5D
Figure 2: Valence average and variability across levels of user popularity on Twitter (average #likes on posts).
consistent across ranges average likes, this UED is most likely not influenced by this social aspect.
Thus, overall, we observe that for the valence UEDs which showed significant differences across MHCs and control, these differences do not disap- pear when accounting for popularity measures such as the average number of likes user's posts get. Similar trends overall were found for arousal and dominance (Appendix E and F). UED for all three emotion dimensions appear to robust indicators for various mental health conditions, even when users have some varying social characteristics.
Average Valence: In the previous section, we showed that average valence is markedly lower for ADHD, MDD, and PTSD. From Figure 2 (a), we observe that the average valence for these MHCs is markedly lower than control for various Average- Likes bins. (We also observe markedly greater dif- ferences when average likes is [6,8) range.)
Valence Variability: In the previous section, we showed that valence variability is markedly higher for all MHCs except PPD. From Figure 2 (b), we observe that the valence variability for these MHCs is markedly higher than control for various Average-Liie Likes bins. (We also observe markedly lower variability for control when average likes is in the [6,8) range.)
Conclusion
We showed for the first time that there are sig- nificant relationships between patterns of emo- tion change in text written by individuals with self-disclosed MHC compared to control group. Specifically, we found significant differences four utterance emotion dynamics (UED) metrics (average emotion, emotion variability, rise rate, and recovery rate) across three emotion dimensions (va- lence, arousal, and dominance) for 7 MHCs. Our findings provide important contextual information of overall well-being and supporting indicators (in addition to other assessments) to clinicians for di- agnosis detection and management.
Valence Rise Rate: the previous section, we showed that valence rise rate is markedly higher for MDD, PTSD, and Depression. From Figure (a) we observe that the valence rise rate for these MHCs is markedly higher than control for various Average-Likes bins. (We also observe markedly lower valence rise rate for control when average user likes in the (6,8) range.)
Valence Recovery Rate: In the previous section, we showed that valence recovery rate is markedly higher Bipolar, MDD, PTSD, and Depression. From Figure (b) we observe that the valence re- covery rate for these MHCs is markedly higher than control for various -irrage-Likss bins. (We also observe markedly lower recovery rate for control when average user likes is in the (6.8) range.)
Looking ahead, we plan to explore UED metrics across different textual genres, regions, languages, and demographics (such as socioeconomic status), in collaboration with clinicians. Through such a partnership. UED metrics could be studied in the context of clinical information well. Lastly, ex- ploring new UED metrics that correlate with psy- chopathology is a promising direction for future work.

---

Limitations
tener, or someone mentioned in the utterance. However, is not sufficient for accurately de- termining any of their momentary emotional states. Deciphering the true momentary emo- tional state of an individual requires extra- linguistic context and world knowledge. Even then, one can be easily mistaken.
In this study, we used NLP techniques to com- pare UED across different psychopathologies and a control group. It is important to be cautious of interpreting these results due to natural limitations within the dataset. Due to the high rates of co- morbidity with mental health disorders (Druss and Walker, 2011) examining users who only disclosed one diagnosis may not be representative of the pop- ulation. Furthermore, it is also possible that the dataset included users with more than one disorder but only disclosed one (e.g., a user may have been diagnosed with depression and ADHD but only tweeted "diagnosed with ADHD" or vice versa). Self-disclosure of diagnoses may also be inaccu- rate due reasons such as impression management (Leary, 2001) or social desirability (Latkin al., 2017) where users may disclose having a diagnosis without having received a formal diagnosis. Al- ternatively, there may have been users included in the control group who have formal diagnosis of one or more mental health disorders but did not dis- close this Twitter. Overall, despite the dataset creators' best efforts collect data accordingly, the users in each group may not be representative of the mental disorders. Future research could replicate this study using a sample of users with confirmed formal diagnoses.
The inferences we draw in this paper are based on aggregate trends across large populations. We do not draw conclusions about specific individuals or momentary emotional states.
Acknowledgements
Many thanks to Krishnapriya Vishnubhotla for the Emotion Dynamics codebase, which set the ground- work for computing emotion dynamics, and for in- sightful discussions. This research was supported by NSERC, SSHRC, Digital Research Alliance of Canada (alliancccan.cu)) Alberta Innovates, Deep- Mind, and CIFAR. Alona Fyshe holds a Canada CIFAR AI Chair. This research project is funded by the Bavarian Research Institute for Digital Transfor- mation (bidt), an institute of the Bavarian Academy of Sciences and Humanities. The author is respon- sible for the content of this publication.
References
Karla V. Ballman. 2015. Biomarker: Predictive or prog- nostic? Journal of Clinical Oncology, 33(33):3968- 3971. Publisher Copyright: 2015 American Soci- ety of Clinical Oncology. All rights reserved.
Ethics Statement
Our research interest study emotions at an ag- gregate/group level. This has applications in emo- tional development psychology and in public health (e.g., overall well-being and mental health). How- ever, emotions are complex. private, and central to an individual's experience. Additionally, each indi- vidual expresses emotion differently through lan- guage, which results in large amounts of variation. Therefore, several ethical considerations should be accounted for when performing any textual anal- ysis of emotions (Mohammad, 2022, 2023). The ones we would particularly like to highlight are listed below:
Laura Calzà, Gloria Gagliardi, Rema Rossini Favretti, and Fabio Tamburini. 2021. Linguistic features and automatic classifiers for identifying mild cognitive impairment and dementia. Computer Speech & Lan- guage, 65:101113.
Cheryl M. Corcoran, Vijay A. Mittal, Carrie E. Bear- den, Raquel E. Gur, Kasia Hitezenko, Zarina Bil- grami. Aleksandar Savie, Guillermo A. Cecchi, and Phillip Wolff. 2020. Language as a biomarker for psychosis: A natural language processing approach. Schizophrenia Research, 226:158-166, Biomarkers in the Attenuated Psychosis Syndrome.
Our work on studying emotion word usage should not be construed as detecting how peo- plc feel; rather, emotions that are conveyed by users via the language that they use.
Colleen M. Cummings. Nicole E. Caporino, and Philip C. Kendall. 2014. Comorbidity of anxiety and depression in children and adolescents: 20 years after. Psychological Bulletin, 140(3):816-845.
draw inferences the
Vittoria Cuteri, Giulia Minori, Gloria Gagliardi, Fabio Tamburini, Elisabetta Malaspina, Paola Gualandi, Francesca Rossi, Milena Moscano. Valentina Fran- cia, and Antonia Parmeggiani. 2022. Linguistic fea- ture of anorexia nervosa: a prospective case-control
The language used in an utterance may con- vey information about the emotional state (or perceived emotional state) of the speaker, lis-

---

pilot study. Eating and weight disorders: EWD, 27(4):1367-1133 1375.
Aaron Heller, Andrew S Fox, and Richard J Davidson. 2018. Parsing affective dynamics to identify risk for mood and anxiety disorders. Emotion, 19(2):283- 291.
Richard J. Davidson. 1998. Affective style and affective Perspectives from affective neuroscience. Cognition and Emotion, 12(3):307-330.
disorders:
Will Hipson and Saif M. Mohammad. 2021. Emotion dynamics in movie dialogues. PLOS ONE, 16(9):1- 19.
Munmun De Choudhury, Scott Counts, and Eric Horvitz.
2013. Social media as measurement tool of de- pression in populations. In Proceedings of the 5th Annual ACM Web Science Conference, WebSci '13. page 47-56, New York, NY, USA. Association for Computing Machinery.
Robert M. A. Hirschfeld. 2001. The comorbidity of major depression and anxiety disorders: Recogni- tion and management in primary care. Primary care companion to the Journal of clinical psychiatry, 3(6):244 254.
Mummun De Choudhury, Michael Gamon, Scott Counts, and Eric Horvitz. 2021. Predicting depression via social media. Proceedings of the International AAAI Conference on Web and Social Media, 7(1):128-137.
Marlies Houben, Wim Van Den Noortgate, and Peter Kuppens. 2015. The relation between short-term emotion dynamies and psychological well-being: A meta-analysis.
Benjamin Druss and Elizabeth Reisinger Walker. 2011. Mental disorders and medical comorbidity. The Synthesis project. Research synthesis report, (21):1-26. 26.
Sanne Koops, Sanne G Brederoo, Janna N de Boer, Femke G Nadema, Alban E Voppel, and Iris E Sommer. 2023. Speech as biomarker for depres- sion. CNS& neurological disorders drug targets. 22(2):152-15)
Gloria Gagliardi and Fabio Tamburini. 2021. Linguistic biomarkers for the detection of mild cognitive im- pairment. Lingue e linguaggio, Rivista semestrale, (1/2021):3-31.
Philip A. Kragel, Ahmad R. Hariri, and Kevin S. LaBar. 2022. The temporal dynamics of spontancous emo- tional brain states and their implications for mental health. Journal of cognitive neuroscience, 34(5):715- 728. May, 2022.
Gloria Gagliardi and Fabio Tamburini. 2022. The auto- matic extraction of linguistic biomarkers as viable solution for the early diagnosis of mental disorders. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 5234-5242, Mar- seille, France. European Language Resources Asso- ciation.
Peter Kuppens, Iven Van Mechelen, John B Ne- zlek, Dorien Dossche, and Tinneke Timmermans. 2007. Individual differences in core affect variabil- ity and their relationship to personality and psycho- logical adjustment. Emotion (Washington, D.C.). 7(2):262-2) 274.
George Gkotsis, Anika Oellrich, Tim Hubbard, Richard Dobson, Maria Liakata. Sumithra Velupillai, and Rina Dutta. 2016. The language of mental bealth problems in social media. In Proceedings of the Third Workshop on Computational Linguistics and Clinical Psychology. pages 63-73, San Diego, CA, USA. Association for Computational Linguistics.
Peter Kuppens and Philippe Verduyn. 2015. Looking at emotion regulation through the window of emotion dynamics. Psychological Inquiry, 26(1):72-79,
Peter Kuppens and Philippe Verduyn. 2017. Emotion dynamics. Current Opinion in Psychology, 17:22-26. Emotion.
Jack M. Gorman. 1996. Comorbid depression and anx- iety spectrum disorders. Depression and Anxiety, 4(4):160-168.
Carl A. Latkin, Catie Edwards, Melissa A. Davey- Rothwell, and Karin Tobin. 2017. The relation- ship between social desirability bias and self-reports of health, substance use, and social network factors among urban substance users baltimore, maryland. Addictive Behaviors, 13:133-136.
Paulo A. Graziano, Rachael D. Reavis, Susan P. Keane, and Susan D. Calkins. 2007. The role of emotion reg- ulation in children's 's early academic success. Journal of School Psychology, 45(1):3-19
M.R. Leary. 2001. Impression management, psychol- ogy of. In Neil J. Smelser and Paul B. Baltes, editors. International Encyclopedia of the Social Behay- ioral Sciences, pages 7245-7248. Pergamon, Oxford.
Daniel Graziotin, Xiaofeng Wang, and Pekka Abrahams- son. 2015. Do feelings matter? the correlation of affects and the self assesaseedd productivity in software engincering, J. Softw: Evol. Process, 277744677-487
Palaniyappan Lena. 2021. More than a biomarker: could language be biosocial marker of psy- chosis? NPJ Schizophrenia. 7(1). Copyright @ The Author(s) 2021. This work is published under http://creativeeoommnns wwwrraarecommooooooolieeens////o O/ (the "Li- cense"). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance
Melisa Gumus, Danielle D DeSouza, Mengdan Xu, Celia Fidalgo, William Simpson, and Jessica Robin. 2023. Evaluating the utility of daily speech assess- ments for monitoring depression symptoms. DIGI- TAL. HEALTH, 9:20552076231 180523.

---

with the terms of the License: Last updated 2023- 02-22.
Rachel M. Ranney, Evelyn Behar, and Alissa S. Ya- masaki. 2020. Affect variability and emotional re- activity in generalized anxiety disorder. Journal of Behavior Therapy and Experimental Psychiatry, 18:10154.
David E Losada, Fabio Crestani, and Javier Parapar. 2017. crisk 2017: Clef lab on carly risk predic- tion on the internet: experimental foundations. In Experimental IR Meets Multilinguality, Multimodal- ity, and Interaction: 8th International Conference of the CLEF Association, CLEF 2017. Dublin, Ire- land, September 11-14, 2017, Proceedings 8. pages 146-660. Springer.
Andrew Reagan, Lewis Mitchell, Dilan Kiley, Christo- pher M. Danforth, and Peter S. Dodds. 2016. The emotional ares of stories are dominated by six basic shapes. EPJ Data Science, 5(1):1-12. Copyright EPJ Data Science is copyright of Springer, 2016; Last updated 2017-02-06.
David E Losada, Fabio Crestani, and Javier Parapar.
James A Russell. 2003, Core affect and the psycholog- ical construction of emotion. Psychological review. 110(1):145
2018. Overview of erisk: early risk prediction on the internet. In Experimental IR Meets Mulrilingual- it, Multimodality, and Interaction: 9th International Conference of the CLEF Association, CLEF 2018, Avignon, France, September 10-14, 2018, Proceed- ings 9. pages 343-361. Springer.
Elizabeth M Seabrook. Margaret L Kern, Ben D Fulcher. and Nikki S Rickard. 2018. Predicting depression from language-based emotion dynamics: Longitudi- nal analysis of facebook and twitter status updates. J Med Internet Res, 20(5):e168.
Saif Mohammad, 2011. From once upon time to happily ever after: Tracking emotions in novels and fairy tales. In Proceedings of the 5th ACL-HLT Work- shop on Language Technology for Cultural Heritage, Social Sciences, and Humanities, pages 105-114. Portland, OR, USA. Association for Computational Linguistics.
Jennifer S. Silk, Erika E. Forbes, Diana J. Whalen, Jen- nifer L. Jakubcak, Wesley K. Thompson, Neal D. Ryan, David A. Axelson, Boris Birmaher, and Ronald E. Dahl. 2011. Daily emotional dynamics in depressed youth: A cell phone ecological momentary assessment study. Journal of Experimental Child Psychology, I 10(2))241 1-257. Special Issue: Assess- ment of Emotion in Children and Adolescents.
Saif Mohammad. 2023. Best practices in the creation and use of emotion lexicons. In Findings of the Asso- ciation for Computational Linguistics: EACL 2023, pages 1825-1836, 1836, Dubrovnik, Croatia. Association for Computational Linguistics.
Linda Sosa-Hernandez. McLennon Wilson, and Heather A Henderson. 2022. Emotion dynamics among preadolescents getting to know each other: Dyadic associations with shyness. Emotion (Wash- ington, D.C.).
Saif M. Mohammad. 2018. Obtaining reliable human ratings of valence, arousal, and dominance for 20,000 english words. In Proceedings of The Annual Confer- ence of the Association for Computational Linguistics (ACL), Melbourne, Australia.
Sarah H. Sperry, Molly A. Walsh, and Thomas R. Kwapil. 2020. Emotion dynamics concurrently and prospectively predict mood psychopathology. Jour- nal of Affective Disorders, 261:67775.
Saif M. Mohammad. 2022. Ethies sheet for automatic emotion recognition and sentiment analysis. To Ap- pear in Computational Linguistics.
Brian Stasak, Julien Epps, Nicholas Cummins, and Roland Goecke. 2016. An investigation of emotional speech in depression classification. In Understand- ing speech processing in humans and machines: 17th Annual Conference of the International Speech Com- munication Association (INTERSPEECH 2016), San Francisco, California, USA, 8-12 September 2016; Volume
Katerine Osatuke, James K. Mosher, Jacob Z. Gold- smith, William B. Stiles, David A. Shapiro, Gillian E. Hardy, and Michael Barkham. 2007. Submissive voices dominate in depression: Assimilation analysis of a helpful session. Journal of Clinicul Psychology, 133-)1153-164
Andrew Stickley, Ai Koyanagi, Hidetoshi Takahashi, Vladislav Ruchkin, Yosuke Inoue, Aki Yazawa, and Yoko Kamio. 2018. sttention-effiidyyypeeccii disorder symptoms and happiness among adults in the general population. Psychiatry Research, 265:317-323.
Louise H Phillips. Louise H Phillips. Rebecca Bull. Ewan Adams, and Lisa Fraser. 2002. Positive mood and executive function: evidence from stroop and fluency tasks. Emotion (Washington, D.C.). 2(1):12 22.
Mark H Pollack. 2005. Comorbid anxicty and depres- sion. Journal of Clinical Psychiatry, 66:22.
Suhavi, Asmit Kumar Singh, Udit Arora, Somyadeep Shrivastava, Aryaveer Singh, Rajiv Ratn Shah, and Ponnurangam Kumaraguru. 2022. Twitter-sthhd: An extensive user-level datahase of multiple men- tal health disorders. Proceedings of the Interna- tional AAAI Conference on Web and Social Media, 182-119821119
Cameron P. Pugach, Casey I. May, and Blair F. Wisco. 2023. Positive emotion in posttraumatic stress disor- der: A global or context-specific problem? Journal of Traumatic Stress, 36(2):444

---

Daniela Teodorescu, Alona Fyshe, and Saif Mohammad. 2023. Utterance emotion dynamics in children's 's po- ems: Emotional changes across age. In Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, pages 401-414 -01- Toronto, Canada. Association for Computational Linguistics.
B Statistical Results
B.1 Levene's Test
Levene's test indicated that the assumption for ho- mogencity of variance was violated for the effect of diagnosis on all UED metrics (emotional aver- age, emotional variability, rise rate, and recovery rate) across all three dimensional emotions (va- lence, arousal, and dominance). We show the re- sults in Table 3.
Daniela Teodorescu and Saif Mohammad. 2023, Eval- uating emotion arcs across languages: Bridging the global divide in sentiment analysis. In The 2023 Con- ference on Empirical Methods in Natural Language Processing.
B.2 Welch's ANOVA
Krishnapriya Vishnubhotla and Saif M. Mohammad. 2022a. Tweet Emotion Dynamics: Emotion word us- age in tweets from US and Canada. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 4162-411 1176, Marseille, France. Eu- ropean Language Resources Association.
In Table 4 we show the results for Welch's ANOVA for valence, arousal, and dominance.
C Data Descriptives for Distributions of Social Aspects
Krishnapriya Vishnubhotla and Saif M. Mohammad. 20226. Tweet emotion dynamics: Emotion word us- age in tweets from us and canada. Proceedings of the Thirleenth International Conference on Language Resources and Evaluation (LREC 2022), Marseille, France.
In Table 5 we show some key details about the distribution of the popularity aspect on Twitter: average likes per Tweeter.
D UED Across Average Likes per Tweeter: Valence
APPENDIX
A Twitter-STMHD Dataset
In Figure 3, we show UED metrics (e.g., rise rate and recovery rate) for valence, across various bins of average likes received for each group.
Suhavi et al. (2022) created a regular expression pattern to identify posts which contained a self- disclosure of diagnosis and the diagnosis name (using lexicon of common synonyms, abbrevi- ations, etc.) such 'diagnosed with X'. They collected large set of tweets using the regex. This resulted in a preliminary dataset of users with po- tential MHC diagnoses. To handle false positives (e.g., 'my family member has been diagnosed with X', or 'I was not diagnosed with X¹), the dataset was split into two non-overlapping parts, one of which was manually annotated, and the other using updated and high-precision regex. In the part that was annotated by hand, each tweet was anno- tated by two members of the team. A user was only included in the dataset if both annotations were positive as self-disclosing for a particular class. A licensed clinical psychologist found the 500-tweet sample to be 99.2% accurate. The manual anno- tations were used to refine the regular expressions and diagnosis name lexicon. This updated scarch pattern was applied to the other dataset split. To verify the quality of the updated regex, the authors applied it to the manually annotated dataset split. When considering the manual annotations as cor- rect, the regex was found to be 94% accurate.
E
UED Across Average Likes per Tweeter: Arousal
In Figure 4, we show the four UED metrics (e.g., average emotion intensity, emotional variability, rise rate, and recovery rate) for arousal, across vari- ous bins of average likes received for each group. Average Arousal: In Section 5, we showed that aver- age arousal is markedly lower for ADHD, Bipolar, OCD, PPD, and Depression. From Figure 4 (a), we observe that the average arousal for these MHCs is markedly lower than control for various Average- Likes bins. (We also observe slightly smaller dif- ferences for Depression and OCD when average likes is in the [6,8) range, perhaps pointing the potential for average arousal to be influenced when a post is considered quite popular.)
Arousal Variability: In Section 5, we showed that arousal variability is markedly higher for all MHCs. From Figure (b), observe that the arousal vari- ability for these MHCs is markedly higher than con- trol for various Average-Likes bins. (We also ob- serve markedly lower variability for control when average likes is in the [6,8) range.)

---

Emotion Valence
UED Metric
dff
df2
F-statistic
P-value
average emotion
14158
60.50
p<.0001
emotional variability
14158
66.33
pc.001
rise rate recnvery rate average emotion emotional variability rise rate recovery rate average emotion emotional variability
14156
77.35
p<,001
14156
72.58
ps,001
Arousal
14158
37.60
p<,001
7
14158
22.38
p<,001
7
14155
34.76
P<,001
7
14155
41.28
P<,001
Dominance
7
14158
61.86
Ps.001
7
14158
72.21
P<.001
rise rate recovery rate
7
14150
37.69
P<.001
7
14154
35.64
P<.001
Table 3: The degrees of freedom, F-statistic, and p-value in Levene's test of Homogeneity of Variances for each UED metric and emotion.
Emotion Valence
UED Metrie
df1
df2
F-statistic
P-value
Effeet Size (est wr)
average emotion
1021.65
14.79
p<.001
0,0068
emotional variability
1021.20
70.30
P<.001
0,0331
rise rate recovery rate average emotion
1026.32
9.93
p<,001
0.0044
1023.62
8.86
P<.001
0,0039
Arousal
1024.41
33.24
p<,001
0.0157
emotional variability
1029.77 77
66.23
P<,001
0,0312
rise rate recovery average emotion emotional variability rise recovery
1025.85
2.84
p= 006
0.0009
1026.95
5.19
P<.001
0,0021
Dominance
1020.10
$6.69
P<.001
0.0268
1023.12
40.50
P<.001
0,0191
1025.35
6.31
Ps.0011
0.0026
1022.99
9.94
PS2.001
0.0044
Table 4: The degrees of freedom (for the numerator and denominator), F-statistic, p-value, and effect size in Welch's ANOVA test for differences between groups, for each UED metric emotion (valence, arousal, and dominance).
Popularity Aspect Avg. Likes per Tweeter
Average
Std. Dev.
25th Percentile
Median
75th Percentile
2.06
8.33
0.67
1.20
2.11
Table 5: Key statistics of the distribution of the popularity aspect: Avg. Likes per Tweeter.
Arousal Rise Rate: In Section 5, we showed that arousal rise rate is markedly higher for PTSD. From Figure (c), we observe that the arousal rise rate for PTSD does cross the control line at bin with [4-6) likes. This points to the potential for arousal rise rate to be slightly influenced by a popularity aspect such as the average number of likes received, (We also observe markedly lower arousal rise rate for control when average user likes is in the (6.8) range.)
Thus, overall, we observe that for the arousal UEDs which showed significant differences across MHCs and control, these differences still do appear (although slightly less for arousal rise and recovery rates) when accounting for popularity measures such as the average number of likes a user's posts get.
F UED Across Average Likes per Tweeter: Dominance
Arousal Recovery Rate: Section 5, we showed that arousal recovery rate is markedly higher for ADHD, MDD, OCD, PTSD, and Depression. From Figure (d), we observe that the arousal recovery rate for these MHCs is markedly higher than con- trol for various Average-Likes bins (slightly less for ADHD). (We also observe markedly lower arousal recovery rate for control when average user likes is in the (6,8) range.)
In Figure 5, we show the four UED metries (e.g., av- erage emotion intensity, emotional variability, rise rate, and recovery rate) for dominance, across vari- ous bins of average likes received for each group. Average Dominance: In Section average dominance is markedly lower for all MHCs. From Figure 5 (a), we observe that the average dominance for these MHCs is markedly lower than control for various Average-Liee bins.
showed that

---

Valence Recovery Rate
0.132
0.12
Valance Rise Rate
0.110
0.10
1118
0106
0.106
0.106
0.104
0.104
0.102
24
46
6-0
2-
66
6-0
Average Likes
Average Likes
Cantrol
Hipllar
MEDD
1913
Zontel RDMD
Monllrr
MIDD
1933
ADMD
pepresion
P15D
Depression
PI5
Figure 3: Valence rise and recovery rates across levels user popularity on Twitter (average #likes on posts).
Dominance Variability: In Section 5, we showed that dominance variability is markedly higher for all MHCs. From Figure (b), observe that the dominance variability for these MHCs is markedly higher than control for various Average-Likes bins. (We also observe markedly lower variability for control when average likes in the [6,8) range.)
appear (although less for dominance recovery rates)
when accounting for popularity measures such the average number of likes user's posts get.
G UED Mean Differences
Table 6 and Table show the mean difference in UEDs between the control group and an MHC if the difference was statistically significant in the pairwise comparison shown in Section 5.
Dominance Rise Rate: In Section 5, we showed that dominance rise rate is markedly higher for ADHD, OCD and Depression. From Figure (c), we observe that the dominance rise rate for these MHCs is markedly higher than control for various Average- Likes bins. (We also observe markedly lower dominance rise rate for control when average user likes is in the [6,8) range.)
Dominance Recovery Rate: In Section 5, showed that dominance recovery rate is markedly higher for ADHD, MDD, OCD, PTSD, and Depres- sion. From Figure (d), we observe that the domi- nance recovery rate for these MHCs higher than control for various Average-Likes bins although not for ADHD, PTSD which intersect the control line at bin with [4-6) likes. We also notice that the difference dominance recovery rate for OCD and control is less at bin 3 with [4-6) likes. This points to the potential that dominance recovery rate may be influenced by popularity measure such as the average number of likes received. (We also observe markedly lower dominance recovery rate for control when average user likes is in the [6.8) range.)
Thus, overall, we observe that for the dominance
UEDs which showed significant differences across MHC's and control, these differences still do largely

---

0201
-0.01
Arousal Average
Arousal Varabiltty
0.200
0.258
0.356
74
46
68
02
2-1
46
69
Average Likes
Average Likes
Hpolar
NDD
Depression
MD
40HD
Depression
OCD
PT5D
0.108
0.102-
Arousal Recovery Rate
0.106
Arousal Rise Rata
0.101
0.104
0.100
0.102-
0.09
0.098
0.100
0.297
0.688
0.66
0.996
0.155
02
2-1
44
6/8
02
2-1
46
6-8
Average Likes
Average Likes
Control A3ND
Bipllar Deprissios
Comtel 43RD
Bigolar
MDD
OC33
PISD
Deprissson
OCCO
P1533
Figure 4: UED metrics for arousal across various levels user popularity on Twitter (average #likes on posts).
Average Emotion
Emotion Variability
Dataset
UHHC-Control
A
D
V
A
D
Sriute-STTMD
ADHD-contro
-0.018
00021
-0.030
0.008
0.007
0.006
Bipolar-control
-0.013
-0.029
0.015
0.007
0.008
control
-0.027
-0.059
0.017
0.009
0.009
OCD-control
-0.023
-0.033
0.010
0.007
0.005
PPD control
-0.046
-0.051
0.006
0.006
PTSD PTSD-control
-0.023
-0.018
0.015
0.007
0.006
Depression-comte control
-0.015
-0.040
0.013
0.007
0.006
Reddit eRisk
Depression-control
-0.041
0.014
0.006
Table 6: The mean difference in UED metrics between each MHC group and the control if there was a significant difference between groups. E.g. -0.018 ADHD-contte and average emotion "V" means that the ADHD group has significantly lower average valence by 0.018 than the control group.

---

(a)
(b)
0.292
Dominance Average
Dominance Varability
0.250
0.18
0.48
6.16
0.266
0.14
Average Likes
Average Likes
Contra
Mpollrr
MDD
Zontrll
Bisolar
MI0
AD1
OCD
(c)
(d)
0.096
Dominance Recovery Rate
Dominance Rise Rate
0.894
0.044
0.892
6.200
0.99
0.88-
0684
Average Likes
Average Likes
Control
Figure 5: UED metrics for dominance across various levels of user popularity on Twitter (average #likes posts).
Rise Rate
Recovery Rate
Dataset
MoC-Control
D
V
A
D
Swiller-STMHDD
ADHD-cootrol
0.001
0.001
0.002
Bipolar-control
0.002
MDD-control
0.004
0.004
0.003
0.005
OCD-control
0.002
0.002
0.002
PPD-control
PTSD-control
0.002
0.001
0.002
0.001
0.001
Depression control
0.002
0.002
0.002
0.001
0.002
Reddit eRisk
Depression-contru
Table 7: The mean difference in UED metrics between each MHC group and the control if there was significant difference between groups. E.g., 0.004 MDD-control and rise rate "V" means that the MDD group has significantly higher rise rate for valence by 0.004 than the control group.


Reviews:
[{"review_id": "zwfLonmpe0", "reviewer": "Reviewer_Gsi5", "summary": "", "questions": "In relation to the ANOVA test: why not actually test for normality instead of assuming it, at least by \"manual\" inspection of the scores histograms? (which are already illustrated in the paper)", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer,\n\nThank you very much for your feedback and thorough review.\n\nRe: Stated goal of using emotion dynamics as \"biosocial markers\" is insufficiently addressed via the methodology, which doesn't focus on the social aspect almost at all\n\nWe fully agree with you that more can be done regarding the social aspects of biomarkers. We would like to clarify that the primary goal of this work is to assess the utterance emotion dynamics in language used by various populations associated with mental health conditions. That said, regardless of the extent to which social aspects are experimented with in a work, we think it is always important to refer to such linguistic markers as biosocial. Not treating them so has often, in the past, led to over generalized claims and one-size fit all solutions, at the expense of marginalized communities. Thus, our introduction situates the description of UED markers firmly as bio social. We do not make any claims that this work substantially covers social aspects associated with UED. We will make this clear in the revised version. We also point out below some challenges with addressing social aspects responsibly. Further, we did not want to force-fit more aspects when there wouldn't be adequate space to describe them. We hope for more careful and thoughtful work in this space in the future. \n\nControlling for social aspects:\n\nIn our work, we control for popularity measures on social media as one such aspect. Some other aspects worth exploring include demographics such as age, gender, region, and culture as you are mentioning. We could control for gender, however this has become a sensitive topic as of late. Twitter’s current policy states:\n\n“You should be careful about using Twitter data to derive or infer potentially sensitive characteristics about Twitter users. Never derive or infer, or store derived or inferred, information about a Twitter user’s: Health (including pregnancy), Negative financial status or condition, Political affiliation or beliefs, **Racial or ethnic origin**, or beliefs, Sex life or sexual orientation, Trade union membership, Alleged or actual commission of a crime.” \n\nRe: Text processing approach may be too weak:\n\nWhile ML methods often outperform the lexicon approach at the instance level, in tasks where information is aggregated from many instances (as is the case in generating emotion arcs) the lexicon approaches perform so well that the improvement from ML methods is very small. By obtaining emotion information from many instances in a bin, past work (Teodorescu and Mohammad, 2022 and in Öhman, 2021) has shown that lexicon approaches obtain highly accurate emotion arcs. For example, Teodorescu and Mohammad (2022) show through experiments on 40+ datasets that predicted emotion arcs with these methods obtain ~98.5% correlation with gold arcs. More importantly, with this approach, we are able to optimize interpretability and ease of computing (no training data or heavy compute power is needed). \n\nTeodorescu, Daniela, and Saif M. Mohammad. \"Frustratingly easy sentiment analysis of text streams: Generating high-quality emotion arcs using emotion lexicons.\" arXiv preprint arXiv:2210.07381 (2022).\n\nÖhman, E., 2021, December. The validity of lexicon-based sentiment analysis in interdisciplinary research. In Proceedings of the Workshop on Natural Language Processing for Digital Humanities (pp. 7-12).\n\nText processing details:\n\nWe have added into the main text details on the text processing in depth. Extra figures and diagrams will be added to the appendix as well. Thank you for pointing it out. \n\nWe have also included a formal test for normality in the paper and inspected the data through plots.\n\nThank you for sharing the reference to \"An emotion and cognitive based analysis of mental health disorders from social media data\". We have added it in.\n\nWe thank you for your thoughtful review and engaging with our work. We see the time and thought you have put into this and very much appreciate it.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Interesting goal and some useful results. Novel notion of \"biosocial marker\" well justified, relatively new metrics of emotion dynamics from text used in relation to mental health diagnosis\n- Many different disorders included\n- Clear and convincingly written, especially the Introduction\n- Sound and thorough statistical analysis", "reasons_to_reject": "- Stated goal of using emotion dynamics as \"biosocial markers\" is insufficiently addressed via the methodology, which doesn't focus on the social aspect almost at all\n- Text preprocessing details not sufficiently explained\n- Potential weakness, but not a serious reason to reject: text processing methods might be too weak to measure emotion dynamics correctly", "questions_to_authors": "In relation to the ANOVA test: why not actually test for normality instead of assuming it, at least by \"manual\" inspection of the scores histograms? (which are already illustrated in the paper)", "paper_topic_and_main_contributions": "This paper proposes a study on the relationship between emotion dynamics, as reflected in texts posted on social media, and a self-stated diagnosis of a mental disorder, including several different mental disorders.\nThe novelty is in the focus on emotion variability (aside from the usual emotional valence) in relation to mental health, measured with different metrics. The paper also introduces the term \"biosocial marker\", referring to how emotional dynamics can be useful as a diagnostic factor for mental health, but only seen in a social and cultural context (manifestations can be different depending on culture or demographics). The introduction is interesting and well developed.\n\nThe study uses an existing Twitter dataset. The methods are relatively simple in terms of natural language processing (measures of different emotion metrics based on lexicons), but very robust in terms of statistical analysis of the data.\nSome of the results are novel, especially given the new metrics used compared to the usual studies; the inclusion of several different disorders is also a strength.\n\nWhile the authors insist on the \"biosocial marker\" motivation in the introduction and explain it well, their methodology and discussion does little to accomplish the goal of studying the problem from this perspective, in my opinion. The contextualization of the relation between emotion dynamics and mental disorder diagnosis wrt culture or demographics is very brief. Generally, the experiments compute aggregates across the entire group, and some distinction between different demographic groups is made only in the Discussion section. The population is grouped only based on likes and followers, not on any other demographic criteria. \nIt would have been useful to at least have some more data on the demographics found in the posters in the dataset, and the issue of controlling for demographic/cultural criteria mentioned in the Discussion or Future Work sections.\n\nThe text processing methods might not be powerful enough to measure the emotion metrics properly, since, as far as I understand, they are based on lexicons and the analysis is done at word unigram level.\nSome more details regarding preprocessing methods are necessary: how was the text tokenized, what was done about emojis and punctuation, what other tokens were removed, how big is the vocabulary? These are useful especially given that the linguistic analysis is mostly vocabulary based.\n\n ", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "There are at least a couple of other studies which have discussed the idea of using emotion variability over time as a criteria for detecting mental health disorders, even if not using the same metrics:\n\nUban, Ana-Sabina, Berta Chulvi, and Paolo Rosso. \"An emotion and cognitive based analysis of mental health disorders from social media data.\" Future Generation Computer Systems 124 (2021): 480-494.\n\n\n", "Typos_Grammar_Style_and_Presentation_Improvements": "- The acronyms in the abstract are only introduced after\n- Line 199: \"As well...\" - incomplete sentence\n- Lines 203-206 are unclear to me\n- Line 250: \"negatively correlated\" is meant to be \"positively correlated\"?\n- Footnote on page 6: \"small\" effect defined as two different intervals\n- Line 525: \"diagnoses\"\n- Figure 1 caption: Same caption for subfigures (a) and (b) ?\n- Line 647: \"aspect\" -> \"aspects\""}}, {"review_id": "Jh1kUsMAxO", "reviewer": "Reviewer_9kEM", "summary": "", "questions": "Questions:\n\nA Line 621-624 - Emotions also vary due to one's age, gender and culture. E.g. Young adults are more likely to express arousal. Pride in Western society has higher positive valence and arousal compared to Japan. It is thus important to also control for age, gender and location of users. There are ML based methods to predict user demographics which authors may explore. \n\nSee Batja Mesquita, Nico H Frijda, and Klaus R Scherer.\n1997. Culture and emotion. Handbook of crosscultural psychology, 2:255–297\n\nEmi Furukawa, June Tangney, and Fumiko Higashibara.\n2012. Cross-cultural continuities and discontinuities\nin shame, guilt, and pride: A study of children residing in japan, korea and the usa. Self and Identity,\n11(1):90–113. \n\nB. In Sec 5.4, the high recovery rate for MHC group and similar arousal for emotional rise-rate - raise the question if there is an underlying platform effect (Twitter) for emotion variability. The authors might consider analysis on another social media platform.  \n", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer,\n\nThank you very much for your feedback and questions. We appreciate your time in reviewing our work. We address them below.\n\nWe fully agree with you that more can be done regarding the social aspects of biomarkers. We would like to clarify that the primary goal of this work is to assess the utterance emotion dynamics in language used by various populations associated with mental health conditions. That said, regardless of the extent to which social aspects are experimented with in a work, we think it is always important to refer to such linguistic markers as biosocial. Not treating them so has often, in the past, led to over generalized claims and one-size fit all solutions, at the expense of marginalized communities. Thus, our introduction situates the description of UED markers firmly as bio social. We also point out below some challenges with addressing social aspects responsibly which we will add in the revised version. Further, we did not want to force-fit more aspects when there wouldn't be adequate space to describe them. We hope for more careful and thoughtful work in this space in the future.\n\nControlling for age, gender, culture:\n\nIn our work, we control for popularity measures on social media as one such aspect. Some other aspects worth exploring include demographics such as age, gender, region, and culture as you are mentioning. We could control for gender, however this has become a sensitive topic as of late. Twitter’s current policy states:\n\n“You should be careful about using Twitter data to derive or infer potentially sensitive characteristics about Twitter users. Never derive or infer, or store derived or inferred, information about a Twitter user’s: Health (including pregnancy), Negative financial status or condition, Political affiliation or beliefs, **Racial or ethnic origin**, or beliefs, Sex life or sexual orientation, Trade union membership, Alleged or actual commission of a crime.” \n\nEmotion Variability on Twitter:\n\nYes, exactly as you are pointing out. Variability could have an impact on some measures. In fact, looking into this further after the submission deadline we found what is referred to as the “inertia-instability paradox” in psychology (Koval et al., 2013; Bos et al., 2019). When controlling for variability it appears that these two emotion dynamics metrics are no longer associated with depressive symptoms (Koval et al., 2013; Bos et al., 2019). We will add in analyses controlling for variability to see what role variability plays in UED.\n\nKoval, Peter et al. “Affect dynamics in relation to depressive symptoms: variable, unstable or inert?.” Emotion (Washington, D.C.) vol. 13,6 (2013): 1132-41. doi:10.1037/a0033579\n\nBos, Elisabeth H et al. “Affective variability in depression: Revisiting the inertia-instability paradox.” British journal of psychology (London, England : 1953) vol. 110,4 (2019): 814-827. doi:10.1111/bjop.12372\n\nWe thank you for your thoughtful review and engaging with our work. We see the time and thought you have put into this and very much appreciate it.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Emotion variability as a marker of mental health disorders, is intuitive yet understudied. The authors provide a detailed analysis highlighting its potential for detecting MHC which could be of interest to research community working on similar problems. ", "reasons_to_reject": "See Questions for the authors.", "questions_to_authors": "Questions:\n\nA Line 621-624 - Emotions also vary due to one's age, gender and culture. E.g. Young adults are more likely to express arousal. Pride in Western society has higher positive valence and arousal compared to Japan. It is thus important to also control for age, gender and location of users. There are ML based methods to predict user demographics which authors may explore. \n\nSee Batja Mesquita, Nico H Frijda, and Klaus R Scherer.\n1997. Culture and emotion. Handbook of crosscultural psychology, 2:255–297\n\nEmi Furukawa, June Tangney, and Fumiko Higashibara.\n2012. Cross-cultural continuities and discontinuities\nin shame, guilt, and pride: A study of children residing in japan, korea and the usa. Self and Identity,\n11(1):90–113. \n\nB. In Sec 5.4, the high recovery rate for MHC group and similar arousal for emotional rise-rate - raise the question if there is an underlying platform effect (Twitter) for emotion variability. The authors might consider analysis on another social media platform.  \n", "paper_topic_and_main_contributions": "The authors demonstrate correlation between patterns of emotion change learnt from language and mental health disorders on an existing Twitter dataset with a focus to examine if UED can be considered a biosocial marker for mental health. ", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Figure 1 - the title is the same for (a) and (b). "}, "extra_text": {}}, {"review_id": "f29Insom17", "reviewer": "Reviewer_Vx9h", "summary": "", "questions": "* Did you apply a Bonferroni correction (for example) to adjust for your large number of comparisons? \n\n* For Table 3, what does \"across groups\" mean? I thought groups meant the MHCs. \n\n* Footnote 2 (\"An effect size...\"), where did you get these levels?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer, \n\nThank you very much for your feedback and questions. We address them below.\n\nRe VAD and V-A, A-D, V-D correlations:\n\nWe appreciate you bringing these points up. The VAD Lexicon is a widely used resource in NLP, Psychology, and many other disciplines. It is especially popular in works exploring mental health and language where interpretability is of utmost importance. It has close to 500 citations since the 5 years of its publication at ACL 2018 main conference, including many in the CL Psych literature. \n\nRe I would imagine the correlations between the V, A, and D scores are very high.\n\nThis is not the case. The inter-dimension correlations listed in the 2018 ACL paper are:\nV–A: -0.26\nA–D: -0.30\nD–V: 0.48\nThese indicate a weak inverse correlation for V–A and A–D, and a moderate correlation for D–V. (These inter-dimension correlations are similar to what Warriner et al. 2014 found in their lexicon.) This demonstrably shows the V, A, and D are measuring different things. \n\nDominance being the dimension of helplessness – having a feeling of control, \n\nAnd\n \nArousal being the dimension of calmness/sluggishness – excitement/anxiety \n\nhave direct relevance to mental health conditions (the feelings of helplessness, dullness, and anxiety are known correlates with conditions such as depression, anxiety disorder, etc.)\n\nJohnson, S. L., Leedom, L. J., & Muhtadie, L. (2012). The dominance behavioral system and psychopathology: evidence from self-report, observational, and biological studies. Psychological bulletin, 138(4), 692–743. https://doi.org/10.1037/a0027503\n\nMehrabian, A. (1997). Comparison of the PAD and PANAS as models for describing emotions and for differentiating anxiety from depression. Journal of Psychopathology and Behavioral Assessment, 19(4), 331–357. https://doi.org/10.1007/BF02229025\n\nMehrabian, A. (1995). Distinguishing depression and trait anxiety in terms of basic dimensions of temperament. Imagination, Cognition and Personality, 15(2), 133–143. https://doi.org/10.2190/JB3J-LL1E-GYGY-D0RJ\n\nWork in Psychology has largely only explored valence – notably because it has long been shown the most important emotional signal, and most easy to obtain through self-reports (compared to other dimensions of emotions). Since the UED approach and the VAD lexicon allow exploration of arousal and dominance and because these two emotional dimensions are highly relevant to mental health conditions, we explore them here in this work. \n\nUnpacking results:\n\nWe will add in a table highlighting the expected associations based on literature in psychology to help better contextualize our findings.\n\nWe are interested in whether UED is a meaningful indicator of mental health and well-being. While that is our overarching research question, we explore this through the various metrics in the UED framework (e.g., average emotion intensity, variability, rise rate and recovery rate). Analyzing the relationship for each UED marker between MHCs and the control then become subquestions, allowing us to answer our overarching question of if UED are biosocial markers for mental health.\n\nQ: Did you apply a Bonferroni correction (for example) to adjust for your large number of comparisons?\n\nA: Yes, we applied the Games Howell correction for multiple comparisons. We will state it explicitly in the paper.\n\nQ: For Table 3, what does \"across groups\" mean? I thought groups meant the MHCs.\n\nA: By “across groups”, this is referring to each of the MHCs. We will update this caption to make it more clear as the following: \n\n“Valence: The mean UED metrics for each MHC. Those with an asterisk were significantly different from the control group.”\n\nQ: Footnote 2 (\"An effect size...\"), where did you get these levels?\n\nA: These levels are from Fields, 2013. Andy Fields is a well-known and respected statistician. However there are also guidelines by Cohen, 1992. As selecting effect size guidelines is largely dependent on context and customs in the field, we select Fields’ guidelines since we are performing exploratory analyses and predict that UED will be more difficult to predict.  \n\nField, Andy. 2013. Discovering Statistics Using IBM SPSS Statistics. sage. \nCohen, Jacob. 1992. “A Power Primer.” Psychological Bulletin 112 (1): 155.\n\nThank you for those references. They are relevant as you are mentioning and we have added them in. Also, we’ve addressed the typos & comments in the paper. \n\nWe thank you for your thoughtful review and engaging with our work. We see the time and thought you have put into this and very much appreciate it.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The authors address an important question. Humans (and their language) are temporally dynamic (as well as many other things, e.g. hierarchical) and most NLP methods to measure emotion do not take this into consideration. This is especially important as it relates to mental health.", "reasons_to_reject": "The paper doesn't feel fully fleshed out in terms of its implications. We have 4 metrics, 3 emotions, and 7 MHCs, which is a fairly large space. Do we have any reason to suspect that there will be differential patterns to the results which depend on metric/emotion/MHC? For example, should rise rate for dominance matter for depression? \n\nIf \"yes\", then I think there are some issues. First, methodologically, the main weakness of the paper is in the VAD lexicon. I'm not convinced there are any meaningful differences between V, A, and D. For example, looking at the columns in Table 4 or Tables 3/5/6. I would imagine the correlations between the V, A, and D scores are very high. This is most likely a problem with the model, but, again, if your claim is that are differential patterns to these dynamics, then you need to show that this model is measuring three different things. \n\nSecond, your results need to be unpacked more. How does this all relate back to psychopathology? Are there any takeaways other than \"we should measure dynamics\"? A table highlighting *expected* associations (based on literature or a preregistration) would be super helpful. \n\nIf \"no\", then it's unclear to me that the 4 research questions are actually separate questions. The authors simply restate the same question but use a different UED. It seems like there is one central question: do dynamics differ between controls and MHCs? To be fair, the authors claim \"this work provides important early evidence\", so I don't think the authors are actually over-claiming anything, but I do think the paper could be reframed.", "questions_to_authors": "* Did you apply a Bonferroni correction (for example) to adjust for your large number of comparisons? \n\n* For Table 3, what does \"across groups\" mean? I thought groups meant the MHCs. \n\n* Footnote 2 (\"An effect size...\"), where did you get these levels?", "paper_topic_and_main_contributions": "The paper attempts to measure emotion dynamics (average emotion, emotion variability, rise rate, and recovery rate) using a data set of Twitter users who have self disclosed various mental health conditions (MHCs). The authors then show that emotion dynamics (where emotions are defined as valence, arousal, and dominance) tend to be different when comparing Twitter users with MHCs to those who do not.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "The 2022 CLPsych shared task seems super relevant:\n\nTsakalidis, A., Chim, J., Bilal, I. M., Zirikly, A., Atzil-Slonim, D., Nanni, F., ... & Liakata, M. (2022, July). Overview of the CLPsych 2022 shared task: Capturing moments of change in longitudinal user posts. In Proceedings of the Eighth Workshop on Computational Linguistics and Clinical Psychology (pp. 184-198).\n\nThis also seems relevant, as they show language predicting mental health diagnoses is better when the language is measured closer in time to the diagnosis:\n\nEichstaedt, J. C., Smith, R. J., Merchant, R. M., Ungar, L. H., Crutchley, P., Preoţiuc-Pietro, D., ... & Schwartz, H. A. (2018). Facebook language predicts depression in medical records. Proceedings of the National Academy of Sciences, 115(44), 11203-11208.\n\nAlso, there is some work that shows that emotion and mental health can be tracked across *populations* temporally. See Hannah Metzler & David Garcia for great work in this area as well as:\n\nMangalik, S., Eichstaedt, J. C., Giorgi, S., Mun, J., Ahmed, F., Gill, G., ... & Schwartz, H. A. (2023). Robust language-based mental health assessments in time and space through social media. arXiv preprint arXiv:2302.12952.", "Typos_Grammar_Style_and_Presentation_Improvements": "Figure 1 (a) and (b) have the same caption. \n\nTable 4: You are already using \"*\" to mark significance in Table 3, so please don't use that symbol for something else. \n\nTable 4: I think this table could be more readable if you simply used V, A, and D instead of *, +, and a triangle. \n\nLine 634: \"valence, arousal, and, dominance\" should have one less comma"}}]

Author rebuttal:

Meta-review: This paper studies the relationship between mental health disorders and social media post emotion dynamics (tweets). The work highlights correlations between mental health diagnoses and linguistic cues about emotion dynamics. The reviewers are in agreement that the topic is interesting and that the work is generally sound. The reviewers' concerns seem to have been addressed during the discussion period.
Decision: Accept-Main