A Psychologically Informed Part-of-Speech Analysis of Depression in Social Media (2021)

Abstract: In this work, we provide an extensive part-of-speech analysis of the discourse of social media users with depression. Research in psychology revealed that depressed users tend to be self-focused, more preoccupied with themselves and ruminate more about their lives and emotions. Our work aims to make use of large-scale datasets and computational methods for a quantitative exploration of discourse. We use the publicly available depression dataset from the Early Risk Prediction on the Internet Workshop (eRisk) 2018 and extract part-of-speech features and several indices based on them. Our results reveal statistically significant differences between the depressed and non-depressed individuals confirming findings from the existing psychology literature. Our work provides insights regarding the way in which depressed individuals are expressing themselves on social media platforms, allowing for better-informed computational models to help monitor and prevent mental illnesses.

## introduction mental health disorders are a common problem in our world. currently, mental health issues are on the rise: there is a 13% increase in the past decade according to world health organization (who) foot_0 , with depression being at the forefront. many mental illnesses remain undiagnosed due to social stigma, leading people to live 1 in 5 years of disability in their lifetime. with the rise of social media websites, interdisciplinary researchers in natural language processing, psychology and network analysis have turned their attention to automatically detect and monitor mental health manifestations through users' individual activity on social platforms (e.g. facebook, twitter, reddit). the research is primarily focused on analyzing users' texts from posts and comments and determining, through computational linguistics models, the risk for various mental conditionsself-harm, depression, addictions etc. research is fulled through curated datasets @xcite @xcite with texts from primarily reddit and twitter. at the forefront of incentivising interdisciplinary research on monitoring mental health on social media are workshops such as the early risk prediction on the internet (erisk) workshop foot_1 and the workshop on computational linguistics and clinical psychology (clpsych) 3 . clpsych and erisk are two significant initiatives focusing on the interdisciplinary research area between computational linguistics and psychology. the erisk workshop, from the conference and labs of the evaluation forum (clef), focuses on the technologies that can be used for early risk detection of different pathologies or safety threats @xcite . in five years of existence, the workshop addressed multiple mental health problems: pathological gambling, depression, self-harm and anorexia. the clpsych workshop was co-located with several international conferences on natural languages processing, the last edition was co-located with the annual conference of the north american chapter of the association for computational linguistics (naacl). throughout the seven editions of this workshop, it hosted shared tasks on depression, post-traumatic stress disorder (ptsd) @xcite , labeling crisis posts from the peer-support forum reachout foot_3 @xcite , predicting current and future psychological health from childhood essays @xcite , the degree of suicide risk (no risk, low, moderate, or severe risk) @xcite and suicide risk prediction from real data donated through ourdata-helps foot_4 @xcite . in the present study, we perform a part-of-speech analysis and contribute to the understanding of the differences in social media discourse between depressed and non-depressed individuals. we focus on the differences in part-of-speech use and ground them in the existing literature from psychology researchers. we aim to answer the following two research questions: rq1: are there significant differences in part-ofspeech use between individuals with self-reported depression diagnosis and control? rq2: can these part-of-speech features be used alone to differentiate individuals with depression from controls? ## related work detecting the manifestations of mental health disorders from social media is an interdisciplinary research problem for researchers from psychology, natural language processing and network analysis. the two main approaches used to detect cues of depression from online users are: extracting linguistic features for a quantitative analysis or using automated models for classification. the differences in language between depressed and non-depressed individuals focus on greater use of negative words, the personal pronoun "i" @xcite , more words with negative polarity, frequent dichotomous expressions (e.g. always, never) @xcite , cues of rumination (reflected in greater use of past tense verbs) @xcite in texts from depressed individuals. most linguistic features are extracted using the linguistic inquiry and word count (liwc) lexicon @xcite . liwc provide a list of dictionary words for more than seventy categories: part-of-speech (e.g. personal pronouns, first-person personal pronouns, nouns, present/future/past verbs, adjectives), psychological processes (e.g. social, affective, cognitive processes), personal concerns (e.g. work, money, religion, death), etc. it is used to detect cues of depression @xcite @xcite , neuroticism @xcite , to explore the language of suicide poets @xcite , etc. other approaches to mental illnesses detection from text rely on character and word n-grams @xcite or topic modelling techniques @xcite . recent studies analyzing the online discourse of social media users with depression have focused on other particularities of language, such as offensive language. birnbaum et al. (2020) found that depressed individuals use more swear words in their facebook messages compared to controls. @xcite apply offensive language identification techniques and show that users with depression diagnosis use more offensive language in their reddit posts, individuals manifesting signs of depression in their posts having a more profane language and fewer insults targeted towards other individuals or groups. computational models used to detect the cues of depression from social media texts rely on traditional machine learning classifiers (e.g. svm, naïve bayes) @xcite @xcite , cnns @xcite , rnns @xcite or transformer models @xcite @xcite . in the multimodal framework (involving text, voice and visual cues), the use of syntactic features (e.g. pronouns, adverbs usage) seems to improve the performance in depression detection, further emphasizing the relationship between linguistic features and depression @xcite . recently, researchers began exploring the interpretability and explainability of the decisions made by automatic classifiers for mental illnesses detection from social media to further understand the manifestations of different mental disorders in written language @xcite . ## data in our experiments, we use the dataset from the erisk workshop containing posts written in english from the social media platform reddit. the erisk 2018 dataset @xcite was created for the early detection of depression task. it contains two classes of users, depression and control. users from the depression class were annotated by their mention of diagnosis in their posts (e.g. "i was diagnosed with depression"), but expressions such as "i have de-pression", "i am depressed" were not taken into account. the authors removed the mentions of diagnosis from the dataset. users from the control group are random individuals who do not have any mention of diagnosis in their post, including those active in the depression subreddit foot_5 . the training dataset provided by the organizers contains 135 depressed users and 752 controls, while the test dataset contains 79 depressed users and 741 controls. we use both train and test splits in our exploration, consisting of a total of approximately 90,000 submissions from users with a depression diagnosis and over 985,000 in the control group. ## methods part-of-speech analysis for each post in our dataset, we use the spacy foot_6 part-of-speech tagger to extract the corresponding tags and also morphological features (e.g. person and number for pronouns) for each word. we extract the universal pos tags foot_7 and the ones from the penn treebank tagset @xcite . we use the latter to explore the differences in the verb tenses. we assign the tenses according to their tags: vbd and vbn corresponding to past tense, vbg, vbz and vbp corresponding to present tense, and md tag before a vb corresponds to the future tense @xcite . from the morphological features provided by the spacy tagger, we extract the person and the number for all the pronouns. after this analysis, we use the following features in our exploration: for each of these features, we compute their frequency for each post in the dataset. for the universal pos, the frequency is computed as the number of occurrences of a specific tag normalized by the total number of tags in a post. for verb tenses, the frequency of each tense is calculated as a percentage of the total number of verb occurrences. for the three kinds of personal pronouns, each frequency is computed as a percentage of the number of all personal pronouns. the frequency of singular and plural first-person pronouns is computed as a percentage of all first-person pronouns. to further explore the part-of-speech usage in the social media dataset, we also use some special measurements @xcite : pronominalisation index (pi): reflecting the usage of pronouns, instead of another part-of-speech (e.g. nouns). it is computed as the number of pronouns divided by the number of nouns @xcite . formality metric @xcite : moreover, we test the discriminatory power of both pos tags frequency usage in users' texts and the specific computed indices. for this, we employ a random forest classifier on the train set of the erisk 2018 dataset on the aforementioned features. to interpret the trained model and to estimate the importance of each feature, we employ shapley additive explanations (shap) (lundberg and lee, 2017) to measure each feature's contribution to the classifier decision. shap offers a game-theoretic approach to quantify feature importance, aligned with human intuition. classification we opted for a simple random forest model, trained with 50 estimators and a max depth of 15, to avoid overfitting, with balanced class weights, since the dataset is heavily imbalanced. on the test set for the erisk 2018 dataset, we obtain a weighted f 1 -score of 78.37% (with balanced class weights) and a macro f 1 -score of 51.93%. while the classification task is difficult, we are interested in exploring the feature importances of the model, which shed light on the model behaviour and provide us with insights regarding which pos tag is most discriminatory. we further present our findings and provide interpretations and discussions based on recent findings in psychology. ## results and discussion addressing our rq1, we compute the welch t-test for all our features and demonstrate that there are statistically significant differences (@xmath0.001) in part-of-speech usage between depressed and non-depressed individuals. in this section, we present these differences and their interpretation from the psychology research. content part-of-speech in figure 1 , we present the usage of content words for the two classes from the erisk 2018 dataset. individuals diagnosed with depression tend to use fewer common and proper nouns in comparison with control users. they also use more verbs and adverbs in their posts. the discourse is focused around actions, but with fewer entities (e.g. nouns), showing a defective linguistic structure with less interest in the environment (e.g. people, objects) @xcite . to further understand these differences, we pay a closer look at the frequencies of nouns and verbs in the social media discourse. we compute the keyness score @xcite for verbs and nouns separately. in the keyness analysis, we compare the frequencies of nouns and verbs from the posts written by individuals with depression diagnosis (target corpus) in comparison to the posts from control users (reference corpus). in figure 2 , we present the top 20 verbs and nouns from each corpus, ordered by their log-likelihood ratio (g 2 ) @xcite . rumination is a cognitive process focusing on past and present negative content and resulting in emotional distress @xcite . it is present in several mental health disorders (e.g. depression, anxiety, obsessive-compulsive disorder, post-traumatic stress disorder). in depression, rumination is defined as behaviours and thoughts that focus one's attention on one's depressive symptoms and on the implications of these symptoms @xcite . the rumination, as a response to depression, focuses the person's attention on their emotional state and inhibits the actions necessary to distract them from their mood. in figure 3 , when comparing the usage of the three verb tenses (present, past and future) between the two groups, we expected that signs of rumination would be present in our analysis of verb tenses, with texts from depressed users being shifted into the past @xcite , but this result is not found in this sample of individuals. regarding the usage of future tense, depressed people have a lower frequency of verbs portraying future actions. this result may be a consequence of anhedonia, people suffering from depression reporting lower anticipatory pleasure, and thus talking online less about their future plans. anhedonia, defined as markedly diminished interest or pleasure in all, or almost all activities most of the day, nearly every day @xcite , is a common symptom of depression. the higher frequency of cognitive verbs (e.g. feel, think, know) in the texts written by depressed individuals indicates the cognitive impairments and judgement issues specific to depression. people with depression have cognitive deficits and biases in the processing of emotional information and they are unable to adaptively regulate their emotions @xcite . individuals with or without depression may not differ in their initial response to an adverse event. still, they differ in their ability to recover once they have experienced the negative emotion. depressed individuals are not able to repair their mood. instead, they remain in a negative state of mind, which is related to increased negative thoughts, selective attention to negative stimuli and greater accessibility of negative recollections @xcite . in comparison, the individuals from the control group use more action verbs (e.g. vote, lead, show, begin, create. in addition, depressed individuals are more passive; they have a lower level of general activity, consistent with symptoms of depressive disorder @xcite . being an online social media platform similar to forums, reddit is organized in subreddits with specific topics. it also has several communities dedicated to mental health problems. compared to other social media platforms that require realname authentication (e.g. facebook), reddit affords users anonymity or pseudo-anonymity. complete anonymity is almost impossible, users providing bits of information with every interaction on the platform (e.g. comments, posts). reddit allows users to create throwaway accounts to engage temporary without revealing their identity @xcite . these types of accounts are used to discuss sensitive information or stigmatizing problems. de choudhury and de (2014) study the mental health discourse on reddit and show that its communities allow a high degree of information exchange related to mental health. users use reddit to self-disclose the challenges faced in their daily lives or in personal relationships. they also seek emotional support or specific informa-tion about mental illnesses diagnosis and treatment. their study demonstrates that reddit fills the gap between social media platforms like twitter and facebook and online health forums regarding mental health discourse. examining the frequencies of nouns in the erisk 2018 dataset, we show in figure 2 that the users with self-reported depression diagnosis use their reddit account to disclose and discuss their mental health problems (e.g. depression, anxiety, therapist) or their personal relationships (e.g. friend, boyfriend, relationship, mom, dad). the process of seeking online support is also shown in the frequency of verbs: feel, talk, diagnose, help. even if the dataset contains control users active in the depression subreddit, the majority of control users seem to post on other themed subreddits (e.g. politics). bucur and dinu (2020) perform a topic modelling analysis and show that texts from control users are found in topics related to their hobbies, as opposed to depressed people, who are more focused on their feelings and life events. our results are in line with this study, the users from the control group use more politics-related words (e.g. trump, government, president, news, vote). functional part-of-speech in figure 4 , we present the frequencies of functional part-of-speech for depressed and control users. depressed individuals generally use fewer functional words in their texts in comparison with control users, with the exception of pronouns. neurons involved in content words processing are equally distributed over both hemispheres, while function words are processed mainly in the left hemisphere @xcite . lower preposition usage may be an outcome of deficient activation of the left hemi-sphere regions, responsible for producing more abstract lexical units @xcite . function words are highly social, the capacity to use function words requires social skills @xcite . the figure 5 shows the differences in personal pronouns used by the two groups. the high frequency of first-person singular pronouns indicates a higher self-preoccupation in depressed individuals, as opposed to the first-person plural pronouns, which shows collective attention. second and thirdperson pronouns indicate social interactivity and contain references to other people or things in the environment @xcite . depressed users use more first-person singular pronouns. the frequencies of first-person plural pronouns are inversely proportional to the firstperson singular pronouns frequencies. this result is in line with the self-focused attention tendency (sfa) in depressed individuals. sfa is a cognitive bias linked to depression; the high frequency of first-person singular pronouns in speech or written text is considered a linguistic marker of sfa @xcite . individuals with depression have deficits in other-focused social cognitions, they are impaired in theory of mind reasoning and empathy. theory of mind enables people to make inferences on the behaviour of others and their own @xcite . erle et al. (2019) show that individuals exhibiting high levels of depressive symptoms were impaired on tasks involving overcoming their egocentrism. the usage of fewer first-person plural pronouns in texts from users diagnosed with depression may be a sign of a lesser sense of belonging. the information-processing biases of depressed individuals make it hard for them to perceive cues of acceptance and belonging in social interactions, and to view ambiguous social interactions as being negative @xcite . figure 7: feature importances of the randomforest model for depression classification using pos frequencies and pos indices. as shown in our analysis, the usage of pronouns (1st person singular/plural) and proper nouns is the most discriminatory. pronominalisation and formality indices in figure 6 we present the results for the two metrics computed on part-of-speech tags. depressed individuals have a higher pronominalisation index, using more pronouns in spite of nouns. this finding is also found in the language of people with self-destructive behaviour, insufficient activation of the cerebellum being associated with suicidal behaviour @xcite . depressive individuals also use less formal language. they have a more contextualized discourse on social media, providing information about the context in order to avoid ambiguity @xcite . addressing our rq2, we show in figure 7 the shapley values for a random sample of 1500 posts from the erisk 2018 test set. a higher shapley value corresponds with a higher importance in the final model decision based on the feature value. we used all computed pos features and indices in our model, but show only the top 20 for clarity. it is evident that from the summary plot, the absence of proper nouns is the most discriminatory factor in the decision to classify a person as depressive. moreover, the use of pronouns (also evident in the pronominalisation index) is highly correlated with positive model output. the high usage of first-person singular pronouns and low usage of first-person plural pronouns confirms both our findings in the exploratory analysis and psychology literature. ## conclusion in this work, we provide an extensive analysis of part-of-speech usage in social media texts from depressed and non-depressed users. our findings are confirmed by studies in psychology and show that individuals diagnosed with depression use more pronouns (especially first-person singular pronouns) and verbs, and fewer common and proper nouns. their social media discourse revolves around their life experiences and sentiments, as opposed to control users who are not interested in discussing their problems online. moreover, we also provided insights into the discriminatory power of pos frequencies by employing shap, a game-theoretic approach for model interpretation. through this, we showed that depressive users can be characterized most easily, primarily through their usage of pronouns and proper nouns.