Mental Disorder Classification via Temporal Representation of Text (2024)

Abstract: Mental disorders pose a global challenge, aggravated by the shortage of qualified mental health professionals. Mental disorder prediction from social media posts by current LLMs is challenging due to the complexities of sequential text data and the limited context length of language models. Current language model-based approaches split a single data instance into multiple chunks to compensate for limited context size. The predictive model is then applied to each chunk individually, and the most voted output is selected as the final prediction. This results in the loss of inter-post dependencies and important time variant information, leading to poor performance. We propose a novel framework which first compresses the large sequence of chronologically ordered social media posts into a series of numbers. We then use this time variant representation for mental disorder classification. We demonstrate the generalization capabilities of our framework by outperforming the current SOTA in three different mental conditions: depression, self-harm, and anorexia, by an absolute improvement of 5% in the F1 score. We also investigate the situation when current data instances fall within the context length of language models and present empirical results highlighting the importance of temporal properties of textual data. Furthermore, we utilize the proposed framework for a cross-domain study, exploring commonalities across disorders and the possibility of inter-domain data usage.

## introduction in the contemporary world, mental health plays a crucial role in a person's overall well-being. the world health organization (who) 1 highlights the intensity of this matter by reporting that globally, one in every eight individuals suffers from a mental disorder. a comprehensive study @xcite reveals that over 50% of people worldwide confront a mental health issue at some point in their lives. the scarcity of adequately trained professionals hinders access to timely and effective intervention, motivating the need for automated mental disorder detection. social media platforms, like reddit, have become a widespread outlet for self-expression, thus becoming a source of user-generated content that may provide valuable insights into individuals' mental states. the mental state of a patient varies from time to time, depending on the severity of symptoms. figure 1 shows an example of a situation with such behavioural variation. the post at time t i+2 shows a peak in depression in the person, whereas the post at time t i+1 can be seen as the onset of depression. posts at time t i and t i+3 are indicative of the general behaviour of the person. these rise and falls in the sentiment polarity of social media posts represent temporal properties that can serve as important information for detecting mental disorders. these clues can be used to differentiate patients with mental disorders from healthy subjects efficiently. figure 2 : this figure shows the overall pipeline of our approach. here, (1) shows the generation of the anchor embedding from rmhd, (2) shows the creation of temporal representations of social media posts of an individual, and (3) depicts the classification of the temporal representations as control or condition. generating the anchor embedding is the first step which is followed by representing the posts in a temporal manner. these temporal representations are then used to train the time series classification model to detect the presence of a disorder. the central idea of recent language model-based approaches @xcite is to concatenate multiple texts to form a chunk and then apply a predictive model to each chunk separately. the final prediction is positive if most chunks are classified as positive. the major issues in such approaches are: 1) loss of temporal information: applying the predictive model on chunks separately leads to the loss of order and inter-post dependencies. 2) lack of global view: the concentration of positive class text in only a few of the chunks can lead to misclassification because of majority voting. 3) semantic noise: merging posts to form a single chunk with different semantics can diminish the significance of a single post and introduce noise in the data. for instance, a post indicating severe depression could be concatenated with the subsequent post discussing the positive outcomes after the treatment. to handle these issues, we must account for two aspects: 1) preserving the post-identity and order and 2) global classification i.e. classifying the entire sequence of posts together instead of classifying chunks of social media text. using current llms here can be challenging due to computational costs and the high context length of social media posts. for instance, the average context length of a subject is 11.6k words (max 89.4k) hence, the chunking-based methods can not address these concerns, while other llm-based ap-proaches may prove infeasible within the given context. this motivates the need for a representation technique capable of compressing information while retaining the time-variant properties of social media posts. we build upon these insights and propose a framework to account for these issues. we perform rigorous analyses to establish the importance of temporal properties in mental disorders (section 7.3). we also discuss the performance of the current language models when the input length is within the capacity of the language model (section 8). our source code is available on github foot_0 for academic purposes. our contributions are: in the subsequent sections, we discuss recent works that are relevant to our contributions. we then discuss the dataset used and the proposed method, followed by the insights from our experiments. ## related work in this work, control class refers to the class of healthy individuals not diagnosed with a psychiatric disorder whereas condition class refers to the class of individuals diagnosed with a psychiatric disorder. for our study, we focus on two mental disorders, anorexia and depression, as well as a mental condition, self-harm. for brevity, we collectively refer to them as mental disorders (like @xcite . mental disorder prediction from social media has seen significant development in the last decade. early works transitioned from the use of low-level handcrafted features like linguistic inquiry and word count (liwc) @xcite ; @xcite ; @xcite ) to high semantic features like word or document embeddings @xcite @xcite . this was succeeded by representation learning-based approaches @xcite @xcite , which operate on user-level prediction and eliminate the need for explicit feature engineering. other works leverage longitudinal data to capture unique patterns of emotional transitions shown by mental patients. these approaches use chunking to process m words ( trotzek et al. (2018 2021); jamil (2017 ) related to a specific subject. however, these approaches fail to incorporate the temporal variations between the posts of a subject because of the usage of chunking and majority voting (see section 1). a few studies closely align with our approach to constructing temporal representations of social 2020 ) created a time series representation of the mood profile using traditional sentiment retrieval models. @xcite proposed a multi-task framework aimed at predicting suicidality for bipolar disorder patients by categorizing individual posts into different suicidality levels. @xcite introduced graphbased approaches to model user interaction and the temporality of posts, while @xcite modeled "phases" at the post level, requiring postlevel annotations for suicide ideation detection. a significant limitation of these approaches is either their reliance on low-level features, which fail to capture the deeper semantic understanding of emotional aspects in human language, or the need for user interaction data and post-level annotations, which are not available in our setup. @xcite fine-tuned bert to generate emotional post representations and derived emotion transition matrices. however, emotional states were extracted individually from each post, overlooking inter-post dependencies. another drawback is their reliance on the first-order markov assumption for prediction. recent works like @xcite pre-trained a bert model on the social media and mental health dataset. @xcite performed a double domain adaptation (extending the pre-training process) on bert, incorporating data from reddit and mental health sources. these approaches again rely on chunking and majority voting while facing the limitation of high expense. in this work, we propose a novel framework which captures the temporal patterns of social media posts without the use of majority voting and chunking. ## datasets we use two datasets in this work: erisk evaluations and reddit mental health dataset (rmhd). ## baseline approaches we use all the baseline approaches introduced by @xcite for comparing our approach. mentalbert @xcite pre-trained bert on social media and mental health text, and disor-bert @xcite applied double-domain adaptation with random and guided masking. additionally, we use mpnet @xcite as a baseline in both zero-shot and fine-tuned settings. to compare the results with current sota language models, we report results on gpt3.5 turbo @xcite and mentalllama-chat-13b @xcite instruction tuned on large mental health data. we follow @xcite for evaluation and training. all the baseline methods used the chunking (@xmath0. the f1 score of the positive class serves as our standard metric for assessing the experimental results. we provide a detailed explanation for baselines in appendix b and present experimental results on additional baselines in ## results and observations the results of our experiments and the baseline results are summarized in the poor performance of gpt-3.5-turbo and mentalllama-chat-13b ( ## analysis our approach demonstrates improvement in selfharm and depression prediction while showing a competitive performance in anorexia prediction ( the social media posts of individuals diagnosed with self-harm use more provocative and intense language compared to posts about other mental conditions. for example, "i sliced my arm with a box cutter. it almost seeped through my long sleeve since it bled so much." these strong linguistic indications cause a huge contrast between the control subject's posts and the condition subject's posts, leading to a 9% improvement over sota. the low improvement in anorexia can be attributed to the high noise in the test set. in most posts by the condition class, individuals have a lot of irrelevant information. for example, "to be honest, i have no problem paying a small bit extra for amazon. their client service is really great." these kinds of instances result in noisy test data, resulting in a low improvement over sota. ## full context analysis in this section, we compare our approach with llms in the scenario when the data instances have context length within the capacity of the llm. the main purpose of this experiment is to understand if llms can understand and capture the temporality directly from the natural language. we perform this experiment using mentalllama @xcite because of its superior performance in mental disorder identification over gpt and other llms. for this experiment, we only consider the data instances with word count within the capacity of mentalllama (i.e. 2048 tokens). we report the data statistics in the ## transfer-learning setting this section aims to study the similarity between linguistic cues shared across mental disorders by applying the proposed framework. various psychological studies @xcite @xcite highlight the possible coexistence of multiple disorders in one person. overall, 25%-55% of patients with eating disorders showed self-harming behaviour @xcite ; at least 64% of the patients with anorexia experience co-morbid major depressive disorder @xcite and 36.6% of individuals with depression attempted suicide @xcite . the similarity between anorexia and depression can be explained by their common behavioural symptoms like feelings of low selfesteem and greater self-criticism @xcite . for example: "feeling overwhelmed today. skipped breakfast again; just couldn't face it. trying to stay positive, but some days are harder than others." this person shows depression and anorexia symptoms at the same time. the relationship between self-harm and depression can be understood by their shared symptoms like rumination, shame, guilt, regret, etc. @xcite . for example, a post like "it's hard to fight this guilt daily. i wish to forget this pain.", shows that the person is feeling depressed and has a self-harming tendency. notably, depression is commonly observed in patients diagnosed with self-harm and anorexia. however, the converse is not necessarily true. anorexia and self-harm are both considered maladaptive coping mechanisms occurring when an individual can not figure out healthy ways to channel their thoughts, resulting in harming oneself by self-starvation @xcite . to understand the commonality of the disorder d 1 with respect to disorder d 2 , we take the cosine similarity between the anchor embedding of the d 1 disorder with the post embeddings of d 2 disorder arranged in chronological order. after obtaining this series of cosine similarities (section 4), we use it as the train set and evaluate the test set of the disorder d 1 . this step aims to extract shared information about the mental disorder under consideration from the data of other disorders. for example, to study the linguistic similarity of depression with self-harm, we take the cosine similarity between the anchor embedding of depression and social media posts embedding from self-harm. using the acquired series as the training set, we report the test set results from depression. we observe that the results are significantly better than the random baseline and competitive with the sota in certain cases. we present results on all three pairs (6 combinations) in ## conclusion and future work this work proposes a novel framework to incorporate temporal representation of textual data for the identification of anorexia, depression, and selfharm from social media data. our methodology utilizes fundamental deep-learning architecture and surpasses llm-based baselines by accounting for temporality and the full context of the input data. our transfer-learning analysis highlights the overlapping linguistic cues among the disorders and hints at the possibility of leveraging data from different mental disorders. our work can be extended by exploring more complex mental disorders such as schizophrenia, personality disorders, bipolar disorder etc. the task of mental disorder classification can be enhanced by including other modalities like audio and visual signals that give insights into behavioural patterns. the proposed framework can be also extended beyond the mental health domain in scenarios which require a temporal understanding of natural language data.