Are You Really Okay? A Transfer Learning-based Approach for Identification of Underlying Mental Illnesses (2022)

Abstract: Evidence has demonstrated the presence of similarities in language use across people with various mental health conditions. In this work, we investigate these correlations both in terms of literature and as a data analysis problem. We also introduce a novel state-of-the-art transfer learning-based approach that learns from linguistic feature spaces of previous conditions and predicts unknown ones. Our model achieves strong performance, with F1 scores of 0.75, 0.80, and 0.76 at detecting depression, stress, and suicidal ideation in a first-of-its-kind transfer task and offering promising evidence that language models can harness learned patterns from known mental health conditions to aid in their prediction of others that may lie latent.

## introduction mental health conditions are a pervasive but historically often overlooked societal and individual concern @xcite . in recent decades their study has gained increasing priority, and within the past decade this study has extended to techniques for automated analysis and detection of mental health conditions, including through patterns detected in written and spoken language @xcite . most work on automated assessment of mental health seeks to identify and possibly alleviate specific mental health conditions. researchers have focused on a myriad of target illnesses and diagnoses such as depression @xcite , schizophrenia @xcite , or even suicideal ideation foot_0 @xcite . however, to date they have not yet examined the overlap or interplay between these target illnesses. this overlap may present a valuable source of information, particularly in the resource-poor settings common in mental health and healthcare applications more generally. in this paper we ask three important research questions centered on the interplay between the linguistic footprints of known and latent mental health conditions (mhcs), foot_2 and present answers to them with evidence. our first question relates to the linguistic markers of mhcs. we comprehensively examine existing psycholinguistic and mental health research to search for common underlying threads ( §3). to answer our second question, we investigate the relation between the identified features using well defined and trusted nlp baselines ( §4). finally, we answer our last question by experimentally determining the success with which we can use similar and dissimilar linguistic feature spaces to predict the presence of latent mhcs ( §5). to do so, we leverage transfer learning to achieve a strong benchmark accuracy of 85%. ## background according to the national institute of mental health, 43.6 million adults (nearly 18.1% of the u.s. population) experience mental health conditions in a given year. 3 oftentimes, symptoms may be recognizable when interacting with close relations @xcite or even on social media @xcite . @xcite investigate the popularity of social media as an outlet for mental health discussion at length, finding reasons including anonymity, sense of empowerment, sense of community, and perceptions of the internet as a safe space. a growing number of approaches have sought to leverage social media data to aid in the automated identification of specific mhcs, with work to date including automated detection of depression @xcite @xcite , post-traumatic stress disorder @xcite , anxiety @xcite , and stress @xcite . however, these approaches have lagged behind the state of the art in more fundamental nlp tasks. in particular, work harnessing high-powered transfer learning models has remained either scarce or singularly focused on one illness @xcite . we aim to fill this translational gap by synthesizing fundamental progress with the applied problem of detecting the presence of underlying mhcs. we follow blodgett et al. (2020)'s lead and model our approach not only on existing nlp models, but on findings from pyscholinguistic and other domainspecific literature as well, including those correlating retention @xcite , cognitive attention and complexity @xcite , reasoning @xcite , and problem-solving skills @xcite with specific mental health conditions. little has been done towards this problem with rqs of multi-task learning from social media being very recent @xcite . this work, to the best of our knowledge, is the first of its kind to study correlation among diseases in both theory and practice. we examine prior literature to identify correlating themes across illnesses, analyze language data from individuals with different mental health conditions to find practical correlations and trends, and present transfer learning-based classification models to identify undiagnosed illnesses given known features grounded in mental health and psycholinguistic theory and nlp practice. ## feature correlation across varying mental health conditions a natural question that arises when using social media data is its reliability as an information source. social media is increasingly seen as a popular choice and acceptable platform for healthcare information exchange @xcite , and its use has been investigated in numerous predictive healthcare tasks. classical models (e.g., support vector machines) trained on simple textbased features have reliably predicted mental health emergencies @xcite . audio features have also been found to be excellent markers of mood or other prosodic signals, including for automated detection of @xcite . language models have demonstrated an ability to learn powerful, quantifiable signals from tweets to predict users' mental states @xcite , and more clinically advanced mental health conditions such as psychosis have also been detected using short appraisals of social media posts @xcite . predicting depression on social media is a long standing research track @xcite , and social media has also shown that signals to identify suicidal ideation can be traced with high efficacy @xcite . platforms like reddit foot_4 can be instrumental in terms of support, resources, and self-disclosure about mental health @xcite . one of the first traceable thematic identifications of correlated, quantifiable information regarding mental state and wellbeing was by @xcite , suggesting that a lack of social support combined with social isolation was present in patients showing signs of depression or post-partum depression. the same work also identified effects of psychological stress on attitude, emotion, and behavior. the relationship between social isolation, loneliness, and clinical depression was later also validated by @xcite , and the relationship between latent stress and surface depression has since persisted as a recurring theme across mental health literature @xcite . @xcite found that high levels of stress or distress are related to higher levels of suicidal ideation. @xcite also pointed to trepidation, frustration, annoyance, helplessness, and again stress as major themes correlating with expression of mental illness. depression and stress co-exist in latent forms for other surface illnesses such as schizophrenia as well, as demonstrated by @xcite who extracted liwc @xcite features from social media data to detect advanced psychosis and schizophrenia in social media. perhaps one of the most interesting finds in translational mental health research is the direct relationship between depression, suicidal ideation, and stress @xcite . @xcite provide evidence that depressive language correlates with sustained periods of low sentiment and has similar topical themes to language produced by suicidal or dysphoric individuals. although nlp researchers have experimented with a wide range of linguistic features for mental health assessment and analysis, several have emerged as being particularly discriminating. metadata such as hashtags or the name of a forum @xcite can be powerful features to detect mental health conditions such as suicidal ideation @xcite . specific words or hashtags can be used to identify personality profiles, as well as stigma or awareness of mental health conditions on social media @xcite . degrading or negative n-grams (e.g., crazy, mad, or nuts) can distinguish personality types and mental health outlook (hwang and hollingshead, 2016), and part-of-speech (pos) tags can also be informative in social media data @xcite . @xcite characterize speech at a granular level with social and personal profiles, and present liwc, a powerful tool to extract such features @xcite . n-grams have been powerful markers of depression or ptsd @xcite , and can be valuable tools for feature discovery @xcite . lexicon-based features, word embedding features, or annotated posts from social media are also informative @xcite . across this systematic review of mental health within nlp literature, the following key relations become evident: we experiment further with these features in the following subsections. ## results and discussion our experiments offer a first-of-its-kind examination of transfer learning across multiple mhcs. since there are no directly comparable transfer learning models, we compare individual task performance to the respective benchmarks established by the dataset creators using task-specific models. these models leverage many architectures and feature types, intersecting in their use of vector representations and liwc features. specifically, we compare to the following: shing et al. (2018) use liwc features, word2vec embeddings, bag-of-words features, lda features, and nrclex features with a cnn classifier. model depression stress suicide losada and crestani (2016) 0.66 --turcan and mckeown (2019) -0.79 -shing et al. (2018) --0.42 ours 0.75 0.80 0.76 we also experiment with an alternative model grounded in psychological evidence that suicide may occur as a natural escalation from stress and then depression. we train our same core cnn model first on stress, then on depression, and then on suicide and achieve an 85% accuracy at the target task of suicide. our bilstm model achieves an accuracy of 75% on depression when first trained on stress, and then an accuracy of 76% on suicide when subsequently trained on depression, echoing this trend albeit to a lesser degree. the strong performance of this technique further supports our finding that shared language characteristics across mhcs make this a promising and impactful sandbox for experiments with transfer learning. ## research answers in §1, we asked three important research questions. following our analyses, we present concrete answers to them in this section. how do features relate across multiple mhcs? mental health conditions have similar manifestations in language, and correspondingly in their linguistic signatures. we provide evidence for this in our literature review ( §3) and analyses ( §4). although we cannot through linguistic analysis conclusively measure the similarity of two mhcs, we can discern that the language usage and its features have significant overlap across mhcs (see figure 1 and tables 1 and 2 , and other figures and tables in the appendix). can we represent different mhcs under the same feature spaces and find relations? yes, using semantically descriptive features such as liwc it is possible to find relations ( §4). we demonstrate that using standard nlp tools such as lda or n-gram language modeling it is possible to see similar themes and topical relationships ( §4). can we identify underlying mhcs using the language of known ones? yes and no! while models trained on one task and transferred efficiently can predict unseen mhcs with a higher accuracy then when predicting them using only target domain data, these are linguistic classifications only ( §5). ai models are still far from being able to conclusively identify mhcs, and should not be considered as replacements for professional mental health care. given these research answers, we close by discussing how we can carry this forward and what it means for nlp in mental health. ## conclusion and future directions in this work, we examine the utility of transfer learning for the identification of three mhcs: depression, stress, and suicidal ideation. these mhcs vary in their clinical classification and severity. depression is formally defined as a mental illness @xcite , stress is a process which may ultimately result in mental illness @xcite , and suicidal ideation is classified as a disorder @xcite . although we achieve promising performance in detecting these conditions, nothing-not even actual diagnosis by a human expert-can conclusively identify a mental illness with 100% certainty @xcite . we presented a qualitative exploration of the overlap and interplay between language and mental health across multiple mhcs, and also presented quantitative correlations among words, tokens, themes, topics, and large feature space representations using well-known, established nlp methods. finally, we introduced a transfer learning model to predict unseen mental health conditions using similar and dissimilar feature spaces, the first of its kind. our model outperforms the baselines established by benchmark models for detecting depression, stress, and suicide with percent increases in measured performance of 13.64%, 1.27%, and 80.95%, respectively. the model also achieved an 85% accuracy at detecting suicidal ideation in a psychologically informed model that trains on datasets in an order established by clinical evidence, with stress followed by depression foot_8 and then ultimately suicide @xcite . although this paper demonstrated preliminary evidence that similarities in feature spaces can be leveraged to better predict unknown mhcs, in the future we wish to explore this further with a larger variety of models. we also plan to further examine the role that transfer learning order has in establishing performance. 9 other work has found that social media-based models do not always generalize and may incur substantial performance losses @xcite , and other factors such as social concerns, self-disclosure bias, and temporal artifacts may also influence model performance @xcite . we hope that researchers will use our findings to explore new ways to increase the efficiency and usefulness of ai-supported treatment and diagnosis of mhcs @xcite . dataset size labeling scheme labels used in our experiments baseline f 1 depression 531,453 posts from 892 users binary binary 0.66 stress 187,444 posts binary binary 0.79 suicide 11,129 initial users, downsampled to 934 categorical (4 categories) aggregated binary 0.42