Symptom Identification for Interpretable Detection of Multiple Mental Disorders on Social Media (2022)

Abstract: Mental disease detection (MDD) from social media has suffered from poor generalizability and interpretability, due to lack of symptom modeling. This paper introduces PsySym, the first annotated symptom identification corpus of multiple psychiatric disorders, to facilitate further research progress. PsySym is annotated according to a knowledge graph of the 38 symptom classes related to 7 mental diseases complied from established clinical manuals and scales, and a novel annotation framework for diversity and quality. Experiments show that symptom-assisted MDD enabled by PsySym can outperform strong pure-text baselines. We also exhibit the convincing MDD explanations provided by symptom predictions with case studies, and point to their further potential applications.

## introduction mental health has been a significant challenge in global healthcare. nearly 1 in 5 u.s. adults live with a mental illness or condition @xcite , and there are about 1 billion people suffering from mental disorders worldwide @xcite . due to the stigma of mental disorders and lack of professional mental health services, many people cannot receive proper diagnose or treatment for their conditions. social media can be a promising source for studies on this problem, as we may detect hidden traces of mental disorders from the symptoms that users may reveal in their free sharing, and provide pertinent help for those in need. consequently, mental disease detection (mdd) from social media has received increasing attention @xcite . however, most automatic mdd methods still struggle in this task, especially for their unsatisfying generalizability and explainability. firstly, these models may learn dataset-specific spurious correlations between certain words and the labels (usually diseases), and thus fail to generalize @xcite . moreover, most deep learning based methods work as black boxes, and cannot provide explanations for their prediction, which differs from the clinical practice which leverages symptom-based diagnostic criterions from authoritative manuals like dsm-5 @xcite . therefore, it may be hard for current mdd methods to gain trust from their users. to tackle these issues, there has been a rising interest in utilizing symptoms for mdd, as they are the bases that human psychiatrists use to make diagnoses. pioneering research has shown their potential benefits of improving the accuracy, generalizability and interpretability of mdd @xcite @xcite . nevertheless, due to the lack of large-scale annotated corpus for supervised learning, they can only extract symptom features with unsupervised/weakly supervised methods or simple pattern matching, which may not guarantee the quality of the extracted features. moreover, most of these works only focus on the detection of depression. however, many mental disorders may share similar set of symptoms. for example, depressed mood can not only be seen on the patient of depression, but also those suffering from bipolar disorder, anxiety, etc. jointly modeling the symptoms of multiple diseases may enhance the performance on all classes. these limitations call for the establishment of a large-scale, multi-disease annotated dataset for symptom identification, which will face many novel challenges. initially, the symptoms of multiple diseases are scattered over the different chapters of dsm-5 and other materials. similar symptoms would have varied expressions in different places, which causes difficulty in setting up the annotation standard. furthermore, the free and diverse language style on social media @xcite can make the retrieval of candidate posts for annotation difficult. last but not least, the relatively large amount of symptoms from different disorders and the nuanced differences and similarities between them make it hard to get high-quality annotations (i.e. inter-rater agreement will be low). in this work, we propose a novel data annotation framework, and introduce the first multidisease symptom identification dataset based on social media posts, psysym (psychiatric-disorder symptoms), which contains the multi-label annotations of 38 symptom classes from 7 mental diseases on 8,554 reddit post sentences. we establish our annotation target (symptom classes) mainly based on the diagnostic criterions from dsm-5, with symptom descriptions on clinical questionnaires as supplementary. we leverage embeddingbased retrieval methods @xcite instead of keyword matching to get the candidate sentences for annotation, which can effectively constrain our efforts to a precise but diverse subset of posts for efficient annotation. to guarantee the quality of data, we apply several quality control approaches, and divide the annotation tasks by separate diseases so as to reduce the cognitive burdens of annotators, resulting in high inter-rater agreement. finally, we propose a symptom-assisted mdd framework (figure 1 ). we use models trained on psysym to extract symptom features for mdd, outperforming strong bert-based baseline @xcite . we also explore the interpretability of symptom identification for mdd. we find that they can reasonably provide dsm-5 compliant expla-nations for diagnosed patients. we will also show that symptom-based interpretations can also help us find incorrect labels in automatically constructed mdd datasets, indicating their further potential. our contributions are: symptom identification there have been some pioneering attempts to leverage symptom-related features for mdd. @xcite uses manually complied lexicon to detect symptoms, and aggregate them into a score for the detection of depression. @xcite and @xcite leverage the embedding similarity between a post and symptom-related descriptions to decide the presence or risk of a symptom. @xcite uses regular expressions and heuristics to automatically build weakly-supervised training data for symptom identification. these methods have exhibited superior generalizability and interpretability. nevertheless, the efforts on establishing annotated corpus for symptom identification are still limited @xcite , which may hinder the potential of symptom-assisted mdd methods for leveraging stronger supervised symptom models. ## dataset construction in this section, we introduce the construction of psysym, the first annotated multi-disease symptom identification dataset based on social media posts. ## models this section will introduce the proposed models for the two sub-tasks of symptom identification: relevance judgment and status inference that can leverage the data from multiple diseases simultaneously, and how to leverage the above models for mental disease detection. ## experiments in this section, we present experimental results to: ## conclusions in this work, we introduce psysym, the first annotated multi-disease symptom identification dataset based on social media posts. a novel annotation framework is proposed to guarantee the diversity and quality of the annotations. psysym defined two sub-tasks of symptom identification: relevance judgment and status inference, to enable a more comprehensive analysis. strong baselines are established on the two sub-tasks with multi-disease modeling techniques that can properly handle missing labels and distribution-targeted learning to deal with the ambiguity in status inference. with psysym-trained models, symptom-assisted mdd method can outperform strong pure-text baselines. qualitative examples also demonstrate the explainability enabled by symptom predictions for mdd.