E8- IJS @ LT - EDI - ACL 2022 - BERT , A uto ML and Knowledge-graph backed Detection of Depression (2022)

Abstract: Depression is a mental illness that negatively affects a person’s well-being and can, if left untreated, lead to serious consequences such as suicide. Therefore, it is important to recognize the signs of depression early. In the last decade, social media has become one of the most common places to express one’s feelings. Hence, there is a possibility of text processing and applying machine learning techniques to detect possible signs of depression. In this paper, we present our approaches to solving the shared task titled Detecting Signs of Depression from Social Media Text. We explore three different approaches to solve the challenge: fine-tuning BERT model, leveraging AutoML for the construction of features and classifier selection and finally, we explore latent spaces derived from the combination of textual and knowledge-based representations. We ranked 9th out of 31 teams in the competition. Our best solution, based on knowledge graph and textual representations, was 4.9% behind the best model in terms of Macro F1, and only 1.9% behind in terms of Recall.

## introduction depression is a type of mental illness that affects a large part of our society and is one of the most complex challenges facing our humanity. since depression is a disease that, if left untreated, can lead to serious consequences such as suicide over time, its early detection is crucial. since people with depression typically do not open up in person very often, they often see social media as a way to express their thoughts and feelings @xcite . this trend increased rapidly with the covid-19 pandemic due to restrictive measures that encouraged people to use social media as a means of expression. as the number of posts on social media has increased rapidly in recent years, there is a need to process them automatically to extract valuable information such as signs of depression. for this task, detecting signs of depression is a standard multi-class classification problem, where each post can be assigned to one of three classes ("non-depressive", "moderate", and "severe"). increasing predictive accuracy may be critical for psychiatrists to detect the early signs of major depression to prevent further consequences. the remainder of this article is organized as follows: in section 2, we discuss approaches to solve multi-class text classification problems and related work with data sets of social media posts on depression, in section 3, we present the statistics of our given data, in section 4, we explain the methods we used to solve the given problem, in section 5, we present and analyze the results, and in section 6, we provide the conclusion and present our plans for future work. ## related work in related work, we can find various approaches to detecting depression from textual data including various social media. one of the frequently used sources is twitter. in one of the earlier studies of data from twitter users who have attempted to take their life, @xcite propose a logistic regression classifier using character n-gram character features. @xcite tackled the task of detecting depression in spanish tweets and used occurrences of negative and positive words determined by spanish sentiment lexicon @xcite and the spanish senticon lexicon @xcite as additional features. for arabic, @xcite addressed the task of classifying twitter posts as depressed or non-depressed. after text preprocessing and cleaning, the authors extract sparse features from tweets to construct feature vectors. the classification is done using four popular models: random forest @xcite , naïve bayes @xcite , adaboostm1 @xcite and liblinear svm @xcite . recently, there were also deep learning based approaches proposed for depression detection on twitter data. for example, @xcite use a bidirectional long short term memory recurrent neural network with attention @xcite which produces a high performance on a data set consisting of over 30,000 english tweets. in the last years, also multimodal approaches have been explored. for example, @xcite propose the combination of visual and textual information in order to achieve better results. they model the problem as markov decision process, solving it in reinforcement learning manner. for the text feature extraction, they consider learning custom embeddings via a bidirectional gru network, for images they used the pre-trained vggnet @xcite . another source of data is reddit, as in our shared task. @xcite use term-frequency inverse-document-frequency features and various classification models, such as support vector machine with stochastic gradient descent and multinominal naive bayes @xcite . reddit was also the source of a study by wolohan (2020) about the linguistic characteristics of depression during global pandemics. the authors analyze the increase of depression on the social platform and model the problem with fasttext @xcite embeddings and employ lstm networks to learn to detect depression in the data. next, the authors have also used facebook data. for example, @xcite decided to generate content features by lstm neural network expanding the quantity of information represented in the feature vectors. merging these features with additional content, behavior, and living-based features are used for the construction of a standard deep neural network. finally, the authors also use blogs. yuka niimi (2021) tackled the problem of depression detection ## data description the data set @xcite that is provided by the task's organizers consists of english posts from the reddit social media platform, which includes more textual data compared to other social media platforms @xcite . the posts belong to one of three given classes: "not depressed", "moderate" and "severe". we use three data splits one for training, one for development, and one for testing. we used the development set for the internal evaluation of various models. ## methodology in the following section, we will present the methods that were used along with the evaluation measures. for the given task we have developed three independent methods. ## evaluation in this section, we represent the evaluation of our proposed approaches. we first showcase the evaluation of the development data set, followed by the evaluation of methods on the test split. ## conclusion and future work in this paper, we explored three different approaches to address the task of detecting depression in a social media text. first, we leveraged the bert family models, as they represent the state of the art for many text classification problems. next, we investigated how automl approaches such as autobot perform on this task, where features are iteratively generated and selected via evolution strategies, and finally combined in the final representation. finally, we studied how knowledgebased representations based on knowledge graph concepts that occur in a text and textual representations such as lsa and sentence transformers perform. we show that combined knowledge and textual representations outperform any of our other combinations and perform best. our proposed solution yields good results in terms of macro f1, but it is 4.9% behind the leading model. for future work, we propose to develop ensembles of the previously created models to see how the combination of different feature types affects performance. next, we propose to explore larger (like conceptnet @xcite ) or more specific knowledge graphs (like medical knowledge graph @xcite ) to improve the results. finally, we propose the use of feature importance methods to see how outcomes are affected by each family of features and what insights this can give us about depression.