T ech SSN 4@ LT - EDI -2023: Depression Sign Detection in Social Media Postings using D istil BERT Model (2023)

Abstract: As world population increases, more people are living to the age when depression or Major Depressive Disorder (MDD) commonly occurs. Consequently, the number of those who suffer from such disorders is rising. There is a pressing need for faster and reliable diagnosis methods. This paper proposes the method to analyse text input from social media posts of subjects to determine the severity class of depression. We have used the DistilBERT transformer to process these texts and classify the individuals across three severity labels - ‘not depression’, ‘moderate’ and ‘severe’. The results showed the macro F1-score of 0.437 when the model was trained for 5 epochs with a comparative performance across the labels.The team acquired 6th rank while the top team scored macro F1-score as 0.470. We hope that this system will support further research into the early identification of depression in individuals to promote effective medical research and related treatments.

## introduction the role of deep learning (dl) is growing in importance when it comes to automating the diagnosis and treatment of diseases, particularly in the field of mental health. mental health disorders are a major cause of disability globally, as reported by the world health organization (who). depression, or major depressive disorder (mdd), is denoted by symptoms such as low mood, loss of interest in activities, and negative effects on thoughts, behavior, motivation, and emotions, which can even lead to suicide. the causes of depression can be attributed to various factors, including biological, social, and psychological influences. according to who's 2022 report @xcite , approximately 970 million people worldwide are living with mental disorders, of which 28.9% are depressive disorders. while there are different treatment options available including psychological therapies like behav-ioral activation and problem-solving therapy, selfcare plays a significant role in addressing depression. accurate identification of mental health conditions is vital for effective care, especially considering the shortage of psychiatrists in many areas. depression, as a prevalent mental health disorder, requires early identification and a comprehensive approach combining various treatment options to tackle its significant impact on individuals and society. the following paper proposes a method to analyse text input from social media posts of subjects to determine the severity class of depression. our work covers the research question of how the application of a developed typology for social media texts, which aims to detect depression severity, contribute to effectively identifying subtle signs of depressive disorders in tweets. we have used the distilbert transformer to process these texts and classify the posts across three severity labels -'not depression', 'moderate' and 'severe'. ## related work studies have been carried out to detect the presence or absence of depression in individuals by analysing any one or a combination of text, audio and video inputs. islam et al. ( 2018 ) present their efforts to analyze depression based on facebook data obtained from an online public source utilizing machine learning (ml) techniques. their findings resulted in a substantially enhanced accuracy and reduced classification error rates, demonstrating that decision tree (dt) models outperform other simpler ml methods. @xcite attempt to overcome the shortcomings in accurately assessing model latency during depression detection. they have identified concerns regarding the widely used erde metric and put forth an alternative measure called latencyweighted f1, which effectively addresses these is-sues. subsequently, this evaluation approach is used to assess multiple models as part of the erisk 2017 shared task on depression detection. their results showed more effective distinctions captured between systems by using this metric. another study merges posts of users from two platforms, twitter and facebook, in order to detect the level of depression @xcite . this research employs ml techniques to classify data using support vector machine (svm) and naïve bayes algorithms and potentially identify depression. a survey-based study carried out by zafar and chitnis (2020) indicated the increasing interest in data-mining and analysis of information from social networking sites for recognizing depression in users. @xcite use tweets from twitter to examine users' expressions and gain insights into their emotional states. their use of the distilbert model for a binary classification of depressed or non-depressed subjects resulted in an enhanced accuracy. with respect to dl models, a comparison study was carried out by @xcite . they considered three variants of bert and four ensemble models of these variants in classifying depression using transcripts of responses to 12 clinical interview questions. their findings reveal that the utilization of ensembles leads to improved mean f1 scores. as part of depsign-lt-edi@acl-2022, janatdoust et al. (2022) present a predictive ensemble model that leverages the fine-tuned contextualized word embeddings from distilbert, bert, roberta and albert base models. their findings demonstrated a performance surpassing the baseline models across all evaluated metrics, achieving an impressive 61% accuracy and f1 score of 54%. a predictive model from text has also been developed using long-short term memory (lstm) and recurrent neural network (rnn) models @xcite . the rnn is trained on text-based data to recognize depression using semantics and written content. with a 99.0% accuracy rate, this framework performed better than frequency-based dl models for textual detection, and has a lower false positive rate. @xcite use a transformer encoder model for their research on voice-base depression detection. in order to address the limited size of the available dataset, the researchers extracted low-level features from audio recordings and applied augmentation techniques. by leveraging these approaches, their network achieved a recognition accuracy of 73.51% on the e-daic database. anantharaman et al. (2022 ) uses bert model for detecting depression from text while esackimuthu et al. (2022) uses albert model for depression detection. ## proposed system our depression detection system involves data preprocessing, encoding, model building and predicting the unseen test samples. various experiments were conducted and it was concluded that the distil-bert model outperformed other models in terms of accuracy, leading to its selection for text processing. the model is implemented using python's torch library based on the transformer architecture. it takes the encoded input text and utilizes a pretrained "distilbert-base-uncased" model to generate contextualized embeddings for each token. in order to mitigate the risk of overfitting, a dropout layer is included, employing a dropout rate of 0.1. additionally, a linear classification layer is employed to map the hidden state size of the distilbert model to the desired number of output classes. as a result, logits are obtained, representing raw scores that indicate the likelihood of the input belonging to each output class. our system comprises of the distilbert model trained using the train and dev sets provided, over 5 epochs. the trained model was subsequently evaluated on the test dataset, yielding an accuracy of 47.9%. ## results and discussion the performance of distilbert model trained on depression dataset is evaluated on test dataset to predict the output depression class labels. the overall performance of the system was determined across several metrics including accuracy and weighted and macro averaged values of precision, recall and f1-score and is tabulated in accuracy measures the effectiveness of classification models by determining the proportion of accurate predictions out of the total predictions made, represented as a percentage while f1 score gives the blend of precision and recall. the class-wise accuracy and f1-scores of the model are depicted in the confusion matrix was also visualized over the results for a better class-wise comparison as seen in ## conclusion and future work our distilbert model was trained by combining both the train and dev sets of data, but other approaches could consider these separately and improve the model through a validation phase post training. another technique that can improve training is running the model for a larger number of epochs, allowing for more iterations and potentially better learning. prior to training and testing, it is also crucial to ensure the cleanliness of the text data, as incorrect encoding and decoding can lead to the loss of important information. while this study focuses on detecting depression through social media postings it can be extended for analysis of longer texts that could also be speech transcripts. however, analyzing longer sequences of text poses a challenge as prevalent models in nlp typically have limited input lengths. while mental health awareness has increased, there is still a stigma associated with depression. the implications for society with the availability of a rapid and reliable depression detection system are immense. early identification of such conditions can result in enhanced treatment results and a higher standard of living for individuals impacted by them.