JA - NLP @ LT - EDI -2023: Empowering Mental Health Assessment: A R o BERT a-Based Approach for Depression Detection (2023)

Abstract: Depression, a widespread mental health disorder, affects a significant portion of the global population. Timely identification and intervention play a crucial role in ensuring effective treatment and support. Therefore, this research paper proposes a fine-tuned RoBERTa-based model for identifying depression in social media posts. In addition to the proposed model, Sentence-BERT is employed to encode social media posts into vector representations. These encoded vectors are then utilized in eight different popular classical machine learning models. The proposed fine-tuned RoBERTa model achieved a best macro F1-score of 0.55 for the development dataset and a comparable score of 0.41 for the testing dataset. Additionally, combining Sentence-BERT with Naive Bayes (S-BERT + NB) outperformed the fine-tuned RoBERTa model, achieving a slightly higher macro F1-score of 0.42. This demonstrates the effectiveness of the approach in detecting depression from social media posts.

## introduction depression is a prevalent mental health disorder affecting people of all ages from diverse backgrounds, irrespective of their socioeconomic status or cultural circumstances. the world health organization (who) approximates that there are over 264 million individuals globally who suffer from depression, underscoring its significant impact on public health. depression exhibits a higher occurrence rate in women, surpassing that in men by approximately 50%. alarmingly, suicide claims the lives of over 700,000 people each year, positioning it as the fourth leading cause of death among individuals aged 15 to 29 foot_0 @xcite . the emergence of social media platforms has revolutionized online communication, information sharing, and self-expression. by 2021, facebook alone had reported a staggering 2.8 billion monthly active users, while other platforms like twitter, instagram, and reddit also maintained substantial user bases. this widespread adoption of social media has presented researchers with an extensive pool of user-generated content to investigate, including its application in mental health analysis and other diverse endeavors @xcite @xcite . there are several advantages of detecting signs of depression from social media posts. firstly, it offers a large-scale and easily accessible data source, allowing for real-time analysis of a significant number of individuals. secondly, social media text often reflects individuals' genuine emotions and experiences, as they express themselves freely and spontaneously on these platforms. additionally, social media data can be collected over time, enabling the monitoring of changes in individuals' mental well-being as it evolves. in recent years, there has been increasing interest in analyzing social media texts to detect signs of depression. research has revealed that individuals suffering from depression often demonstrate specific language patterns in their online posts @xcite . studies have found that depressed individuals tend to utilize more self-referential pronouns (such as "i" and "me"), expressing a higher frequency of negative emotions and words, and exhibit reduced engagement in positive social interactions @xcite . however, the detection of depression signs from social media texts is not without its challenges. while specific linguistic patterns associated with depression have been identified @xcite @xcite , it is important to consider individual differences, personality traits, cultural variations, and context-specific factors that can influence the use of language @xcite . by exploring diverse approaches, techniques, challenges, and ethical considerations (al-sagri and ykhlef, 2020; @xcite , our aim is to contribute to the continuous efforts of enhancing mental health outcomes through the innovative and responsible use of social media data. in accordance with existing research, this study introduces a refined roberta model to detect depression in social media posts. additionally, sentence-bert is employed to convert social media posts into a consistent encoded vector. these vectors are subsequently utilized as input for various well-known classical machine learning classifiers. the remaining paper is organized as follows: section 2 briefs the related work, section 3 discusses the methodology adopted to perform the work, section 4 highlights the results obtained, and finally section 5 concludes the overall work. ## related works this section provides insights into the field of detecting signs of depression from social media texts. these highlights the use of linguistic analysis, sentiment analysis, machine learning, psychological frameworks, and natural language processing, providing a comprehensive understanding of the field @xcite @xcite . machine learning algorithms play a vital role in identifying depression through text extracted from social media. various supervised learning techniques, such as support vector machines (svm), naïve bayes, and random forests, have been utilized to classify text as depressed users or non-depressed (alsagri and ykhlef, 2020; @xcite . moreover, deep learning approaches like recurrent neural networks (rnns) and convolutional neural networks (cnns) have demonstrated encouraging outcomes in capturing intricate linguistic patterns to detect depression @xcite . @xcite explores the use of twitter data to quantify mental health signals, including depression. in this study, linguistic features including words, phrases, and linguistic structures are employed to detect mental health signals linked to depression, post-traumatic stress disorder (ptsd), and other mental health conditions. to accomplish this, machine learning techniques are utilized to classify tweets and evaluate the models' accuracy in predicting mental health states. in addition, the paper discusses the challenges faced when quantifying mental health signals within twitter data, such as the presence of inherent noise and bias in social media posts. further, by incorporating various data sources, the research strives to provide a more comprehensive and robust analysis of mental health indicators. the authors in @xcite with an objective of gaining insights into the expression and analysis of depressive emotions in the realm of social media, employed a large dataset of twitter posts, and leverage natural language processing techniques to identify patterns and sentiments related to depressive language. their study focuses on understanding how individuals express and communicate their feelings of depression through social media platforms like twitter. the authors examine linguistic features in twitter, such as words, emoticons, and grammatical structures, to understand the occurrence and expression of depressive moods. they discover distinct language patterns associated with depressive emotions. however, relying solely on twitter data limits the study's representativeness and generalizability to diverse demographics. in their study, @xcite conducts a systematic analysis of various studies utilizing social media data to detect depression and other mental health conditions. they provide a comprehensive overview of the research conducted in the field of detecting depression and mental illness through social media analysis by providing an indepth assessment of their strengths and limitations. it examines the linguistic, behavioral, and contextual features utilized to detect signs of depression in social media posts from the application point of view of machine learning and natural language processing techniques. the challenges encompass concerns related to data privacy, the representativeness of social media users, biases arising from selfdisclosure, and the requirement for more accurate ground truth labels to effectively train models. reece and danforth (2017) takes a unique approach by leveraging the vast amount of usergenerated data on instagram. their primary goal is to uncover predictive indicators of depression. to achieve this, the researchers meticulously analyzed a substantial number of instagram photos posted by participants, along with the accompanying captions and metadata. the study revealed a significant pat-tern where participants diagnosed with depression exhibited a strong preference for darker, grayer, and bluer tones with less likely smiling faces in their instagram photos, while those without depression tended to favor brighter, warmer colors. further, the study also revealed that individuals with depression tended to engage more in frequent posting for social validation and support. additionally, the researchers identified distinct textual markers like increased utilization of filters and more frequent references to feelings of sadness, anxiety, and loneliness. these findings provide valuable insights into the relationship between depression and online behavior, particularly in terms of text-based expression. the research primarily relies on self-reported diagnoses of depression and lacks a comprehensive clinical assessment conducted by mental health professionals. furthermore, the study's findings are confined to data obtained solely from instagram, which may not provide a complete representation of the broader population. @xcite focuses on characterizing mental health conditions by leveraging informed deep learning models to analyze user-generated content, including posts and interactions on social media platforms. the results demonstrate promising capabilities in detecting symptoms related to depression, ptsd, self-harm, and more. additionally, the study uncovers distinct linguistic patterns and behavioral markers associated with different mental health conditions, encompassing variations in word usage, sentiment analysis, and linguistic styles employed by individuals facing diverse mental health challenges. however, the research relies on publicly available social media data, which may not fully represent the entire population. furthermore, considerations regarding privacy, data accuracy, and generalizability need to be addressed. @xcite adopts an innovative approach by examining the language patterns of adults with attention deficit hyperactivity disorder (adhd) in their social media interactions. it uncovers distinct linguistic characteristics, including a higher frequency of self-referential pronouns, words related to time urgency, emotional language, and references to cognitive processes. these findings highlight the potential of language analysis to identify and understand communication patterns associated with adhd in digital environments. however, the study relies on self-reported diagnoses and focuses on language patterns within specific social media platforms. therefore, it may not fully encompass the experiences of all adults with adhd or capture their interactions across a wide range of digital platforms. @xcite explores the intriguing interplay between personality traits, gender, age, and language usage on social media platforms. employing an open-vocabulary approach, the authors delve into an extensive analysis of large-scale social media data to uncover patterns and correlations between language use and individual characteristics. the linguistic analyses identified distinct linguistic markers associated with various personality traits, including extraversion, neuroticism, and agreeableness. further, the study uncovers correlations between gender and linguistic choices, revealing variations in language use between male and female users. notably, age-related disparities in language were also observed, indicating that language patterns on social media may undergo transformations as individuals grow older. the study relies on self-reported personality assessments that focus on language patterns within specific social media platforms. as a result, the study may not fully encompass the complete spectrum of personality traits, gender identities, or age groups within the broader population. de @xcite aims to characterize and predict postpartum depression by utilizing data shared on facebook. the authors explore different variables including language patterns, linguistic styles, social interactions, and behavioral cues exhibited by users, with the goal of identifying significant markers associated with postpartum depression. further, they discover distinct language patterns like increased use of first-person pronouns, negatively affecting words, feelings of loneliness and isolation, etc. they also highlight the importance of social interactions and behavioral cues, such as changes in posting frequency and decreased engagement with friends, as potential indicators of postpartum depression. however, the study's reliance on facebook data may limit the representation and understanding of individuals beyond the scope of the platform itself. the authors in @xcite address the prevalence of suicide and the growing influence of social media platforms in shaping public discourse. the paper emphasizes the significance of automated systems in monitoring and identifying individuals who may be at risk. their research un- ## methodology the overall flow diagram of the proposed work is illustrated in figure 1 . in addition to the fine-tuned roberta model, a total of eight different models were developed, namely: (i) fine-tuned roberta, (ii) sentence-bert + support vector machine (s-bert + svm), (iii ## result the performance evaluation of the proposed model encompasses various metrics, including precision (p), recall (r), f 1 -score (f 1 -score), accuracy (acc), weighted precision, weighted recall, weighted f 1score, macro-precision, macro-recall, macro-f 1score, confusion matrix, and auc-roc curve. the results of validating different deep learning models on both the testing and validation datasets are presented in in the case of sentence-bert combined with classical machine learning models, s-bert + lr outperformed the classical machine learning approach, achieving a macro f 1 -score of 0.51 for the development dataset. the confusion matrix and auc-roc curve for the s-bert + lr model on the development dataset can be observed in figures 4 and 5, respectively. likewise, when considering the testing dataset, the proposed fine-tuned roberta model again demonstrated remarkable performance, attaining a notable macro f 1 -score of 0.41. the corresponding confusion matrix and auc-roc curve for the finetuned roberta model on the testing dataset are presented in figures 6 and 7, respectively. regard-0.0 0.2 0.4 0.6 0.8 1.0 false positive rate 0.0 0.2 0.4 0.6 0.8 1.0 true positive rate receiver operating characteristic curve micro-average roc curve (@xmath0.80) macro-average roc curve (@xmath1.73) moderate (@xmath2.66) not depression (@xmath3.70) severe (@xmath4.84) figure 3: roc curve for the proposed fine-tuned roberta model (validation dataset)