Overview of the shared task on Detecting Signs of Depression from Social Media Text (2023)

Abstract: Social media has become a vital platform for personal communication. Its widespread use as a primary means of public communication offers an exciting opportunity for early detection and management of mental health issues. People often share their emotions on social media, but understanding the true depth of their feelings can be challenging. Depression, a prevalent problem among young people, is of particular concern due to its link with rising suicide rates. Identifying depression levels in social media texts is crucial for timely support and prevention of negative outcomes. However, it’s a complex task because human emotions are dynamic and can change significantly over time. The DepSign-LT-EDI@RANLP 2023 shared task aims to classify social media text into three depression levels: “Not Depressed,” “Moderately Depressed,” and “Severely Depressed.” This overview covers task details, dataset, methodologies used, and results analysis. Roberta-based models emerged as top performers, with the best result achieving an impressive macro F1-score of 0.584 among 31 participating teams.

## introduction depression is considered a common mental disorder that involves mood swings and a lack of interest in any activities 1 . various aspects of life may be affected by depression. depression may develop in people who undergo abuse, severe losses, or other stressful events. depression has emerged as a worldwide concern for public health @xcite . even though a lot many people suffer from depression, adequate treatment is received by only a fraction of them. detecting and diagnosing depression is often delayed, imprecise, and missed. depression is also considered an important cause 1 https://www.who.int/news-room/factsheets/detail/depression of suicide. based on the severity and impact of the symptoms, they can be categorized as mild, moderate, or severe. almost 72% of the population is found to be active on social media foot_0 which provides an opportunity for early detection of depression, particularly in young adults. social media posts act as important information in identifying people at risk of depression or other mental disorders. there is a lot of research being carried out in the field of detecting depression from social media texts. to analyse the massive amount of social media text, machine learning has been considered one of the most efficient approaches. the probability of the existence of depression had been predicted using different machine learning algorithms @xcite . detection of depression from social media text had also been implemented using long-short term memory (lstm) model with recurrent neural network (rnn) @xcite . with all these informations in mind, the shared task on detecting the levels of depression from social media posts has been organised which is a continuation of the shared task depsign-lt-edi@ @xcite conducted during 2022. the shared task depsign-lt-edi@ranlp-2023 aims to detect levels of depression in reddit posts. ## task description the objective of depsign-lt-edi@ranlp-2023 is to identify indicators of depression in individuals based on their posts on social media platforms. by analyzing the language, feelings, and emotions expressed in these posts, the system aims to categorize individuals into three levels of depression: "not depressed," "moderately depressed," and "severely depressed." the system works specifically with english social media postings to detect signs of depression. ## data description to determine the level of depression based on social media data, a dataset was created by scraping and labeling posts from reddit. the dataset is the extension of depsign-lt-edi@ @xcite . it consists of three class labels: "not depressed," "moderately depressed," and "severely depressed." detailed guidelines and annotation instructions can be found in the publication by @xcite . the dataset is provided in a "tab separated value" format and is divided into three sets: the training set, evaluation set, and test set. the distribution of data across these sets is illustrated in ## methodology a total number of 62 submissions were submitted by 31 teams. the first run was based on logistic regression, the second run used the random forest classifier, and the third run applied voting to logistic regression and random forest classifier. ## evaluation the evaluation encompassed the utilization of all performance metrics available in sklearn. to account for the dataset's inherent imbalance, the submitted runs were assessed and ranked primarily based on the macro f1 score. the teams' rankings are presented in a notable observation from the table is that the system developed by the deeplearningbrasil team excelled, achieving the highest macro f1 score of 0.584 and the top accuracy score of 0.565. ## analysis and discussion early detection of depression is crucial as it is a prevalent mental illness that profoundly affects an individual's mood and emotions. failure to recognize and address depression at its early stages can have severe consequences. the depsign-lt-edi@ranlp-2023 shared task focused on utilizing reddit postings to detect various levels of depression. the detection process involved assigning three labels to individuals: "not depressed," "moderately depressed," and "severely depressed." by analyzing the content of these posts, the aim was to identify and categorize individuals based on their level of depression. ## conclusion depression is a widespread mental health issue that profoundly affects a person's emotions and well-being. detecting it early is crucial to prevent potential harm. the depsign-lt-edi@ranlp-2023 challenge aimed to identify depression levels from reddit posts, categorizing them as "not depressed," "moderately depressed," or "severely depressed." in this shared task, 31 teams participated, employing various models, with a strong focus on transformer-based methods and machine learning. the systems were assessed using the macroaveraged f1-score. remarkably, team deeplearn-ingbrasil excelled by combining roberta in an ensemble approach, achieving an impressive f1 score of 0.584. this underscores the promise of advanced natural language processing techniques in early depression detection.