ML & AI _ IIITR anchi@ LT - EDI -2023: Hybrid Model for Text Classification for Identification of Various Types of Depression (2023)

Abstract: DepSign–LT–EDI@RANLP–2023 is a dedicated task that addresses the crucial issue of identifying indications of depression in individuals through their social media posts, which serve as a platform for expressing their emotions and sentiments. The primary objective revolves around accurately classifying the signs of depression into three distinct categories: “not depressed,” “moderately depressed,” and “severely depressed.” Our study entailed the utilization of machine learning algorithms, coupled with a diverse range of features such as sentence embeddings, TF-IDF, and Bag-of- Words. Remarkably, the adoption of hybrid models yielded promising outcomes, culminating in a 10^{th} rank achievement, supported by macro F1-Score of 0.408. This research underscores the effectiveness and potential of employing advanced text classification methodologies to discern and identify signs of depression within social media data. The findings hold implications for the development of mental health monitoring systems and support mechanisms, contributing to the well-being of individuals in need.

## introduction depression is a widespread mental health condition that affects a significant number of individuals worldwide. the identification of depression symptoms in individuals is a crucial step in providing timely support and intervention. given the growing use of social media platforms as channels for emotional expression and experience sharing, there is an increasing interest in leveraging these platforms to detect indications of mental health issues, including depression. accordingly, the depsign-lt-edi@ranlp-2023 task focuses on the detection of depression signs in individuals based on their social media posts. past works in this field offer valuable insights into the importance of monitoring mental health and the challenges associated with identifying and addressing mental health problems. the global health data exchange (ghdx) by the institute of health metrics and evaluation (glo) serves as a valuable resource for accessing global health data and understanding the prevalence of mental health disorders. additionally, the study conducted by @xcite highlights socioeconomic disparities in the treatment gap for individuals with anxiety, mood, and substance use disorders, underscoring the need for targeted interventions and support. the erisk 2017 study @xcite carried out further emphasizes the importance of early risk prediction and underscores the experimental foundations in the field. this research aims to employ machine learning algorithms and text classification techniques to identify signs of depression in social media posts. by leveraging features such as sentence embeddings and utilizing feature representations like the tf-idf, and bag-of-words, we aim to develop an effective classification model capable of categorizing signs of depression into three distinct labels: "not depressed", "moderately depressed", and "severely depressed". the hybrid models utilized in our study incorporate a combination of these features, enabling enhanced classification performance. the ultimate objective of this research is to contribute to the field of mental health monitoring and support systems by providing a reliable and efficient method for detecting signs of depression through social media analysis. by harnessing the abundant data available on social media platforms, we aim to facilitate timely intervention and support for individuals experiencing depressive symptoms. through the depsign-lt-edi@ranlp-2023 task, we aim to address the urgent need for inno-vative approaches to mental health detection and intervention. by utilizing advanced machine learning algorithms and integrating diverse features, our goal is to offer an easy to use but effective solution for identifying signs of depression in social media data. although berts and other large language models (llms) are available for the classification task a major drawback that holds them back is their massive size and training time. not every individual or organization possesses the capability to leverage the llm power and supremacy for text classification. also, there is not much data present to perform the fine-tuning task and get great results. to overcome this and still utilize the abilities of the llm, we were motivated to get a way around by making use of sentence embeddings that are generated from the llm but are of smaller size and can be computed in a limited amount of time. the results obtained from our study, as evidenced by our 10th rank and an f1 score of 0.408, demonstrate the potential and promise of our proposed approach. this paper presents the related work section 2, dataset description section 3, task description section 4, methodology and validation results section 5, result section 6, and conclusions section 7 of our research, shedding light on the effectiveness of text classification techniques in detecting signs of depression. furthermore, the findings contribute to the broader field of mental health research and pave the way for the development of scalable and efficient solutions for mental health monitoring and support systems. ## related work in the past many efforts have been made in the field of depression identification from social media texts. the authors of @xcite were among the pioneers in researching the detection of depression through social media posts. their study focused on twitter users who had been diagnosed with depression, and they collected one year's worth of posts from this group. using this dataset, they developed a statistical classifier that aimed to estimate the risk of depression. the classifier utilized various linguistic and behavioral features extracted from the users' posts to predict the likelihood of depression. this research marked an important milestone in the field, providing insights into the potential of social media data for identifying mental health conditions. by employ-ing a rigorous statistical approach, it also laid the groundwork for subsequent studies on detecting depression through social media, contributing to the advancement of research in this domain. the authors of @xcite conducted a study that explores the use of machine learning techniques to detect depression from social media posts. by employing linguistic features and classifiers, the authors achieve promising results, demonstrating the potential of utilizing social media data for identifying signs of depression. they compare various machine learning algorithms and feature selection methods to determine the most effective combination. the study highlights the importance of pre-processing techniques and feature engineering in improving classification performance. the findings contribute to the development of reliable and efficient methods for detecting signs of depression through social media analysis, supporting timely intervention and support for individuals experiencing depressive symptoms. the authors of @xcite carried research that focuses on the scientific aspect of detecting depression from social media data using machine learning techniques and text mining. the authors propose a novel approach that combines text and image information for improved depression detection accuracy. they extract textual features from posts and visual features from associated images, and then employ a multi-modal fusion model to integrate these features. the study evaluates the proposed approach on a large-scale dataset and compares it with existing methods, demonstrating its superior performance. the findings highlight the importance of considering both textual and visual cues for accurate depression detection, offering valuable insights for developing effective mental health monitoring systems. observing the multi modality of the task, advancements have been made to make the text based identification more robust. work presented by the authors of @xcite have led emphasis on detecting the linguistic features from the text corpora generated from reddit, so as to classify them into the categories of depression based on lexical and predictive analysis. the introduction of deep learning architectures like transformer models have also significantly improvised the results @xcite presents work that leverages the power of transfer learning to classify text into depression. the authors of @xcite pre-sented a thorough search of the literature, the authors found 34 primary papers that satisfied their inclusion requirements. the studies employed several techniques, such as language feature extraction, machine learning, and statistical analysis, to find symptoms of depression on social media. the research findings were conflicting, but taken together, they imply that social media can be used to identify depression symptoms with some degree of accuracy. our work has been inspired by past related works and motivated us to develop a simple system to test and identify depression. past methods have inculcated various experimentation's like statistical analysis, pre-trained model-based research, and employing classical machine learning algorithms on various features; considering them, we have devised a solution that involves sentence embeddings as features and further classified them using multiple machine learning classifiers. ## dataset the english-language postings in the dataset for the competition were taken from the reddit platform. "not depression", "moderate", or "severe" are the three labels that have been manually added to each post in the dataset @xcite @xcite . when there were no signs of depression found in the post, the label "not depression" is used to denote those cases. alternatively, the terms "moderate" and "severe" denote increasing degrees of depression symptoms in the text. the training set, the development set, and the test set were the three separate parts of the dataset that were divided up for the competition. it is important to note that the labels for the test set were withheld by the competition's administrators because this section was only used to assess the competitors' solutions. notably, the training set contains a larger number of instances compared to the development set. this discrepancy in size enables more effective finetuning of hyperparameters for both machine learn-ing algorithms and deep learning neural networks. the availability of a larger training set facilitates more robust model optimization, ultimately leading to improved performance and generalization capabilities. ## task description the objective of this task is to develop a system that can effectively classify signs of depression within social media postings written in english. the system should be able to categorize these signs into one of three distinct labels: "not depressed", "moderately depressed", and "severely depressed". by automating this classification process, the system can provide valuable insights into individuals' mental well-being and potentially facilitate timely intervention and support. the dataset provided for this task consists of a collection of social media posts expressed in the english language. each post has been carefully annotated with one of the three aforementioned labels, indicating the level of depression detected within the content. this labeling scheme enables the development of a comprehensive classification system that can effectively gauge the severity of depression symptoms expressed in social media postings. the authors of @xcite , have shown depression identification with the use of deep visualtextual multimodal learning approach which embarks a great development in this field and simultaneously introduces the domain where machine learning approaches can be applied where the features can be multimodal, meaning it can be in the form of text, video or both. the authors of @xcite ; alsagri and ykhlef, 2020) have performed on a similar task where they have used large pre-trained models to make accurate predictions. they finally used the features and greatly performed using an average ensemble approach. the text cited acted as our motivation for this task, and we leveraged various machine-learning techniques and methodologies. these include traditional machine learning algorithms, deep learning models, or a combination of both. by training and fine-tuning such models on the provided dataset, we aim to create a robust classifier capable of accurately predicting the level of depression exhibited in unseen social media posts. ## methodology and validation results the methodology employed in the classification task comprises several sequential steps: data preprocessing, tf-idf features based classification, bag of words features based classification, and the utilization of sentence embeddings for classification. in the initial data pre-processing step, the raw english social media postings underwent a series of text cleansing procedures. these include the removal of punctuation, stop words, and special characters, as well as tokenization and stemming operations to standardize the textual data. this preprocessing stage ensures the text is appropriately formatted for subsequent analysis. subsequently, tf-idf based classification is implemented. the tf-idf (term frequency-inverse document frequency) technique is employed to represent each document, i.e., social media post, as a numerical feature vector. following tf-idf based classification, the methodology incorporates bag of words classification. in this approach, the text data is represented using a bag of words model, which establishes a vocabulary consisting of unique words derived from the corpus. finally, sentence embeddings are leveraged for classification. sentence embeddings aim to capture the semantic meaning of a sentence through a compact vector representation. by adhering to this methodology, which encom-passes data pre-processing, tf-idf and bag of words classification, and the utilization of sentence embeddings, a classical and effective approach was formulated for accurately categorizing signs of depression within english social media postings. ## conclusion the successful completion of this task holds great potential for various applications and benefits. by accurately classifying the signs of depression within social media postings, the developed system can aid in the identification of individuals who may be at risk or in need of mental health support. this classification system could be integrated into social media platforms or utilized as a standalone tool to provide real-time insights into users' mental well-being. early identification and intervention can play a crucial role in promoting mental health and well-being, and the system developed through this task has the potential to contribute significantly in this regard.