Interns@ LT - EDI : Detecting Signs of Depression from Social Media Text (2023)

Abstract: This submission presents our approach for depression detection in social media text. The methodology includes data collection, preprocessing - SMOTE, feature extraction/selection - TF-IDF and Glove, model development- SVM, CNN and Bi-LSTM, training, evaluation, optimisation, and validation. The proposed methodology aims to contribute to the accurate detection of depression.

## introduction depression is one of the most severe mental-health diseases right now and it is important to detect the signs early on and take actions to stop troubled individuals from taking their own life and provide them with the help they need. most methods we rely on today are medical procedures in clinics or actual human interaction which is highly ineffective leading to the high rates of suicide and a significant percentage of individuals affected by depression. according to the world health organization(who) almost 280 million people in the world suffer from depression and on an average about 703,000 people take their lives around the world as on 2022. however with the huge online presence in various social media platforms and individuals giving daily updates through social media posts it would make detecting depression very easy if we had a model that uses the social media texts and classifies it into multiple levels of depression. by using natural language processing, machine learning, and data mining techniques, researchers have made significant strides in automating the detection of depression in social media data. these include traditional machine learning algorithms like svms, random forest or naive bayes algorithms in the past and advanced deep learning algorithms in the recent times like recurrent neural networks(rnn), convolutional neural networks(cnn), long short-term memory (lstm) and generative adversarial networks (gan) to name a few @xcite . in this paper we present our model for detecting depression for the competition shared task on detecting signs of depression from social media text at lt-edi@ranlp 2023. this paper has 6 sections. section 2 contains a short summary of the related works on depression detection using social media text referred to by the authors. section 3 contains the details of the datasets used and any pre-processing done on them. section 4 is on the methodology of our model. section 5 shows the experiments done and the results obtained. section 6 gives the conclusion of the paper and future work. ## dataset the datasets provided for the competition includes train data, development data and test data @xcite . train data and development data contain reddit posts in english language and each post is annotated with one of the labels: 'not depression', 'moderate' and 'severe'. the first label indicates absence of any sign of depression in the post. the second label indicates the presence of some signs of depression and the third one indicates severe depression. the distribution indicates an imbalance among the classes, with the 'moderate' label being the most prevalent, followed by 'not depression', and 'severe' being the least represented category. class imbalance can pose challenges during model training, as the minority class (in this case, 'severe') may have fewer samples to learn from, potentially resulting in biased predictions. to address this class imbalance we used smote oversampling for training dataset on the class 'severe'. smote works by creating synthetic minority class examples that are located between existing minority class examples. this is done by randomly selecting a minority class example and then selecting one of its k nearest neighbors. a new synthetic example is then created at a point along the line connecting the two selected examples. to understand the datasets better and prepare the data for analysis, several preprocessing steps were undertaken. firstly, duplicates were removed from both the training and development datasets to ensure the uniqueness of the samples. additionally, the word '[removed]' was eliminated from the datasets as it indicated that certain posts were removed by reddit moderators for various reasons, such as containing inappropriate or banned content. we will be using the development dataset as test dataset for the sake of the paper to compare the models on the basis of f1-score which won't be possible for test data as the posts are not labelled. ## methodology for depression detection from text a few steps are involved. this includes data preprocessing which involves cleaning and preprocessing the collected data, feature extraction which involves identifying the most informative features that are likely to be associated with depression and model development which involves choosing an appropriate algorithm to develop a predictive model. for data preprocessing we are cleaning the data and using smote to balance the dataset. ## results and dicussion table-3 shows the results of the three models. the svm model with textblob (tf-idf) was the best in terms of accuracy (0.603) and f1-score (0.479). following that was the cnn model with textblob (tf-idf) with an accuracy of 0.584 and f1-score of 0.442. finally the model based on bi-lstm model with textblob and glove embeddings had an accuracy of 0.47 and f1-score of 0.34. the macro f1-score for each of them were 0.48, 0.44 and 0.34 respectively. the main challenge faced in this shared task was the imbalanced dataset. the reason for the better performance of the svm model could be because of the simplicity of the problem. to compare the models the metrics used during the experiments are accuracy, precision, recall and macro f1-score and weighted f1-score for all the classes. the macro f1-score is the main metric to evaluate the models. model accuracy f1 score svm model with textblob(tf-idf) 0.603 0.479 cnn model with textblob(tf-idf) 0.584 0.442 bi-lstm model with textblob and glove embeddings 0.47 0.34 ## conclusion and future works in this paper, we presented three solutions to the shared task on detecting signs of depression from social media text at lt-edi@ranlp 2023. the svm model with textblob for feature extraction proved to be the best one among the three and the cnn model with a close second. for future works we plan to implement models better suited for both feature extraction and classification and use a better split training data and testing data. also the accuracy could be improved by balancing the dataset better by using other methods.