MUCS @Text- LT - EDI @ ACL 2022: Detecting Sign of Depression from Social Media Text using Supervised Learning Approach (2022)

Abstract: Social media has seen enormous growth in its users recently and knowingly or unknowingly the behavior of a person will be reflected in the comments she/he posts on social media. Users having the sign of depression may post negative or disturbing content seeking the attention of other users. Hence, social media data can be analysed to check whether the users’ have the sign of depression and help them to get through the situation if required. However, as analyzing the increasing amount of social media data manually in laborious and error-prone, automated tools have to be developed for the same. To address the issue of detecting the sign of depression content on social media, in this paper, we - team MUCS, describe an Ensemble of Machine Learning (ML) models and a Transfer Learning (TL) model submitted to “Detecting Signs of Depression from Social Media Text-LT-EDI@ACL 2022” (DepSign-LT-EDI@ACL-2022) shared task at Association for Computational Linguistics (ACL) 2022. Both frequency and text based features are used to train an Ensemble model and Bidirectional Encoder Representations from Transformers (BERT) fine-tuned with raw text is used to train the TL model. Among the two models, the TL model performed better with a macro averaged F-score of 0.479 and placed 18th rank in the shared task. The code to reproduce the proposed models is available in github page1.

## introduction a person feeling unimportant, useless, or unhappy may be a sign of depression. depression is one of the most severe mental health conditions which may be unnoticed, undiagnosed and untreated in many cases. people worldwide suffer from depression and the affected person may operate poorly at work, studies, and in the community. recent research studies have shown that the popularity of social media networks in one's life is increasing day by day. people are using social media to share their thoughts, feelings, emotions and sentiments @xcite . knowingly or unknowingly the behavior of a person will be reflected in the comments she/he posts on social media @xcite @xcite @xcite . social media users who are usually in depression try to seek the attention and sympathy of others by posting negative and disturbing messages or requesting help. some have even reached to the extent of going live on social media before taking drastic steps such as suicide @xcite @xcite . due to all these issues, understanding mental health on social media has become a popular field of study @xcite . studies have indicated that the analysis of the messages posted on social media platforms by the users can help to predict the sign of depression @xcite of the users and the early prediction can help the users to get through the situation. researchers are exploring to analyze social media content to predict the mental health of users in order to lend a helping hand to the needy at the earliest. in this paper, we -team mucs, describe the models submitted to depsign-lt-edi@acl 2022 2 shared task to detect signs of depression in social media text and classify them into into three categories: "not depressed", "moderately depressed", and "severely depressed". two models: i) an ensemble of ml classifiers, namely: random forest (rf), multinomial naive bayes (mnb), multi-layer perceptron (mlp), and gradient boosting (gb) with soft voting ii) tl model with bert, are proposed to classify the given input into one of the three predefined categories. the rest of the article is structured as follows: a review of relevant work is included in section 2, and the methodology is discussed in section 3. experiments, results, and error analysis are described in section 4 followed by concluding the paper with future work in section 5. ## literature review researchers have experimented various methodologies to build systems capable of detecting the signs of depression in social media content and a few of the relevant ones are described below: to analyze suicide ideation symptoms on reddit social media, tadesse et al. ( 2020) even though several techniques have been developed to detect the sign of depression in social media text, there are no full-fledged models for all datasets. further, the trend in posting comments on social media changes frequently because of creative users. hence, this necessitates the need for the development of new models to detect the sign of depression in a social media text. ## methodology the proposed methodology includes two distinct models namely: i) ensemble of ml classifiers and ii) tl model with bert, for detecting the sign of depression in social media text. description of the two models are given below: 3.1 ensemble of machine learning classifiers the proposed ensemble of ml classifiers consists of pre-processing, feature extraction and model building steps and the framework of the proposed model is shown in figure 1. each of the steps are explained below: pre-processing -dataset is pre-processed to remove punctuation, digits, and stopwords, as they do not contribute to the classification task. the english stopwords list available at natural language tool kit (nltk) library 6 is used to remove stop-words and porter stemmer 7 is used to reduce the words to their stems. feature extraction -as the given dataset is imbalanced, resampling is carried out using randomoversampling 8 technique to bring balance in the dataset. frequency based features, namely: tf-idf of character bigrams and trigrams and word unigrams and text based features, namely: count of words and characters followed by the count of adjectives, adverbs, nouns, and pronouns are extracted. these features are combined and used to train the ensemble of ml classifiers. the number of character bigrams and trigrams extracted amounts to 9,024 and word unigrams amounts to 13,169. model building -ml classifiers are generally ensembled by making use of the strength of one classifier to overcome the weakness of another classifier to improve the results. rf, mlp, mnb, and gb classifiers are ensembled to detect the sign of depression in social media text and soft voting is used to predict the category of the test set. the rf algorithm consists of a set of decision trees, each of which is trained with a random subset of features, and the prediction is carried out based on majority voting of all the trees in the forest @xcite . the mlp classifier is widely used in classification as they are simple and easy to implement. it is a feed-forward neural network which consists of three layers, namely: input layer, an output layer, and one or more hidden layers @xcite . the mnb model is a popular ml classifier because of its computing efficiency and relatively good predictive performance @xcite . gb classifier will benefit the regularization methods that penalize different parts of the algorithm and improve the overall performance by reducing overfitting @xcite . ## experiments and results several experiments were conducted with different resampling techniques and various combinations of features and classifiers and the models that gave good performance on the development set are applied for the test set. the dataset provided by the organisers to detect the sign of depression consists of social media comments in english @xcite . in spite of resampling the data using random over sampling to balance the dataset, the results are still low. this may be because the random over sampling technique duplicates features from the minority classes resulting in overfitting for some models @xcite . ## conclusion and future work this paper describes the models submitted by our team -mucs to depsign-lt-edi@acl-2022 shared task to detect signs of depression from social media text in english. the two proposed models are: i) ensemble of ml classifiers trained with the combination of frequency and text based features and ii) tl model with bert. resampling is also explored to handle the data imbalance problem. the tl model outperformed ensemble model with a macro averaged f-score of 0.479 securing 18 th rank in the shared task. future research will explore different sets of features and feature selection algorithms for detecting sign of depression from social media text.