MUCS @ LT - EDI 2023: Detecting Signs of Depression in Social Media Text (2023)

Abstract: Depression can lead to significant changes in individuals’ posts on social media which is a important task to identify. Automated techniques must be created for the identification task as manually analyzing the growing volume of social media data is time-consuming. To address the signs of depression posts on social media, in this paper, we - team MUCS, describe a Transfer Learning (TL) model and Machine Learning (ML) models submitted to “Detecting Signs of Depression from Social Media Text” shared task organised by DepSign-LT-EDI@RANLP-2023. The TL model is trained using raw text Bidirectional Encoder Representations from Transformers (BERT) and the ML model is trained using Term Frequency-Inverse Document Frequency (TF-IDF) features separately. Among these three models, the TL model performed better with a macro averaged F1-score of 0.361 and placed 20th rank in the shared task.

## introduction depression is a mental health condition resulting in feelings like sorrow, emptiness, loss of interest or distress and these feelings vary from individual to individual @xcite . in severe conditions, depression may cause thoughts of suicide or death. the user-friendly social media platforms allow users to share their posts or seek help from the online community @xcite . depressed people might feel more ease in sharing their feelings, difficulties, and experiences, via posts on social media. further, some people find it therapeutic to talk about their problems with others to get guidance, mental support, and sympathy from their online community. according to research, it might be possible to predict the signs of depression an online user is facing by reading the content of such users' posts on social media sites @xcite . as depression can lead to significant changes in the individuals' posts on social media, analysing such posts can help in identifying the signs of depression in users. once identified, any help or support can be extended to the users suffering from depression. the increase in the number of social media users is increasing the user-generated text drastically @xcite . such user-generated text consists of hashtags, emojis, alphanumeric characters, slangs, short forms, etc. in addition to the actual content. processing and analyzing this complex social media text using conventional text analysis techniques to get valuable insights into the data is challenging. this necessitates the need for automated tools/approaches to process social media text. the automated approaches can identify depressive symptoms by systematically examining signs of depression in social media texts, providing a possibility for timely intervention and support for depressed individuals @xcite . though researchers have explored several techniques to detect signs of depression in social media text, it still remains a challenge because of the complexities of social media text. to address the challenges of identifying signs of depression in social media text, in this paper, we -team mucs, describe the models submitted to "detecting signs of depression from social media text" shared task, organised by depsign-lt-edi@ranlp-2023 1 @xcite . the goal of this task is to detect the signs of depression from social media posts and classify them into one of three categories: "not depressed", "moderately depressed", and "severely depressed". the shared task is modeled as a multi-class text classification problem and two approaches: i) tl model with fine-tuning bert and ii) ml classifiers (lr and mnb) trained on tf-idf word n-grams, are proposed for the task. the rest of the paper is as follows: related work is contained in section 2 followed by the methodology in section 3. experiments with their results are explained in section 4 and the paper concludes with future work in section 5. ## related work researchers have experimented many techniques to recognise signs of depression in social media content, and the description of some of the most useful studies are given below: tadesse et al. (2020) developed ml models (support vector machine (svm), naive bayes (nb), random forest (rf), extreme gradient boosting) trained with tf-idf and statistical features and deep learning (dl) models (combination of long short term memory and convolutional neural network (lstm+cnn) model) trained with word2vec embeddings. among the proposed models, lstm+cnn model obtained an accuracy of 93.8%. to identify the signs of depression in social media posts, hegde et al. (2022a) used two learning models: i) tl model with fine-tuning bert and ii) ensemble (rf, multilayer perceptron, mnb, and gradient boosting (gb)) model with soft voting. as the dataset is imbalanced resampling technique (i.e. randomoversampling) is used to balance the dataset. among the two models, the tl model performed better with a macro-average f1-score of 0.479. aswathy et al. (2019) utilized word2vec for generating word embeddings to train lstm+cnn 1 https://codalab.lisn.upsaclay.fr/competitions/11075 and svm models to identify the signs of depression from the tweets. their models obtained the weighted average f1-scores of 0.97 and 0.85 for the lstm+cnn and svm models respectively. mowery et al. (2016) , developed ml (decision tree, linear perceptron, rf, lr, svm, and nb) models for determining whether a tweet represents evidence of depression or not and experimented on "depressive symptoms and psychosocial stressors associated with depression (sad)" dataset which contains 9,300 tweets. to train their models, they used features including unigrams, emoticons, age, gender, linguistic inquiry word counts, etc. and obtained 0.52 average f1-score for svm classifier. @xcite proposed an ensemble of fine-tuned bert models (a lite bert (al-bert), distilbert, robustly optimized bert (roberta), and bert base model) with majority voting and experimented on social media comments in english @xcite and obtained a macro f1 score of 0.54. to summarize, different learning approaches including ml, dl, and tl models are explored for detecting signs of depression in social media text. though several techniques are experimented to detect the signs of depression in social media text in english, not all models have performed well. further, the dynamic nature of user-generated content on social media makes the task more challenging. this emphasises the need for developing models to enhance the performance of identifying depressive symptoms from social media text. ## methodology to detect the signs of depression in social media texts, the proposed methodology comprises of two learning models: i) tl model with fine-tuning bert and ii) ml model trained with tf-idf ngrams. description of the two models are given below: ## experiments and results the proposed models aim to detect the signs of depression from the social media posts and classify them into one of levels of the signs of depression: "not depressed", "moderately depressed", and "severely depressed". the statistics of the dataset provided by the shared task organizers @xcite is shown in predictions obtained from the proposed tl model and ml models are evaluated by the shared task organizers based on macro-averaged f1-score. the performances of the proposed models on development and test sets are shown in the performances of the proposed models are influenced by issues like: the imbalanced dataset, an incorrect spelling of words, and limited vocabulary in the dataset. further, the given train set consists of very less number of samples for 'severe' class compared to the other classes. as a result, the proposed model failed to understand the features and patterns associated with the 'severe' class during the training process. few misclassified samples in the test set along with the actual and predicted labels and remarks are shown in ## conclusion and future work in this paper, we describe two models: tl model with fine-tuning bert and ml models (lr and mnb), for detecting signs of depression in social media text and classify them into one of three categories: "not depressed", "moderately depressed", and "severely depressed". these models are submitted to the "detecting signs of depression from social media text" shared task at lt-edi@ranlp2023. among proposed models, the tl model outperformed the ml models with a macro-averaged f1-score of 0.361. efficient techniques will be explored to handle the imbalanced dataset and improve the performance of the proposed models.