IDIAP Submission@ LT - EDI - ACL 2022: Detecting Signs of Depression from Social Media Text (2022)

Abstract: Depression is a common illness involving sadness and lack of interest in all day-to-day activities. It is important to detect depression at an early stage as it is treated at an early stage to avoid consequences. In this paper, we present our system submission of ARGUABLY for DepSign-LT-EDI@ACL-2022. We aim to detect the signs of depression of a person from their social media postings wherein people share their feelings and emotions. The proposed system is an ensembled voting model with fine-tuned BERT, RoBERTa, and XLNet. Given social media postings in English, the submitted system classify the signs of depression into three labels, namely “not depressed,” “moderately depressed,” and “severely depressed.” Our best model is ranked 3^{rd} position with 0.54% accuracy . We make our codebase accessible here.

## introduction depression is a common mental illness that involves sadness and lack of interest in all day-to-day activities foot_1 . detecting depression is essential as it has to be observed and treated at an early stage to avoid severe consequences @xcite . depression implies mental disorder which may cause disability @xcite @xcite , very few people are able to receive treatment @xcite . it is far more difficult for the people with low socioeconomic status or people living in low economic conditions @xcite , even adjusting for disorder severity @xcite . consequently, there is a need to detect these signs of depression early in time to avoid further repercussions. in this work, we detect the signs of depression, namely in "not depressed," "moderately depressed," and "severely depressed" from person's social media postings where people share their feelings and emotions. there are dataset available for detecting depression task from social media platform such as twitter @xcite @xcite @xcite , reddit (de jesús @xcite @xcite @xcite , facebook @xcite @xcite , instagram @xcite , weibo @xcite and nhanes, k-nhanes @xcite . the linguistic feature extraction methods used for detecting depression signs on social media such as word embedding @xcite , n-grams @xcite , tokenization @xcite , bag of words @xcite , @xcite , emotion analysis @xcite @xcite , part-of-speech (pos) tagging @xcite , behavior features @xcite and sentiment polarity @xcite ). ## related work there have been several attempts to use machine learning algorithms as svm @xcite @xcite , logistic regression @xcite @xcite , neural networks @xcite , random forests @xcite , bayesian statistics @xcite , decision trees @xcite al., label train dev total not depressed 3801 1830 5631 moderately depressed 8325 2306 10631 severely depressed 1261 360 1621 ## shared task description the shared task urges to detect the signs of depression of a person from the social media post where people share their feeling and emotions. its aim is to detect speech for equality, diversity, and inclusion (depsign-lt-edi@acl-2022)(??). the goal was to classify the sign of depression into three labels, namely, "not depressed," "moderately depressed", and "severely depressed" for a given social media posting. the dataset @xcite ) contains 8891, 4496, 3245 comments for training, development, and test set, respectively, annotated with three different labels for the english language. the detailed distribution of the dataset based on labels can be seen in the organizers have provided a baseline code using state-of-art machine learning techniques along with the dataset. ## methodology firstly, we pre-process the social media tweets with the basic nltk library @xcite for stop words removal, emojis removal, and punctuation removal. secondly, we extract the features by tokenizing all the sentences and mapping those tokens with the word ids. for every sentence in the dataset, we follow a series of steps (i) tokenize the sentences (ii) prepend the [cls] token to the start (iii) append the [sep] token to the end (iv) map the token to their ids (v) pad or truncate the sentenced to max length (vi) mapping of attention masks for [pad] tokens. we padded and truncated the @xmath0. the generated sequence sentences are passed for encoding with its attention mask (simply differentiating padding from nonpadding). finally, we predicted the labels using ensembles voting model for bert @xcite , xlnet @xcite and roberta model @xcite . bert is bi-directional encoder representation from transformers (bert), involving pre-training bi-directional transformers for language understanding from an unlabelled text by jointly conditioning left t-right context for all layers. fine-tuning of a pre-trained bert model can be easily done with just one additional output layer for developing a state-of-art model for a wide range of nlp tasks without substantial taskspecific architecture modifications. robustly optimized bert approach has emphasized data being used for pre-training and the number of passes for training. the bert model is optimized with dynamic masking, more extended training with big batches over more data, removing the next prediction objective, and dynamically changing masking patterns for training data. the model achieved state-of-art results on glue, race, and squad without multi-task finetuning for glue or additional data for squad. ernie 2.0 is another continual pre-training framework that efficiently supports customized training tasks in multi-task learning incrementally. the pre-trained model is finetuned to adapt to various language understanding tasks. the framework has demonstrated significant improvement over bert and xlnet on approximately 16 tasks, including glue. we take each label to the sentence and number of labels with the highest vote if chosen as the final label in ensemble voting @xcite . ## conclusion in this paper we present our system paper submission for depsign-lt-edi@acl-2022. we aim to detect the signs of depression of a person from their social media postings wherein people share their feelings and emotions. the proposed system is an ensembled voting model with fine-tuned bert, roberta, and xlnet. given social media postings in english, the submitted system classify the signs of depression into three labels, namely "not depressed," "moderately depressed," and "severely depressed." our best model is ranked 3 rd position with 0.54% accuracy . the system performs quite well to recognize the comments for depression comments; in the future, we intend to work on a multitask learning framework to handle all kinds of depression or illness and even the severity of depression. we also aim to detect multilingual depression speech in the code-mixing scenarios.