IISERB @ LT - EDI - ACL 2022: A Bag of Words and Document Embeddings Based Framework to Identify Severity of Depression Over Social Media (2022)

Abstract: The DepSign-LT-EDI-ACL2022 shared task focuses on early prediction of severity of depression over social media posts. The BioNLP group at Department of Data Science and Engineering in Indian Institute of Science Education and Research Bhopal (IISERB) has participated in this challenge and submitted three runs based on three different text mining models. The severity of depression were categorized into three classes, viz., no depression, moderate, and severe and the data to build models were released as part of this shared task. The objective of this work is to identify relevant features from the given social media texts for effective text classification. As part of our investigation, we explored features derived from text data using document embeddings technique and simple bag of words model following different weighting schemes. Subsequently, adaptive boosting, logistic regression, random forest and support vector machine (SVM) classifiers were used to identify the scale of depression from the given texts. The experimental analysis on the given validation data show that the SVM classifier using the bag of words model following term frequency and inverse document frequency weighting scheme outperforms the other models for identifying depression. However, this framework could not achieve a place among the top ten runs of the shared task. This paper describes the potential of the proposed framework as well as the possible reasons behind mediocre performance on the given data.

## introduction early prediction of mental illness over social media is a new research area potentially applicable to a wide variety of situations such as identifying people having anxiety and depression over social media @xcite . depression is a common mental illness that involves sadness and lack of interest in day to day activities @xcite . poor recognition and late treatment of depression may have serious consequences like heart failure @xcite . early detection of depression is thus necessary. the information available over social media is a rich source for sentiment analysis or inferring mental health issues @xcite . many research works have been done in the last few years to examine the potential of social media as a tool for early detection of depression @xcite @xcite @xcite . in the last few years, erisk group, has organized a series of nlp shared-task for early prediction of different types of mental illnesses @xcite @xcite @xcite . as part of these shared tasks, many machine learning based frameworks have been proposed for early prediction of depression using social media posts. oliveira @xcite proposed a model using svm classifier with different types of hand-crafted features (i.e. bag of words, lexicons and behavioural patterns) to estimate the level of depressin using posts over reddit. @xcite used different pre-trained language models and random forest classifier for early detection of depression over reddit posts @xcite . guangyao shen proposed a new multimodal depressive dictionary learning model to detect depressed users on twitter, and compared their solution with naive bayes classifier and multiple social networking learning @xcite . the depsign-lt-edi-acl2022 shared-task is focused on early prediction of severity of depression using reddit posts, a popular social media @xcite . the organizers released the reddit posts of a set of users and the ground truths based on the severity of depression of a portion of the data were also released @xcite . there are three levels of severity, viz., no depression, moderate, and severe. the data with ground truths were further divided into training and validation set to build the model. the rest of the data without grounds truths were used as test set to evaluate the performance of the model. we developed a generic text classification framework to identify the severity of depression from the given data without having any additional inputs from the clinical experts. the contributions of this paper are (a) explore the performance of different feature engineering schemes to derive relevant features from given reddit posts, and (b) presenting a generic text classification framework to generate potential features from the given data to help improve the quality of severity of classification of depression. ## methodology the text data contain the chats of different social media users over a period of time. the proposed framework relies on deriving textual features from the given data, and consists of two major steps as described below. ## conclusion the proposed framework relied on extracting different types of relevant text features, including unigrams, bigrams, trigrams and document embeddings to identify the scale of depression. the lr classifier using bow features following tf-idf based term weighting scheme achieved the best performance on validation and test data in terms of macro-averaged f-measure. the performance of different models indicate that the combinations of different types of features are important rather than using a single type of feature set. it has been observed from the experimental results that the conventional bow model performs better than the document embeddings on the test data. note that we have developed the document embeddings based on the given training corpus, which has reasonably low number of documents in compare to the other pretrained deep learning based word embeddings e.g., glove, which were trained on huge text collections. as a result the doc2vec model cannot properly identify the semantic interpretations of the given data and hence its performance is not as good as the bow model. in future we plan to develop some pretrained transformer based embeddings for depression and other mental disorders by collecting documents over social media, wikipedia and other relevant resources to improve the performance of such classification tasks.