T ech W hiz@ LT - EDI -2023: Transformer Models to Detect Levels of Depression from Social Media Text (2023)

Abstract: Depression is a mental fitness disorder from persistent reactions of unhappiness, void, and a deficit of interest in activities. It can influence differing facets of one’s life, containing their hopes, sympathy, and nature. Depression can stem from a sort of determinant, in the way that ancestral willingness, life occurrences, and social circumstances. In current years, the influence of social media on mental fitness has become an increasing concern. Excessive use of social media and the negative facets that guide it, can exacerbate or cause impressions of distress. The nonstop exposure to cautiously curated lives, social comparison, cyberbullying, and the pressure to meet unreal standards can impact an individual’s pride, social connections, and overall well-being. We participated in the shared task at DepSignLT-EDI@RANLP 2023 and have proposed a model that identifies the levels of depression from social media text using the data set shared for the task. Different transformer models like ALBERT and RoBERTa are used by the proposed model for implementing the task. The macro F1 score obtained by ALBERT model and RoBERTa model are 0.258 and 0.143 respectively.

## introduction social media has transformed the way we combine, correspond, and share facts. while it has led to numerous benefits, there is an increasing concern regarding its negative effect on mental health, specifically when it comes to depression @xcite . the loyal uncovering of carefully curated lives, social comparison, and cyberbullying are just instances of how social media can cause feelings of depression. one of the negative effects of social media is the phenomenon of social comparison. platforms like instagram and facebook frequently present idealized interpretations of crowd's lives, stressing their realizations, travels, and happy moments. this never ending risk to seemingly perfect lives can lead human beings to compare themselves negatively, developing feelings of failure, envy, and depression @xcite . cyberbullying is another important concern on social media podiums. the obscurity and distance given by these platforms can encourage human beings to undertake harmful behaviors, like spreading rumors, making cruel comments, or posting offensive content @xcite . such experiences can lead to increased social isolation, reduced pride, and depression. to address the negative impact of social media on mental health, it is owned by adopting healthy habits and practices. firstly, limiting social media use can help humble exposure and counter excessive comparison or rumination. engaging in offline activities, spending time accompanying loved ones, and the following amusement can determine a much-needed break from the in-essence globe. cultivating a healthful online atmosphere is important. this includes being aware of the content we consume and share, encouraging positiveness and support, and vigorously combating cyberbullying. building a forceful support network online and offline can supply more emotional support and neutralize the negative effects of social media. while social media has transformed communication, it is crucially expected to be aware of its potential negative effects on mental health, particularly depression. depression is considered as one of the most severe mental health diseases, as it often leads to suicide. hence identifying and summarizing existing evidence concerning depression from data provided by users on social media has become important @xcite . the shared task on detecting signs of depression from social media text @xcite was a part of ranlp 2023 which is based on english comments. the task of detecting signs of depression from social media text is a multi-class classification problem, in which the model has to predict the la-bel associated with the text as severe, moderate, or not depression. for example, the text "i didn't deserve all of this: i have been suffering from depression for 5 years following personal traumas, i am surviving, in certain situations, i put on an infinite sadness, the panic disorder truncates all my attempts at recourse" represents severe depression. the text, "any advice? : so... i don't know where to start and even if i should post this here is moderate and insecurities, fuck em. : i constantly feel like anyone i talk to at all, or act like myself around is just trying to get me to shut up." represents a not depressed case 2 related works a gold standard data set had been developed by @xcite to classify the text based on the levels of depression. an empirical analysis using different traditional machine learning algorithms had been presented. the problem of data imbalance had been overcome using data augmentation. the model with word2vec victories and random forest classifier on augmented data had outperformed the other models. salas-zárate et al. ( 2022 ) summarized different works on detecting depression from social media posts. it had been identified that twitter was the most studied social media for depression sign detection, and word embedding was the most prominent linguistic feature extraction method. support vector machine (svm) was the most used machinelearning algorithm. long-short term memory (lstm) model with two hidden layers and large bias together with recurrent neural network (rnn) with two dense layers had been used by @xcite to predict depression from text and had provided better results. different transformer models like distilbert, roberta and albert had been used by @xcite to classify social media posts based on the severity of depression associated with them. the detection of mental illness including depression had been implemented as a multi-class classification problem by @xcite . the use of traditional machine learning, deep learning, and transfer learning-based methods had been explored and the pre-trained roberta transfer learning model resulted in better outcomes. the strengths of the sequence model and transfigure 2: training dataset statistics former model had been consolidated by zhang et al. (2022). the model used a robustly optimized bert approach to map words into word embedding space and a bidirectional long short-term memory model to capture the long-distance contextual semantics. ## dataset description the data set that is used to implement the depression detection was the training, evaluation and test dataset that was provided by the organizers of the shared task. each instance of the training dataset had a label specifying whether the text is moderate, severe, or not depression. the previous version of the task (s et al., 2022) had used a dataset in which the social media text were classified as one of the same three categories. the data distribution of the training and development dataset for the task is shown in the data distribution of different classes of data in the training data is represented in figure 2 and the development dataset is represented in figure 3 . ## system description initially, the three datasets provided by the task organizers, namely the training dataset, development dataset, and testing dataset were collected. the training dataset is preprocessed where unnecessary digits, characters, and white spaces are removed using tokenization and it is followed by an encoding process. then the model is created. in this system, two pre-trained transformer models were used namely albert and roberta. the preprocessed dataset along with the model created is used for the training phase. each model is then evaluated using the development dataset. the albert model that provided the highest accuracy is taken as the final run for submission and was used to find the predictions for the testing dataset. the proposed architecture is represented in figure 4 . the removal of unnecessary information is taken care of by the preprocessing phase. all three datasets namely the training, evaluation, and test dataset are preprocessed. this is followed by the process of model building, where trained models namely albert and roberta were used. in the training phase, the pre-trained models are trained using the preprocessed training dataset. the evaluation of the trained model is carried out in the evaluation phase using the evaluation dataset which makes use of the accuracy as the parameter of evaluation. fine-tuning of hyper-parameters is performed to improve the accuracy of the proposed system. the labels for the text in the dataset are predicted during the testing phase. contextual embeddings are generated and are used during the training of the model. ## results the metrics that were considered for the evaluation of the task was the macro-f1 score and accuracy. the f1 score is an overall measure of a model's accuracy that merges precision and recall. an extreme f1 score represents that the classification has happened accompanying the reduced number of false positives and low false negatives. the values of the performance metrics particularly the f1 score and accuracy acquired for various models are shown in ## error analysis the f1 score obtained for the task using the proposed albert model shows that more false positive and false negative classification has occurred. one reason for this could be considered as the data imbalance nature of the dataset. considering the number of instances for the class labeled severe is higher, and the f1 score, precision, and recall associated with this class are high when compared to figure 6: classification report -albert model the class, not depression. this represents that the number of misclassifications increases when the number of instances for training is lower, which is associated with data imbalance. data augmentation could be considered to improve the model's performance. examples of texts that are misclassified are shown in ## conclusions depression detection has become an important area of research as it is interlinked with different application areas. having this in mind ranlp 2023 had come up with the task of depression detection where the text is classified into moderate, severe, and not depression. the exploration of detecting signs of depression from social media text using the albert and roberta models at lt-edi@ranlp 2023 demonstrates the significance of leveraging advanced natural language process-s.no. text predicted label actual label 1 i hate that people moderate severe don't understand that i don't want to kill myself, i just don't want to be alive anymore 2 but here i am, severe moderate 24 years old man and doing exactly that 3 i'm trapped inside. severe moderate does anyone else get that feeling? my memories from the past few years are shoddy at best. i think i'm losing it future enhancement to this work can be associated with handling contextual information which can help in effectively detecting depression. the usage of hybrid approaches where different deep learning models are combined can also facilitate the efficient detection of depression from the text.