CIMAT - NLP @ LT - EDI -2023: Finegrain Depression Detection by Multiple Binary Problems Approach (2023)

Abstract: This work described the work of the team CIMAT-NLP on the Shared task of Detecting Signs of Depression from Social Media Text at LT-EDI@RANLP 2023, which consists of depression classification on three levels: “not depression”, “moderate” depression and “severe” depression on text from social media. In this work, we proposed two approaches: (1) a transformer model which can handle big text without truncation of its length, and (2) an ensemble of six binary Bag of Words. Our team placed fourth in the competition and found that models trained with our approaches could place second

## introduction approximately 280 million persons suffer depression around the world, and suicide is the fourth cause of death according to world health @xcite . other studies have highlighted the impact of social media on adolescents, introducing a phenomenon known as "facebook depression" @xcite . this term refers to the symptoms of depression that young people may experience when they spend significant time on social media platforms. additionally, a study on college students in afghanistan revealed a correlation between social media addiction and depression @xcite . the findings suggested that individuals experience more severe symptoms of depression as their social media usage increases. given the increasing number of people affected by depression, developing systems to detect individuals with this mental illness is crucial. one notable effort in this direction is the shared task on detecting signs of depression from social media texts at @xcite . our team, cimat-nlp, proposed two approaches for this task. firstly, we divided significant texts into sub-packages and utilized the roberta transformer @xcite . secondly, we employed an ensemble of six binary bags of words (bow) models with different characteristics. the remaining sections of this paper are organized as follows: section 2 discusses related works on detecting depression on social media. section 3 provides an overview of the competition and data distribution. in section 4, we describe the methods we developed for the task. section 5 presents the results obtained by our models. finally, in section 6, we draw conclusions based on our work. ## related work detecting depression presents a challenging task due to the intricate nature of this mental disorder. the complexity of this mental illness makes screening for depression a demanding task. in this field, various workshops are dedicated to this cause, such as the early internet risk prediction workshop (clef erisk) @xcite . this workshop focuses on developing methods for automatic systems for online risk prevention. in the context of erisk, proposals have predominantly focused on the use of bag of words (bow)-based machine learning models together with svm classifiers or deep neural networks, due to the proven effectiveness of both approaches. notable examples include the top three best ranks in the 2018 erisk competition @xcite ,funez et al. (2018) ), demonstrating the effectiveness of bow in representing text for tasks related to detecting mental illness. our approach follows suit, as we choose to implement bow with several specialized classifiers, targeting different levels of depression. it is evident that transformers have surpassed the state of the art in various nlp tasks. however, a drawback of transformers is their inability to process large inputs, which is a common scenario in author profiling tasks, due to the high computational resources required. in @xcite , this issue was addressed by segmenting the text into subchunks per category during training and averaging the prediction probabilities of these subchunks for an overall prediction. another limitation of transformers lies in the variability of their predictions, which stems from variations in their initialization seeds during training. to mitigate this variability and leverage the benefits of these results, multiple transformer ensemble techniques have been proposed. poświata and perełkiewicz (2022 the depsign-lt-edi@ranlp-2023 dataset consists of texts collected from various social media networks. all the texts are in english and have been classified into three labels: "not depression," "moderate," and "severe." the competition was divided into two phases. in the first phase, the organizing committee provided participants with the training and development data to work. in the second phase, participants were given the test data to make predictions and submit their results. ## 228 89 ## method in this section, we described the approaches used for the task. in the first approach, we proposed a transformer model. because the text is in english, we used @xcite , roberta @xcite , mentalbert and mentalroberta @xcite for our experiments. the second approach is an ensemble of six bags of words, each specializing in diverse detection with different characteristics. ## results in this section, we present the results in the dev dataset for each approach to choosing the model and hyperparameters for the submissions in the competitions. in the second part of this section, we present the competition results for the two models. figure 3 : in this ensemble, we refer to "severe" as s, "moderate" as m , and "not depression" as n . the counter variables cs, cm and cc are for "severe", "moderate", and "not depression", respectively. the counter variables, voting scheme and final prediction are per user. ## ethical issues the automatic detection of mental illness, such as depression, using user-generated data raises several critical ethical considerations. one key aspect is the need to prioritize and ensure the anonymity of the users whose text is recorded for training and development purposes. crowd-sourcing is commonly employed in the context of labelling the data, where multiple annotators assess and assign labels to the instances. however, this process introduces a level of subjectivity, and there is no guarantee of perfect accuracy or consistency in the labelling. annotators may have different interpretations or judgments, leading to potential discrepancies in the assigned labels. consequently, the reliability and consistency of the labelled data may be influenced by the subjective opinions of the annotators. the data used for automatic detection should ideally be collected with explicit user consent, where individuals knowingly and willingly provide their data for such purposes. however, in some cases, the data might have been obtained without users' explicit permission or awareness. this raises concerns about privacy violations and the potential discomfort or distress users may feel upon discovering their data is being used without their knowledge. it is essential to prioritize data anonymization and protection of user identities throughout the entire data collection and storage process. furthermore, efforts should be made to obtain explicit user consent when collecting data, ensuring individuals are fully aware of how their data will be used and can deny the use if they wish. ## conclusion as we see in the previous subsection, multiclass classification is a difficult task for the complexity of depression detection. our proposed models obtained performances similar to other teams in better places in the competition. the bow ensemble obtained an f1-macro score close to our best-proposed model using less computational resources than a transformer model, as this model does not need gpu or a large amount of storage to be used. the bow-multiclass surpassed too the transformer model and the ensemble, even though the dev dataset did not surpass the proposed models; this could be because the test dataset is smaller than the dev dataset, and machine learning models as bow tend to function better with fewer data. in future work, we plan to explore better strategies for the values in the weights for the ensemble models of bow and the rules made for the final predictions.