SSN @ LT - EDI - ACL 2022: Transfer Learning using BERT for Detecting Signs of Depression from Social Media Texts (2022)

Abstract: Depression is one of the most common mentalissues faced by people. Detecting signs ofdepression early on can help in the treatmentand prevention of extreme outcomes like suicide. Since the advent of the internet, peoplehave felt more comfortable discussing topicslike depression online due to the anonymityit provides. This shared task has used datascraped from various social media sites andaims to develop models that detect signs andthe severity of depression effectively. In thispaper, we employ transfer learning by applyingenhanced BERT model trained for Wikipediadataset to the social media text and performtext classification. The model gives a F1-scoreof 63.8% which was reasonably better than theother competing models.

## introduction one of the crucial modern world problem that needs attention today is mental health and its wellness. according to ghdx, around 5% of all young adults have depressive disorders @xcite . about half of them never get it diagnosed or treated. since the advent of the internet, people have felt more comfortable discussing topics like depression and stress online due to the anonymity it provides @xcite . people have come forward to share their mental struggles with others and are ready to seek help. this sharedtask has used data scraped from various social media sites like twitter, reddit to detect signs and the severity of depression symptoms. the main target of this task was to identify deep learning models that performed well in classifying tweets based on the level of severity of depression. the term severity here is based on the presence of certain words and their inclining in the word spectrum for mental health. the textual data contains many hidden patterns and styles that distinctly identifies signs of depression @xcite . in this paper, we designed a transformer model with significant fine-tuning to predict if the context shows moderate, severe or no signs of depression. ## bert based transfer learning bert (bidirectional encoder representations from transformers) is a pre-trained deep learning model that functions on the sequence to sequence learning of text. bert models have found to be performing well in understanding textual data on depression identification @xcite . the types of bert model differs based on the number of transformer layers, self-attention layers, number of parameters, types of fine tuning, masking and word embedding and so on. the classifier uses google's bert-small model foot_0 from tfhub pretrained for seqtoseq task and adapts it for text classification with both pooled and sequence type outputs. the model used in this classification system is very similar to @xcite . the difference occurs in fine-tuning of the model, where in addition to the dropout layer, a dense classifier layer is added to obtain the final label. the bert model for depression detection is shown in figure 1 . the training phase consists of the following steps: ## experimental setup in this section we will see the experimental arrangement of the model, their preprocessing steps and their results and discussion. ## conclusion detecting the signs of depression form just a collection of words is a huge accomplishment in the