SSN _ MLRG 1@ LT - EDI - ACL 2022: Multi-Class Classification using BERT models for Detecting Depression Signs from Social Media Text (2022)

Abstract: DepSign-LT-EDI@ACL-2022 aims to ascer-tain the signs of depression of a person fromtheir messages and posts on social mediawherein people share their feelings and emo-tions. Given social media postings in English,the system should classify the signs of depres-sion into three labels namely “not depressed”,“moderately depressed”, and “severely de-pressed”. To achieve this objective, we haveadopted a fine-tuned BERT model. This solu-tion from team SSN_MLRG1 achieves 58.5%accuracy on the DepSign-LT-EDI@ACL-2022test set.

## introduction depression is a frequently found mental illness that involves sadness and lack of interest in all day-to-day activities. it is vital to detect and treat depression at an early stage to avoid consequences. treatment involves diagnosis of patient who might have depression, but patient would have have to initiate contact in order to receive this opportunity. it has been proven by multiple studies that depression is preventable and early stage detection and the most severe effect of this disease can be mitigated by quick treatment. however, openly accessible tools to this end are very few and very rare. the rise of social media as one of humanity's most important public communication platforms presents a potential prospect for early identification and management of mental illness @xcite . people's daily lives are increasingly dominated by social media @xcite . on social media, a lot of multimedia content, mostly brief words and photographs, is constantly exchanged @xcite . information put on the internet, as opposed to conventional human contact, may be swiftly disseminated by acquaintances and accessed by strangers @xcite @xcite @xcite . this method allows users to avoid direct interaction with individuals while also increasing their urge to convey their emotions. this task 4 in second workshop on language technology for equality, diversity, inclusion (lt-edi) aims to detect depression from english text @xcite . this research article evinces how a bert transformer model can effectively classify social media texts into 3 classes "not depressed", "moderately depressed", and "severely depressed". the model is trained on social media texts from various sources, labelled as above. the process involves 2 subtasks -preprocessing and training. in subtask-a, the text is cleaned up, and converted to a format more suitable for context and sentiment analysis for depression detection. in subtask-b a simple transformer bert classifcation model is trained on the task data, and the performance is evaluated. ## system overview the first step in the system flow is preprocessing the data. the aim is to remove any unnecessary elements from the text, and transform the data given into a more uniform form. this involves the following steps: (i) extend contractions -a contraction is an abbreviated version of a word, such as don't, which stands for do not, and aren't, which stands for are not. in order for the model to perform better, we need to broaden this contraction in the text data. the next part of the system is the bert classification model. the pre-trained bert model from simpletransformer api has been used in this model. the bert model is fine tuned on the processed data, to give a 3-class classification model capable of effectively classifying new data encountered into various classes. the working of bert model is shown in figure 1 . it has two phases as pre-training and finetuning. ## experimental setup the data is imported as a pandas dataframe. this dataframe is first passed to a function to expand all contractions, this is done with a precollected dictionary of contractions. next, the sentence is converted to lower case, and punctuation's are removed using a regex compiled expression. the data at this point is parsed for english stopwords, a list of which are obtained from the natural language tool kit (nltk) in python. these stopwords are removed. stemming and lemmatization are also done using the python ntlk. white spaces are removed using a regex expression. nlpaug library is used for data augmentation to balance the data between the 3 classes since the data is imbalanced across the labels. a bert model is trained on the above processed data, multiple parameters have been tested using wandb sweeps, and the highest scoring configuration has been used. a learning rate of 1e-4 and a batch size of 16 were used. ## results the efficacy of this model has been proven by the results given below: we have obtained an accuracy of 59% and the top rank team has achieved 66% accuracy. further improvement in the system can be achieved by tweaking the hyper parameters. ## conclusion in summation, our research work presents a bert model for classification of social media texts into the 3 target classes. the current model does not perform very well on the given data. one reason that can be attributed to this is the complexity of different texts, with various parts involving various sentiments. future models, will aim to remedy this through splitting the sentences based on their complexity, and using different models for different levels of complexity. the other reason for the low results may be the different manifestations of depression symptoms in different people, this will be remedied by using various other features along with social media texts. in the future, a classifier to segregate texts based on complexity and the number of sentiment expressions may be supplied to further improve the efficiency of the classifier.