BERT 4 EVER @ LT - EDI - ACL 2022-Detecting signs of Depression from Social Media:Detecting Depression in Social Media using Prompt-Learning and Word-Emotion Cluster (2022)

Abstract: In this paper, we report the solution of the team BERT 4EVER for the LT-EDI-2022 shared task2: Homophobia/Transphobia Detection in social media comments in ACL 2022, which aims to classify Youtube comments into one of the following categories: no,moderate, or severe depression. We model the problem as a text classification task and a text generation task and respectively propose two different models for the tasks. To combine the knowledge learned from these two different models, we softly fuse the predicted probabilities of the models above and then select the label with the highest probability as the final output. In addition, multiple augmentation strategies are leveraged to improve the model generalization capability, such as back translation and adversarial training. Experimental results demonstrate the effectiveness of the proposed models and two augmented strategies.

## introduction this paper includes a review and explanation of bert4ever's ideas and experiments for the lt-edi-2022 shared task2 @xcite : homophobia/transphobia detection in social media comments in acl 2022. in this task, participants would be given sentences from social media comments and then predict whether they contain any form of homophobia/transphobia. the homophobia/transphobia detection dataset @xcite , a collection of comments from youtube, serves as the task's seed data. the comments were manually annotated to show whether the text contained homophobia/transphobia. the label annotation consists of three categories: no depression, moderate depression and severe depression. this is a comment/post level classification task. for this task, our solution consists of two main blocks. prompt-learning is a new paradigm in modern natural language processing to adapt pretrained language models (plms) to downstream nlp tasks, which modifies the input text with a textual template and directly uses plms to conduct pre-trained tasks. it directly adapts plms to cloze-style prediction, autoregressive modeling, or sequence to sequence generation, resulting in promising performances on various tasks such as text classification @xcite , named entity typing @xcite and relation extraction @xcite . in addition, given the limited amount and category imbalance of training data available, we experiment with multiple augmentation techniques, including continual pre-training @xcite , back translation, adversarial training @xcite , easy ensemble @xcite . considering the combination of the knowledge learned from the different models, we softly fuse the predicted probabilities of the two models above and then select the label with the highest probability as the final output. we conduct extensive experiments on the given dataset and achieves comparable results which reaches 0.5818 on the test set and 0.5426 in the online evaluation. in summary, our contributions are as follow: the rest of this paper is organized as follows: section 2 describes the details of the proposed method in this paper. section 3 elaborates the experimental setup and analyzes the experimental results. finally, conclusions are drawn in section 4. ## conclusion we present our submission to the shared task on homophobia/transphobia detection in social media comments at lt-edi 2022-acl 2022. we model the problem as two different tasks and propose two novel methods respectively for the tasks. in addition, focusing on two problems of small data size and category imbalance in the original training set, we leverage multiple augmentation strategies to enhance the model performance. we softly fuse the predicted probabilities of different models and output the label with highest probability. we evaluate each single model on the validation set drawn from the training set and provide some explanations. to justify our design choices, we conduct an ablation study. overall, we achieve the 4th macro averaged f-score of the dataset on the online evaluation. in the further work, on one hand, we would explore how sentiment knowledge and prompting technique can further improve the model performance. on the other hand, we would expand our emotion lexicon and create more fine-grained representations of word-emotion clusters, allowing the classification model to learn more about emotions.