Transformers at # SMM 4 H 2024: Identification of Tweets Reporting Children’s Medical Disorders And Effects of Outdoor Spaces on Social Anxiety Symptoms on R eddit Using R o BERT a (2024)

Abstract: With the widespread increase in the use of social media platforms such as Twitter, Instagram, and Reddit, people are sharing their views on various topics. They have become more vocal on these platforms about their views and opinions on the medical challenges they are facing. This data is a valuable asset of medical insights in the study and research of healthcare. This paper describes our adoption of transformer-based approaches for tasks 3 and 5. For both tasks, we fine-tuned large RoBERTa, a BERT-based architecture, and achieved a highest F1 score of 0.413 and 0.900 in tasks 3 and 5, respectively.

## introduction the past few years have witnessed an exponential rise in the use of social media. people can voice their views and opinions on social media platforms such as facebook, reddit, and twitter. twitter and reddit have become major platforms for people seeking help and sharing their medical problems. they also take to these platforms to share their health regimen and medical concerns. thus, reddit and twitter are indispensable resources that aid in better comprehension of health services and exploration of avenues for improvement in health services. the recent developments in the field of natural language processing (nlp) have garnered great interest from the healthcare research community. some of the major breakthroughs include long short term memory @xcite , and gated recurrent units @xcite . however, the advent of transformers @xcite led to a significant improvement in the performance. the social media mining for health applications (smm4h) @xcite unites researchers from across the globe with the objective of developing and sharing nlp methods for mining, representation, and analysis of health-related data. this year, smm4h hosted seven shared tasks involving extraction, classification, and large language model (llm) identification. our team participated in two of the classification tasks, namely, task 3 and task 5. task 3 is a multi-class classification task aimed to qualitatively evaluate the impact of outdoor spaces on social anxiety disorder (sad). around one-third of the people with sad report showing symptoms for ten years before seeking medical help. however, people do share their symptoms on platforms such as reddit to discuss and share their symptoms and seek help to alleviate those symptoms. task 3 focuses on classifying such posts on reddit into one of the four categories, 'positive effect', 'neutral or no effect', 'negative effect', and 'unrelated'. task 5 aims at distinguishing tweets that mention a disorder such as delayed speech from the tweets whose users reported their pregnancy on twitter and also reported having a child with attention-deficit/hyperactivity disorder (adhd), autism spectrum disorders (asd), delayed speech, or asthma. this task facilitates the use of twitter as a medium not just for epidemiological studies but also to investigate the parents' experiences and directly target support interventions. ## methodology transformers have shown great potential in various nlp classification tasks @xcite . for the purpose of performing classification in both the tasks, various transformers were tested. however, roberta showed the best performance for both the multi-class and binary classification tasks. the roberta transformer model was first introduced by facebook in 2019 @xcite . roberta has been trained on 160gb of uncompressed text leading to improved performance on classification tasks. in this work, we present our approach to finetuning roberta for the smm4h classification tasks. a pre-trained model, roberta, was trained in a self-supervised manner, i.e., only raw texts were used to train it without the involvement of human intervention for labeling. ## results and discussion an in-depth analysis was performed on the performance of the large roberta transformer model in the work. the performance was analyzed in different scenarios, both with and without pre-processing. the results obtained on the test data, along with the mean and median of the overall performance of all teams, have been summarised in to fine-tune the transformers, we first perform under-sampling on the data to avoid bias in the model. then, we use two approaches for training using the balanced data. in the first approach, we use the text as is, without any pre-processing, and in the second approach, we perform pre-processing as described in section 2.1. the early stopping approach was used to determine the number of epochs. in task 3, large roberta without preprocessing performed better and achieved an f1score of 0.413 than large roberta with preprocessing, which achieved an f1-score of 0.383. however, in task 5, large roberta with preprocessing achieved an f1-score of 0.900, whereas large roberta without pre-processing achieved only 0.870. ## conclusion and future work in this work, we present our adoption of the large roberta transformer model to perform classification on social media data sourced from reddit and twitter. in task 3, we use the model to perform multi-class classification on the effects of outdoor spaces on social anxiety using reddit posts. in task 5, we use the model to perform binary classification of english tweets to classify whether or not they report medical disorders in children. we also analyze the performance of the transformer, both with and without pre-processing. in the future, an ensembling approach can be implemented. this approach can help amalgamate the results of different transformers, which may lead to improved results @xcite @xcite .