Experimenting with Transformer-based and Large Language Models for Classifying Effects of Outdoor Spaces on Social Anxiety in Social Media Data (2024)

Abstract: Social Anxiety Disorder (SAD) is a common condition, affecting a significant portion of the population. While research suggests spending time in nature can alleviate anxiety, the specific impact on SAD remains unclear. This study explores the relationship between discussions of outdoor spaces and social anxiety on social media. We leverage transformer-based and large language models (LLMs) to analyze a social media dataset focused on SAD. We developed three methods for the task of predicting the effects of outdoor spaces on SAD in social media. A two-stage pipeline classifier achieved the best performance of our submissions with results exceeding baseline performance.

## introduction social anxiety disorder (sad) is a prevalent anxiety disorder that affects up to 12% of the population at some point in their lives @xcite . interestingly, social media platforms like reddit have become a space for people with sad to connect, share their experiences, and seek advice on managing symptoms. while research suggests that spending time outdoors in natural environments can be beneficial for alleviating anxiety in general @xcite , little is known about the specific impact of such environments on sad. this study investigates the effect of outdoor spaces on social anxiety using social media posts as a source of data. ## data and task description this work leverages the dataset provided by the organizers of smm4h @xcite . the data originates from the r/socialanxiety subreddit on reddit and consists of 3,000 annotated social media posts. the training set comprises 1,800 posts, each labelled with a code based on the user's sentiment towards the mentioned nature-related spaces or activities. a four-class labelling scheme was employed: positive: the space/activity benefits the user's well-being; neutral: mention of nature without a clear impact; negative: the space/activity has a negative impact; and unrelated: posts where the nature keyword is metaphorical or unclear in meaning. the task is four-class classification on these labels. ## results the evaluation metric for this task is the macroaveraged f1-score over all 4 classes. based on these results, we selected roberta for the final submission employing the four-class classification approach. additionally, the pipeline approach for the submission utilized mentalbert for the initial binary classification of posts into "related" and "unrelated" categories and roberta model for the following three-class classification into positive, neutral, and negative. roberta was also employed in the multi-task approach. results of our submission provided by task organizers is shown in ## conclusion this study investigated the connection between social anxiety and outdoor spaces on social media. we employed transformer-based and llms classifiers on the provided dataset. our results exceeded the baseline, and the pipeline approach achieved the highest performance as evaluated by the task organizers.