DILAB at # SMM 4 H 2024: Analyzing Social Anxiety Effects through Context-Aware Transfer Learning on R eddit Data (2024)

Abstract: This paper illustrates the system we design for Task 3 of the 9th Social Media Mining for Health (SMM4H 2024) shared tasks. The task presents posts made on the Reddit social media platform, specifically the *r/SocialAnxiety* subreddit, along with one or more outdoor activities as pre-determined keywords for each post. The task then requires each post to be categorized as either one of *positive*, *negative*, *no effect*, or *not outdoor activity* based on what effect the keyword(s) have on social anxiety. Our approach focuses on fine-tuning pre-trained language models to classify the posts. Additionally, we use fuzzy string matching to select only the text around the given keywords so that the model only has to focus on the contextual sentiment associated with the keywords. Using this system, our peak score is 0.65 macro-F1 on the validation set and 0.654 on test set.

## introduction analyzing health-related topics from social media data, such as twitter, facebook, and reddit, to gauge public sentiment is an area of significant research interest. the smm4h 2024 @xcite shared tasks encourage researchers to address some of these research problems. we decided to take part in task 3. the focus of task 3 is on social anxiety disorder (sad) @xcite , and the motivation behind it is that while a significant number of people may experience sad in their lives, they may experience its symptoms for much longer before actually seeking professional help. however, people often turn to social media to discuss symptoms of sad such as the r/socialanxiety subreddit. in particular, this task aims to understand the effects of outdoor activities on the symptoms of sad. our approach to this involved stripping text from each post so that only the context surrounding the keywords was fed into our model. our model consisted of a roberta backbone, followed a dense layer, a batch normalization layer, a dropout layer, and finally a classification layer, which will classify the post into one of four categories depending on the effect of the outdoor activity on symptoms of sad : positive, negative, no effect, or keyword is not an outdoor activity. we then proceeded to fine-tune this model on that data provided. keyword text class run 21/m. i want to experience young love, but i've never had a relationship before... (continued) 0 pre-processing with fuzzy string matching. using the fuzzywuzzy library, we select all instances of each keyword that appeared in the posts using fuzzy string matching (fsm) using levenshtein distance @xcite . to limit false positives, we only select matched keywords whose length was at least 3, and whose similarity_score returned from the fuzzywuzzy.process.extract func model. our model relies on a roberta 1 backbone @xcite , from huggingface transformers library @xcite . each sentence is first passed into the roberta backbone. since, we treat the task as a simple sequence classification task, we perform mean-pooling on the 768dimensional embeddings (1024 if roberta-large) generated by the roberta model. then, we pass the pooled embedding into 768-dimensional (1024 if roberta-large) dense layer followed by a batch normalization and relu layer. we then pass the output through a dropout layer before passing it through a dense layer, which classifies it as one of the four categories. implementation details. our tokenizer employs a maximum token length of 256 tokens with lowercase text processing. we fine-tune models for 20 epochs using a batch size of 8, a learning rate of 1e -5 with the adam optimizer (kingma and ba, 2017) and dropout @xmath0.4, and the cross entropy loss function. all trials use a fixed random seed of 42, and no additional data is used. ## conclusion in this work, we study social anxiety disorder using reddit data to identify individuals who may experience its symptoms for a prolonged period before seeking professional help. by applying fuzzy string matching to find contexts and transfer learning with roberta, we achieve a validation set macro-f1 score of 0.65 and a test set score of 0.654, outperforming the task mean and median scores. we also have analyzed errors in our best performing model with possible explanations and reasoning.