T hang DLU at # SMM 4 H 2024: Encoder-decoder models for classifying text data on social disorders in children and adolescents (2024)

Abstract: This paper describes our participation in Task 3 and Task 5 of the #SMM4H (Social Media Mining for Health) 2024 Workshop, explicitly targeting the classification challenges within tweet data. Task 3 is a multi-class classification task centered on tweets discussing the impact of outdoor environments on symptoms of social anxiety. Task 5 involves a binary classification task focusing on tweets reporting medical disorders in children. We applied transfer learning from pre-trained encoder-decoder models such as BART-base and T5-small to identify the labels of a set of given tweets. We also presented some data augmentation methods to see their impact on the model performance. Finally, the systems obtained the best F1 score of 0.627 in Task 3 and the best F1 score of 0.841 in Task 5

## introduction social disorders are significantly influencing a large proportion of young people globally. social anxiety disorder (sad) typically emerges during early adolescence and is characterized by excessive anxiety in social situations @xcite . although spending time outdoors in green or blue environments has been shown to alleviate symptoms of various anxiety disorders, limited research has explored its impact specifically on sad. meanwhile, numerous children receive diagnoses of conditions that can significantly affect their daily functioning and persist into adulthood. the commonly diagnosed childhood disorders are attention-deficit/hyperactivity disorder (adhd) @xcite , autism spectrum disorders (asd) @xcite , speech delay, and asthma. datasets related to these disorders are usually extracted from tweets or user posts on social platforms such as twitter and reddit. users describe their disorders daily and receive feedback from the community or comment on what they have passed. as a text classification problem, the most advanced and popular methods of social disorder identification use deep learning networks such as bert @xcite , roberta @xcite , transformer models @xcite , and their variants. in participating in the #smm4h 2024 workshop @xcite , we apply transfer learning from two pre-trained models (bart-base @xcite and t5-small @xcite ), which follow the architecture of transformer and sequence-to-sequence. additionally, we exploited two data augmentation methods -(1) false inferred data and (2) paraphrased data extracted from chatgpt -to supplement the training set and see their impact on model performance. because the organizers hide the final ranking table, we can only present our results compared to the mean and median values by metrics (f1, precision, recall, and accuracy) they provided. ## methodology after conducting several initial experiments on training the dataset with and without preprocessing steps, we observed that training without any preprocessing yielded better performance. it is assumed that every token within the data can positively impact the model's performance. therefore, we fed raw data directly into the model in the training process. we applied transfer learning from two pretrained models based on the architecture of the transformer and sequence-to-sequence: bartbase @xcite and t5-small @xcite for task 3 and task 5, respectively. they are encoder-decoder language models, producing outputs that can be unknown labels. from a given text, the models must detect its label, and any out-of-scope label will be set automatically to a default label, which we choose as "0" for both tasks. the best models were saved based on their 1 performance on the validation set by f1-macro for task 3 and the f1 score on the positive class for task 5. two data augmentation methods complement new data to improve mode performance. first, the false inferred data in the validation set were utilized when inferring them in a trained model with the default training data. second, paraphrased data by chatgpt is taken from the training and validation sets at an insignificant cost. ## conclusion this paper introduced our approach, utilizing pretrained encoder-decoder models with two data augmentation methods to address task 3 and task 5 of the #smm4h 2024 workshop. our findings underscore the advantages of encoder-decoder models in text classification problems when they offer a strong baseline performance. furthermore, it is beneficial to exploit data augmentation methods to enhance the model's performance by comple-menting paraphrased texts from chatgpt. in the experiments, we achieved the highest f1 score of 0.627 for task 3 and 0.87 for task 5. in the future, we will investigate further how large language models' outputs like chatgpt can positively impact downstream classification tasks' performance.