CIC NLP at SMM 4 H 2022: a BERT -based approach for classification of social media forum posts (2022)

Abstract: This paper describes our submissions for the Social Media Mining for Health (SMM4H) 2022 shared tasks. We participated in 2 tasks: a) Task 4: Classification of Tweets self-reporting exact age and b) Task 9: Classification of Reddit posts self-reporting exact age. We evaluated the two( BERT and RoBERTa) transformer based models for both tasks. For Task 4 RoBERTa-Large achieved an F1 score of 0.846 on the test set and BERT-Large achieved an F1 score of 0.865 on the test set for Task 9.

## introduction social media platforms have become more integrated in this digital era, and have impacted various people's perceptions of networking and socializing. the social media mining for health applications (smm4h) shared task involves natural language processing (nlp) challenges of using social media data for health research, including informal, colloquial expressions and misspellings of clinical concepts, noise, data sparsity, ambiguity, and multilingual posts @xcite . as computational analysis opens up new opportunities for researching complex topics using social media data, models are being developed to automatically detect demographic information such as users' age @xcite , language @xcite (aroyehun and gelbukh, 2020), gender @xcite , medical history @xcite , and so on. more people are using social media in various ways to interact with others, share information, and express their own thoughts. social media platforms like twitter and reddit have cutting-edge technology and are rich with raw, unprocessed data that can be analyzed and transformed into meaningful information. social media research in different fields, including health (aroyehun and gelbukh, 2019), politics (mckeon and gitomer, 2019), economics @xcite , have included demographic variables. we participated in the social media mining task 4 of the smm4h 2022 shared task which centered on automatically identifying tweets that self-report the user's exact age from those that do not. in task 9 of the same challenge, we also attempted to automatically classify reddit posts that self-report the actual age of the online user at the time of posting from those that do not. a detailed overview of the shared tasks in the 7th edition of the workshop can be found in @xcite . we applied two (bert-large and roberta-large) transformer models using hugging face foot_0 library. the paper is organized as follows: section 2 describes task 4 objective, system description, experiment and result. section 3 describes task 9 objective, system description, experiment and result. finally, section 4 concludes the paper and sheds some light on possible future work. ## conclusion in this work, we describe our team submission for social media mining for health applications shared task 2022. we have explored an application of roberta and bert language models to the task of classification of tweets self-reporting exact age and classification of reddit posts selfreporting exact age. our experiments have shown that roberta-large outperforms bert-large in classification of tweets self-reporting exact age (task 4) in both validation and test set. for classification of reddit posts self-reporting exact age (task 9), we found that roberta-large outperforms bert-large in validation set. in the future, we will explore the effect of class imbalance on the performance of classification models, apply different methods to solve class imbalance and their effect on model performance.