LHS 712 NV at # SMM 4 H 2024 Task 4: Using BERT to classify R eddit posts on non-medical substance use (2024)

Abstract: This paper summarizes our participation in the Shared Task 4 of #SMM4H 2024. Task 4 was a named entity recognition (NER) task identifying clinical and social impacts of non-medical substance use in English Reddit posts. We employed the Bidirectional Encoder Representations from Transformers (BERT) model to complete this task. Our team achieved an F1-score of 0.892 on a validation set and a relaxed F1-score of 0.191 on the test set.

## introduction substance use, whether prescription or illicit, poses a significant public health challenge, contributing to addiction, overdose, and various associated health concerns @xcite . understanding the clinical and social ramifications of nonmedical substance use is crucial for enhancing the treatment of substance use disorder, informing intervention strategies, and developing preventive measures @xcite . this paper addresses a named entity recognition (ner) task focused on identifying two key entity types: clinical and social impacts. clinical impacts refer to the effects of substance use on individuals' health and well-being, while social impacts encompass broader societal consequences @xcite . the social media mining for health (#smm4h) shared tasks aim to leverage natural language processing (nlp) techniques to extract valuable health insights from social media data. in #smm4h 2024, we participated in task 4 on extraction of the clinical and social impacts of non-medical substance use from reddit posts. we were particularly motivated to participate in this task because of the pressing public health concern surrounding nonmedical substance use. understanding the clinical and social impacts of such usage is crucial for developing effective interventions and prevention strategies. by leveraging advanced, deep-learning based nlp models, we aimed to contribute to this understanding and advance the field of health informatics through innovative data analysis techniques. ## conclusion this paper discusses our submission for #smm4h 2024 task 4 on named entity recognition (ner) for identifying clinical and social impacts of substance use in reddit posts using large language models (llms), particularly the bert model. while the bert model demonstrated high overall performance, there were notable disparities in its efficacy across different metrics and datasets. challenges remain in accurately identifying social impact instances, highlighting the need for further refinement. despite these challenges, the bert model shows promise in automating the detection of clinical impacts in reddit posts. to facilitate continued development, we have shared the implementation code for our system participation on github. 1