BERT based Transformers lead the way in Extraction of Health Information from Social Media (2021)

Abstract: This paper describes our submissions for the Social Media Mining for Health (SMM4H) 2021 shared tasks. We participated in 2 tasks: (1) Classification, extraction and normalization of adverse drug effect (ADE) mentions in English tweets (Task-1) and (2) Classification of COVID-19 tweets containing symptoms (Task-6). Our approach for the first task uses the language representation model RoBERTa with a binary classification head. For the second task, we use BERTweet, based on RoBERTa. Fine-tuning is performed on the pre-trained models for both tasks. The models are placed on top of a custom domain-specific pre-processing pipeline. Our system ranked first among all the submissions for subtask-1(a) with an F1-score of 61%. For subtask-1(b), our system obtained an F1-score of 50% with improvements up to +8% F1 over the median score across all submissions. The BERTweet model achieved an F1 score of 94% on SMM4H 2021 Task-6.

## introduction social media platforms are a feature of everyday life for a large proportion of the population with an estimated 4.2 billion people using some form of social media @xcite . twitter is one of the largest social media platforms with 192 million daily active users @xcite . the 6th social media mining for health applications workshop focuses on the use of natural language processing (nlp) for a wide number of tasks related to health informatics using data extracted from twitter . our team, tensorflu, participated in 2 tasks, (1) task-1: classification, extraction and normalization of adverse effect (ae) mentions in english tweets and (2) task-6: classification of covid-19 tweets containing symptoms. a detailed overview of the shared tasks in the 6th edition of the workshop can be found in @xcite . the classification and extraction of adverse drug effects (ade) on social media can be a useful indicator to judge the efficacy of medications and drugs while ensuring that any side effects that previously remained unknown can be found. thus social media can be a useful medium to judge gauge patient satisfaction and well being. according to the report in @xcite , 15% of american adults get their news on twitter while 59% of twitter users get their news on twitter itself. thus during the spread of a pandemic like covid-19, tracking reports by users as well as news mentions from local organizations can perform the function of tracking the spread of the disease in new regions and keep people informed. similar to the last edition of the workshop, the top performing model @xcite for task-1 with the highest score this year was roberta @xcite . the biggest challenge while dealing with the dataset provided for this years competition was the huge class imbalance. the proposed approach handles this by the use of random sampling (abd elrahman and abraham, 2013) of the dataset during finetuning. named entity recognition (ner) for the extraction of text spans was performed using the roberta based model provided in the spacy @xcite en_core_web_trf pipeline. for the classification of tweets with covid-19 symptoms, we used a model called bertweet @xcite trained on 845 million english tweets and 23 million covid-19 related english tweets as of the latest publically available version of the model. finetuning was performed on the pretrained models for both tasks. section 2 summarizes the methodology and results obtained for task-1, while section 3 summarizes the methodology and results for task-6. ## task-6: classification of covid-19 tweets containing symptoms the goal of this task is to classify tweets into 3 categories: (1) self-reports (2) non-personal reports (3) literature/news mentions. ## acknowledgements for task 1a, sidharth r, abhiraj tiwari, parthivi choubey and saisha kashyap contributed equally to the work. for task 1b, sidharth r, abhiraj tiwari, sahil khose, and kumud lakara contributed equally to the work. all authors contributed equally to the work done for task 6. we would also like to thank mars rover manipal and project manas for providing the necessary resources to train our models.