Innovators @ SMM 4 H ’22: An Ensembles Approach for self-reporting of COVID -19 Vaccination Status Tweets (2022)

Abstract: With the Surge in COVID-19, the number of social media postings related to the vaccine has grown, specifically tracing the confirmed reports by the users regarding the COVID-19 vaccine dose termed “Vaccine Surveillance.” To mitigate this research problem, we present our novel ensembled approach for self-reporting COVID-19 vaccination status tweets into two labels, namely “Vaccine Chatter” and “Self Report.” We utilize state-of-the-art models, namely BERT, RoBERTa, and XLNet. Our model provides promising results with 0.77, 0.93, and 0.66 as precision, recall, and F1-score (respectively), comparable to the corresponding median scores of 0.77, 0.9, and 0.68 (respec- tively). The model gave an overall accuracy of 93.43. We also present an empirical analysis of the results to present how well the tweet was able to classify and report. We release our code base here https://github.com/Zohair0209/SMM4H-2022-Task6.git

## introduction and motivation covid-19 pandemic has unprecedented challenges and also advances to humanity. it provides the scientific community to advance science with wider access to openly available data @xcite . vaccine surveillance became a pressing research issue with the widespread roll-out of covid-19 vaccines. social media platforms such as twitter and facebook contain abundant text data that can be utilized for research purposes @xcite . in this paper, we describe our submission for the shared task foot_0 , for vaccine surveillance which is a very pressing issue. we aim to target people who report adverse events via their healthcare providers to systems like vaccine adverse event reporting system (vaers) @xcite , or are found documented in their electronic health record (ehr), a more robust and convenient method could be devised using self-reports from social media. in this task, we experiment with a dataset of twitter users personally reporting vaccination status and users discussing vaccination status but not revealing their own. this task is tricky because users discuss the vaccination status of others or from news reports in similar ways than they discuss their own at a higher rate (1 to 8 on average). the dataset presents as the positive class, unambiguous tweets of users clearly stating that they have been vaccinated. all other tweets are of users discussing vaccination status. this task involves the identification of self-reported covid-19 vaccination status in english tweets. as a two-way classification task, the two classes in the training dataset correspond to vaccination confirmation and vaccine-related chatter. in this task, there is a class imbalance of roughly 1 to 8, implying that 1,496 tweets correspond to vaccination confirmation, and 12,197 tweets correspond to vaccine chatter. ## conclusion in this paper, we present our system paper submission for innovators @ smm4h'22. we aim to classify user tweets, indicating whether they represent self reports for the covid-19 vaccines or are a part of general vaccine chatter. the proposed system is an ensembled voting model with fine-tuned bert, roberta, and xlnet. given tweets in english, the submitted model classifies each vaccinerelated tweet into one of the two labels: "vaccine chatter" and "self report." the system performs quite well to accomplish the desired task with an accuracy of 93.43%. in the future, we intend to work on a multitask learning framework to handle social media postings related to other medical advancements, apart from the covid-19 vaccines. we also aim to develop models for multi-lingual postings featuring similar scenarios.