AIR - JPMC @ SMM 4 H ’22: BERT + Ensembling = Too Cool: Using Multiple BERT Models Together for Various COVID -19 Tweet Identification Tasks (2022)

Abstract: This paper presents my submission for Tasks 1 and 2 for the Social Media Mining of Health (SMM4H) 2022 Shared Tasks competition. I first describe the background behind each of these tasks, followed by the descriptions of the various subtasks of Tasks 1 and 2, then present the methodology. Through model ensembling, this methodology was able to achieve higher results than the mean and median of the competition for the classification tasks.

## introduction social media has grown to be an essential communication platform. massive platforms like facebook have billions of users @xcite , enabling it to gauge public sentiment about a plethora of topics, two of them being pharmacovigilance and health mandates. pharmacovigilance is the assessment of adverse effects from medical drugs @xcite . in the past, pharmacovigilance information were gathered from marketing clinical trials, or postmarketing reporting by physicians, which pose limitations on information quantity. @xcite . platforms like twitter are also a key tool to gauge sentiment about public events, such as health mandates during a public health crisis. understanding public sentiment is pertinent to get everyone on the same page. the issue with twitter data, however, is its lack of uniformity, making it hard to automate sentiment classification. this is where natural language processing (nlp) and this competition comes into play. hosting for the seventh time @xcite , this shared tasks competition, like in previous years, tackles many social media data related tasks, including pharmacovigilance @xcite . this system builds upon state-of-the-art models to further improve results for classification tasks. ## results however, task 1b (span prediction) performed at or above the mean in overlapping metrics while worse in strict metrics. this concurs with the submitted system, as it only takes the start index of the first word and last index of the last word in the predicted token list to build the predicted span, meaning that any gaps in between the predicted tokens are included in the final span. this was done to account for multiple spans in the same tweet in the training data. for task 1c (entity normalization), the first-term approach, as expected, performed worse than the mean metrics across the board. ## conclusion as shown in the results section, better results were achieved than the baseline through model ensembling. this resulted in our system performing better than the mean and median metrics of the competition for all of the classification tasks and on-par with the overlapping metrics for task 1b. further work can be done to improve this system. for the classification tasks, pre-processing of the dataset, such as removing extraneous punctuation or hashtags, may generate better results. for task 1b, keeping the best epoch based on f1-score in addition to getting rid of the gaps in the span prediction may help especially in the strict metrics. lastly, having the opportunity to conduct entity normalization properly may help improve task 1c metrics as well.