BOUN - TABI @ SMM 4 H ’22: Text-to-Text Adverse Drug Event Extraction with Data Balancing and Prompting (2022)

Abstract: This paper describes models developed for the Social Media Mining for Health 2022 Shared Task. We participated in two subtasks: classification of English tweets reporting adverse drug events (ADE) (Task 1a) and extraction of ADE spans in such tweets (Task 1b). We developed two separate systems based on the T5 model, viewing these tasks as sequence-to-sequence problems. To address the class imbalance, we made use of data balancing via over- and undersampling on both tasks. For the ADE extraction task, we explored prompting to further benefit from the T5 model and its formulation. Additionally, we built an ensemble model, utilizing both balanced and prompted models. The proposed models outperformed the current state-of-the-art, with an F1 score of 0.655 on ADE classification and a Partial F1 score of 0.527 on ADE extraction.

## introduction this paper describes the models developed for the classification and extraction of adverse drug event (ade) mentions in english tweets (tasks 1a and 1b) in the social media mining for health (smm4h) 2022 shared task @xcite . we viewed the ade classification and extraction tasks as sequence-to-sequence problems and fine-tuned the text-to-text transfer transformer (t5) model @xcite , which is a pretrained model where the input and the output are both represented as text rather than class labels or spans. observing a significant class imbalance in the training data, we made use of over-and undersampling to balance out positive and negative class instances. after this operation, with an equal number of positive and negative instances (1:1 ratio), we obtained an 8% improvement in f1 score for ade classification, and a 2.2% improvement in * equal contribution: order determined by a coin flip. ## conclusion in this study, we applied the t5 model and its problem formulation (i.e. viewing tasks as sequenceto-sequence problems) to ade classification and extraction tasks. we used over-and undersampling to address class imbalance, a major challenge in ade extraction from social media. in addition, we explored prompting methods to further benefit from the t5 language model and its sequence-tosequence formulation. our official submission for the ade classification task made use of data balancing via over-and undersampling, and achieved an f1 score of 0.655, outperforming the state-ofthe-art. for the ade extraction task, we ensembled our balanced and prompted models to increase generalization for both partial and exact matches. our submission of the ensemble model achieved a partial f1 score of 0.527, beating the current stateof-the art and performing at par with the mean of all submissions. in the future, we intend to experiment with other methods of dealing with imbal-anced data, such as data augmentation. the ade classification and extraction tasks is also planned to be used in a multi-task learning scenario with the t5 model.