H ale L ab_ NITK @ SMM 4 H ’22: Adaptive Learning Model for Effective Detection, Extraction and Normalization of Adverse Drug Events from Social Media Data (2022)

Abstract: This paper describes the techniques designed for detecting, extracting and normalizing adverse events from social data as part of the submission for the Shared task, Task 1-SMM4H’22. We present an adaptive learner mechanism for the foundation model to identify Adverse Drug Event (ADE) tweets. For the detected ADE tweets, a pipeline consisting of a pre-trained question-answering model followed by a fuzzy matching algorithm was leveraged for the span extraction and normalization tasks. The proposed method performed well at detecting ADE tweets, scoring an above-average F1 of 0.567 and 0.172 overlapping F1 for ADE normalization. The model’s performance for the ADE extraction task was lower, with an overlapping F1 of 0.435.

## introduction social media data is extensively used for a wide variety of informed decision-making applications in varied domains like e-business, healthcare, policy making etc @xcite @xcite . analyzing drug post-marketing management policies is a major aspect of pharmacovigilance @xcite and has received significant research attention in recent years. as part of the shared task, participants were provided with twitter data @xcite that includes ade mentions for detection, extraction and normalization purposes to aid pharmacovigilance studies. while detecting adverse reactions from social data having informal language is a complex task, the availability of valid ade samples is scarce, making the data highly imbalanced @xcite @xcite . each of the tasks are correlated, thus, it is necessary to ensure that authentic ade tweet samples are detected before utilizing them for ade extraction and normalization processes. with these objectives, our work encompasses the design of an adaptive learning mechanism for the foundation model for effective ade identification, extraction and normalization. ## methodology extracting ades from informal social data is challenging due to the short text length and style-variant nature @xcite . as stated earlier, we focused on developing an efficient adaptive learning method for foundation models due to the association between the three subtasks. we adopted roberta @xcite as the foundation model for applying adaptive learning techniques. the empirical and theoretical aspects of the data-specific features learnt by the higher-order layers were explored, for understanding the contribution of different layers in a foundation model and relearning task-specific requirements. during knowledge distillation, it is essential to ensure that the pre-trained weights are not affected by learning divergence, due to absence of previously learned weights. with this in mind, we employed mixout @xcite with the weight reinitialization technique @xcite , which helps adaptively retrain the last few layers. while mixout helps retain the weights from the base foundation model without losing the pre-trained features, weight reinitialization helps relearn these specific weights for the higher order layers to prethe span extraction task was approached as a question answering (qa) task, rather than as a conventional named entity recognition (ner) task @xcite . to extract the side effects or adverse medical events from the adaptive learner output, roberta-base @xcite fine-tuned on the squad2.0 dataset @xcite was trained for a batch size of 96 for two epochs, a learning rate of 3e-5, handling a maximum sequence length of 386 and query length of 96. since all the spans here represent ades on individuals, we considered a fixed query ("what are the side effects?") to extract spans from ade tweets. due to fewer number of ade tweets with varied meddra codes 1 @xcite (no duplicates), formulating it as a supervised classification problem is practically unfeasible. in view of this, the conventional fuzzy matching algorithm built on levenshtein distance @xcite based similarity measure was used for mapping the extracted spans to meddra codes. we created two dictionaries: meddra terms and their codes (preferred terms -pt) from the meddra database and the dictionary from the gold standard train data with spans and meddra codes. the spans extracted from the qa model were matched against the two dictionary terms, and the meddra code for one that weighted more was used as the label for the ade spans. ## experiments and observation the smm4h'22 task organizers provided data @xcite for this experimentation. we observed that the data was highly imbalanced along with a unique 1 https://www.meddra.org/ set of ade events, thus increasing the learning complexity of a system. for ade detection, experimentation on weight retraining varied between 1 to 4 layers and on mixout from 0.5 to 0.7 were performed. empirically, we chose to reinitialize the top 4 layers by fixing the mixout to 0.7. while approaching the entity recognition task as qa, we found that it returned context chunks comprising adverse events rather than short entities. since the meddra database holds proper clinical expressions, we believe that including the gold standard data covering the colloquial mentions of adverse effects as another dictionary helps improve the span normalization. ## conclusion for addressing the smm4h'22 shared task 1, we concentrated on developing an adaptive learning technique for the foundation model using roberta base for ade detection. while achieving a promising f1 measure of 0.72 on the validation set, the same model on the test set produced an aboveaverage f1 of 0.567. we approached ade span extraction as a qa task that showed degraded results due to the return of contextual chunks rather than word entities. fuzzy matching for span normalization gave above-average overlapping p, r and f1 scores. in future, we plan to fine-tune the qa model to return adverse word level entities for improved performance in the span prediction task.