P ing A n T ech at SMM 4 H task1: Multiple pre-trained model approaches for Adverse Drug Reactions (2022)

Abstract: This paper describes the solution for the Social Media Mining for Health (SMM4H) 2022 Shared Task. We participated in Task1a., Task1b. and Task1c. To solve the problem of the presence of Twitter data, we used a pre-trained language model. We used training strategies that involved: adversarial training, head layer weighted fusion, etc., to improve the performance of the model. The experimental results show the effectiveness of our designed system. For task 1a, the system achieved an F1 score of 0.68; for task 1b Overlapping F1 score of 0.65 and a Strict F1 score of 0.49. Task 1c yields Overlapping F1 and Strict F1 scores of 0.36 and 0.30, respectively.

## introduction mining adverse drug events (ades) from social media is one of the most researched topics in the field of social media pharmacovigilance. to promote the research on this topic, the health language processing lab of the university of pennsylvania organized social media mining for health applications (smm4h) shared tasks. this year, the smm4h shared tasks included ten subtasks @xcite . our team focused on three subtasks in task 1, which are (1) classifying tweets reporting ades (adverse drug events); (2) detect ade spans in the tweets; (3) map these colloquial mentions to their standard concept ids in the meddra vocabulary. the main challenges of this task are as follows: (1) how to handle unbalanced data and (2) the presence of a large amount of noise in the data, e.g., emojis, redundant punctuation, desensitized usernames, some link addresses, etc. in addition, medical expressions in the text are often expressed in non-professional colloquial expressions, which are common problems in social media data and usually mislead the trained model. to address these issues, we use pre-trained language models as the basis; many text pre-processing and adversarial training are described in detail in the following sections, with corresponding experimental results. ## task 1a: classify tweets reporting ades this subtask aims to identify whether sentences contain adverse drug reactions, which can be modeled as a classification task. there are a total of 17385 samples in this dataset, where the positive to negative ratio is 1:11 (mentions of ades are labeled as 0), a total of 915 samples in the validation set (87 ade labels and 828 noade labels), and 10,984 samples in the test set @xcite . ## task 1b: ade span detection this subtask task extracts the location of ade entities from the text and can be considered a sequence annotation task. data distribution is the same as task 1a. ## task 1c: normalization the task is to extract the ade keywords in tweet data and match these ade keywords to the correct standard ade terms. there are two difficulties in this task. first, there are only 1000+ tweet data containing ade keywords, which constrains us not from using overly complex models. secondly, there are more than 30,000 standard ade terms, and most of the standard ade terms are not match the tweet data. ## conclusion in this paper, we use task-specific methods for each of the three subtasks of classification, extraction, and normalization. we enhance the effectiveness of our model through various strategies and demonstrate the effectiveness of the model. in future work, we will specifically target pre-training tasks in the negative drug effect vertical, such as how to make the model acquire prior knowledge of drugs. .