ISL ab System for SMM 4 H Shared Task 2020 (2020)

Abstract: In this paper, we described our systems for the first and second subtasks of Social Media Mining for Health Applications (SMM4H) shared task in 2020. The two subtasks are automatic classi-fication of medication mentions and adverse effect in tweets. Our systems for both subtasks are based on Robustly optimized BERT approach (RoBERTa) and our previous work at SMM4H’19. The best F1-scores achieved by our systems for subtask 1 and 2 were 0.7974 and 0.64 respec-tively, which outperformed the average F1-scores among all teams’ best runs by at least 0.13.

## introduction nowadays, social media are often being used by general public to create and share public messages related to their health. with the global increase in social media usage, there is a trend of posting information related to adverse drug reactions (adr). mining social media data for this type of information is helpful for pharmacological post-marketing surveillance and monitoring. in order to facilitate the use of social media for health monitoring and surveillance, we participated in the social media mining for health applications (smm4h) shared task to develop systems that can automatically identify tweets conveying medications and adverse effects. ## methods for each task, we developed three systems; the first and the second was based on robustly optimized bert pretraining approach (roberta) (y. @xcite and the third was based on the method proposed in our previous work @xcite ; @xcite . ## results ## discussion for both subtasks, the distribution of binary class is highly imbalanced. when we used the traditional machine learning methods like svm, even we have tried to deal with the data imbalance problem by rus_wesmote, the performance is still far below that of the system with the pre-trained roberta. on the other hand, for the roberta based systems, we didn't apply any imbalance techniques but they still get compatible and even better precision, recall and f-score. it is surprising to see that the system based on the tweet-retrained roberta model didn't outperform the original pre-trained roberta model. ## conclusion we demonstrated that the system based on the pre-trained roberta model outperformed traditional svm-based method and the re-trained roberta model. we will conduct error analysis to interpret the results of the two roberta-based models to figure out the reason why the performance of the re-trained roberta model get worse in the future.