HITSZ - ICRC : A Report for SMM 4 H Shared Task 2020-Automatic Classification of Medications and Adverse Effect in Tweets (2020)

Abstract: This is the system description of the Harbin Institute of Technology Shenzhen (HITSZ) team for the first and second subtasks of the fifth Social Media Mining for Health Applications (SMM4H) shared task in 2020. The first task is automatic classification of tweets that mention medications and the second task is automatic classification of tweets in English that report adverse effects. The system we propose for these tasks is based on bidirectional encoder representations from transformers (BERT) incorporating with knowledge graph and retrieving evidence from online information. Our system achieves an F1 of 0.7553 in task 1 and an F1 of 0.5455 in task 2.

## introduction with greatly increasing number of social media users, plenty of personal information has been posted in social networking websites such as twitter, including massive data about health and medicine. social media has attracted more and more attention from researchers and industrials in the health and medicine domain. to utilize the imbalanced and informal expressions of clinical concepts, a system should be able to alleviate the noise of social media posts and detect medical mentions which requires professional knowledge. in 2020, the health language processing lab at university of pennsylvania organized the social media mining for health applications (smm4h) shared task about using social media mining for health monitoring, which includes 5 subtasks. we participate in the first two subtasks: (1) automatic classification of tweets that mention medications, and (2) automatic classification of tweets in english that report adverse effects (aes). in this report, we briefly introduce our system developed for the two subtasks. the system is based on pre-trained language model bert @xcite with external knowledge. ## data and pre-processing the 2020 smm4h organizers provide a dataset of tweets with a label indicating whether the tweets mention medications for task1 (1-yes, 0-no), and a dataset of tweets with a label indicating whether the tweets mention aes for task2. each tweet includes text, a date and an identified id. ## system description our system for both task 1 and task 2 is based on bert, which is used to represent tweet. below we describe in detail the methods for the two tasks one by one. ## experiments for task1, we set batch size to 64, learning rate to 2e-5 and sequence length to 64 when training models. for task2, the batch size and sequence length of bert are the same as the ones of task1 while the learning rates of bert and attention layer are set to 1e-5 and 5e-5 respectively. we train all models using 5-fold cross validation on the training set. 2 , where na denotes that we do not submit results as internet problem, bert_merged is the model that merges results of bert_strategy1 and bert_strategy2 by voting. our system achieves the highest f-score of 0.7553 on task1 and 0.5455 on task2. for task 1, on the training set, bert_med achieves much higher recall, but much lower precision than bert. for task2, bert_strategy1 and bert_strategy2 perform roughly the same on the training and test sets. bert_merged performs slightly worse than them on the training set, but a little higher on the test set. the difference between performances of our system on the training set and test set is very large.