Overview of the Fifth Social Media Mining for Health Applications (# SMM 4 H ) Shared Tasks at COLING 2020 (2020)

Abstract: The vast amount of data on social media presents significant opportunities and challenges for utilizing it as a resource for health informatics. The fifth iteration of the Social Media Mining for Health Applications (#SMM4H) shared tasks sought to advance the use of Twitter data (tweets) for pharmacovigilance, toxicovigilance, and epidemiology of birth defects. In addition to re-runs of three tasks, #SMM4H 2020 included new tasks for detecting adverse effects of medications in French and Russian tweets, characterizing chatter related to prescription medication abuse, and detecting self reports of birth defect pregnancy outcomes. The five tasks required methods for binary classification, multi-class classification, and named entity recognition (NER). With 29 teams and a total of 130 system submissions, participation in the #SMM4H shared tasks continues to grow.

## introduction the aim of the social media mining for health applications (#smm4h) shared tasks is to take a community-driven approach to addressing natural language processing (nlp) challenges of utilizing social media data for health informatics, including informal, colloquial expressions of clinical concepts, noise, data sparsity, ambiguity, and multilingual posts. the fifth iteration of the #smm4h shared tasks consisted of five tasks involving mining health-related information from twitter data (tweets): automatic classification of tweets that mention medications (task 1), automatic classification of multilingual tweets that report adverse effects of a medication (task 2), with sub-tasks for distinct sets of tweets posted in english (task 2a), french (task 2b), and russian (task 2c), automatic extraction and normalization of adverse effects in english tweets (task 3), automatic characterization of chatter related to prescription medication abuse in tweets (task 4), and automatic classification of tweets self-reporting a birth defect pregnancy outcome (task 5). teams could register for one or multiple tasks. in total, 57 teams registered for at least one task. to develop their systems, teams were provided with annotated training and validation sets of tweets for each task. for the final evaluation, teams were provided with an unlabeled test set for each task, and were allowed up to four days to submit the predictions of their systems to codalab foot_0 -a platform that facilitates data science competitions. each team was allowed to submit up to three sets of predictions per task. in total, 29 of the 57 registered teams submitted at least one set of predictions. more specifically, 16 teams participated in task 1 (40 submissions), 17 teams in task 2a (35 submissions), 5 teams in task 2b (7 submissions), 7 teams in task 2c (14 submissions), 7 teams in task 3 (15 submissions), 3 teams in task 4 (9 submissions), and 4 teams in task 5 (10 submissions). in section 2, we will briefly describe the tasks. in section 3, we will present the performance and a brief summary of each team's best-performing system for each task. appendix a provides the system description papers corresponding to the team numbers used in section 3. ## conclusion this paper presented an overview of the #smm4h 2020 shared tasks. with 29 teams and a total of 130 system submissions, participation in the #smm4h shared tasks continues to grow. all of the teams with the best-performing system for each task used deep learning-based systems, most of which were transformer-based architectures. the system description papers that are cited in appendix a were each peer-reviewed by two reviewers and provide further details about 26 teams' systems.