Overview of the Sixth Social Media Mining for Health Applications (# SMM 4 H ) Shared Tasks at NAACL 2021 (2021)

Abstract: The global growth of social media usage over the past decade has opened research avenues for mining health related information that can ultimately be used to improve public health. The Social Media Mining for Health Applications (#SMM4H) shared tasks in its sixth iteration sought to advance the use of social media texts such as Twitter for pharmacovigilance, disease tracking and patient centered outcomes. #SMM4H 2021 hosted a total of eight tasks that included reruns of adverse drug effect extraction in English and Russian and newer tasks such as detecting medication non-adherence from Twitter and WebMD forum, detecting self-reported adverse pregnancy outcomes, detecting cases and symptoms of COVID-19, identifying occupations mentioned in Spanish by Twitter users, and detecting self-reported breast cancer diagnosis. The eight tasks included a total of 12 individual subtasks spanning three languages requiring methods for binary classification, multi-class classification, named entity recognition and entity normalization. With a total of 97 registering teams and 40 teams submitting predictions, the interest in the shared tasks grew by 70% and participation grew by 38% compared to the previous iteration.

## introduction the social media mining for health (#smm4h) shared tasks aim to foster community participation in tackling natural language processing (nlp) challenges in social media texts for health applications. the tasks hosted annually attract newer methods for extraction of meaningful health related infor-mation from noisy social media sources such as twitter and webmd where the information of interest is often sparse and noisy. the nlp methods required for the eight tasks spanned the categories of text classification, named entity recognition and entity normalization. systems developed for the tasks often require the use of nlp techniques such as noise removal, class weighting, undersampling, oversampling, multi-task learning, transfer learning and semi-supervised learning to improve over traditional methods. the sixth iteration of #smm4h hosted eight tasks with a total of twelve individual subtasks. similar to previous years, the most tasks centered around pharmacovigilance (i.e. ade extraction, medication adherence) and patient centered outcomes (i.e. adverse pregnancy outcomes, breast cancer diagnosis). this year the shared tasks featured the addition of covid-19 related tasks such as detection of self reported cases of covid-19 and symptoms of covid-19, as well as extraction of professions and occupations for the purposes of risk analysis. the individual tasks are listed below: teams interested in participating were allowed to register for one or more tasks/subtasks. on successful registration, teams were provided with annotated training and validation sets of tweets for each task. in total, 97 teams registered for one or more tasks. the annotated datasets contained examples of input text and output labels which the participants could use to train their methods. during the final evaluation period which lasted four days for each task, teams were provided with a evaluation datasets which contained only the input texts. participants were required to submit label predictions for the input texts which would be evaluated against the annotated labels. the submissions were facilitated through codalab foot_0 and participants were allowed to make up to two prediction submissions for each of the subtasks. of the 97 registered teams, 40 teams submitted one or more predictions towards the shared tasks. the remainder of the document is as follows, in section 2, we briefly describe the individual task objectives and research challenges associated with them. in section 3, we present the evaluation results and a brief summary of each team's bestperforming system for each subtask. appendix a provides the system description papers corresponding to the team numbers. ## conclusion this paper presents an overview of the sixth smm4h shared tasks held in 2021. the shared tasks hosted a total of eight tasks with 12 individual tasks in total. with 40 teams participating in the shared tasks, we find that interest in tasks grew by 38% from the previous year. analyzing the methods in the submitted systems, we find that the best systems used transformer based models such as bert and roberta with various techniques for addressing class imbalance. details of individual systems are available as system description papers cited in appendix a. appendix a. team numbers and system description papers