Extracting a Knowledge Base of COVID -19 Events from Social Media (2022)

Abstract: We present a manually annotated corpus of 10,000 tweets containing public reports of five COVID-19 events, including positive and negative tests, deaths, denied access to testing, claimed cures and preventions. We designed slot-filling questions for each event type and annotated a total of 28 fine-grained slots, such as the location of events, recent travel, and close contacts. We show that our corpus can support fine-tuning BERT-based classifiers to automatically extract publicly reported events, which can be further collected for building a knowledge base. Our knowledge base is constructed over Twitter data covering two years and currently covers over 4.2M events. It can answer complex queries with high precision, such as “Which organizations have employees that tested positive in Philadelphia?” We believe our proposed methodology could be quickly applied to develop knowledge bases for new domains in response to an emerging crisis, including natural disasters or future disease outbreaks.

## introduction since december 2019, the novel coronavirus rapidly spread across the world, and consequently, a flood of covid-19 related information has appeared on social media. this includes reports on public figures who have tested positive/negative for the virus, which often break first on twitter, such as bill gates's announcement as shown in figure 1 . besides public figures, individual users and organizations on twitter also report covid-19 events around the world. for example in january 2021, many sources in different countries reported an increasing number of new cases exported from the uk (figure 2 ). being able to gather this information can potentially help experts and the general 1 our corpus (with user-information removed), automatic extraction models, and the corresponding knowledge base are publicly available at https://github.com/viczong /extract_covid19_events_from_twitter . public to quickly identify issues and assess the situation near real-time, complementing officially reported data which may take longer to obtain, and does not include information at the same level of granularity as that reported in natural language on news and social media. in this paper, we present an empirical study on the extraction of large quantities of structured knowledge related to an ongoing pandemic from twitter. to achieve this, we construct a corpus of 10,000 tweets with rich linguistic annotations, covering five event types: positive tests, negative tests, denied access to testing, deaths, claimed methods of cure and prevention. more specifically, we annotate fine-grained semantic information for each event type by designing slot-filling questions and asking annotators to highlight text spans as answers. we show that our corpus can support training bert-based classifiers to extract structured information automatically from twitter. while slot f1 scores vary from 0.3 to 0.9 in individual tweets (most f1 scores are greater than 0.5), we show it is possible to achieve very high accuracy by aggregating extractions over a large corpus, exploiting redundancy of information that arises when events are widely discussed on twitter. although many twitter datasets have emerged after the covid-19 outbreak, to the best of our knowledge, our work is the first to provide complex linguistic annotations to support structured information extraction. to demonstrate the utility of our dataset, we built covidkb, a knowledge base that supports ## related work event extraction from twitter. there has been much interest in extracting events from twitter. for example, @xcite built a system for open domain event extraction. recent work also explored extraction of cybersecurity events @xcite , including denial of service attacks @xcite and software vulnerabilities @xcite . @xcite use a nonparametric bayesian mixture model for event extraction. in this work, we design event types and attributes that are specific for covid-19 and develop automatic nlp tools for extracting structured information from tweets. ## an annotated corpus for covid-19 event extraction to extract structured knowledge from tweets, we formulate the problem as a supervised slot filling task @xcite @xcite . specifically, given a tweet, annotators are asked to first identify whether it contains a relevant event, then highlight the text spans of answers that correspond to a list of pre-defined questions for each event type (detailed questions are in table a2 ). ## automatic event extraction we now use our annotated corpus to train and evaluate supervised learning methods for automatic covid-19 event extraction. each slot filling question is treated as a binary classification task: given a tweet t and the candidate span c, the classification model f e,s (t, c) → {0, 1} predicts whether c correctly answers the question for the slot s of event type e. ## covidkb knowledge base we have built models that can extract structured information related to covid-19 from individual tweets. to demonstrate the utility of our annotated dataset and models, we create a knowledge base (figure 2 ) that enables structured search over covid-19 events that are automatically extracted from twitter. ## conclusion in this paper, we presented a corpus of 10,000 tweets annotated with 5 types of events and 28 slots. we showed that our corpus supports automatic extraction of covid-19 events using supervised learning. by aggregating extractions over millions of tweets, our approach can accurately answer a range of structured queries about events that are publicly reported in real-time on twitter. our knowledge base could be a useful tool for epidemiologists, journalists and policymakers to more efficiently track the spread of this new disease. this work also presents a case-study on how an information extraction system can be rapidly developed for a new domain in response to an emerging crisis. for example, our methodology could be applied to develop knowledge bases for natural disasters @xcite or future disease outbreaks.