Collecting Verified COVID -19 Question Answer Pairs (2020)

Abstract: We release a dataset of over 2,100 COVID19 related Frequently asked Question-Answer pairs scraped from over 40 trusted websites. We include an additional 24, 000 questions pulled from online sources that have been aligned by experts with existing answered questions from our dataset. This paper describes our efforts in collecting the dataset and summarizes the resulting data. Our dataset is automatically updated daily and available at https://github.com/JHU-COVID-QA/ scraping-qas. So far, this data has been used to develop a chatbot providing users information about COVID-19. We encourage others to build analytics and tools upon this dataset as well.

## introduction with the quick spread of covid19, misinformation has rapidly spread. @xcite misinformation around the use of certain drugs for the prevention of covid-19 has had fatal outcomes, and stigmatization guided by misinformation about certain communities as vectors of virus undermines the long-term welfare of our society. we are developing a natural language processing (nlp) backed-informational chatbot targeted at comprehensive covid-19 information and misinformation. users can interact with our chatbot on different platforms to access information about covid-19, available care, and other topics of interest. @xcite to aid in this effort, we aggregate factual information in the form of verified questions and answers to help answer frequently asked questions about the pandemic. we employ three main aggregation efforts in tandem: 1) generating high quality and accurate information from domain experts, i.e. public health researchers at johns hopkins university; 2) automatically scraping frequently asked questions and answers from online trusted sources, e.g. newspapers and government agencies; and 3) automatically ranking and manually aligning additional questions from social media with the scraped questions and answers in our dataset. this paper primarily describes our efforts to extract high quality content from trustworthy websites and domain experts. our effort has resulted in a publicly available dataset that currently contains over 2,100 questions and answers from more than 40 webpages. the dataset is available at https://covid-19-infobot.org/data/ . since we are actively scraping more websites and rescrape all sites at least once a day, these numbers are updated daily. @xcite ## creating our faq dataset we create our publicly available dataset of over 2,100 question-answer pairs by aggregating faqs from trusted news sources. @xcite we choose websites to scrape based on three broad criteria: 1) the informativeness and trustworthiness of the website; 2) the ease of scraping frequently asked questionanswer pair from the website; and 3) the number of questions and answers on the website. we use a straightforward scraping process that enables undergraduate students to contribute to our efforts. we developed a python library for students to easily add scrapers to our project. as demonstrated in the example in figure 1 , our library requires each question-answer (and metadata) to be stored as a simple dictionary. the library automatically adds this information to our set of question-answer pairs. additionally, the library accordingly handles updating answers to questions in our dataset if a previously scraped website updates its information. this has enabled students to efficiently join the project and contribute immediately. further documentation is available at https://github.com/ jhu-covid-qa/scraping-qas and we encourage others to join our efforts. ## data the described effort resulted in a dataset that is evolving daily. the june 15th version contains over 2,100 questions and answers scraped from 40 websites. we list the number of question-answer pairs extracted from each source in websites might update or change how they store information. this is why the current version of our dataset contains just 1 example from the delaware state government webpage. the may 20th version of our dataset contains 22 examples from this website. ## manually aligning additional questions and answers since the internet contains many more questions that are not answered, we additionally collected questions and align them with the question-answer pairs in our dataset. we leverage information retrieval techniques to match these unanswered questions with questions in our dataset and then rely on domain experts to verify each aligned questionquestion-answer (qqa) pair. in this section, we provide details for each of these steps. ## conclusion we have presented our growing dataset of over 2,100 question-answers that has been created by scraping over 40 websites. we also discussed other data we collected and annotated that may be beneficial to others in the community as well. our evolving dataset is complementary to other recent covid-19 qa datasets, e.g. tang et al. (2020 )'s 124 question-article pairs, wei et al. (2020) 1,690 questions and 403 answers, and möller et al. (2020)'s dataset.