Answering Questions on COVID -19 in Real-Time (2020)

Abstract: The recent outbreak of the novel coronavirus is wreaking havoc on the world and researchers are struggling to effectively combat it. One reason why the fight is difficult is due to the lack of information and knowledge. In this work, we outline our effort to contribute to shrinking this knowledge vacuum by creating covidAsk, a question answering (QA) system that combines biomedical text mining and QA techniques to provide answers to questions in real-time. Our system also leverages information retrieval (IR) approaches to provide entity-level answers that are complementary to QA models. Evaluation of covidAsk is carried out by using a manually created dataset called COVID-19 Questions which is based on information from various sources, including the CDC and the WHO. We hope our system will be able to aid researchers in their search for knowledge and information not only for COVID-19, but for future pandemics as well.

## introduction the most recent pandemic to affect humankind is the coronavirus disease 2019 (covid-19), which has infected nearly 36 million people worldwide as of oct 9th, 2020 foot_1 and has led researchers and scientists to scramble to find a solution. despite such efforts being made and experimental results being released, finding a viable treatment or vaccine for covid-19 still seems far off. one of the biggest hurdles to this process is the lack of knowledge regarding covid-19 and the difficulty of finding relevant and reliable information in a timely fashion. taking these difficulties into consideration, creating a real-time question answering (qa) system would be able to greatly aid the efforts of researchers to effectively combat the current pandemic. recently, qa models have made significant developments in terms of both performance and throughput. such improvements can be attributed to the introduction of large-scale qa datasets @xcite and deep learning models @xcite , while recent trends also consider the efficiency of such models @xcite . the development of qa models has also benefited other domains. for example, biomedical qa @xcite has seen many advances due to neural qa models @xcite . despite such progress, creating a qa model specific to covid-19 poses several challenges. the first challenge is that there are almost no qa datasets that are tailored specifically to covid-19 with a few recent exceptions @xcite . this virtually means that models will have to be evaluated in a zero-shot setting. we take the approach of using a real-time qa model @xcite and evaluate its transferability from existing qa datasets to a covid-19 dataset that we coin as the covid-19 questions dataset. covid-19 questions is created by using known facts and experimental results of covid-19. the second challenge is the incorporation of traditional biomedical text mining tools into existing qa models. in order to address this, we shift our focus to a key feature used in biomedical text mining: biomedical named entities. named entities provide important information in text, and this is especially so in the biomedical domain. taking this into account, we use biobert-based models @xcite to extract and normalize biomedical named entities found in documents that contain answers. this approach would allow researchers to easily navigate named entities that are linked to their respective concept unique ids (cuis) such as medical subject headings (mesh). to improve the stability of our approach, we incorporate a biomedical entity search engine @xcite that provides relevant named entities to entity-level queries. the contributions of our paper are three-fold. first, we present covidask for real-time qa on coronaviruses to aid researchers and scientists in effectively navigating resources; second, we incorporate various techniques from traditional biomedical text mining (e.g., biomedical named entity linking (nel)) to enhance the usability of our model; and third, we evaluate covidask on our covid-19 questions dataset and publicly release the source code of covidask and covid-19 questions for future work. 3 ## system desiderata we first specify the overall goals of our system since qa is very broad and has many details to consider (e.g., types of questions, types of answers, etc.). in this section, we detail how we take such desiderata into consideration, and in section 3 we elaborate on how we adapted covidask to address each desideratum. ## architecture the overall layout of our system is illustrated in figure 1 and the hosted web service is shown in figure 2. we pre-index all phrases in research papers contained in @xcite and use them to build the denspi model @xcite . we also highlight and use biomedical named entities in pubmed foot_4 for building best @xcite . for given questions, covidask returns two different lists of answers from both denspi and best. ## discussion since the outbreak of covid-19, there have been various efforts being made from multiple angles in order to handle it. among those efforts, research focused on building effective qa & ir systems @xcite and dataset curation for such systems @xcite @xcite have been especially promising as they can most directly aid researchers in narrowing the knowledge gap. taking into consideration the lack of datasets for covid-19, many ir models for covid-19 perform in an unsupervised or zero-shot setting. however, unlike ir where unsupervised models and algorithms such as the bm25 @xcite often produce satisfying results, many qa models rely on a large amount of rich annotations @xcite @xcite . while covidask uses qa models trained on squad and natural questions and returns zeroshot results on covid-19 questions, we believe that covidask could benefit from recent discoveries in unsupervised qa @xcite , zeroshot qa @xcite , and question generation @xcite as well. further evaluation and fine-tuning could also be conducted using the aforementioned new covid-19 qa datasets. during the development of covidask, we faced several challenges that are difficult to tackle considering the current state of our model. these include: these challenges are also actively studied in many nlp researches and we hope to tackle them in near future. ## conclusion in this work, we presented covidask in an attempt to assist the process of collecting and curating much needed knowledge by providing highly accurate answers to questions in real-time. while covidask is an ongoing endeavor that will be updated accordingly, we provide a cornerstone of constructing a covid-19 qa system and also make public the source code and evaluation dataset. we plan to further automate and refine our system so that it will be able to be adapted to provide aid in future pandemic situations as well.