M ulti S pan QA : A Dataset for Multi-Span Question Answering (2022)

Abstract: Most existing reading comprehension datasets focus on single-span answers, which can be extracted as a single contiguous span from a given text passage. Multi-span questions, i.e., questions whose answer is a series of multiple discontiguous spans in the text, are common real life but are less studied. In this paper, we present MultiSpanQA, a new dataset that focuses on multi-span questions. Raw questions and contexts are extracted from the Natural Questions dataset. After multi-span re-annotation, MultiSpanQA consists of over a total of 6,000 multi-span questions in the basic version, and over 19,000 examples with unanswerable questions, and questions with single-, and multi-span answers in the expanded version. We introduce new metrics for the purposes of multi-span question answering evaluation, and establish several baselines using advanced models. Finally, we propose a new model which beats all baselines and achieves state-of-the-art on our dataset.

## introduction the task of reading comprehension, where models are required to process a text and answer questions about it, has seen rapid progress in recent years. as systems have increasingly matched humans on popular datasets @xcite , researchers have developed newer, more complex formulations of the task, such as very long contexts and answers @xcite , multi-hop reasoning @xcite , and discrete operations over the content of paragraphs @xcite . one thing these datasets have in common is that the answer is constrained to be a single span that can be extracted or computed from the context. however, in practice, the answer to a question will often consist of multiple parts. as in the example in figure 1 , the answer set contains 10 countries, some of which are discontiguous in the passage. such cases are largely ignored in existing reading comprehension research, in part because there are no datasets of multi-span questions. in this paper, we introduce multispanqa, a new reading comprehension dataset consisting of 6,536 multi-span examples. the raw questions and passages are extracted from natural questions ("nq": @xcite al. (2019) ), a large-scale opendomain qa dataset. trained annotators were asked to identify question-passage pairs where the answer was multi-span, and annotate the spans. in addition to the basic version of the dataset consisting entirely of multi-span answers, we also prepare an expanded version with a selection of unanswerable questions, and questions with single-and multispan answers, intended to reflect a more realistic qa setup. we further classify answer semantics into 5 categories, and manually label the logical structure of the answer spans. we introduce metrics to evaluate multi-span qa systems across these different tasks. we propose several baselines, and a new model which casts the task as a sequence tagging problem. the proposed model combines a sequence tagger with a span number predictor, span structure predictor, and span adjustment module. experimental results show that the proposed model surpasses all baselines and achieves 59.28% exact-match f1 score and 76.50% partial-match f1 score. to summarize, our contributions are: ## dataset construction and composition in this section, we describe how we construct mul-tispanqa, and provide a statistical breakdown of its composition. ## models formally, given a question and passage pair q, p , the task of multi-span qa involves finding all answer spans s 1 , s 2 , ...s n , which are neither duplicated nor overlap with each other, as well as predict the answer structures. ## conclusion we present multispanqa, a reading comprehension dataset where answers consist of multiple discrete spans. as part of this, we proposed a method for classifying the semantic structure of answers, based on the semantic relation between answer spans. we also provide an expanded version of the dataset which includes unanswerable questions and single-answer questions, to make it both more challenging and more realistic. we additionally presented a number of models for multi-span qa extraction, and found that the best-performing model was sequence tagging-based, augmented by a span number prediction module and span adjustment module.