A Simple and Effective Model for Answering Multi-span Questions (2020)

Abstract: Models for reading comprehension (RC) commonly restrict their output space to the set of all single contiguous spans from the input, in order to alleviate the learning problem and avoid the need for a model that generates text explicitly. However, forcing an answer to be a single span can be restrictive, and some recent datasets also include multi-span questions, i.e., questions whose answer is a set of non-contiguous spans in the text. Naturally, models that return single spans cannot answer these questions. In this work, we propose a simple architecture for answering multi-span questions by casting the task as a sequence tagging problem, namely, predicting for each input token whether it should be part of the output or not. Our model substantially improves performance on span extraction questions from DROP and Quoref by 9.9 and 5.5 EM points respectively.

## introduction the task of reading comprehension (rc), where given a question and context, one provides an answer, has gained immense attention recently. in most datasets and models @xcite @xcite @xcite , rc is set up as an extractive task, where the answer is constrained to be a single span from the input. this makes learning easier, since the model does not need to generate text abstractively, while still being expressive enough to capture a large set of questions. however, for some questions, while the answer is indeed extractive, i.e., contained in the input, it is not a single span. for example, in figure 1 the answer includes two people who appear as noncontiguous spans in the context. existing models @xcite are by design unable to provide the correct answer to such multispan questions. while most work has largely ignored this issue, recent work has taken initial steps towards handling multi-span questions. @xcite proposed to predict the number of output spans for each question, and used a non-differentiable inference procedure to find them in the text, leading to a complex training procedure. @xcite proposed a merge operation that merges spans, but is constrained to at most 2 spans. @xcite proposed a non-differentiable symbolic approach which outputs programs that compose single-span extractions. in this work, we propose a simple and fully differentiable architecture for handling multi-span questions that evades the aforementioned shortcomings, and outperforms prior work. similar to @xcite , who used a linear model over treebased features, we cast question answering as a sequence tagging task, predicting for each token whether it is part of the answer. at test time, we decode the answer with standard decoding methods, such as viterbi. we show the efficacy of our approach on spanextraction questions from both the drop @xcite and quoref @xcite datasets. replacing the single-span architecture with our multi-span approach improves performance by 7.8 and 5.5 em points respectively. com-bining the single-span and multi-span architectures further improves performance by 2.1 em on drop, surpassing results by other span-extraction methods on both datasets. first, we encode the question and context with a pre-trained language model, such as bert @xcite : @xmath0, c]), where @xmath1, . . . , h m ) is a sequence of contextualized representations for all input tokens. then, two parameterized functions (feed-forward networks), f start (h i ) and f end (h i ), are used to compute a score for each token, corresponding to whether that token is the start or the end of the answer. last, the start and end probability for each token i is computed as follows: ## "multi-head" models some rc datasets contain questions where the output is not necessarily a span. for example, in drop, the answer to some questions is a number that is not in the text, but can be computed by performing arithmetic operations. to handle such cases, many models @xcite employ a multi-head architecture. in these models, each head z is a small module that takes the contextualized representations h as input and computes a probability distribution over answers p z (a | q, c) = p z (a | h). for example, in @xcite , there are two heads that output spans, and three heads that output numbers. to determine which head to use for each question, an additional module is trained: p head (z | q, c) = p head (z | h). thus, the model probability for an answer is: with this architecture, we can seamlessly integrate our multi-span approach into existing rc models. specifically, a model can include both a single-span head and a multi-span head, dynamically deciding which span extraction method to utilize based on the input. ## empirical evaluation experimental setup as an encoder, we use the hugging face implementation of roberta large @xcite , which produces the representations h. for drop, we add the arithmetic and count heads from @xcite to handle non-span questions. full details of the experimental setup are in appendix a. ## conclusion in this work, we cast the task of answering multi-span questions as a sequence tagging problem, and present a simple corresponding multispan architecture. we show that replacing the standard single-span architecture with our multispan architecture dramatically improves results on multi-span questions, without harming performance on single-span questions, leading to state-of-the-art results on quoref. in addition, integrating our multi-span architecture into existing models further improves performance on drop, as is evident from the leading models on drop's leaderboard. our code can be downloaded from https://github.com/eladsegal/tag-based-multispan-extraction .