Question Answering with Long Multiple-Span Answers (2020)

Abstract: Answering questions in many real-world applications often requires complex and precise information excerpted from texts spanned across a long document. However, currently no such annotated dataset is publicly available, which hinders the development of neural question-answering (QA) systems. To this end, we present MASH-QA, a Multiple Answer Spans Healthcare Question Answering dataset from the consumer health domain, where answers may need to be excerpted from multiple, non-consecutive parts of text spanned across a long document. We also propose MultiCo, a neural architecture that is able to capture the relevance among multiple answer spans, by using a query-based contextualized sentence selection approach, for forming the answer to the given question. We also demonstrate that conventional QA models are not suitable for this type of task and perform poorly in this setting. Extensive experiments are conducted, and the experimental results confirm the proposed model significantly outperforms the state-of-the-art QA models in this multi-span QA setting.

## introduction developing neural networks for question answering (qa) has become an important and fast-growing area of research in the nlp community. interest in this area is largely driven by the importance and effectiveness of such systems in virtual assistants and search engines. driven by the development of large-scale datasets such as squad @xcite , most of the work in this domain focuses on the task of machine reading comprehension, where the objective is to find a single short answer span-typically ranging from a few what are tips for managing my bipolar disorder? along with seeing your doctor and therapist and taking your medicines, simple daily habits can make a difference. start with these strategies. (22 words truncated) pay attention to your sleep. this is especially important for people with bipolar disorder... (178 words truncated) eat well. there's no specific diet... (29 words truncated) focus on the basics: favor fruits, vegetables, lean protein, and whole grains. and cut down on fat, salt, and sugar. tame stress. (81 words truncated) you can also listen to music or spend time with positive people who are good company. (73 words truncated) limit caffeine. it can keep you up at night and possibly affect your mood. (47 words truncated) avoid alcohol and drugs. they can affect how your medications work. (118 words truncated) figure 1 : an example of a question and its corresponding answer (highlighted) from mash-qa. the answer consists of multiple sentences from the context. all the highlighted sentences will form the comprehensive answer. the context here is 632 words long, so we truncate a few portions of it. existing qa datasets mainly consist of questions with short answers-typically ranging from a few words to a sentence-from the context document. even though natural questions dataset @xcite provided paragraphlength answers for certain questions, these long answers are generally the paragraphs that contain the short answers, making most of the information supplemental (not critical) in nature. moreover, because of the open-ended nature of many questions, the final comprehensive, succinct and correct answers may need to be extracted from multiple spans or sentences from the document. this problem is exacerbated when several spans that contain the answer are not in the vicinity of each other. especially, this is often the case in domains such as healthcare, where people seek information regarding their specific health conditions, and the precise answer for their queries usually come from multiple sections or spans of a document. in this work, we introduce mash-qa, a largescale dataset for question-answering, with many answers coming from multiple spans within a long document. mash-qa is based on questions and knowledge articles from the consumer health domain, where the questions are generally non-factoid in nature and cannot be answered using just a few words. fig. 1 shows an example question, and its corresponding context and answer from our dataset, which poses several unique challenges. first, the contexts are comprehensive healthcare articles, which can typically contain tens of paragraphs and hundreds of lines. context of such length is challenging for existing neural qa models. second, the answers are typically several sentences long, while current span extraction models usually predict very short spans. another challenge in this setting raises from the fact that answers can consist of multiple sentences from nonconsecutive parts of a document, which can often be many sentences or even paragraphs apart. this results in sparsely-scattered patterns of semantic relevance in the context with respect to the query. this means that even if the answer comes from different parts of the document, which might be surrounded by the text that have limited relevance to the question, different answer snippets have some form of semantic relevance with each-other, and are centered around the same topic as the question. although our dataset is from the healthcare domain, we believe that this problem setting can be generalized to other domains, where the questions typically require long and detailed answers. considering all these challenges, we formulate our question-answering task as a sentence selection task, which should also model the semantic relevance existing between different answer sentences, even when they are not adjacent to each-other in the context. hence, we also propose multico, a novel neural architecture that can address the challenges discussed above. our model utilizes xlnet @xcite , which incorporates transformer-xl units @xcite to give semantic representations that capture the long-range dependencies existing in the long document context. we also use a sparsified attention mechanism, to ensure that the representations of sparsely scattered answer units are compactly aligned with each-other. the main contributions of this paper can be summarized as follows: to the best of our knowledge, this is the first work that introduces the qa setting with multiple discontinuous answer spans from a long document. techniques earlier works in qa used similarity based models for classifying answers based on their semantic similarity with the document @xcite . the public release of squad dataset motivated the development of attention-based neural models @xcite @xcite . with the advancements in language modeling (lm) techniques such as bert @xcite and xlnet @xcite , lm-based techniques have gained more popularity in recent times. ## the proposed multico model given a query and a document, the goal of our multico model is to select the sentences that can accurately answer the query. an intuitive way to solve this problem would be to use a text matching model that takes the query and a sentence as the input, and predicts their relevance. however, as shown later, this approach does not capture the overall context of the sentences. hence, in our problem setting, where multiple sentences from a document can belong to the answer, it gives poor results. hence, our proposed approach uses the concept of query-based contextualized sentence selection from the document. ## conclusion we proposed a novel form of question-answering, where answers to a question consist of multiple sentence-level spans from a long document. to support this task, we introduce mash-qa, a novel and challenging qa dataset from the consumer health domain. mash-qa consists of questions that can be answered using information from multiple spans from the document. to motivate further research in multi-span qa, we also propose a novel qa architecture called multico, that uses query-based contextualized sentence selection approach for finding multi-span answers from long documents. by using a sentence-selection based objective, our model outperforms the existing stateof-the-art qa models by a wide margin.