A nswer Q uest: A System for Generating Question-Answer Items from Multi-Paragraph Documents (2021)

Abstract: One strategy for facilitating reading comprehension is to present information in a question-and-answer format. We demo a system that integrates the tasks of question answering (QA) and question generation (QG) in order to produce Q&A items that convey the content of multi-paragraph documents. We report some experiments for QA and QG that yield improvements on both tasks, and assess how they interact to produce a list of Q&A items for a text. The demo is accessible at qna.sdl.com.

## introduction automated reading comprehension is one of the current frontiers in ai and nlp research, evidenced by the frequently changing state-of-the-art among competing approaches on standard benchmark tasks (e.g. @xcite . these systems aim to reach the standard of human performance, but they also have the potential to further enhance human reading comprehension. for instance, many demonstrations of reading comprehension involve eliciting answers to questions about a text. meanwhile, educational research and conventional writing advice indicate that structuring information in a question-and-answer format can aid comprehension @xcite . accordingly, systems that present content in this format by automatically generating and answering relevant questions may help users better understand the content. the two nlp tasks essential to this objective, question answering (qa) and question generation (qg), have received a lot of attention in recent years. recent work has started to explore the intersection of qa and qg for the purpose of enhancing performance on one or both tasks @xcite @xcite . among application interfaces 3 current affiliation: ebay inc., san jose, ca, usa that demo these tasks, most have focused on either one or the other @xcite . @xcite presented a system that integrated these tasks to simulate a pedagogical approach to human reading comprehension. in our work, we demo an end-to-end system that applies qa and qg to multi-paragraph documents for the purpose of user content understanding. the system generates a catalog of q&a items that convey a document's content. this paper first presents some focused contributions to the individual tasks of qa and qg. in particular, we examine the challenging task of qa applied to multi-paragraph documents and show the impact of incorporating a pre-trained text encoding model into an existing approach. additionally, we report a new set of results for qa that assesses generalizability between datasets that are typically evaluated separately. for qg, we demonstrate the benefit of data augmentation by seeding a model with automatically generated questions, which produces more fluent and answerable questions beyond a model that observes only the original human-authored data. in combining the two tasks into a single pipeline, we show that the information given by the generated q&a items is relevant to the information humans target when formulating questions about a text. the demo is implemented as a web application in which users can automatically generate q&a pairs for any text they provide. the web application is available at qna.sdl.com, and our code is at github.com/roemmele/answerquest. ## generating q&a pairs we combined our best-performing qg and qa models into a system that takes a text as input and returns a list of q&a pairs. our web demo illustrates this system (see appendix a.4 for details). for our evaluations in section 3, the qg models observed annotated answer spans upon which the generated questions were conditioned. however, these annotations are obviously not available by default for any arbitrary text. consequently, after splitting the text into sentences, we automatically identify syntactic chunks and named entities as candidate answers to questions (see appendix a.5 for details). for each candidate answer in a sentence, we produce an input consisting of that sentence annotated with that span. we also include sentences with no answer annotations as inputs, since they are not formally required by the model. we provide all inputs for a given sentence to the augmented qg model to get a list of questions that can be passed to the qa component. note that even though some of the questions are already associated with annotated answers, we still apply qa as an additional means of verifying their answerability, and defer to the qa-predicted answer. to prepare the qa inputs, for each sentence-question item, we extend the sentence to include the sentences immediately preceding and following it, so each question becomes aligned with a 3-sentence passage. this enables the qa system to possibly retrieve additional context beyond the sentence that it may deem as part of the answer span. we provide these passage-question pairs to the bert shared-norm megaqa model, then retain output items for which answers are found. we reduce the redundancy of items by filtering those with duplicate questions or answers, as well as items where the question and answer concatenated share 60% or more of the same tokens. in these cases, we only retain the item with the highest qg probability. we used our system to generate q&a pairs for ten texts from the squad test set. appendix ## conclusion and future work in this paper, we present a system that automatically produces q&a pairs for multi-paragraph documents. we report some novel experiments for qa and qg that motivate techniques for improving these tasks. we show that combining these components can produce informative q&a items. our future work will focus on more advanced modeling of information structure in documents. for example, the ideal design of q&a items varies by domain (e.g. news stories vs. financial reports vs. opinion editorials), and items should target the content readers find most substantial in each domain. website foot_5 . this included adam optimization with β 1 = 0.998, gradient re-normalization for norms exceeding 0, glorot uniform parameter initialization, 0.1 dropout probability, noam decay, 8000 warmup steps for decay, learning @xmath1, and label @xmath2.