Addressing Semantic Drift in Generative Question Answering with Auxiliary Extraction (2021)

Abstract: Recently, question answering (QA) based on machine reading comprehension has become popular. This work focuses on generative QA which aims to generate an abstractive answer to a given question instead of extracting an answer span from a provided passage. Generative QA often suffers from two critical problems: (1) summarizing content irrelevant to a given question, (2) drifting away from a correct answer during generation. In this paper, we address these problems by a novel Rationale-Enriched Answer Generator (REAG), which incorporates an extractive mechanism into a generative model. Specifically, we add an extraction task on the encoder to obtain the rationale for an answer, which is the most relevant piece of text in an input document to a given question. Based on the extracted rationale and original input, the decoder is expected to generate an answer with high confidence. We jointly train REAG on the MS MARCO QA+NLG task and the experimental results show that REAG improves the quality and semantic accuracy of answers over baseline models.

## introduction question answering (qa) has come a long way from answer sentence selection, relationship qa to machine reading comprehension (mrc). recently, qa has become an essential problem in natural language understanding and a major milestone towards human-level machine intelligence. current mainstream approaches @xcite @xcite treat mrc as a process of extracting a consecutive piece of text from a document to a given question. despite the great success in extractive mrc @xcite , in real-world applications, correct answers may span different ## conclusion and future work this paper presents a novel model reag that is designed to incorporate an extractive mechanism into a generative qa model. reag introduces a new task on the encoder to extract rationales. based on these rationales and original input, a rationaleenriched decoder is proposed to generate an answer with high confidence. the experimental results show that reag significantly improves the quality and semantic accuracy of generated answers over state-of-the-art models.