Fluent Response Generation for Conversational Question Answering (2020)

Abstract: Question answering (QA) is an important aspect of open-domain conversational agents, garnering specific research focus in the conversational QA (ConvQA) subtask. One notable limitation of recent ConvQA efforts is the response being answer span extraction from the target corpus, thus ignoring the natural language generation (NLG) aspect of high-quality conversational agents. In this work, we propose a method for situating QA responses within a SEQ2SEQ NLG approach to generate fluent grammatical answer responses while maintaining correctness. From a technical perspective, we use data augmentation to generate training data for an end-to-end system. Specifically, we develop Syntactic Transformations (STs) to produce question-specific candidate answer responses and rank them using a BERT-based classifier (Devlin et al., 2019). Human evaluation on SQuAD 2.0 data (Rajpurkar et al., 2018) demonstrate that the proposed model outperforms baseline CoQA and QuAC models in generating conversational responses. We further show our model’s scalability by conducting tests on the CoQA dataset. The code and data are available at https://github.com/abaheti95/QADialogSystem .

## introduction factoid question answering (qa) has recently enjoyed rapid progress due to the increased availability of large crowdsourced datasets (e.g., squad @xcite , ms marco @xcite , natural questions @xcite ) for training neural models and the significant advances in pre-training contextualized representations using massive text corpora (e.g., elmo @xcite , bert @xcite ). building on these successes, recent work examines conversational qa (convqa) systems capable of interacting with users over multiple turns. 1 the code and data are available at https://github.com/abaheti95/qadialogsystem . large crowdsourced convqa datasets (e.g., coqa @xcite , quac @xcite ) consist of dialogues between crowd workers who are prompted to ask and answer a sequence of questions regarding a source document. although these convqa datasets support multi-turn qa interactions, the responses have mostly been limited to extracting text spans from the source document and do not readily support abstractive answers @xcite . while responses copied directly from a wikipedia article can provide a correct answer to a user question, they do not sound natural in a conversational setting. to address this challenge, we develop seq2seq models that generate fluent and informative answer responses to conversational questions. to obtain data needed to train these models, rather than constructing yet-another crowdsourced qa dataset, we transform the answers from an existing qa dataset into fluent responses via data augmentation. specifically, we synthetically generate supervised training data by converting questions and associated extractive answers from a squadlike qa dataset into fluent responses via syntactic transformations (sts). these sts over-generate a large set of candidate responses from which a bert-based classifier selects the best response as shown in the top half of figure 1 . while over-generation and selection generates fluent responses in many cases, the brittleness of the off-the-shelf parsers and the syntatic transformation rules prevent direct use in cases that are not well-covered. to mitigate this limitation, we generate a new augmented training dataset using the best response classifier that is used to train end-toend response generation models based on pointer-generator networks (pgn) @xcite and pre-trained transformers using large amounts of dialogue data, dialogpt (d-gpt) @xcite . in §3.2 and §3.3, we empirically demon-q: where did hizb ut-tahrir fail to pull off a bloodless coup in 1974 ? a: egypt parser + syntactic rules r 1 : he failed to egypt to pull off a bloodless coup r 2 : they failed to pull off a bloodless coup in 1974 egypt … r m : he failed to pull off a bloodless coup egypt best response classifier r m : he failed to pull off a bloodless coup egypt q: where did hizb ut-tahrir fail to pull off a bloodless coup in 1974 ? a: egypt sequence-to-sequence r: they failed to pull off a bloodless coup in egypt augment training data for end-to-end setup over-generate and select the best response: end-to-end response generation: pointer generator network (pgn) or dialogpt (d-gpt) figure 1 : overview of our method of generating conversational responses for a given qa. in the first method, the syntactic transformations (sts) over-generate a list of responses (good and bad) using the question's parse tree and the best response classifier selects the most suitable response from the list. our second method uses this pipeline to augment training data for training a seq2seq networks pgn or d-gpt ( §3.1). the final seq2seq model is end-to-end, scalable, easier to train, and performs better than the first method exclusively. ## generating fluent qa responses in this section, we describe our approach for constructing a corpus of questions and answers that supports fluent answer generation (top half of figure 1 ). we use the framework of overgenerate and rank previously used in the context of question generation @xcite . we first overgenerate answer responses for qa pairs using sts in §2.1. we then rank these responses from best to worst using the response classification models described in §2.2. later in §3, we describe how we augment existing qa datasets with fluent answer responses using sts and a best response classifier. this augmented qa dataset is used for training the pgn and transformer models. ## data-augmentation and generation seq2seq models are very effective in generation tasks. however, our 2028 labeled question and response pairs from the sg train set ( to increase the size of ss training data, we take the qa pairs from natural questions @xcite and harvestingqa foot_2 @xcite and add q, a, r instances using the same sts+bert classifier technique. these new pairs combined with ss result in a dataset of 1,051,938 q, a, r instances, referred to as the ss+ dataset. ## conclusion in this work, we study the problem of generating fluent qa responses in the context of building fluent conversational agents. to this end, we propose an over-generate and rank data augmentation procedure based on syntactic transformations and a best response classifier. this method is used to modify the squad 2.0 dataset such that it includes conversational answers, which is used to train seq2seq based generation models. human evaluations on squad-dev-test show that our models generate significantly better conversational responses compared to the baseline coqa and quac models. furthermore, the d-gpt model with oracle answers is able to generate conversational responses on the coqa dev set 77 % of the time showcasing the model's scalability.