C on Q uest: Contextual Question Paraphrasing through Answer-Aware Synthetic Question Generation (2021)

Abstract: Despite excellent performance on tasks such as question answering, Transformer-based architectures remain sensitive to syntactic and contextual ambiguities. Question Paraphrasing (QP) offers a promising solution as a means to augment existing datasets. The main challenges of current QP models include lack of training data and difficulty in generating diverse and natural questions. In this paper, we present Conquest, a framework for generating synthetic datasets for contextual question paraphrasing. To this end, Conquest first employs an answer-aware question generation (QG) model to create a question-pair dataset and then uses this data to train a contextualized question paraphrasing model. We extensively evaluate Conquest and show its ability to produce more diverse and fluent question pairs than existing approaches. Our contextual paraphrase model also establishes a strong baseline for end-to-end contextual paraphrasing. Further, We find that context can improve BLEU-1 score on contextual compression and expansion by 4.3 and 11.2 respectively, compared to a non-contextual model.

## introduction in recent years, transformer-based architectures have made enormous progress in neural question answering (qa) @xcite (izacard and grave, 2020). however, these models are still sensitive to query syntax and ambiguity @xcite . while data augmentation and lexical normalization are popular approaches to account for syntactic variations of input texts, progress on generating diverse and meaningful question variants (i.e., question paraphrasing (qp)) has been limited. the main challenge in qp remains the lack of large-scale question paraphrase datasets. one potential solution is to use a qg model to generate multiple questions based on a given context and use these as paraphrased versions of each other. the challenge with this approach is the difficulty in generating questions that are diverse, fluent, and consistent. further, these questions should be of different lengths to make sure that the qa models learn which information in the question to consider or neglect. this can be achieved by considering question expansion/compression tasks. however, in order to generate such questions, the context surrounding the source question and answer becomes important. the expansion task requires associating sparse information in a short-form question with details from the context to reformulate a richer question. conversely, compression requires synthesizing extraneous details in a long-form question into a shorter form that is still consistent with the given passage and answer. here, we introduce conquest, a framework for generating synthetic contextual question paraphrase datasets for tasks such as question compression and expansion. specifically, given a passage as context, we first employ an answer-aware sequence-to-sequence (seq2seq) model to generate a diverse set of question variants, each consistent with a common answer span. then, we pair the shortest and longest variants to form a novel contextual paraphrasing dataset suitable for question expansion and compression tasks. being answeraware, this qg model accounts for contextual information when linking question pairs (unlike existing paraphrase methods using back-translation @xcite or term replacement @xcite ). using this synthetic dataset, the final module of our framework is a seq2seq model for contextual question expansion and compression. by considering the context, this module addresses the unexplored challenge of contextual question reformulation. further, we describe a multi-tag encoding scheme for compression and expansion to improve the quality of generated questions of dif-ferent length by providing an effective method to disentangle their representations while leveraging their shared properties. to evaluate our framework, we use questions from squad v1.1 @xcite , a common question answering dataset. we measure performance of each component in our framework across a number of automatic and human-based metrics. we will release our question paraphrasing dataset and code upon acceptance. ## related work in this section, we review previous results and challenges in question generation and reformulation, and discuss novel aspects of our contextual question paraphrasing dataset. ## method in this section, we describe conquest, our contextual question paraphrasing framework. con-quest has two main modules: 1) paraphrase question generation where answer-aware question generation is used to create a question-pair dataset and 2) contextual paraphrasing to train a generative model to compress and expand these questions. these two modules and their components are shown in figure 1 . ## conquest evaluation to understand the performance of conquest, we have conducted a set of experiments using squad v1.1, a commonly-used english questionanswering dataset. in this section, we discuss the evaluation results for paraphrase question generation and contextual paraphrasing. ## conclusion in this paper, we introduce conquest, a framework for generating synthetic contextual question paraphrase data. conquest first employs an answeraware question generation model to create a dataset of diverse question-pairs. then, it trains a multi-tag contextualized question paraphrasing model, which is able to control the length of the paraphrased questions. we have extensively evaluated conquest using the squad v1.1 dataset. the results show that contextualized question paraphrasing results in higher performance across various automatic metrics (e.g., 4.3 and 11.2 points improvement in bleu-1 compared to non-contextualized baseline). further, our answer-aware qg-based data generation model achieves greater diversity, naturalness and consistency than previous question paraphrase models. finally, we have performed and presented various ablation studies.