C hain CQG : Flow-Aware Conversational Question Generation (2021)

Abstract: Conversational systems enable numerous valuable applications, and question-answering is an important component underlying many of these. However, conversational question-answering remains challenging due to the lack of realistic, domain-specific training data. Inspired by this bottleneck, we focus on conversational question generation as a means to generate synthetic conversations for training and evaluation purposes. We present a number of novel strategies to improve conversational flow and accommodate varying question types and overall fluidity. Specifically, we design ChainCQG as a two-stage architecture that learns question-answer representations across multiple dialogue turns using a flow propagation training strategy. ChainCQG significantly outperforms both answer-aware and answer-unaware SOTA baselines (e.g., up to 48% BLEU-1 improvement). Additionally, our model is able to generate different types of questions, with improved fluidity and coreference alignment.

## introduction conversational systems are important in many realworld applications, including personal assistants, educational tutors @xcite , customer service (el @xcite , and increasingly, entertainment. a key component of these systems is the ability to interpret a search query and retrieve information from different sources as naturally and efficiently as possible. in analogous human interactions, such a search generally occurs through conversation. in this context, a conversation consists of a sequence of dialogue turns during which the search objective becomes clearer over time. the applications mentioned above could benefit greatly from this type of multi-turn interaction, enabling conversational agents to accurately predict intent, request additional information, and better understand ambiguous followup questions and comments. in an applied setting, meaningful and natural conversations are important features of virtual entities as a means to establish trust and improve usability. here, we are motivated by the challenging task of conversational question answering (cqa). current open-source datasets such as coqa @xcite and quac @xcite provide strong baselines for this task. however, these datasets have limited applicability in practical settings, because 1) they are created from domainagnostic source material, and 2) they do not necessarily consider the full diversity of question types and vernacular that may be encountered in natural dialogue. creating realistic, domain-specific datasets to train cqa models is notoriously costly and time-consuming. as such, we focus on the related task of question generation as a means to generate synthetic conversational questions and subsequently, create new datasets or augment existing ones. this will ultimately enable training cqa models in closed-loop, simulation environments, as well as allow machines to initiate dialogue and engage in information-seeking behavior. while qa models have been studied previously @xcite @xcite @xcite ), the qg task, which is the focus of this paper, has received less attention. qg models in the answer-unaware setting aim to predict a question given the source passage, while in the answer-aware setting, the target answer and rationale are included as inputs as well. most qg-related literature has focused on single-turn question generation using question-answer datasets such as squad @xcite , and other textual sources like wikipedia articles @xcite . ## related work in this section, we first explore previous approaches to question generation and then discuss outstanding research challenges in conversational question generation. ## the chaincqg framework chaincqg learns the question-answer representations jointly using two modules: answer encod-ing (ae) and question generation @xcite . encoding the answer based on the passage and dialogue history improves the answer understanding within the qg module, which in turn improves the generated questions. in the rest of this section, we provide a description of the input pre-processing steps, the general cqg problem formulation, and the ae and qg modules. ## discussion and ethical issues the results presented here demonstrate the efficacy of modern transformer-based architectures, and specifically chaincqg, in producing conversational questions on a challenging dataset. while answer-aware qg is our focus here, we plan to expand this in future work to include answer-unaware and open-ended qg, multi-task nlg involving qa, and domain-specific dialogue simulation. the flexibility of the chaincqg architecture lends itself well to each of these problems, as representations from different inputs and tasks can be shared easily between modules. as for the practical implications of our qg work, a number of applications mentioned in previous sections could immediately take advantage of qg features, either for training qa models or generating user-facing questions. in the former setting, generation models, such as chaincqg, risk polluting the training dataset with examples that are noisy or inconsistent with the target answers, which can cause undesirable effects at inference time. in the latter setting, generation models may suffer from bias based on the questions available for training, which may lead to misrepresentation of application domains and individual users. additional work is required to understand the extent of these issues in real-world applications, and identify corrective measures to ensure model robustness and diversified training distributions. ## conclusion in this paper, we introduce chaincqg, an answeraware conversational question generation model that outperforms all baselines on both automatic and human evaluation metrics on the inverted coqa dataset (e.g., bleu-1 improvement of 48% and 28% with gpt medium compared to redr and cfnet, respectively). we have designed a twostage gpt-2-based architecture that jointly learns passage and dialogue history representations via a flow propagation training method. chaincqg produces high-quality questions in multi-turn dialogue, addressing previous sota issues such as question type fidelity, question-answer inconsistency and coreference misalignment. finally, we have performed and presented extensive ablation studies for various aspects of our approach.