Consecutive Question Generation via Dynamic Multitask Learning (2022)

Abstract: In this paper, we propose the task of consecutive question generation (CQG), which generates a set of logically related question-answer pairs to understand a whole passage, with a comprehensive consideration of the aspects including accuracy, coverage, and informativeness.To achieve this, we first examine the four key elements of CQG, i.e., question, answer, rationale, and context history, and propose a novel dynamic multitask framework with one main task generating a question-answer pair, and four auxiliary tasks generating other elements. It directly helps the model generate good questions through both joint training and self-reranking. At the same time, to fully explore the worth-asking information in a given passage, we make use of the reranking losses to sample the rationales and search for the best question series globally.Finally, we measure our strategy by QA data augmentation and manual evaluation, as well as a novel application of generated question-answer pairs on DocNLI. We prove that our strategy can improve question generation significantly and benefit multiple related NLP tasks.

## introduction question generation (qg) is an important and promising task in natural language generation (nlg). it has long served as an effective way to improve other nlp tasks. the applications of synthetic questions have expanded from qa data augmentation @xcite to building tutoring or dialogue systems @xcite , self-assessing the ability of language models @xcite , and checking the faithfulness of an abstract summary @xcite , etc. in such cases, we propose the task of consecutive question generation (cqg), which automatically produces a set of well-ordered and logically related question-answer (q-a) pairs to help understand a given passage (or story). with these demands, we propose an integrated dynamic multitask framework, with five unified seq2seq generation tasks. one main task generates q-a pairs and four auxiliary tasks make full use of the generation of four key cqg elements (i.e., question, answer, rationale, and context history). we link the qualities of key aspects with the inference losses of four auxiliary tasks respectively. based on it, we then design four distinct methods to improve the model performance from all aspects and from all stages during training and inference. the five tasks are jointly trained in one model to help it learn from different views. in inference, the main task generates candidates and then the auxiliary tasks self-rerank them, improving q-a accuracy, coverage, and informativeness all-roundly. to fully exploit the worth-asking information in each sentence and generate questions properly and dynamically, we propose a novel rationale sampling method and sentence-level beam-search. we recompose the context history reranking losses to measure the information in each rationale, and then design a sample probability to guarantee that the more information a rationale leaves, the more likely it is asked once again. to relieve the error cascade and guide the direction of a q-a flow, we reinvent beam-search to sentence-level, which rearranges the total reranking results and seeks the global optimum q-a series for a whole passage. finally, we conduct abundant experiments to augment various qa datasets, only using the model trained on coqa. we also make a manual evaluation and propose a novel zero-shot method for document-level nli task @xcite using question generation. successfully, we promote the performance on multiple qa scenes and prove the expansibility of our model on different nlp tasks. ## related work question generation is a promising task which has been well studied in many researches. initially, rule-based or traditional machine learning methods are widely used in producing questions. heilman and smith (2010) adopt verb transformations and @xcite use semantic parsing to synthesize questions. recently, deep learning techniques have given a further development of question generation. @xcite use an lstm @xcite model, and sultan et al. (2020) adopt roberta (liu et al., 2019) model to generate questions. at the same time, the strategies like multitask learning and self-training have been applied to improve the quality of generated questions. zhou et al. (2019) and ma et al. (2020) employ a multitask structure to generate coherent and fluent questions. sachan and xing (2018) and rennie et al. (2020) adopt a self-training strategy to jointly learn to ask and answer questions. alberti et al. (2019) use roundtrip consistency to filter out inconsistent results. shinoda et al. (2021) generate noisy data and sultan et al. (2020) employ nucleus sampling @xcite to improve the diversity of questions. however, they mainly focus on only one quality aspect and most of them are based on pre-defined answers or original data. as qg can produce meaningful questions, it has been widely used to promote other nlp tasks. @xcite use a constrained question rewriting way to generate new data for qa tasks. @xcite and @xcite check the faithfulness of summaries through answering generated questions. @xcite generate question-answer pairs and convert them for fact verification. nevertheless, the researches above mainly produce each question independently and ignore the connections between questions. as for generating a set of questions over a specific passage, @xcite propose a pipelined system to ask different levels of questions from general to specific. @xcite use conditional variational autoencoder to generate multiple robust questions for a given paragraph. similar to us, chai and wan (2020) generate sequential and related questions under dual-graph interaction, but use ground-truth answers. to the best of our knowledge, we are the first to consecutively synthesize a series of connected questionanswer pairs to understand an entire passage, with the comprehensive consideration of accuracy, coverage, and informativeness. ## multitask framework in our cqg strategy, the foundation is five various but unified tasks. the effects of these tasks are dynamically spread throughout our whole strategy. ## training and inference based on the dynamic multitask framework, we jointly train a bart @xcite model. in inference, we use the main task to generate several candidates and self-rerank them using the auxiliary tasks. with the reranking losses, we design a formula to assess the information and automatically sample the rationales. globally, we beamsearch for the best q-a series on sentence level. ## conclusion in this paper, we propose the consecutive question generation task, which synthesizes mutually connected question-answer pairs to fully explore the information in a passage. by constructing a novel multitask framework with one main task and four unified auxiliary tasks, we generate optimum q-a series using four sub-methods, which help "generate good questions" as well as "find worthasking information". with extensive experiments, we prove that our model is able to generate highquality q-a pairs to understand a whole passage and has the power to benefit various nlp tasks.