Improving Question Generation with Multi-level Content Planning (2023)

Abstract: This paper addresses the problem of generating questions from a given context and an answer, specifically focusing on questions that require multi-hop reasoning across an extended context. Previous studies have suggested that key phrase selection is essential for question generation (QG), yet it is still challenging to connect such disjointed phrases into meaningful questions, particularly for long context. To mitigate this issue, we propose MultiFactor, a novel QG framework based on multi-level content planning. Specifically, MultiFactor includes two components: FA-Model, which simultaneously selects key phrases and generates full answers, and Q-Model which takes the generated full answer as an additional input to generate questions. Here, full answer generation is introduced to connect the short answer with the selected key phrases, thus forming an answer-aware summary to facilitate QG. Both FA-Model and Q-Model are formalized as simple-yet-effective Phrase-Enhanced Transformers, our joint model for phrase selection and text generation. Experimental results show that our method outperforms strong baselines on two popular QG datasets. Our code is available at https://github.com/zeaver/MultiFactor.

## introduction question generation (qg) is a crucial task in the field of natural language processing (nlp) which focuses on creating human-like questions based on a given source context and a specific answer. in recent years, qg has gained considerable attention from both academic and industrial communities due to its potential applications in question answering @xcite , machine reading comprehension @xcite , and automatic conversation @xcite . effective content planning is essential for qg systems to enhance the quality of the output questions. this task is particularly important for generating complex questions, that require reasoning over long context. based on the content granularity, prior research @xcite can be broadly categorized into two groups: phrase-level and sentence-level content planning. on one hand, the majority of prior work @xcite @xcite @xcite has focused on phrase-level planning, where the system identifies key phrases in the context and generates questions based on them. for instance, given the answer "28 january 1864" and a two-paragraphs context in figure 1 , we can recognize "english inventor," "the oil engine," "herbert akroyd stuart" as important text for generating questions. for long context, however, it is still challenging for machines to connect such disjointed facts to form meaningful questions. on the other hand, sentence-level content planning, as demonstrated by @xcite , aims at automatic sentence selection to reduce the context length. for instance, given the sample in figure 1 , one can choose the underscored sentences to facilitate qg. unfortunately, it is observable that the selected sentences still contain redundant information that may negatively impact question generation. therefore, we believe that an effective automatic content planning at both the phrase and the sentence levels is crucial for generating questions. in this paper, we investigate a novel framework, multifactor, based on multi-level content planning for qg. at the fine-grained level, answer-aware phrases are selected as the focus for downstream qg. at the coarse-grained level, a full answer generation is trained to connect such (disjointed) phrases and form a complete sentence. intuitively, a full answer can be regarded as an answer-aware summary of the context, from which complex questions are more conveniently generated. as shown in figure 1 , multifactor is able to connect the short answer with the selected phrases, and thus create a question that requires more hops of reasoning compared to vanilla qg. it is also notable that we follow a generative approach instead of a selection approach @xcite to sentence-level content planning. figure 1 demonstrates that our generated full answer contains more focused information than the selected (underscored) sentences. specifically, multifactor includes two components: 1) a fa-model that simultaneously selects key phrases and generate full answers; and 2) a qmodel that takes the generated full answer as an additional input for qg. to realize these components, we propose phrase-enhanced transformer (pet), where the phrase selection is regarded as a joint task with the generation task both in fa-model and q-model. here, the phrase selection model and the generation model share the transformer encoder, enabling better representation learning for both tasks. the selected phrase probabilities are then used to bias to the transformer decoder to focus more on the answer-aware phrases. in general, pet is simple yet effective as we can leverage the power of pretrained language models for both the phrase selection and the generation tasks. our main contributions are summarized as follows: ## related work early question generation (qg) systems (mostow and @xcite @xcite followed a rule-based approach. this approach, however, suffers from a number of issues, such as poor generalization and high-maintenance costs. with the introduction of large qa datasets such as squad @xcite and hot-potqa @xcite , the neural-based approach has become the mainstream in recent years. in general, these methods formalize qg as a sequence-to-sequence problem @xcite , on which a number of innovations have been made from the following perspectives. ## conclusion and future works this paper presents multifactor, a novel qg method with multi-level content planning. specifically, multifactor consists of a fa-model, which simultaneously select important phrases and generate an answer-aware summary (a full answer), and q-model, which takes the generated full answer into account for question generation. both fa-model and q-model are formalized as our simple yet effective pet. experiments on hotpotqa and squad 1.1 demonstrate the effectiveness of our method. our in-depth analysis shows that there is a lot of room for improvement following this line of work. on one hand, we can improve the full answer generation model. on the other hand, we can enhance the q-model in multifactor either by exploiting multiple generated full answers or reducing the error propagation.