Synthesize, Prompt and Transfer: Zero-shot Conversational Question Generation with Pre-trained Language Model (2023)

Abstract: Conversational question generation aims to generate questions that depend on both context and conversation history. Conventional works utilizing deep learning have shown promising results, but heavily rely on the availability of large-scale annotated conversations. In this paper, we introduce a more realistic and less explored setting, Zero-shot Conversational Question Generation (ZeroCQG), which requires no human-labeled conversations for training. To solve ZeroCQG, we propose a multi-stage knowledge transfer framework, Synthesize, Prompt, and trAnsfer with pRe-Trained lAnguage model (SPARTA) to effectively leverage knowledge from single-turn question generation instances. To validate the zero-shot performance of SPARTA, we conduct extensive experiments on three conversational datasets: CoQA, QuAC, and DoQA by transferring knowledge from three single-turn datasets: MS MARCO, NewsQA, and SQuAD. The experimental results demonstrate the superior performance of our method. Specifically, SPARTA has achieved 14.81 BLEU-4 (88.2% absolute improvement compared to T5) in CoQA with knowledge transferred from SQuAD.

## introduction question generation (qg) aims to automatically generate questions from the given context and answer. it plays a vital role in knowledge testing @xcite @xcite and information seeking @xcite @xcite by creating quiz questions and spanning question suggestions, respectively. most existing qg research has usually focused on generating single-turn questions, which are formalized as independent interactions @xcite @xcite . however, it is a more natural way to achieve complex information need recent conversational qg models @xcite @xcite which utilize a separate neural encoder to handle the conversation history, have achieved great performance on coqa dataset @xcite . however, these deep models rely heavily on large-scale annotated conversations which provides the dependency between conversation history and the follow-up question. as shown in in this paper, we propose a more realistic and less explored setting, zero-shot conversational question generation (zerocqg), which requires no human-labeled conversational datasets for training. to solve zerocqg, we propose to transfer knowledge from single-turn qg instances and the pre-trained language model (lm). the relation of question to context and answer plays an important role in both single-turn and conversational qg, while single-turn qg instances are often abundant and easier to obtain. however, there is still a significant domain gap between the two qg tasks due to the lack of conversation history in single-turn qg. more recently, pre-trained lms brings remarkable performance improvement on the task of conversational qg @xcite due to their massive amounts of linguistic knowledge and powerful contextual representation capabilities @xcite . however, the different input and output paradigms will also increase the domain gap between the objective of pre-trained lm and the conversational qg. to address these issues, we propose a multi-stage knowledge transfer framework, sythesize, prompt and transfer with pre-trained language model (sparta) to effectively leverage knowledge from single-turn qg instances. (1) synthesize. we synthesize conversation for each single-turn qg instance to alleviate the domain gap between singleturn and conversational qg tasks. specifically, we first retrieve question-answer pairs with similar contextual contents and sequential dependencies from the whole single-turn qg dataset to stimulate history for each single-turn qg instance. then, we incorporate anaphora characteristics into the singleturn question by replacing entity co-occurring in both the question and the simulated history with co-referenced pronouns. (2) prompt. we propose conversation prompting to alleviate the domain gap between the objective of pre-trained lm and conversational qg. specifically, this prompting method reformulates the conversational qg as a masked question-filling task similar to t5 @xcite where the input and output are organized by prompt templates with semantic prefixes to better steer the expressive power of pre-trained lm. (3) transfer. we fine-tune pre-trained lm on the synthetic dataset with conversation prompting. then, the fine-tuned pre-trained lm with the same conversation prompting are directly applied for inference of conversational qg without using any annotated conversations for training. to validate the zero-shot performance of our proposed sparta, we conduct extensive experiments on three conversational datasets: coqa @xcite , quac @xcite and @xcite ) by transferring knowl-edge from three single-turn datasets: ms marco @xcite , newsqa @xcite and squad @xcite based on different pre-trained lms: t5 @xcite , bart @xcite and pegasus @xcite . the experimental results demonstrate that our proposed sparta significantly improves the performance of zerocqg on most transfer settings. for example, sparta (t5) achieves 14.81 bleu-4 (88.2% absolute improvement compared to t5) in coqa with knowledge transferred from squad, we further conduct extensive ablation studies and discussions to explore the effectiveness of each component of the proposed sparta. we summarize our main contributions as follows: ## problem definition in this section, we first introduce the definition of conversational qg task. given a context c u , a conversation history h u @xmath0, a u 1 ), . . . , (q u t-1 , a u t-1 )}, a answer a u t , the conversational qg task aims to generate a followup question q u t at t-th turn: in which the generated question should be coherent with the conversation history and be conversational. furthermore, the task of zerocqg is defined to generate conversational questions without using any human-labeled conversations. ## methodology in this section, we introduce the proposed sparta which mainly contains three stages: synthesize, prompt, and transfer. the overall framework is illustrated in figure1. ## conclusion in this paper, we introduce a novel task setting, named zerocqg, which requires no humanlabeled conversations for training. to solve ze-rocqg, we propose a multi-stage knowledge transfer framework sparta. specifically, sparta synthesizes conversations for each single-turn qg instance to alleviate the domain gap between the two qg tasks. besides, sparta leverage conversation prompting to reformulate conversational qg into a masked question-filling task similar to t5 to alleviate the domain gap between the objective of pre-trained lm and conversational qg. extensive experiments conducted on the knowledge transfer from three single-turn qg datasets: ms marco, newsqa, and squad to three conversational qg datasets: coqa, quac, and doqa demonstrate the superior performance of our method.