Synthesize, Prompt and Transfer: Zero-shot Conversational Question Generation with Pre-trained Language Model
Year: 2023

Abstract:
Conversational question generation aims to generate questions that depend on both context and conversation history. Conventional works utilizing deep learning have shown promising results, but heavily rely on the availability of large-scale annotated conversations. In this paper, we introduce a more realistic and less explored setting, Zero-shot Conversational Question Generation (ZeroCQG), which requires no human-labeled conversations for training. To solve ZeroCQG, we propose a multi-stage knowledge transfer framework, Synthesize, Prompt and trAnsfer with pReTrained lAnguage model (SPARTA) to effectively leverage knowledge from single-turn question generation instances. To validate the zero-shot performance of SPARTA, we conduct extensive experiments on three conversational datasets: CoQA, QuAC, and DoQA by transferring knowledge from three single-turn datasets: MS MARCO, NewsQA, and SQuAD. The experimental results demonstrate the superior performance of our method. Specifically, SPARTA has achieved 14.81 BLEU-4 (88.2% absolute improvement compared to T5) in CoQA with knowledge transferred from SQuAD.

heading: 1 Introduction
text: Question Generation (QG) aims to automatically generate questions from the given context and answer. It plays a vital role in knowledge testing (Heilman and Smith, 2010; Lindberg et al., 2013; Ghanem et al., 2022) and information seeking (Shum et al., 2018; Rosset et al., 2020; Zamani et al., 2020) by creating quiz questions and spanning question suggestions, respectively. Most existing QG research has usually focused on generating single-turn questions, which are formalized as independent interactions (Zhou et al., 2017; Zhao et al., 2018; Tuan et al., 2020). However, it is a more natural way to achieve complex information need
∗ Corresponding author
through conversations involving a series of interconnected questions (Reddy et al., 2019). Different from single-turn QG, the task of conversational QG (Gao et al., 2019) aims to generate questions which depend on both context and conversation history.
Recent conversational QG models (Gao et al., 2019; Pan et al., 2019; Wang et al., 2022b) which utilize a separate neural encoder to handle the conversation history, have achieved great performance on CoQA dataset (Reddy et al., 2019). However, these deep models rely heavily on large-scale annotated conversations which provides the dependency between conversation history and the follow-up question. As shown in Table 1, we cannot infer who the he in the question is referring to without taking into account the conversation history. Therefore, it is also impossible to generate a conversational question with a referential phenomenon to the history, e.g., Friedrich and he.
In this paper, we propose a more realistic and less explored setting, Zero-shot Conversational Question Generation (ZeroCQG), which requires no human-labeled conversational datasets for training. To solve ZeroCQG, we propose to transfer knowledge from single-turn QG instances and the
8989
pre-trained Language Model (LM). The relation of question to context and answer plays an important role in both single-turn and conversational QG, while single-turn QG instances are often abundant and easier to obtain. However, there is still a significant domain gap between the two QG tasks due to the lack of conversation history in single-turn QG. More recently, pre-trained LMs brings remarkable performance improvement on the task of conversational QG (Do et al., 2022; Fei et al., 2022) due to their massive amounts of linguistic knowledge and powerful contextual representation capabilities (Li et al., 2021). However, the different input and output paradigms will also increase the domain gap between the objective of pre-trained LM and the conversational QG.
To address these issues, we propose a multi-stage knowledge transfer framework, Sythesize, Prompt and trAnsfer with pRe-Trained lAnguage model (SPARTA) to effectively leverage knowledge from single-turn QG instances. (1) Synthesize. We synthesize conversation for each single-turn QG instance to alleviate the domain gap between singleturn and conversational QG tasks. Specifically, we first retrieve question-answer pairs with similar contextual contents and sequential dependencies from the whole single-turn QG dataset to stimulate history for each single-turn QG instance. Then, we incorporate anaphora characteristics into the singleturn question by replacing entity co-occurring in both the question and the simulated history with co-referenced pronouns. (2) Prompt. We propose conversation prompting to alleviate the domain gap between the objective of pre-trained LM and conversational QG. Specifically, this prompting method reformulates the conversational QG as a masked question-filling task similar to T5 (Raffel et al., 2020) where the input and output are organized by prompt templates with semantic prefixes to better steer the expressive power of pre-trained LM. (3) Transfer. We fine-tune pre-trained LM on the synthetic dataset with conversation prompting. Then, the fine-tuned pre-trained LM with the same conversation prompting are directly applied for inference of conversational QG without using any annotated conversations for training.
To validate the zero-shot performance of our proposed SPARTA, we conduct extensive experiments on three conversational datasets: CoQA (Reddy et al., 2019), QuAC (Choi et al., 2018) and DoQA (Campos et al., 2020) by transferring knowl-
edge from three single-turn datasets: MS MARCO (Nguyen et al., 2016), NewsQA (Trischler et al., 2017) and SQuAD (Rajpurkar et al., 2016) based on different pre-trained LMs: T5 (Raffel et al., 2020), BART (Lewis et al., 2020) and PEGASUS (Zhang et al., 2020). The experimental results demonstrate that our proposed SPARTA significantly improves the performance of ZeroCQG on most transfer settings. For example, SPARTA (T5) achieves 14.81 BLEU-4 (88.2% absolute improvement compared to T5) in CoQA with knowledge transferred from SQuAD, We further conduct extensive ablation studies and discussions to explore the effectiveness of each component of the proposed SPARTA.
We summarize our main contributions as follows:
• We introduce a novel task setting, ZeroCQG, which requires no human-labeled conversations for training.
• We propose a multi-stage knowledge transfer framework, SPARTA, which effectively leverages knowledge from single-turn QG instances and pre-trained LM for ZeroCQG.
• We have conducted extensive experiments to demonstrate the superior performance of SPARTA in most transfer settings.

heading: 2 Problem Definition
text: In this section, we first introduce the definition of conversational QG task. Given a context cu, a conversation history hut = {(qu1 , au1), . . . , (qut−1, aut−1)}, a answer aut , the conversational QG task aims to generate a followup question qut at t-th turn:
qut = argmax q P (q|cu, hut , aut ) (1)
in which the generated question should be coherent with the conversation history and be conversational.
Furthermore, the task of ZeroCQG is defined to generate conversational questions without using any human-labeled conversations.

heading: 3 Methodology
text: In this section, we introduce the proposed SPARTA which mainly contains three stages: synthesize, prompt, and transfer. The overall framework is illustrated in Figure1.

heading: 3.1 Conversation Synthesis
text: To alleviate the domain gap between single-turn and conversational QG, we synthesize conversation for each single-turn QG instance.

heading: 3.1.1 History Retrieval
text: History is the most differentiating aspect between single-turn and conversational QG. We retrieve question-answer pairs from the whole dataset to simulate the history for each single-turn QG instance. Specifically, we first retrieve questionanswer pairs with similar contextual content to the context cs of the single-turn question qs as candidates. The similarity score of question-answer pairs is calculated through the dot product between the TF-IDF weighted bag-of-word vectors of the corresponding contextual content. Therefore, the retrieved questions are likely to be answerable given the context cs of the query question qs. Notably, for examples in datasets like SQuAD and NewsQA, we can directly adopt the multiple question-answer pairs corresponding to the same context as simulated history candidates of each other.
Then, we rank the question-answer pairs in the candidate set according to their relevance to qs. Specifically, we leverage the Next Sentence Prediction (NSP) based on the pre-trained BERT (Devlin et al., 2019) to capture the intrinsic sequential dependencies between question pairs. We take the concatenation of the candidate history question q and the query question qs, like “[CLS] q [SEP] qs”, as input to NSP, and obtain the probability that qs can be semantically inferred from q, with label isNext. Then, we select the highest t − 1 question-answer pairs to stimulate conversation history according to this probability. The questionanswer pair with higher probability is closer to qs in the synthesized conversation. Therefore, for each single-turn QG instance (cs, as, qs) ∈ Ds, we can obtain a ranked list of question-answer pairs as the simulated history hs = {(qi, ai)}t−1i=1.

heading: 3.1.2 Anaphora Construction
text: Anaphora is the most common characteristic in conversation systems (Reddy et al., 2019). To incorporate anaphora into the single-turn question and the simulated history, we replace co-occurring entities
with co-referenced pronouns. Specifically, we first concatenate the context cs, the simulated history hs and the question qs into one long text. Then, we employ a pre-trained document-level co-reference resolution model, SpanBERT (Joshi et al., 2020), to cluster mentions in the long text which refer to the same real-world entities. Finally, we transform question qs into qs ′ by replacing the co-occurring entities appearing in both qs and hs with pronouns in the same mention cluster, e.g. Beyoncé and she.
Overall, we can synthesize a conversational QG dataset Ds ′ = {(cs, hs, as, qs′)} with simulated history hs and transformed question qs ′ .

heading: 3.2 Conversation Prompting
text: To alleviate the domain gap between the objective of pre-training LM and the conversational QG task, we reformulate the objective of conversational QG as a masked question-filling task. Specifically, the input and output prompt are detailed as follows: Input Prompt For the conversation history, the template H concatenates h = {(qi, ai)}t−1i=1 into a text sequence where the components are identified by semantic prefixes “question:” and “answer:” respectively, rather than newly introduced tokens. For the masked question, multiple consecutive prompt tokens P(m) = [[MASK]1, · · · , [MASK]m] are replaced in the corresponding position of the conversation after the history, where m is the length of question prompt tokens and each prompt token [MASK]i has trainable parameters equal to the size of embedding vector. Finally, the input prompt is composed of context c, history template H(h), answer a, and question prompts P(m), formalized as I(c,H(h), a,P(m)) with additional semantic prefixes “conversation:” and “context:”. Output Prompt The same question prompt tokens P(m) used in the input are prepended before the target question q as the model output, formalized as O(q,P(m)). A longer sequence of question
prompt tokens means more trainable parameters, and therefore more expressive power to steer pretrained LMs to capture the semantic representation of the question prompt in the corresponding position of the input and provide direct guidance for the generation of output question.

heading: 3.3 Knowledge Transfer
text: SPARTA transfers knowledge from single-turn QG to conversational QG based on pre-trained LM. The training and inference is detailed as follows: Training Our model is continuously trained based on the pre-trained LM as an intermediate task (Pruksachatkun et al., 2020) on the synthesized dataset Ds ′ . Specifically, we leverage conversation prompting to transform each instance (cs, hs, as, qs ′ ) ∈ Ds′ as instantiated input Xs = I(cs,H(hs), as,P(m)) and output Y s = O(qs′ ,P(m)) for training. The model with parameter θ is optimized with negative entropy loss:
L = − lY s∑
i=1
logPθ(Y s i |Xs, Y s<i) (2)
where lY s = m + lq and lq refer to the length of output Y s and question qs ′ respectively. Inference We directly use the fine-tuned model with parameter θ for inference on Du. Specifically, we use the same conversation prompting to transform the input of conversational QG as Xu = I(cu,H(hut ), aut ,P(m)). Then, the output is generated as:
Y u = argmax Y Pθ(Y |Xu) (3)
By removing prompt tokens P(m) from Y u, we can obtain the generated conversational question.

heading: 4 Experiment

heading: 4.1 Datasets
text: We use three single-turn datasets as the source datasets: MS MARCO (Nguyen et al., 2016),
NewsQA (Trischler et al., 2017), and SQuAD (Rajpurkar et al., 2016) and three conversational datasets as the target datasets: CoQA (Reddy et al., 2019), QuAC (Choi et al., 2018), and DoQA (Campos et al., 2020). The processed dataset statistics are displayed in Table 2. More details of datasets are in Appendix A.1.

heading: 4.2 Baselines
text: As this novel task setting of ZeroCQG has not been explored in previous work, there is no existing method to compare with. Therefore, we used three commonly used encoder-decoder style pretrained LMs: T5 (Raffel et al., 2020), BART (Lewis et al., 2020), and PEGASUS (Zhang et al., 2020), as baselines. More details are in Appendix A.2.

heading: 4.3 Main Results
text: Table 3 presents the zero-shot performance comparison on three conversational datasets. From that, we have the following findings:
(1). SPARTA significantly outperforms baseline models across most transfer settings in terms of various metrics. For example, SPARTA (T5) outperforms T5 by a large margin on the transfer from SQuAD to CoQA obtaining 88.2% absolute improvement in BLEU-4, 16.2% absolute improvement in METEOR, and 30.6% absolute im-
provement in ROUGE-L. When transferring from NewsQA to QuAC, SPARTA achieves an absolute improvement of 2.75, 2.02, and 2.67 in BLEU-4 compared to vanilla PEGASUS, BART, and T5, respectively.
(2). T5 has better zero-shot generalization performance on conversational QG task. We have observed that T5 achieves better results than BART and PEGASUS in most transfer settings. Similarly, SPARTA (T5) also outperforms SPARTA (BART) and SPARTA (PEGASUS). This may be because the span corruption object in T5 is more generable compared to the gap-sentence generation object designed for abstractive summarization in PEGASUS and the corrupted text reconstruction object using denoising auto-encoder in BART.
(3). Short answers are easier to understand and thus lead to better transfer results. As shown in Table 2, the average answer lengths LA of NewsQA, SQuAD, and CoQA are shorter than 5, while those of MS MARCO, QuAC, and DoQA are longer than 17. We can observe that the performance of transferring from NewsQA or SQuAD to CoQA is significantly higher than other transfer settings. Among them, the knowledge transferred from MS MARCO has the worst generalization ability. This is probably because the answers in MS MARCO are human-generated, lengthy, and difficult for ma-
chines to understand, and the instance number is much fewer. Besides, models learned from different single-turn data all perform poorly on DoQA. This may be due to the fact that the question-answer pairs in DoQA are all domain-specific, and the length distribution of the question-answer is quite different from the single-turn datasets.
(4). The closer the average question lengths LQ of the single-turn and conversational datasets are, the better the zero-shot generalization performance will be. As shown in Table 2, compared to SQuAD, LQ in NewsQA is closer to LQ in CoQA and QuAC, but farther away from LQ in DoQA. Similarly, as shown in Table 3, we observe that the baselines trained on NewsQA achieve better performance on CoQA and QuAC than that trained on SQuAD, but worse on DoQA.

heading: 4.4 Ablation Studies
text: We conduct ablation experiments over different variants of the best-performing model SPARTA (T5) to better understand the relative importance of the proposed SPARTA framework. As shown in Table 4, most variants lead to worse performance and yet still outperform the baseline model T5. Conversation Synthesis. When we transfer knowledge from the single-turn dataset without using CS, the performance of our model drops significantly. For example, when transferring knowledge from SQuAD, the BLEU-4 score drops from 14.81, 5.85, and 2.66 to 9.79, 4.12, and 2.04 in CoQA, QuAC and DoQA respectively. This confirms that the dependency on conversation history is important to the conversational QG task. This module alleviates the domain gap between single-turn and conversational QG with simulated history and constructed anaphora, thus improving the transfer result. Anaphora Construction. By turning off the AC module, the BLEU-4 score drops to 2.43, 5.69, and 5.47 in QuAC with knowledge transferred from MS MARCO, NewsQA, and SQuAD respectively. The
same performance decrease phenomenon can also be seen in the other transfer settings. This demonstrates that there is a difference between singleturn and conversational questions. Training on synthetic datasets with constructed anaphora characteristics is able to generate more conversational questions. While the AC module is mainly based on the co-reference resolution model, SpanBERT. The ablation of AC also verifies the effectiveness of the co-reference resolution model in understanding anaphora phenomena in conversation. Conversation Prompting The variant without CP formalizes the input similar to that commonly used in conversational question answering systems (Reddy et al., 2019), i.e. appending the conversation history and target answer before the context as 〈a〉 a1 〈q〉 q1 · · · 〈a〉 at−1 〈q〉 qt−1 〈a〉 at 〈sep〉 c. 〈a〉 and 〈q〉 are special tokens used to identify answers and questions, respectively. c is the context. And the question qt is taken as output without using any prompts to guide the decoding process. We can see that this variant leads to a large decrease in BLEU-4 scores on CoQA and QuAC, but a slight increase on DoQA. This may be because DoQA is a domain-specific FAQ dataset with longer questions and answers, which has a larger domain gap with the source datasets than CoQA and QuAC. This result shows that CP can enhance the zeroshot generalization ability of the pre-trained LM when the domains are relevant but has limitations when the domain gap becomes large.

heading: 4.5 Analysis of Question Ranking Method
text: History selection is an important module in conversational systems (Zaib et al., 2022). As shown in Table 5, we have explored different question ranking algorithms to investigate the effectiveness of retrieved question-answer pairs for conversation synthesis. The observations are as followings:
(1). All of these question ranking methods lead to significant performance gains. We observe that
all these variants achieve higher BLEU-4 scores compared to the variant SPARTA (T5) (-w/o CS) shown in Table 4. This also demonstrates the importance and robustness of the conversation synthesis module in ZeroCQG.
(2). NSP is best suited for retrieving questionanswer pairs to simulate conversation history. We observe that NSP achieves the best or second-best performance in all settings. The pre-training objective of NSP (Devlin et al., 2019) is to predict whether two sentences appear consecutively in a document. Thus, NSP is able to capture the intrinsic sequential dependencies between question pairs.
(3). Explicit word overlap facilitates retrieval of questions that are more likely to appear in the conversation history. We observe that TF-IDF and Levenshtein distance had fewer worst scores compared to Dense Retrieval. This may be because explicit word match relates to the paraphrased nature of a question.

heading: 4.6 Analysis of History Turns
text: We have explored how the different number of history turns affect knowledge transfer. From Figure 2,
we obtain the following observations: (1). When training on single-turn datasets without conversation synthesis (retrieved QA pairs = 0), inference with ground-truth conversation history leads to significant performance degradation. And as the turns of ground-truth history increases, the performance drops more severely.
(2). When training on single-turn datasets with conversation synthesis, inference without groundtruth conversation history will also result in a significant performance drop.
(3). The BLEU-4 score increases up to a threshold (15 for CoQA, 2 for QuAC, and 2 for DoQA) as the number of retrieved question-answer pairs increases in single-turn training, and then a slight performance drop occurs. Larger question-answer pairs mean more relevant evidence, while potentially introducing more noise.
(4). The performance increases up to a threshold (9 for CoQA, 5 for QuAC, and 3 for DoQA) as the turns of ground-truth conversation history increase, followed by a very slight fluctuation. The difference here may be reflected in the average history turns in the CoQA, QuAC, and DoQA datasets shown in Table 2, respectively.

heading: 4.7 Analysis of Prompt Design
text: To evaluate the relative importance of the conversational prompt, we explore several variants as shown in Table 6. The observations are as followings:
(1). We have observed that semantic prefixes are more beneficial than introducing new special tokens. Removing semantics prefixes leads to a performance drop in most cases.
(2). Both QPinput and QPoutput contribute to the overall prompt architecture in most cases, with QPoutput contributing more to the CP than QPinput. Table 6 shows that removing QPoutput leads to a larger and consistent performance drop, while removing QPinput even improves the transfer from MS MARCO to CoQA and QuAC. This may be because the trainable question prompts used in the output are closer to the target question and thus can be better optimized to guide the generation process.
(3). It is better for QPinput and QPoutput to be the same than different. We can observe the prompt variant QPsame achieves higher score compared to QPdiff. This result suggests that using the same question prompts in both input and output will further improve the semantic connections and thus enhance QPoutput guidance on question generation.

heading: 4.8 Analysis of Question Prompt Length
text: To study the effects of question prompt length on knowledge transfer, we train the models with prompt length varying in {0, 1, 10, 20, 30, 40, 50}. Figure 3 shows the BLEU-4 score of the different models plotted as a function of the question prompt length. We can observe the optimal prompt length varies across models and datasets. Especially it shows large fluctuations on the DoQA dataset.
In CoQA and QuAC, the BLEU-4 score of SPARTA (T5) increases as the prompt length increases to a threshold (40 for both CoQA, and 20 for QuAC), and then decreases. Similar trends can also be seen on SPARTA (BART) and SPARTA (PEGASUS). Among them, the optimal prompt length of SPARTA (PEGASUS) is shorter than other models. Longer prompts mean more trainable parameters and therefore improve expressiveness. But it also increases the computational and time overhead of both training and inference.

heading: 5 Related Work
text: Conversational Question Generation. Question Generation (QG) aims to generate natural questions with targeted answers from textual inputs. Early works were mainly rule-based systems (Heilman,
2011), using linguistic rules and hand-crafted templates to transform declarative sentences into interrogative sentences. With the popularity of neural networks, many research works (Du et al., 2017; Zhou et al., 2017; Zhao et al., 2018) adopt the encoder-decoder framework which combines attention (Bahdanau et al., 2015) and pointer (See et al., 2017) mechanisms to deal with the question generation problem in an end-to-end fashion.
More recently, conversational QG which involves multi-turn interactions has attracted increasing attention. (Gao et al., 2019) utilized the multisource encoder-decoder model with coreference alignment to refer back and conversation flow to maintain coherent dialogue transition. (Pan et al., 2019) proposed a reinforced dynamic reasoning network to better understand what has been asked and what to ask next with the reward defined by the quality of answer predicted by a questionanswering model. (Gu et al., 2021) designed a twostage architecture that learns question-answer representations across multiple dialogue turns using flow propagation-based training strategy. (Wang et al., 2022b) proposed to distill knowledge from larger pre-trained LM into a tiny answer-guided network for efficient conversational question generation with fewer parameters and faster inference latency. (Do et al., 2022) utilized the top-p strategy to dynamically select the most relevant sentences and question-answer pairs from context and history respectively. (Ling et al., 2022) proposed a review and transit mechanism to identify question-worthy content for informative question generation in opendomain conversations. However, these models rely heavily on large-scale annotated conversations. As far as we know, this is the first research work to explore conversational question generation in the zero-shot learning setting.
Transfer Learning. Transfer learning focuses on adapting knowledge gained while solving one task to a different but related task (Pan and Yang, 2010). Fine-tuning is a commonly used approach in transfer learning, where a pre-trained model is adapted to a new task. The pre-trained models are typically trained on large-scale datasets, which can be either labeled images, such as ImageNet (Deng et al., 2009), or unlabeled text, such as BooksCorpus (Zhu et al., 2015) and Wikipedia. It has been successfully applied to many domains, such as computer vision and Natural Language Processing (NLP). In NLP, the well-known utilization of static
word embedding (Mikolov et al., 2013; Pennington et al., 2014) and contextualized word embedding (Peters et al., 2018; Devlin et al., 2019), also called pre-trained LMs, in downstream task can also be referred as applications of transfer learning. In addition, prompt learning (Liu et al., 2021) is a new paradigm that can enhance the knowledge transfer capability by refactoring downstream tasks into the forms that are close to the pre-training objectives and thus alleviate the domain gap problem. More related works about zero-shot learning are detailed in Appendix C.

heading: 6 Conclusion
text: In this paper, we introduce a novel task setting, named ZeroCQG, which requires no humanlabeled conversations for training. To solve ZeroCQG, we propose a multi-stage knowledge transfer framework SPARTA. Specifically, SPARTA synthesizes conversations for each single-turn QG instance to alleviate the domain gap between the two QG tasks. Besides, SPARTA leverage conversation prompting to reformulate conversational QG into a masked question-filling task similar to T5 to alleviate the domain gap between the objective of pre-trained LM and conversational QG. Extensive experiments conducted on the knowledge transfer from three single-turn QG datasets: MS MARCO, NewsQA, and SQuAD to three conversational QG datasets: CoQA, QuAC, and DoQA demonstrate the superior performance of our method.

heading: 7 Limitations
text: Although our proposed method achieves promising performance in the novel direction of ZeroCQG, it still has the following limitations: (1) retrieval-based conversation synthesis is limited to predefined question-answer pairs and may introduce repeated question-answer pairs with small differences (discussed in Appendix B.1). Future work may include exploring generative-based approaches to generate new and diverse questionanswer pairs for better conversation synthesis. (2) Existing question transformation only explore one of the most common conversational characteristics, anaphora. However, other different characteristics, such as ellipsis, should also be considered in the future. (3) The conversation prompting has limitations when the domain gap becomes large (discussed in Sec. 4.4). More robust prompt learning should be explored in the future.

heading: Acknowledgments
text: This work was supported by National Key Research and Development Program of China (2022YFC3303600), National Natural Science Foundation of China (62137002, 62293553, 62293554, 61937001, and 62176209), Innovative Research Group of the National Natural Science Foundation of China (61721002), “LENOVO-XJTU” Intelligent Industry Joint Laboratory Project, Natural Science Basic Research Program of Shaanxi (2023-JC-YB-293), the Youth Innovation Team of Shaanxi Universities.

heading: A Details of Experiment Setup

heading: A.1 Datasets
text: Single-turn Datasets: (1) MS MARCO v1.1 (Nguyen et al., 2016) contains 100K questionanswer pairs where questions are sampled from Bing’s search query, answers are generated by human and passages are retrieved from Bing search engine. (2) NewsQA (Trischler et al., 2017) contains 120K question-answer pairs based on over 10K CNN news articles. (3) SQuAD v1.1 (Rajpurkar et al., 2016) contains more than 100K question-answer pairs based on 536 Wikipedia articles. We can observe that each passages in the MS MARCO corresponds to only one question-answer pair, while each article in the NewsQA or SQuAD corresponds to multiple question-answer pairs. Conversational Datasets: (1) CoQA (Reddy et al., 2019) contains 8k conversations about text passages from seven diverse domain. It consists of 127K question-answer pairs almost half of which refer back to conversation history using anaphors. (2) QuAC (Choi et al., 2018) contains 14K simulated information seeking dialogues where each interaction is between a student and a teacher on a hidden wikipedia passage. (3) DoQA (Campos et al., 2020) contains 2,437 real domain-specific information seeking dialogues collected from FAQ sites, such as Stack Exchange. This makes DoQA a challenging dataset with more coherent conversations and less factoid questions.
Our experiments are conducted with the accessible part. In particular, we use the validation set of CoQA and QuAC as the test set. For conversational datasets, we remove the examples with too little information in the answer, such as unknown, yes or no, to avoid the generation of questions full of randomness.

heading: A.2 Baselines
text: BART (Lewis et al., 2020) is pre-trained using a denoising auto-encoding task which aims to recover corrupted documents to the original ones. BART can adapt well on both discriminative and generative tasks. T5 (Raffel et al., 2020) is pre-trained to fill in randomly corrupted text spans. T5 is applicable to varies natural language processing problems by formulating tasks in text-to-text format. PEGASUS (Zhang et al., 2020) is pre-trained to generate the most important sentences extracted
from an unlabeled document. PEGASUS is mainly designed for abstractive summarization.
The baseline pre-trained LMs in this paper are all base sizes. When we employ question prompts on the baseline models, the number of extra parameters is equal to the corresponding hidden size multiplied by the length of question prompt tokens.
A.3 Implementation Details We run our experiments on a GTX 3090 GPU. The model is trained with Adam optimizer (Kingma and Ba, 2015). The learning rate is initialized with 1e-4 and decays linearly. The batch size for each update is set as 8 with accumulation steps as 4. The maximum training epochs is 4. Early stopping is performed based on the BLEU score on the development set evaluated every 1000 training steps. We conduct beam search with the beam width 5. Inputs that exceed the maximum input length 768 will be truncated from right. The maximum and minimal decoding steps are equal to the length of question prompt tokens m plus 15 and 1 respectively. Decoding stops when the maximum decoding step is reached or the special end token is generated. After the generation finish, we will remove all the question prompt and special tokens. This paper conduct extensive experiments and ablation studies over different transfer setting varies at single-turn datasets, conversational datasets and pre-trained language models. Performing multiple runs for each experiment was not feasible due to computational and time constraints. Therefore, we use the same seed (42) for all experiments. The metrics, such as BLEU, ROUGE and METEOR, are calculated with the package released by (Sharma et al., 2017).

heading: A.4 Hyperparameters
text: Table 8 shows the optimal maximum number of retrieved question-answer pairs across different transfer settings. These numbers are selected from {0, 1, 2, 5, 10, 15, 20} This corresponds to the main results shown in Table 3.

heading: B Supplementary of Experiment Results

heading: B.1 Case Studies
text: Synthesized Conversation We present two examples each from MS MARCO, NewsQA and SQuAD, as shown in Table 9, Table 10 and Table 11 respectively. We can observe that the retrieved question-answer pairs are correlated and helpful for answering the original single-turn question. By replacing co-occurring entities with pronouns identified by the co-reference tool, we introduce anaphora between the transformed questions and the simulated conversation history. However, we also find the following limitations.
1. Retrieval-based methods may introduce repeated question-answer pairs with small differences. As shown in the first example in Table 9, many retrieved questions are asking for the same thing “how long to cook chicken legs in oven”. For the first example in Table 10, the same answer “European travel guidebooks” are corresponding to three different but highly correlated question q2, q3 an q4. These highly repetitive question-answer pairs rarely appear in a normal conversation.
2. Existing question transformation methods are not applicable when the context and retrieved question-answer pairs provides no co-reference information. For example, the retrieved questionanswer pairs in Table 9 mentioned “chicken legs” and “kidney” multiple times respectively, but the method cannot transform such examples. Generated Question We present ZeroCQG instances generated from different models with knowledge transferred from MS MARCO, NewsQA and SQuAD respectively. As shown in Table 12, there is a large discrepancy between ground truth questions and the questions generated by different models trained on MS MARCO, but relatively small differences between the questions generated by models trained on NewsQA or SQuAD. As shown in Table 13, our proposed SPARTA allows different pre-trained LMs to correctly generate she to refer to Bouvier, while
the baseline model without SPARTA cannot. As shown in Table 14, although the questions generated by different models all mentioned some relevant content, such as “drunk driving”, “traffic”, “new years eve”, etc., there are still some semantic differences compared to the ground-truth question. This may be because the target answer is a long sentence, rather than a short span, thus requiring more complex comprehension.

heading: C Related Work of Zero-shot Learning
text: Zero-shot learning aims to learn a model that can generalize to new classes or tasks for which no training data is available. Large language models pre-trained with self-supervised objectives on unstructured text data have shown promising zeroshot generalization in a wide variety of downstream tasks with properly designed prompts (Radford et al., 2019; Liu et al., 2021; Wang et al., 2022a). (Zhou et al., 2022) proposed prompt consistency regularization on multiple prompts for a single task with unlabeled data to improve the zero-shot performance. (Shu et al., 2022) proposed test-time prompt tuning to learn adaptive prompts on the fly through a single test sample without requiring taskspecific annotations. (Wei et al., 2022) proposed the instruction tuning to improve zero-shot performance of large language model on unseen tasks through natural language instructions. In this paper, we focus on the zero-shot conversational QG with knowledge transferred from single-turn QG and pre-trained LMs with synthesis and prompt strategies respectively.

heading: Context
text: Directions . 1 Arrange chicken thighs/leg quarters skin side up in a shallow baking dish . 2 Sprinkle with garlic powder . 3 Drizzle about 1/2 teaspoon soy sauce on each piece . 4 Bake at 350 degrees Fahrenheit for 45 minutes to an hour , until the skin is crisp and brown and the meat is ready to fall off the bones .

heading: Synthesized Conversation Retrieved QA Pairs
text: q1: how long do you cook chicken thighs on the stove a1: 18 mins q2: how long to cook whole chicken legs in oven a2: 45 to 50 minutes . q3: how long do i need to cook chicken thighs in the oven a3: 30 minutes . q4: how long to bake frozen chicken thighs in oven a4: 375 degrees . q5: how long to boil chicken legs a5: 15 minutes q6: how long to cook chicken legs in oven a6: 35 to 40 minutes q7: how long to cook chicken thighs in oven a7: 45 mins to an hour . q8: how long to cook a chicken thigh in the oven a8: 1 hour ( or up to a day ) . Preheat oven to 375 degrees . q9: how do you prepare chicken legs to bake and how long at what temp . a9: Preheat the oven to 400°F . Bake the chicken , uncovered , for 35 to 40 minutes or until the chicken is no longer pink inside . You can bake the chicken legs in a 375°F . oven , if desired . Increase the baking time to 45 to 50 minutes. 35 to 40 minutes . 400°F . Q10: how long to cook chicken legs in the oven A10: 35 to 40 minutes or until the chicken is no longer pink inside . Question: how long cooking chicken legs in the big easy Answer: 45 minutes to an hour

heading: Context
text: Diabetic kidney disease , or diabetic nephropathy , is a complication of type 1 or type 2 diabetes caused by damage to the kidneys ’ delicate filtering system . Your kidneys contain millions of tiny blood vessel clusters ( glomeruli ) that filter waste from your blood .

heading: Synthesized Conversation Retrieved QA Pairs
text: q1: what is diabetic peripheral neuropathy a1: Peripheral neuropathy is nerve damage caused by chronically high blood sugar and diabetes , it leads to numbness , loss of sensation , and sometimes pain in your feet , legs , or hands , it is the most common complication of diabetes . q2: what is the primary function of nephrons in the kidney a2: Filtering the blood is the primary function of the kidney . q3: what is diabetic retinopathy a3: Diabetic retinopathy is the result of damage caused by diabetes to the small blood vessels located in the retina . q4: what hormones do the kidneys produce and what are their function a4: The kidneys remove waste products and excess water from the body and so help to regulate blood pressure . q5: what is involved in a kidney scan a5: 1 . Assessment of the blood flow through the kidneys . 2 See how a transplanted kidney is working . 3 Check the extent of kidney damage . 4 Find an obstruction in the kidney or ureter 5 Find growths in the kidneys q6: what is polycystic kidney disease a6: Polycystic kidney disease ( PKD ) is an inherited disorder in which clusters of cysts develop primarily within your kidneys . Cysts are noncancerous round sacs containing water-like fluid . q7: what causes kidney and liver failure in dogs a7: Bacteria associated with advanced dental disease enter the blood stream and invades multiple organs , causing irreversible damage to the heart , liver and kidneys . q8: what is polycystic kidney disease symptoms a8: 1 High blood pressure . 2 Back or side pain . 3 Headache . 4 Increase in the size of your abdomen.5 Blood in your urine . 6 Frequent urination . 7 Kidney stones . 8 Kidney failure.9 Urinary tract or kidney infections . q9: what are kidneys made of a9: The kidney is made of a majority of cells called nephrons . q10: what is a kidneys function a10: To remove waste products and excess fluid from the body . Question: what is a diabetic kidney Answer: A complication of type 1 or type 2 diabetes caused by damage to the kidneys ’ delicate filtering system .
Table 9: Our synthesized conversation examples from MS MARCO dataset. The original single-turn questionanswer pair can be treated as turn 11 of the synthesized conversation.

heading: Context
text: You ŕe all alone , surrounded by dank mist and the realization that it was these monks who kept literacy alive in Europe . To give you an idea of their importance , Charlemagne , who ruled much of Europe in the year 800 , imported Irish monks to be his scribes . Rounding Slea Head , the point in Europe closest to America , the rugged coastline offers smashing views of deadly black-rock cliffs and the distant Blasket Islands . The crashing surf races in like white horses , while longhaired sheep graze peacefully on the green hillside . Study the highest fields , untouched since the planting of 1845 , when the potatoes never matured and rotted in the ground . The great famine of that year , through starvation or emigration , nearly halved Ireland ś population . Because its endearing people have endured so much , Ireland is called " The Terrible Beauty . " Take your time at the Gallaras Oratory , circa A.D. 800 , the sightseeing highlight of your peninsula tour . One of Ireland ś best-preserved early Christian churches , its shape is reminiscent of an upturned boat . Its watertight dry-stone walls have sheltered travelers and pilgrims for 1,200 years . From the Oratory , continue up the rugged one-lane road to the crest of the hill and then coast back to Dingle Town – hungry , thirsty , and ready for a pint . Rick Steves writes European travel guidebooks and hosts travel shows on public television and public radio . E-mail him at rick@ricksteves.com, or write to him c/o P.O. Box 2009 , Edmonds , Wash. 98020 . Synthesized Conversation

heading: Retrieved QA Pairs
text: q1: What stations do his TV series air on ? a1: public television q2: What kind of books does Rick Steve write ? a2: European travel guidebooks q3: What types of books does Rick Steves write ? a3: European travel guidebooks q4: What does Rick Steves write ? a4: European travel guidebooks q5: What does Rick Steves ’ company do ? a5: writes European travel guidebooks and hosts travel shows on public television and public radio Transformed Question: Where does his show air ? Answer: public television

heading: Context
text: -LRB- CNN -RRB- – Author Arthur C. Clarke , whose science fiction and non-fiction works ranged from the script for " 2001 : A Space Odyssey " to an early proposal for communications satellites , has died at age 90 , associates have said . Visionary author Arthur C. Clarke had fans around the world . Clarke had been wheelchair-bound for several years with complications stemming from a youthful bout with polio and had suffered from back trouble recently , said Scott Chase , the secretary of the nonprofit Arthur C. Clarke Foundation . He died early Wednesday – Tuesday afternoon ET – at a hospital in Colombo , Sri Lanka , where he had lived since the 1950s , Chase said . " He had been taken to hospital in what we had hoped was one of the slings and arrows of being 90 , but in this case it was his final visit , " he said . In a videotaped 90th birthday message to fans , Clarke said he still hoped to see some sign of intelligent life beyond Earth , more work on alternatives to fossil fuels – and " closer to home , " an end to the 25-year civil war in Sri Lanka between the government and ethnic Tamil separatists . " I dearly wish to see lasting peace established in Sri Lanka as soon as possible , " he said . " But I ḿ aware that peace can not just be wished – it requires a great deal of hard work , courage and persistence . " Clarke and director Stanley Kubrick shared an Academy Award nomination for best adapted screenplay for " 2001 . " The film grew out of Clarke ś 1951 short story , " The Sentinel , " about an alien transmitter left on the moon that ceases broadcasting when humans arrive . As a Royal Air Force officer during World War II , Clarke took part in the early development of radar . In a paper written for the radio journal " Wireless World " in 1945 , he suggested that artificial satellites hovering in a fixed spot above Earth could be used to relay telecommunications signals across the globe . He is widely credited with introducing the idea of the communications satellite , the first of which were launched in the early 1960s . But he never patented the idea , prompting a 1965 essay that he subtitled , " How I Lost a Billion Dollars in My Spare Time . " His best-known works , such as " 2001 " or the 1953 novel " Childhood ś End , " combined the hard science he learned studying physics and mathematics with insights into how future discoveries would change humanity . David Eicher , editor of Astronomy magazine , told CNN that Clarke ś writings were influential in shaping public interest in space exploration during the 1950s and 6́0s . Watch how Clarke stands among sci-fi giants "

heading: Synthesized Conversation Retrieved QA Pairs
text: q1: Arthur C. Clarke dies in Sri Lanka at age 90 , aide says a1: whose science fiction and non-fiction works q2: Who died in Sri Lanka ? a2: Arthur C. Clarke q3: What did he and Stanley Kubrick share ? a3: Academy Award nomination for best adapted screenplay for “ 2001’ q4: Clarked lived in Sri Lanka since when ? a4: the 1950s q5: Where did Arthur C. Clarke die ? a5: Colombo , Sri Lanka Transformed Question: Where did he live ? Answer: Colombo , Sri Lanka
Table 10: Our synthesized conversation examples from NewsQA dataset. The original single-turn question-answer pair can be treated as turn 6 of the synthesized conversation. The co-reference mentions are marked with underline.

heading: Context
text: In December , Beyoncé along with a variety of other celebrities teamed up and produced a video campaign for “ Demand A Plan ” , a bipartisan effort by a group of 950 US mayors and others designed to influence the federal government into rethinking its gun control laws , following the Sandy Hook Elementary School shooting . Beyoncé became an ambassador for the 2012 World Humanitarian Day campaign donating her song “ I Was Here ” and its music video , shot in the UN , to the campaign . In 2013 , it was announced that Beyoncé would work with Salma Hayek and Frida Giannini on a Gucci “ Chime for Change ” campaign that aims to spread female empowerment . The campaign , which aired on February 28 , was set to her new music . A concert for the cause took place on June 1 , 2013 in London and included other acts like Ellie Goulding , Florence and the Machine , and Rita Ora . In advance of the concert , she appeared in a campaign video released on 15 May 2013 , where she , along with Cameron Diaz , John Legend and Kylie Minogue , described inspiration from their mothers , while a number of other artists celebrated personal inspiration from other women , leading to a call for submission of photos of women of viewers ’ inspiration from which a selection was shown at the concert . Beyoncé said about her mother Tina Knowles that her gift was “ finding the best qualities in every human being . ” With help of the crowdfunding platform Catapult , visitors of the concert could choose between several projects promoting education of women and girls . Beyoncé is also taking part in “ Miss a Meal ” , a food-donation campaign , and supporting Goodwill charity through online charity auctions at Charitybuzz that support job creation throughout Europe and the U.S .

heading: Synthesized Conversation Retrieved QA Pairs
text: q1: Beyonce was speaking about whom when she said her gift was “ finding the best qualities in every human being . “ ? a1: her mother q2: Who did Beyoncé work with in 2013 on the Chime for Change campaign ? a2: Salma Hayek and Frida Giannini q3: What is the name of the campaign that Beyoncé and others are involved in that deals with gun control ? a3: Demand A Plan q4: Beyonce is contributing to which food-donation campaign ? a4: Miss a Meal q5: What song did Beyonce contribute to the campaign ? a5: I Was Here Transformed Question: What song did she donate to the 2012 World Humanitarian Day campaign ? Answer: I Was Here

heading: Context
text: New Zealand has a strong hunting culture . The islands making up New Zealand originally had no land mammals apart from bats . However , once Europeans arrived , game animals were introduced by acclimatisation societies to provide New Zealanders with sport and a hunting resource . Deer , pigs , goats , rabbits , hare , tahr and chamois all adapted well to the New Zealand terrain , and with no natural predators , their population exploded . Government agencies view the animals as pests due to their effects on the natural environment and on agricultural production , but hunters view them as a resource .

heading: Synthesized Conversation Retrieved QA Pairs
text: q1: What were the the only land mammal in New Zealand ? a1: bats q2: What was the only land mammal native to New Zealand ? a2: bats q3: Why did the population of pigs and rabbits explode in New Zealand ? a3: no natural predators q4: Game animals were introduced here by whom ? a4: acclimatisation societies q5: Why were game animals introduced by acclimatisation societies ? a5: to provide New Zealanders with sport and a hunting resource Transformed Question: What resulted having no natural predators for them introduced ? Answer: their population exploded
Table 11: Our synthesized conversation examples from SQuAD dataset. The original single-turn question-answer pair can be treated as turn 6 of the synthesized conversation. The co-reference mentions are marked with underline.

heading: Context
text: Kendra and Quinton travel to and from school every day . Kendra lives further from the bus stop than Quinton does , stops every morning at Quinton ’s house to join him to walk to the bus stop . Every afternoon , after school , when walking home from the bus stop they go in for cookies and milk that Quinton ’s mother has ready and waiting for them . Quinton ca n’t eat cheese or cake so they had the same snack every day . They both work together on their homework and when they are done they play together . Kendra always makes sure to leave in time to get home for dinner . She does n’t want to miss story time which was right before bedtime . One morning Kendra walked up to Quinton ’s house , she thought something might be wrong because normally Quinton was waiting outside for her and on this morning he was not to be found . Kendra went up to the door and knocked . She waited and waited and yet no one answered . She saw that Quinton ’s mother ’s car was n’t in their driveway which was weird . She waited for a few bit looking up and down the block and getting worried when Quinton was nowhere to be found . Kendra did n’t want to miss the bus to school and hurried off to make it in time . The bus driver saw that she was upset and that Quinton was not with her that morning . She told him what happened and he said that he was sure that everything would be okay . Kendra got to school , ran to her teacher and told him what happened that morning . The teacher smiled and told her not to worry , Quinton ’s mother had called and he was going to the dentist and would be at school after lunch and that she would see him at the bus stop like normal tomorrow .

heading: Conversation History
text: q1: Where do Quinton and Kendra travel to and from every day ? a1: school q2: What do they do every afternoon after school ? a2: go to Quentin ’s house q3: What does Kendra not want to miss ? a3: story time q4: When is that ? a4: right before bedtime q5: What happened when Kendra knocked on Quinton ’s door ? a5: no one answered q6: What did the bus driver see ? a6: that she was upset
Answer a7: everything would be okay
Ground Truth Question q7: what did he say ?
Generated Question with Knowledge Transferred from MS MARCO
PEGASUS: what happens to quinton when he goes to school BART: what happened to kendra after she got home from school T5: what happened to kendra when quinton was not with her SPARTA (PEGASUS): what happened to quinton SPARTA (BART):what did the bus driver tell kendra that she was missing SPARTA (T5): what did the bus driver see Generated Question with Knowledge Transferred from NewsQA
PEGASUS: what did the bus driver say ? BART: what did the bus driver promise kendra ? T5: what did the bus driver say ? SPARTA (PEGASUS): what did the bus driver say ? SPARTA (BART): what did the bus driver say ? SPARTA (T5): what did the bus driver say ?
Generated Question with Knowledge Transferred from SQuAD
PEGASUS: what did quinton ’s teacher tell him ? BART: what did the bus driver say after kendra told him about T5: what did the bus driver say he was sure of ? SPARTA (PEGASUS): what did the bus driver say ? SPARTA (BART): what did the bus driver say ? SPARTA (T5): what did he say ?
Table 12: An example of generated questions in the CoQA by different models with knowledge transferred from MS MARCO, NewsQA and SQuAD respectively.

heading: Context
text: In the fall of 1947 , Bouvier entered Vassar College in Poughkeepsie , New York . She had wanted to attend Sarah Lawrence College , closer to New York City , but her parents insisted that she choose the more geographically isolated Vassar . Bouvier was an accomplished student who participated in the school ’s art and drama clubs and wrote for its newspaper . Due to her dislike for the college , she did not take an active part in its social life and instead traveled back to Manhattan on the weekends . She had made her society debut in the summer before entering college and became a frequent presence in New York social functions . Hearst columnist Igor Cassini dubbed her the “ debutante of the year ” . Bouvier spent her junior year ( 1949-1950 ) in France – at the University of Grenoble in Grenoble , and at the Sorbonne in Paris – in a study-abroad program through Smith College . Upon returning home , she transferred to George Washington University in Washington , D.C. , graduating with a Bachelor of Arts degree in French literature in 1951 . During the early years of her marriage to John F. Kennedy , she took continuing education classes in American history at Georgetown University in Washington , D.C . While attending George Washington , Bouvier won a twelve-month junior editorship at Vogue magazine ; she had been selected over several hundred other women nationwide . The position entailed working for six months in the magazine ’s New York City office and spending the remaining six months in Paris . Before beginning the job , Bouvier celebrated her college graduation and her sister Lee ’s high school graduation by traveling with her to Europe for the summer . The trip was the subject of her only autobiography , One Special Summer , co-authored with Lee ; it is also the only one of her published works to feature Jacqueline ’s drawings . On her first day at Vogue , the managing editor advised her to quit and go back to Washington . According to biographer Barbara Leaming , the editor was concerned about Bouvier ’s marriage prospects ; she was 22 years of age and was considered too old to be single in her social circles . Bouvier followed the advice , left the job and returned to Washington after only one day of work . Bouvier moved back to Merrywood and was hired as a part-time receptionist at the Washington Times-Herald . A week later , she approached editor Frank Waldrop and requested more challenging work ; she was given the position of “ Inquiring Camera Girl ” , despite Waldrop ’s initial concerns about her competence . The position required her to pose witty questions to individuals chosen at random on the street and take their pictures for publication in the newspaper alongside selected quotations from their responses . In addition to the random “ man on the street ” vignettes , she sometimes sought interviews with people of interest , such as six-year-old Tricia Nixon . Bouvier interviewed Tricia a few days after her father Richard Nixon was elected to the vice presidency in the 1952 election . During this time , Bouvier was also briefly engaged to a young stockbroker , John G. W. Husted , Jr. After only a month of dating , the couple published the announcement in The New York Times in January 1952 . She called off the engagement after three months , because she had found him “ immature and boring ” once she got to know him better .

heading: Conversation History
text: q1: where did she go to College ? a1: Bouvier entered Vassar College in Poughkeepsie , New York .
Answer a2: 1947 ,
Ground Truth Question q2: what year did she go to college ?
Generated Question with Knowledge Transferred from MS MARCO
PEGASUS: when was bouvier born BART: when did jacqueline bouvier go to college T5: when did jacqueline bouvier enter vogue SPARTA (PEGASUS): when did bouvier go to college SPARTA (BART): when did she go to vogue SPARTA (T5): when did she go to college
Generated Question with Knowledge Transferred from NewsQA
PEGASUS: what year did bouvier graduate from george washington university ? BART: when did she work for vogue ? T5: when did bouvier graduate ? SPARTA (PEGASUS): when did she go to college ? SPARTA (BART): when did she go to college ? SPARTA (T5): when did she go to college ?
Generated Question with Knowledge Transferred from SQuAD
PEGASUS: in what year did bouvier enter vassar college ? BART: when did bouvier enter vassar college ? T5: when did bouvier enter vassar college ? SPARTA (PEGASUS): when did she go to vassar college ? SPARTA (BART): when did she enter vassar college ? SPARTA (T5): when did bouvier enter vassar college ?
Table 13: An example of generated questions in the QuAC by different models with knowledge transferred from MS MARCO, NewsQA and SQuAD respectively.

heading: Context
text: It should n’t be any worse than usual - it might even be a bit light ; Larchmont is a ways north of NYC proper , so I would n’t expect significant NYE related backups there . One thing that you should be wary about however , is drunk drivers ! There will probably be more of them on the road than usual that night , so be cautious and alert . ( Similarly , there will probably be an above average number of police along the highway looking to catch said drunk drivers - and they wo n’t bee averse to writing you a citation for any other infraction which they might observe . Drive safely ! )

heading: Conversation History
text: q1: How bad is traffic from Boston to New York City on New Years Eve ? a1: It should n’t be any worse than usual - it might even be a bit light q2: How many hours would it take to go from Boston to Larchmont , NY ? a2: Larchmont is a ways north of NYC proper , so I would n’t expect significant NYE related backups there q3: would traffic be better before or after midnight ? a3: One thing that you should be wary about however , is drunk drivers
Answer a4: There will probably be more of them on the road than usual that night
Ground Truth Question q4: are there a lot of drunk drivers on new years eve ?
Generated Question with Knowledge Transferred from MS MARCO
PEGASUS: how many drunk drivers in nyc BART: drinking drivers in nyc T5: how many drunk drivers in larchmont nyc SPARTA (PEGASUS): how bad is traffic in larchmont ny on new years eve SPARTA (BART): would traffic be bad at larchmont nyc SPARTA (T5): would traffic be worse in larchmont nyc on
Generated Question with Knowledge Transferred from NewsQA
PEGASUS: what should you be cautious about ? BART: what should you be careful about ? T5: what should you be cautious of ? SPARTA (PEGASUS): will traffic be better before or after midnight ? SPARTA (BART): what is the problem with drunk drivers ? SPARTA (T5): would the traffic be better before or after midnight ?
Generated Question with Knowledge Transferred from SQuAD
PEGASUS: what is one thing that you should be wary of ? BART: are there more drunk drivers on the road ? T5: why should you be cautious and alert ? SPARTA (PEGASUS): would there be more or less of them on the road ? SPARTA (BART): are there more drunk drivers on the highways in new york SPARTA (T5): would traffic be better or worse on new years eve ?
Table 14: An example of generated questions in the DoQA by different models with knowledge transferred from MS MARCO, NewsQA and SQuAD respectively.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
Section 7
3 A2. Did you discuss any potential risks of your work? Section 4.7, Section 7
3 A3. Do the abstract and introduction summarize the paper’s main claims? Section 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 7 Did you use or create scientific artifacts? Left blank.
B1. Did you cite the creators of artifacts you used? No response.
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? No response.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? No response.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? No response.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? No response.
B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. No response.
C 3 Did you run computational experiments? Section 4, Appendix B
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Appendix A.2, A.3
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Section 4.5, Section 4.7, Appendix A.4
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Appendix A.3
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Section 4.1, Appendix A.3
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: " His best-known works , such as " 2001 " or the 1953 novel " Childhood ś End , " combined the hard science he learned studying physics and mathematics with insights into how future discoveries would change humanity. David Eicher , editor of Astronomy magazine , told CNN that Clarke ś writings were influential in shaping public interest in space exploration during the 1950s and 6́0s. Watch how Clarke stands among sci-fi giants "