q2d: Turning Questions into Dialogs to Teach Models How to Search
Authors: Yonatan Bitton
Shlomi Cohen-Ganor
Ido Hakimi
Yoad Lewenberg
Roee Aharoni
Enav Weinreb
Venue: EMNLP 2023 Main (2023)
Keywords: Large language models
dialog generation
query generation
external search API
synthetic training data
QReCC dataset
information-seeking dialogs
q2d
data generation pipeline
synthetic dialogs
human-generated dialogs
grounded responses
anaphora
outdated information
hallucinations
factually consistent responses
multi-hop QA
PaLM
TL;DR: q2d, an automatic data generation pipeline that transforms questions into information-seeking dialogs using large language models, effectively training dialog and query generation models, producing dialogs that are natural, factual, and accurate

Abstract: One of the exciting capabilities of recent language models for dialog is their ability to independently search for relevant information to ground a given dialog response. However, obtaining training data to teach models how to issue search queries is time and resource consuming.
In this work, we propose $q2d$: an automatic data generation pipeline that generates information-seeking dialogs from questions. We prompt a large language model (PaLM) to create conversational versions of question answering datasets, and use it to improve query generation models that communicate with external search APIs to ground dialog responses. Unlike previous approaches which relied on human written dialogs with search queries, our method allows to automatically generate query-based grounded dialogs with better control and scale.
Our experiments demonstrate that: (1) For query generation on the QReCC dataset, models trained on our synthetically-generated data achieve 90%-97% of the performance of models trained on the human-generated data; (2) We can successfully generate data for training dialog models in new domains without any existing dialog data as demonstrated on the multi-hop MuSiQue and Bamboogle QA datasets. (3) We perform a thorough analysis of the generated dialogs showing that humans find them of high quality and struggle to distinguish them from human-written dialogs.

q2d: Turning Questions into Dialogs to Teach Models How to Search
Yonatan Bitton Shlomi Cohen-Ganor- Ido Hakimi*
Yoad Lewenberg Roee Aharonit Enav Weinreb
The Hebrew University of Jerusalem; Google Research
@google.com
Abstract
when discussing entities and asking related ques- tions with anaphora. To do this, these models use a query generation component that is trained on seciloo-tt--seach--ge
One of the exciting capabilities of recent lan- guage models for dialog is their ability to in- dependently search for relevant information to ground given dialog response. However, ob- taining training data to teach models how to issue search queries time and resource con- suming. In this work, we propose q2d: au- tomatic data generation pipeline that generates cermmaaion-seeking dialogs from questions. We prompt large language model (PaLM) to create conversational versions question an- swering datasets, and use it to improve query generation models that communicate with ex- ternal search APIs ground dialog responses. Unlike previous approaches which relied on human written dialogs with search queries, our method allows automatically generate query- based grounded dialogs with better control and scale. Our experiments demonstrate that: (1) For query generation on the QReCC dataset, models trained on our synthetically geneerrae data achieve 90%-97% of the performance of models trained on the human-generated data; (2) We can successfully generate data for train- ing dialog models in new domains without any existing dialog data as demonstrated on the multi-hop hop MuSiQue and Bamboogle QA datasets. (3) We perform a thorough analysis of the generated dialogs showing that humans find them high quality and struggle to distin- guish them from human-writte dialogs."
datasets. When the model is triggered with a dialog turn that requires search, it generates query that is used to obtain search result, which is then used to generate grounded response. This allows the model provide rele- vant information about the world in its responses user queries. For example, a model trained in 2021 should be able to provide a factual response to the question "How old Joe Biden?" even 2023. In a conversation, one might discuss an entity (e.g. "Joe Biden") and later ask related question (e.g. "How old is he?") with anaphora. In order to pro- vide reliable and consistent answers, it is necessary to generate decontextualized query (e.g.. "How old is Joe Biden") for search engine.
Using APIs also decouples language and reason- ing from knowledge (Borgeaud et al., 2021; Parisi et al., 2022), which can help prevent errors caused by outdated information being stored in the model's 's parameters. For example, if a model trained at the end of 2021 is asked "How old is the current presi- dent?", it may produce the incorrect query "How old is Donald Trump" if its parameters are outdated or if it provides factually- -nnossssinnnn responses (a.k. "hallucinations".
Query generation datasets have been created us- ing human annotators, limiting them in scale, con- trol, and quality (Komeili et al., 2021). As a result, when a new domain is introduced, a significant amount of human effort is required to create a new query generation dataset for that domain (Gupta et al., 2021; Dziri et al., 2021). The fact that lan- guage models often generate hallucinations (Zhao et al., 2020; Maynez et al., 2020; Lee et al., 2018). especially new domains or dialogs that differ from the training data (Nic et al., 2020; Honovich et al., 2021, 2022a), highlights the need for more
1 Introduction
Recent dialog generation models, such as LaMDA (Thoppilan et al., 2022), BlenderBot3 (Shuster al., 2022b) and Sparrow (Glaese et al., 2022) use an external search API to generate grounded and factually accurate responses (Parisi et al., 2022). This is important for providing reliable and con- sistent answers (Shuster et al., 2022a), especially
Work done during an internship Google Research. We publicly relcase all prompts and will release all gener- ated datasets.

---

Question (c)
What was the name of the trek next generation?
To assess whether the automatically generated di- alogs can replace -ennaan-eeerteedd dialogs, we ex- periment with QReCC NQ (Anantha et al., 2020), a numan-curated dialog dataset. We generate a train- ing set that is the same size as the original dataset, but with synthetic dialogue, and use it to train query generation model. The resulting model obtains 90%-95% of the performance of models trained on the human-generated training data, using the same metrics used evaluate QReCC (Anantha et al., 2020).
LM fow-shot
Question: (q)
Transform
Dialog:
"Witte dialog
A: (turnt)
android Star Trek: The Next Generation was named Data.
betweeen en
B: (tur22)
automated assistant and user...
A: (turn3)
Who played the android?
2. Dialogues
Brent Spiner played Data on Star Trek: The Next Generation
Arrady the enswer
Automatic
Who played Ardre?
Search Query; Who played Ardra star trek the next generation?
Query Generation Dataset: Dialog - Search Query
$
Search
Other than training query generation models, our approach is also useful for training the dialog gen- eration models themselves when no dialog data is available for a new domain. We demonstrate that the domain of multi-hop question answering, where we first show that existing dialog models struggle to perform well on a domain-specific chal- lenge set. We then generate synthetic dialog data from the MuSiQue (Trivedi et al., 2021) multi-hop QA dataset, and show that training a dialog model on this data improves performance.
Figure 1: Left: Our q2d method starts from exist- ing query or question and prompt few-shot language model transform into a dialog. We filter out cases where the intent of the generated dialogue differs from the intent of the initial query and apply additional filters. Right: We take question from the QReCC dataset (sur- rounded by rectangle) and generate an information- seeking dialog with g2d. By starting with query and generating dialog. create (dialogue query] dataset, which is used to train and evaluate query gen- eration models. which communicate with an external search API generate factual responses.
We provide thorough analysis of the quality of the generated datasets, demonstrating that they (a) looks natural, and humans struggle to distinguish the synthetic dialogs from natural; (b) factual: gen- crated and human- annotated answers perform simi- larly in query generation; (c) correct: dataset labels are accurate, and strict filtering improves results. To conclude, our main contributions are:
effective query generation datasets that will foster more grounded and factually consistent models.
In this work, we propose data generation pipeline to improve grounded dialog models with access search engines. To create dialog-to- search-queries dataset for training the query gen- eration component in such models, we reverse the process, starting from a search query and generat- ing an information-seeking dialog that corresponds to that query. Our automatic pipeline, shown in Fig- ure 1, begins with search query or question, and prompts large language model (PaLM: Chowdh- ery et al., 2022) generate conversational dialog that conveys the information need implied by the given query. For example in Figure 1, we take the question "Who played Ardra on star trek the next generation?" from the Natural Questions dataset (Kwiatkowssi et al., 2019) and generate a dialog with similar intent: the correct answer to the orig- inal question ("Marta DuBois") is also correct response to the generated dialog. This process al- lows us to leverage existing question-answering datasets, which are widely available for different domains, and extend them by generating dialogs that preserve the original information need while controlling the dialog domain and style.
We introduce q2d: an automatic method to generate information-seeking dialogs from questions using large language models.
2. We show that our method is beneficial for training query generation and dialog genera- tion, including in different domains like multi- hop QA.
3. A thorough analysis showing that the synthet- ically generated dialogs are natural, factual and correct.
4. Publicly releasing the generated datasets and generation prompts, code, and evaluation pro- tocols.
Generating Dialogs from Questions
In this section, we describe our automatic method, called g2d, for generating dialogs from questions, and the properties of datasets produced by this

---

Algorithm 1 Generate Dialogues from Questions
them is left for future work. First, we filter out dialogs whose intent is different from the origi- nal query by measuring the similarity between the query and its reversed version using SBERT simi- larity (sim((,,9) and comparing it to a threshold (Tquery). If the similarity is below the threshold, the generated query is considered to have differ- ent intent and the dialog is filtered. Appendix A. Section A.2 shows several examples of dialogs, original and reversed query and SBERT semantic similarity. Second, we filter out cases where the answer is included the dialog by measuring the n-gram overlap between the dialog and the answer using the Rouge metric (Lin, 2004). If the over- lap is above threshold (Tanswer). the answer is entailed in the dialog and the example is filtered. For example, if the final answer ("Marta DeBois") would have been already written in the dialog for the role of playing Ardra, the final question ("Who played Ardra") would not make sense. Finally, we filter out cases where the last turn of the dialog is similar (>80%) to the original question using SBERT similarity. These cases include situations where no anaphora is required.
input
Few-Shot Model Ms QA Dataset (Q, A),
Examples Queries S = {(tt, d i=1
Examples Dialogues S [(d,, 4)) i=1
Instructions Query I, Instructions Dialogue Iy.
execute
dataset 0 0
for (q. E (Q, do
dialogue t (S
< (M, I,, dialoguc) if filter(dialogue.qqq] a) then dataset.addl (didddddeoggue q, a))
output
Query Generation Dataset: = {(d;, [
method. Our goal is to reduce the effort associ- ated with generating training dataset for training generation, and to improve queer-generation-- based dialog models with a high-quality training dataset. Query generation can start by extracting queries from existing dialogs. However, our approach is unique in that begins with factual queries or ques- tions, allowing us to leverage existing resources. Any question-answering dataset, queries dataset, queries used in popular web search services or dialog model logs can be used with our algorithm. The algorithm is described in Algorithm 1 and consists of three main steps:
In this work, we use PaLM (Chowdhery et al., 2022), a large language model with 540B parame- ters, as the few-shot language model for generating dialogs with a temperature of 0.6. We provide a fully working code with GPT-3 (Brown et al., 2020) for reproducibility. The set of prompts and instruc- tions can be found in Appendix A, Section A.3. the all- For the similarity metric (sim), Transformers, mpnet-base-r2 model from Sentence with a threshold similarity of Tquery 0.999. This threshold justified through human-evaluation and ablation studies for the filtering in Section 5.3.
1. Starting from a query or question from the set Q, we few-shot model My,, specifically PaLM, and instructions to generate dialog given the query. The few-shot prompts can be manually written to adapt to different conversation styles, or sampled from existing dialogs dataset.
3 Replacing Human-Annotated with Auto-Generated Data
2. Using the same few-shot examples reverse, Sa and I,. we generate a query based on the generated dialog. g'
In this section, we evaluate the extent to which our automatically generated dataset can replace the human-annotated dataset. We use the QReCC NQ dataset (Anantha et al., 2020), which contains (di- alog, query) pairs, and automatically generate a dialog from natural questions. This allows us to create an automatically generated train set of the same size, and compare it to the human-ann aanott-d dataset. An example of human- numan-generated dia- log compared an automatically generated dialog is shown in Figure 2. We use the version of the dataset where the intermediate questions are con-
3. Filtering: we filter dialogs with different in- tent, or dialogs where the dialog answer contained in the dialog. We elaborate on the different filters below.
Filtering. In this part we attempt filter (dia- log, query) samples that would not be beneficial for training or testing. We do it three steps, claborated below. We stress that there are many more filtering strategies possible, and exploring

---

Was film The Vikings based on a novel?
SBERT Similarity
Rouge-1 Recall
Search Reselts
Model
Training Dataset
Recall @ 10
Human Annotsted
Auto Gene--eeee
Human Annotated
92.4
88.1
68.5
Where the castle The Vikings film?
what
film The Vikings relessed?
TS
Auto Generated
87.5 (95%)
83.3 (95%)
(90%)
castie
The Vikings, was the roal Fort de is Latte north east Brittany France
The film The Vikings was released in 1968.
Results the human-annotated QReCC NQ test set, experimenting with replacing the human- annotated data with automatically generated data with the q2d method. Bold shows the percentage of perfor- mance for a model trained with auto geoo--eeertted data out of model trained with human-ane annotated data. Training on the automatically generated data achieves 90%-95% of the model trained on the human annotated results.
Table 1:
Who directed the film?
Who directed the film?
The Vikins ge was directed Richard Fleischer
Richard Fleischer directed the film.
Was film based novel?
Was based
Figure 2: An example of human annotated dialogue from QReCC and an automatically generated dialogue produced for the same question.
an open-source T5-3B model (Raffel et al., 2020) in its original form (referred to as 'None'), by fine- tuning it on the natural QReCC training data and contrasting the results with those obtained from training on the auto-generated QReCC dataset. We use batch size of 32, an Adam optimizer, a learn- ing rate of 0.0001, and fine-tune it for 10,000 steps.
textualized, rather than decontextualized. For ex- amplc, the second and third user turns in the figure are contextualized versions of the decontextualized questions "Who directed the film, The Vikings?" and "Was the film The Vikings based novel?".
Results. Results are presented in Table 1, We observe that by replacing human annotated data with auto generated data we were able to reach 90% 95% of the results with a set of the same size using the same model, demonstrating the efficacy of our q2d approach in minimizing annotation labor and producing synthetic training data that is nearly as effective as human-annotated data.
Dataset Generation. To generate our dataset, we use our q2d method as described in Section 2. For the few-shot examples of queries and dialogs (Sy and S2), we sample 15 examples from QReCC that fit within the maximum input sequence length, These examples are available in Appendix A, Sec- tion A.3. For the base questions (Q), we use the Natural Questions (Kwiatkowski et al., 2019) dataset instead of the QReCC NQ questions to re- duce dependence on QReCC. Importantly, all of the questions and dialogs in the natural and automat- ically generated datasets are disjoint. In total, generate 1 13K samples, the same as the QReCC NQ train set. Full prompts, instructions and examples are available in Appendix A, Section A.I.
Extending Query Generation: Multi-Hop QA
This section shows that our method is effective benehmark and training signal that generalizes to -anan-andotated data. also flexible and able to adapt and improve for specific styles of dialog, even without annotated data. It allows us to create dialogs similar to a target domain and provide a fully labeled query-generation dataset. The gen- erated data is useful for training and evaluation, as well as exploring model performance in new scenarios. We demonstrate this using a multi-hop question answering example.
Metrics and Models. Our metries are the same those used in the QReCC dataset, comparing the original and generated queries. These include Rouge-1 Recall (Lin, 2004) for measuring the sim- ilarity between two text unigrams, and SBERT em- bedding semantic similarity for comparing the se- mantic content of two sentences (same metric as in 2 We also use Recall @ 10 to compare the retrieved URLs for the ground-truth query and the generated query. We conduct experiments using
Manual Dialog Construction. We define a chal- lenging test set for multi-hop dialogs by annotating the Bamboogle dataset (Press et al., 2022), which consists of 125 multi-hop human-constructed ques- tions. We create dialogs that ask the same ques- tions, with the user the information seeker and the assistant as the information provider. The as- sistant should help the user obtain the information they are seeking, clarify any questions, and move
"We replaced USE (Cer al., 2018) SBERT MPNet embeddings which are perform better on the STS benchmark (Cer al., 2017) (75 - 88).
'tt /stp///eeppp..e////////// com/ provides an open API for pop-
ular internet search engine.

---

Q: Who is the female star in with Wind married to?
SBERT
Rouge-1 Recall
Search Results
Figure 3: An example dialog generated by q2d from a MuSiQue multi- hop question. The di- alog's final query has a 0.5034 semantic sim- ilarity score with the original, indicating dif- ferent semantic content and thereby not being fil- tered.
Similarity
Recall 10
Just watched Gone with the Wind again. greet
Model/ Test Set
M
B
M
B
M
B
happy you movie that never get old
WizInt
66
67
40
36
21
21
BlenderBot3
62
69
32
35
19
24
Ch yes!l that they chose Clark Gable es the male sier.
is great
T5 (QReCC)
74
77
70
65
34
37
PaLM 540B
88
81
52
41
Flan- U-PaL.M 540B
89
83
57
Yes, Clark Gabll was perfect! The casting director did job choosing the
T5 (MuSiQue)
97
91
94 80
75
54
Iwas also wondering about the fernale star. Who she 107
Table 2: Performance of language and dialogue models on query generation test sets is shown. 'M'B indicates results on MuSiQue auto-generated Bamboogle man- ually constructed dialogues. (QReCC) and (MuSiQue) indicate fine-tuning on a "q2d" dataset. Best results were achieved by models fine-tuned MuSiQue auto- generated dialogue, which improved T5 results by 14%- 59% on the human-annotated test.
the conversation forward without trying to mimic human-to-hunne to-human interaction. Figure 3 shows a positive example where the generated dialog is not being filtered. On the other hand, negative ex- ample could be a query like "Who is the female star in Gone with the Wind married to?" which closely resembles another query asking about the male star, with a high similarity score of 0.9595, This demonstrates the method's shortcomings in filtering semantically similar queries.
MuSiQue dialogs in the same method presented in Section 3. BlenderBot3 and Wizlnt are publicly available in Parlai (Miller et al., 2017), exact de- tails and versions are described in Appendix A, Section A.7. More details on the instructions for zero-shot models can be found in the Appendix A, Section A.3.
Full instructions, examples and annotated data can be found in the Appendix A, Section A.4, in- cluding examples with model predictions.
Results. Query generation results are presented in Table 2.4 Qualitative examples with T5 model predictions are available in Appendix A, Sec- tion A. The T5 model improves performance on the human-curated Bamboogle test by 14%-59% after fine-tuning on the auto-generated MuSiQue multi-hop dialogues. We show examples for it in Appendix A, Section A.6. This improvement also correlates with improvements on the auto- generated test set, indicating the effectiveness of our method for creating evaluation data. To con- clude, our results show that our datasets are ef- fective as a benchmark for query generation, as well as training data that generalizes to both auto- generated and human-annotate test sets.
Dataset Generation. We use our q2d method as described in Section to generate dialogs that ask multi-hop questions, using the MuSiQue dataset (Trivedi et al., 2021) as the base for the questions (Q). MuSiQue is challenging multi-hop QA dataset that is partially auto-generated, so we gen- erate dialogs from partially generated questions. This illustrates how we can automatically gen- erated data to improve human-annotated data. We use seven few-shot examples (S, and S.). As result, we generate 3K train samples and 480 test samples. Full prompts, instructions and examples are available in Appendix A, Section A.I.
Metrics. The metrics used in this work are the same those described in the previous section: Rouge- Recall, SBERT embedding semantic simi- larity, and Recall @ 10.
Producing a Partially Decomposed Query. Given a multi-hop dialog, query generation models may resolve partial information. For example, if dialog asks "How old is the current US president?", query generation model may produce "How old is Joe Biden?", which is correct at the time but may become outdated in the future, or may pro- duce hallucinations. To prevent this, we can make
Models. We evaluate several ofate-of-the-art lan- guage and dialog models. These include PaL.M
540B (Chowdhery al., 2022), Flan-U-PaLM 540B (Chung et al., 2022), T5-3B (Raffel al.,
2020), BlenderBBr3-3BB (Shuster et al., 2022b),
WizInt Search Engine FiD (Lewis et al., 2019). These models are used in a zero-shot setting, except for T5,which is fine-tuned on the auto-generated
"We show relatively low scores with Wizlnt and Blender- Bot3 that seem to be oriented in finding the topic query rather than concrete questions.

---

two query generation calls (first to discover the current US president and then their age), decouple knowledge from executing (Borgeaud et al., 2021; Parisi et al., 2022), periodically update the model's weights, or disallow the model from making par- tial resolves. This will help ensure that the gener- ated query remains accurate and relevant over time. The fine-tuning technique described in this section uses the last approach to avoid making assumptions about the current president's 's age or identity.
NLI (d, (q. r)) with or dialogue response
Who starred in the greatest show on earth?
Score: 94.8
O
Score: 81.5
Betty Hutton, Cornel Wlide, and Charlton Heston star in The Greatest Show on Earth
The Greatest Show on Earth starred Chariton Heston and Betty Hutton.
directed the film?
Score: 99.3
Score: 99.5
The Greatest Show on Earth is 1952 American drama film produced and directed by Cecil DeMille
5 Intrinsic Evaluation: Naturalness, Factuality and Correctness
Cecil B DeMille directed The Greatest Show on Earth.
this section we perform a thorough analysis of the generated dialogs, focusing on the QReCC NQ dataset which contains human annotated dialogs, and evaluate their naturalness (S5.1), factuality ($5.2) and correctness ($5.3).
Figure 4: Illustration of the response factuality eval- uation. For each turn, produce response with PaLM, and compare the generated response to the hu- annotated response. We
NI.I model whether the response answers the question ("Hypothe- sis: The answer to the question {q] is (x]"") according to the Wikipedia document d used by the human anno- tator in the ground-truhh response generation ("Premise: lower score for the M response because it misses the mention of 'Cornel Wilde' that appears in the document summary.
5.1 Naturalness: Humans Struggle to Distinguish Synthetic Dialogs from Natural
[d]"). the first response there
We define a human-evaluation task to distinguish between naturally generated dialogs and auto- generated dialogs. We sample 100 annotated di- alogs from QRcCC NQ (Anantha et al., 2020) and mix them with 100 dialogs we generated. The an- notators, who are not the authors of the paper and have STEM degree, were asked to mark if the dialog seems to be generated by machine, and 0 otherwise." The labels were hidden. We use three annotators for each sample and select their majority vote as the final answer. The results show that the majority vote achieved success rate of 50.5%, while the random chance is 50%. All in- dividual annotators achieved between 50%-55% in this task. In 26% of the cases there is full agreement between all three annotators. When all agreed, the result improves 51.9%, which still close to random chance. These results indicate that humans struggle to differentiate between natural and auto gentereed dialogs. This suggests that the auto-generee generaeeerrted dialogs are of high quality and are similar to human annotations, and can be used in place of human- senaaneeeeeraed dialogs in certain situa- tions, saving time and resources.
5.2 Factuality: Generated and Human-Annotated Answers Perform Similarly in Query Generation
The q2d method generates a dialog by starting with query and generating series of related questions and answers. However, since the intermediate swers generated by large language model, there is a chance that they may be factually correct or incorrect. This raises the following questions. (1) Are the intermediate answers factually correct? (2) How does the factuality of the generated an- swers affect the results of downstream tasks?
We replace all human annotated answers in the QReCC NQ training split with PaLM generated answers. To produce PaL.M answers, we use a few- shot prompt, where the input is the original dialog ending in a question, and the output is the PaLM response. An example is provided in Figure 4.
Intermediate Answers Factuality According Automatic Metries and Human Raters. To an- swer the first question, we evaluate the factual cor- rectness of the generated answers by using an NLI (Dagan et al., 2005) model presented by Honovich
Full instructions to the annotators are provided in Ap- pendix A, Section A.S.

---

al. (2021). We take the question ("q"), the re- sponse ("r") that may be the ground-truth anno- tated response or the generated response, and the
105
Sa Intert Wescringg
ab
Wikipedia document ("d") summary available in QReCC dataset. We construct the following NLI instance: "premise: { hypothesis: The answer to the question (q) is (r)' and produce NLI scores for the ground-truth responses
70
the generated responses. Figure 4 illustrates our process. The av- erage NLI scores for the human responses are 62%, and for the PaLM responses 38%. However, this measure is biased towards the human responses since we measure it with the Wikipedia document that was used to generate the answer. PaLM might also produce correct answer, that is just not writ- ten in the same exact words in Wikipedia. To test this, conducted an annotation task with an notator that is a part of the paper authors. The annotator was presented with 50 samples of dia- log, query, and two options: A and B. One of the options was the original answer and the other was the generated answer. The annotator's 's task was to mark 0/1 for each answer indicating whether it was factual and relevant for the question. The results are that PaLM responses were marked correct in 82% of the cases, compared 93% correctness of the human responses. This result indicates the factuality and relevancy of the generated responses.
10. (0,0225) 10.25 3.51 (03) 9.753 0.ll
10.3, 0.051
SBERT Semeetic Simberity By Score
Figure 5: mrent-preserrvng annotation task results. The proportion of samples that were annotated as intent- preserving increases with the semantic similarity score.
5.3 Correctness: Generated Datasets Labels are Accurate, and Strict Filtering Improves Results
Our main filter measures similarity between the original query and the reversed query sim(q, g') and compare it threshold T queery We measure its effect in human-evaluation and automatic abla- tion studies. Both experiments indicate the label correctness for the task of predicting the query from a dialog and the value of stricter filtering threshold.
Humans Find that Dialogs Generated by Queries Have the Same Intent. We define hu- annotation task to determine whether the di- alogs are ireenn-preserving. Annotators were asked to mark if the dialog intent- pweettpreservvng, and otherwise. We use three annotators for each sample, and select their majority vote as the fi- nal answer. We follow the notation suggested by (Groenendijk and Stokhof, 1984) about entailment between questions: an interrogative entails an- other d iff every proposition that answers & an- swers as well (Jiang and de Marneffe, 2022). Here, 9 stands for question and d stands for an information-sseking dialog. We defined eight SBERT semantic similarity score buckets, with 15 in each, covering all similarities between 0 and 100. Results are presented in Figure 5. All three anno- tators agree in 88% of the cases. The proportion annotations grows according to the SBERT semantic similarity score, with strong gain between 0.95 and the only bucket with 100% intent- preserving annotations. Accord- ingly, we only select samples that have generated
For Query Generation, Generated Answers Per- form Similar to Human-Annotated. To answer the second question, we replace all of the human annotated answers with automatically generated an- swers, receiving semi-auto aun--eeeeeeedeeddd training set with the same structure and same annotated ques- tions, but with PaLM generated dialogs. Then we train T5-3B (Raffel al., 2020) model on the hu- man annotated and the cemi-auto-generated version and compare the results. For example in Figure 4, the semi-auto-generrtted dialog is the one with the answers on the right side. We train the same way we presented in Section 3. The results are 86.6% Rouge-1 1 Recall with the semi auto aute-generated train- ing set, only small drop (1.5%) from the results of the model trained on the natural data, indicating that although PaLM sometimes (<48%) produce in in-factual responses, it only has negligible effect on the query generation task.
of
FFull instructions to the annotators are provided in Ap- pendix A, Section A.5.

---

SBERT Similarity Filtering Threshold
Filtering Proportion
Query Generation Rouge-1 Recall
Data Generation Several works have used large language models for data generation (Agrawal et al., 2022; Honovich et al., 2022b; Yarom et al., 2023). Dai et al. (2022b) applies this technique to information retrieval, creating retrievers based on generated data that generate queries given the doc- ument. Their method involves round-consistency filtering using large language model, a method similar reverse translation. In the context of dialog generation, Dialog Inpaintint (Dai al., 2022a) starts from a document and generates a dia- log. Moreover, Gekhman et al. (2023a) introduced True Teacher, a synthetic data generation method that employs large language models for annotating model--geeratte summaries, demonstrating its ef- fectiveness over existing techniques. Our approach focuses on generating dialogs from queries, which allows us to leverage the availability of existing QA datasets. This enables us create information- seeking dialogs with the same intent as the original questions, along with automatically generated la- bels for the queries and answers.
0
0
68
0.25
6
69
0.5
16
72
0.75
74
0.8
44
76
0.9
62
79
0.95
72
81
0.99
84
83
0.999
84
Table 3: Reversed queries similarity filter. The similar- ity is measured between the original query q and the reversed query o' predicted with the few-shot model o' + (S I,, dialogue). The higher the filter threshold (strict filter), the better the results.
queries very similar to the original query (20.99) in the filtering step.
Strict Filtering Leads to Higher Quality Data, Resulting in Improved Downstream Results. We measure different thresholds tested on an eval- uation set of 1,000 instances generated from other train queries. We also add another filter- ing method based on an NLI (Dagan et al., 2005) model, given a dialog "d" and a question "q"" we construct the following NLI sample: "premise: (d} hypothesis: The dialog asks the question (q)", with different thresholds. Results are presented in Ta- ble 3. We report the Rouge-1 I recall on the evalua- tion set. We see that performance increases as the reversed similarity threshold rises, and with clear trade-off with the filtering proportion. The more data we generate, we are able to apply a more strict filtering, receiving higher quality data, that leads to better results.? 7 We produced four options for the NLI-based method, with thresholds ranging from 0.65 to 0.82, and above it filtered too much data (below the goal of 13K). The max performance for the 0.82 threshold group is 70%, much lower than the alternative reverse queries filter.
Search Based Query Generation dialog models like LaMDA and BlenderBot use search APIs to generate factual responses. Training and evaluation data for such models is obtained mostly with hu- man annotated data. Previous works (Shuster et al., 20226; Thoppilan al., 2022; Komeili et al., 2021) evaluated only the end-to-end dialog response with- out evaluating the generated query. The evaluation was primarily based on automated metrics of per- plexity and F1, or with human annotations assess- ing whether the model response is sensible, specific, and interesting (SSI), or whether it is correct, en- gaging, and consistent. The evaluated dialogs were general, not necessarily information-sseeingg The focus of this paper is on the query generation task for information-seed -eeking dialogs, with concrete question and an expected response.
Question Rewrite Works like QReCC (Anan- tha et al., 2020), Question Answering in Con- text (QuAC) (Choi et al., 2018), TREC Conversa- tional Assistant Track (CAsT) (Dalton et al., 2020), QuAC and CANARD (Elgohary et al., 2019) in the information retrieval domain use human anno- tated data, that mostly contain follow-up dialogs, questions followed by answers. In the domain of Conversational Question Answering (CQA), a com- prehensive study was conducted on the robustness of dialogue history representation, underscoring
6 Related Work
Our work relates to data generation, query gen- cration for scarch-bascd models, and information retrieval datasets.
'The high filtering proportion means that we simply need to generate more data (requiring more compute time) in order to achieve dataset of the same size without filtering.

---

the significance of evaluations centered on robust- ness (Gekhman et al., 2023b). Our work focuses on the application of dialog models like LaMDA and BlenderBot, which often involve the use of less for- mal language and more human-like conversations. The need for a variety of query generation datasets has motivated us to develop automatic method for generating dialogs for the query generation task, with range of different styles and skills required.
foundation models in query-generation tasks. How- ever, we did not explore the possible improvements to the original foundation models, like PaLM, when fine-tuned using our generated data. This repre- sents a promising avenue for further research.
Ethics and Broader Impact
This paper submitted in the wake of a tragic ter- rorist attack perpetrated by Hamas, which has left our nation profoundly devastated. On October 7, 2023, thousands of Palestinian terrorists infiltrated the Israeli border, launching brutal assault on 22 Israeli villages. They methodically moved from home home brutally torturing and murdering more than a thousand innocent lives, spanning from infants the elderly. In addition to this horrifying loss of life, hundreds of civilians were abducted and taken to Gaza. The familics these abductees have been left in agonizing uncertainty, as no infor- mation, not even the status of their loved ones, has been disclosed by Hamas.
7 Conclusions
We introduced q2d, data generation pipeline that produces dialogs based on questions. We demonstrated that our method can replace human- annotated data to train query query-generation models, and create effective, natural, factual, and accu- rate evaluation and training data in new domains, even when no existing dialogue data is available.
8 Limitations
q2d comes with set of limitations about costs, domain identification and factuality.
The heinous acts committed during this attack, which include acts such as shootings, sexual as- saults, burnings, and beheadings, are beyond any justification.
Computational Costs. The process of auto- generating data with large language models, al- though faster and more scalable than human anno- tations, still incurs significant computational costs. These costs, however, are expected to decrease as Al technologies advance.
In addition to the loss we suffered nation and as human beings due to this violence, many of us feel abandoned and betrayed by members of our research community who did not reach out and were even reluctant to publicly acknowledge the inhumanity and total immorality of these acts.
Domain Identification and Sample Selection. Defining the target domain and selecting represen- tative few-shot examples requires manual oversight. This step, although crucial for ensuring the diver- sity and representativeness of generated dialogs, adds a layer of complexity and time the process.
We fervently call for the immediate release of all those who have been taken hostage and urge the academic community to unite in condemnation of these unspeakable atrocities committed by Hamas, who claim to be acting in the name of the Pales- tinian people. We call all to join us in advocating for the prompt and safe return of the abductees. as stand together in the pursuit of justice and peace.
Factuality of Dialogs. Our method generates di- alogs that are generally factual but occasionally inaccurate. Although our analysis show these dis- crepancies do not impact mnerr-generation tasks, they may challenge tasks where factuality is critical. Future applications should consider this limitation and potentially enhance factuality.
This paper was finalized in the wake of these events, under great stress while we grieve and mourn. It may contain subtle errors.
Scope of Fine-Tuning and Model Improvements. In this research, our primary aim was to show that larger foundation models' generated data, such as PaLM 540B, can significantly benefit more com- pact models like T5-3B. Specifically, when smaller models are fine-tuned on our auto-generated data, they can achieve performance surpassing the larger
References
Priyanka Agrawal, Chris Alberti, Fantine Huot, Joshua Maynez, Ji Ma, Sehastian Ruder, Kuzman Ganchev, Dipanjan Das, and Mirella Lapata. 2022. Qamelcon: Multilingual qa with only examples. arXiv preprint 1.X0v:2211.08264..

---

Raviteja Anantha, Svitlana Vakulenko, Zhucheng Tu, Shayne Longpre, Stephen Pulman, and Srinivas Chappidi. 2020. Open-domain question answering goes conversational via question rewriting. arXiv preprint arXiv:2010.0499)
Few-shot dense retrieval from examples. arXiv preprint urXiv:2209.1775.
Jeffrey Dalton, Chenyan Xiong, Vaibhav Kumar, and Jamie Callan. 2020. Cast-19: A dataset for conver- sational information seeking. In Proceedings of the 43rd International ACM SIGIR Conference on Re- search and Development in Information Retrieval. pages 1985-1988.
Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George van den Driessche, lean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et al. 2021. Improving lan- guage models by retrieving from trillions of tokens. arXiv preprint urXiv:2112.044/
Nouha Dziri, Hannah Rashkin. Tal Linzen, and David Reitter. 2021. Evaluating groundedness in dialogue systems: The begin benchmark. arXiv preprint arXiv:2105.00071.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 18:187771901.
Ahmed Elgohary. Denis Peskov, and Jordan Boyd- Graber. 2019. Can you unpack that? learning to rewrite questions-in-conten Can You Unpack That? Learning to Rewrite Questions in--in--Coo Conteon
Daniel Cer, Mona Diab, Eneko Agirre, Inigo Lopez-
Zorik Gekhman, Jonathan Herzig, Roee Aharoni, Chen Elkind, and Idan Szpektor. 2023a. Trueteacher: Learning factual consistency evaluation with large language models. arXiv preprint arxivv2305.11171
Gazpio, and Lucia Specia. 2017. Se01 task Semantic textual similarity-uultiiin and cross-lingual focused evaluation. arXiv preprint arXiv:1708.00055.
Zorik Gekhman, Nadav Oved, Orgad Keller, Idan Szpek- tor, and Roi Reichart. 2023b. On the robustness of dialogue history representation in conversational question answering: a comprehensive study and new prompt-based method. Transactions of the Asso- ciation for Computational Linguistics, 11:351-366.
Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St John, Noah Constant, Mario Guajardo-Cespedee
Steve Yuan, Chris Tar, al. 2018. Universal sentence encoder for english. In Proceedings of the 2018 conference on empiri- cal methods in natural language processing: system demonstrations, pages 169-174.
Amelia Glaese, Nat McAleese, Maja Trębacz. John Aslanides, Vlad Firoiu, Timo Ewalds. Maribeth Rauh, Laura Weidinger, Martin Chadwick, Phoebe Thacker, et al. 2022. Improving alignment of dialogue agents via targeted human judgements. arXiv preprint arXiv:2209. 240.222991337..
Eunsol Choi, He He, Mohit lyyer. Mark Yatskar, Wen- tau Yih. Yejin Choi, Percy Liang, and Luke Zettle- moyer. 2018. Quae: Question answering in context. arXiv preprint arXiv: 1808. 07036.
Aakanksha Chowdhery, Sharan Narang. Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sehastian Gehrmann, et al. 2022. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311.
Jeroen Antonius Gerardus Groenendijk and Martin Jo- han Bastiaan Stokhof. 1984. Studies on the Seman- rics of Questions and the Pragmatics of Answers. Ph.D. thesis, Univ. Amsterdam.
Prakhar Gupta, Chien-Sheng Wu, Wenhao Liu, and Caiming Xiong. 2021. Dialfact: benchmark for fact-checking in dialogue. arXiv preprint 10.0222110.0822..
Hyung Won Chung, Le Hou, Shayne Longpre, Bar- ret Zoph. Yi Tay. William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2022. Scaling carructionfinnttune language models. arXiv preprint arXiv:2210. 11416.
Or Honovich, Roce Aharoni, Jonathan Herzig, Ha- gai Taitelbaum. Doron Kukliansy, Vered Cohen. Thomas Scialom, Idan Sypektor, Avinatan Hassidim, and Yossi Matias. 2022a. True: Re-evaluating factual consistency evaluation. arXiv preprint wrXiv:2204.04991.
Ido Dagan, Oren Glickman, and Bernardo Magnini. 2005. The pascal recognising textual entailment chal- lenge. In Machine learning challenges workshop, pages 177-190. Springer.
Or Honovich, Leshem Choshen, Roee Aharoni, Ellla Neeman, Idan Szpektor, and Omri Abend. 2021. g2: Evaluating factual consistency in knowledge- grounded dialogues via question generation and ques- tion answering, arXiv preprint arXiv:2104.02200 082.0002.
Zhuyun Dai, Arun Tejasvi Chaganty, Vincent Y Zhao, Aida Amini, Qazi Mamunur Rashid, Mike Green, and Kelvin Guu, 2022a. Dialog inpainting: Turning documents into dialogs. In International Conference on Machine Learning, pages 4558 4586. PMLR.
Or Honovich, Thomas Scialom, Omer Levy, and Timo Schick. 2022b. Unnatural instructions: Tuning lan- guage models with (almost) no human labor. arXiv preprini 22X.v:221..09689.
Zhuyun Dai, Vincent Y Zhao, Ji Ma, Yi Luan, Jianmo Ni. Jing Lu, Anton Bakalov, Kelvin Guu, Keith Hall, and Ming-Wei Wei Chang. 2022b. Promptagator:

---

Nan-Jiang Jiang and 2022. Investigating reasons for disagreement in natural language inference. arXiv preprint arXiv:220.00399.
de Marneffe.
knowledge: Modular search generation for di- alogue and prompt completion. arXiv preprint arXiv:2203. 13224.
Kurt Shuster, Jing Xu, Mojtaba Komeili, Da Ju, Eric Michael Smith. Stephen Roller, Megan Ung. Moya Chen, Kushal Arora, Joshua Lane, et al. 2022b. Blenderhot 3: deployed conversational agent that continually learns to responsibly engage. arXiv preprint arXiv:2208.03
Mojtaba Komeili, Kurt Shuster, and Jason Weston. 2021. Internet-augmnntdd dialogue generation. arXiv preprint urXh:2207.005666
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Ken- ton Lee, et al. 2019. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:453- 466.
Romal Thoppilan, Daniel De Freitas, Jamic Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, al. 2022. Lamda: Language models for dialog applica- tions. arXiv preprint arXiv:2201.0822
Katherine Lee, Orhan Firat, Ashish Agarwal, Clara Fan- njiang, and David Sussillo. 2018. Hallucinations in neural machine translation.
Harsh Trivedi. Niranjan Balasubramanian. Tushar Khot, and Ashish Sabharwal. 2021. Musique: Multi-hop questions via single-hop question composition. arXiv preprint arXiv:2108.00557
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019. Bart: De- noising eequence-to-seqeene pre-training for natural language generation, translation, and comprehension. arXiv preprint urXiv:1910.1346.
Michal Yarom, Yonatan Bitton, Soravit Changpinyo. Roee Aharoni, Jonathan Herzig, Oran Lang, Eran Ofek, and Idan Szpektor. 2023. What you see is what you read? improving text-image alignment evalua- tion. arXiv preprini arXiv:2305. 10400.
7heng Zhao, Shay Cohen, and Bonnie Webher. 2020. Reducing quantity hallucinations in abstractive sum- marization. arXiv preprint arXiv:2009. /3312.
Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches ouf, pages 74811.
Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. On faithfulness and factu- ality in abstractive summarization. arXiv preprint arXiv:2005.00661.
Alexander Miller, Will Feng, Adam Fisch, Jiasen Lu, Dhruv Batra, Antoine Bordes, Devi Parikh, and Jason Weston. 2017. Parlai: A dialog research software platform. arXiv preprint 17544:1705.06476.
Yixin Nie, Mary Williamson, Mohit Bansal, Douwe Kiela, and Jason Weston. 2020. like fish, especially dolphins: Addressing contradictions in dialogue mod- eling. arXiv preprint urXiv:2012.1339.
Aaron Parisi, Yao Zhao, and Noah Fiedel. 2022. Talm: Tool augmented language models. arXiv preprint arXiv:2205.1 12255..
Ofrr Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A Smith, and Mike Lewis. 2022. Measuring and narrowing the compositionality gap in language models. arXiv preprint arXiv:2210.03330.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter Liu, et al. 2020. Exploring the limits of transfer learning with unified text-to-text trans- former. J. Mach. Learn. Res., 21(140):1-67.
Kurt Shuster, Mojtaba Komeili, Leonard Adolphs, Stephen Roller, Arthur Szlam, and Jason We- ston. 2022a. Language models that seek for

---

A
Appendix
Table 4: Random Examples from Auto-Generated QRECC
A.1 Generated Examples
Dialog:
Figure 6 and Figure 7 show cherry picked exam- ples of humann-annttann from Bamboogle/ / MuSiQue.
User: who plays the lead mie wish upon a The cast of Wish Upon Stur include Kutherine Heigl. Danielle Panabaker, and Mary Page Keller.
/ auto- gette-eeed examples
Assistant:
User: who plays haley
Tables 4 and Tables 5 show random examples from the auto-generated QReCC/ / MuSiQue di- alogs.
Question: who plays haley in wish upon star?
Answer: Danielle Harris
Dialog:
User: where is assam
A.2 Filtering Examples
Assistunt: Assum is state in India
User: what is the capital
Table 6 show examples of generated samples with different SBERT similarity scores used for filtering.
Assistant: The capital of Assam is Dispur
User: who lead the first revolt against british in the state
Question: who lead the first revolt against british assam?
Answer:
Gomdhar Konwar
A.3 Prompts and Instructions
Dialng:
Prompts (instruction + few-shot examples) for the auto-generated dialogs are presented Table 7 for MuSiQue, and Table 8 for QReCC NQ. We show three prompts for each dataset due page page length, full prompts files are here: MuSiQue, QReCC.
User: when did last man standing uir
Assistant: Last Man Standing premiered on ABC on October 2011.
User: what the main character's name
Assistant: Tim Allen stars as Mike Baxter, the central charac- ter of the series.
User: what are the other character's 's names
Assistant:
The other characters are Mike's wife and three
daughters.
The reverse generation model uses the same amples but in reversed order (to generate a query given a dialog).
User: who plays cami harris
Question: who plays cami harris on last standing?
Answer: Sarah Gilman
The instruction for the reverse generation model and the PaLM zero-shot is: "Given a dialog that asks an indirect question, extract the concrete ques- tion".
Dialog:
User: who were confederates in civil war Assistant: The Confederates the American Civil War were the Confederate States of America, the South. User: who fought them Question: who fought the confederates in the american civil war?
The instructions for the Flan-U-PaLM--55 are bit more detailed (we found out it worked better for this instruction fine-tuned model): "Your task is reformulate the last conversational query in a dialog to fully specified, context-independent query that can be effectively handled by Google Search.".
and
Answer: The Union
Dialog:
User: when did adele webher first come grey's anatomy Assistant: The churucter of Adele Webber was first introduced in the episode ""Save Me"".
User: who plays her Question: who plays adele webber on grey't 's anatomy? Answer:
Loretta Devine
A.4 Human dialog Generation for Bamboogle
Dialog:
User: what did the effigy mounds look like
Bamboogle human constructed dialogs are avail- able here: Bamboogle dialogs.
Assistant: The effigy mounds are earthen mounds built the shapes unimals, humans, and other objects.
User: when were they discovered
Instructions for annotator: "Your task is write a dialog given a question. You may use the internet in order to add factual information to dialog. The dialog intent should be understandable yet chal- lenging. The responses should be 'assistant like' not trying to mimic human emotions."
Assistant: The first effigy mounds
discovered in the 1700s but were not identified as such until the lute 1800s.
User: when were most of them built
Questions: when were most of the effigy mounds built? Answer: 350 - 1300 CE
A.5 Human Evaluation Instructions
Instructions for the natural vs. auto-generated task: "Can you guess which of the following di- alogs is natural and which is auto-generated by a machine? There are 200 dialogs. 100 are synthetic, and 100 are natural. Enter 1 for Synthetic and for Natural in the "synthetic?" field."
Instructions for the intent-preserving task: "You will receive dialog and query, and need to label the dialog has the same intent as the query. If you are unsure, you can check to see whether both intents return the same Google search results.".

---

Q: Who is the father of the father of George Washington?
Q: The husband of Lady Godiva was Earl of which Angilc kingdom?
Q: Who was the head of NASA curing Apollo 117
Who was George Washington?
Lady Godivaa marry?
Who is the head on NASA?
Yes, and she had son Godwine. His grandson was Haroid Godwinson, the last Anglo- Saxon king of England.
A General the American Revolutionary War, the first President of the United States, and one the Founding Fathers the United States.
The head NASA currently Jim Bridenstine. He is former Navy pilot and U.S. representative from Okdahoma
Was her husband Earl?
Who his father's father?
Yes, he was.
heldi that role during Apollc 11?
of which Anglic kingdom?
Who wes George Washington's father
who was lady godiva
Who held that role during Apollo 11?
Who was George Washington's father's father?
Which kingdom did Lady
who was head of nosa
Godiiva's husband rule?
during apollo
Figure 6: Examples from the human-annotated dialogues for Bamboogle. The model predictions above/below the line are of T5, before/after fine-tuning on MuSiQue dialogues.
Q: Who is the female star in Gone with the Wind marrled to?
Q: What record label did the performer of The Place and the Time belong to?
Q: What is the population of Francls Watson's birthplace?
just watched Gone with the Wind again. It is such greet movie!
heard amazing song that believe was called The Place and the Time. What do you know about the performer?
am a huge fan of the game cricket.
lm happy that you like it! It is a movie that will never get old.
It is fascinating game.
have question about an English cricketer.
know that was performed a telented singer.
yes! think great that they chose Clark Gable the male star.
Which player are you referring to?
just wonder could find more his songs.
Yes, Clark Gable
Well, it is good thing that he belonged to record label that made easy to his work.
Francis Watson
perfectl The casting director did great job choosing the actors in this movie.
He very talented player.
was also wondering about the female star. Who she married to?
What record label did he belong to?
What populetion his birthplace?
who clark gable married
the place and the time singer
what's the population of his birthplace
Who is the female star Gone with the Wind married to?
What record label did the performer The Place and the Time belong to?
What is the population of the birthplace Francis Watson?
Examples from the auto-generated dialogues for MuSiQue. The model predictions above/below the line are of T5, before/after fine-tuning on MuSiQue dialogues.
Figure 7:
A.6 Additional Examples
we used: zoo:sea/bart_fid ftti_ssss///mdeell..... For Blende- Bot3 model we used zoo:bb3/bb3 3333mmd//llll Batad//dbbb33/mmdellllle -o gen/r2c2 bb3. The search engine is: https://
Table 9 shows examples of the ground-truth vs. the predicted queries from samples in the Bamboogle dialogs dataset.
A.7 Parlai Models
We used available versions in Parlai (Miller al., 2017) that can be accessed through an open API for popular search engine. For the WizInt model

---

Table 5: Random Examples from Auto-Generated MuSiQue
Table 6: Examples from the generated QReCC data with different SBERT similarity scores between original Query (Q) and the reversed Query (Rq). The higher the similarity, the more the dialog's 's intent is the same as the original query. We took only dialogs with SBERT similarity 2 0.999.
Dialog:
User: am looking for a movie to watch tonight.
Assistant: think I can help you find a good movic. What are your preferences?
User: like watching action movies.
Assistant: What about The Boure Deception? It very popular action movie.
SBERT Similarity: 0.999
Dialog:
User:
Who plays the main charucter of this movic?
User: who is the chairman of the joint chiefs of staff Assistant: General Joseph Dunford is the current Chairman of the Joint Chiefs of Staff.
Question: Who plays the main character of The Bourne De- ception?
Answer: Matt Damon
User: who does he advise
Q: who does the chuirman of the joint chief's of staff advise
Dialog:
Rg: Who does the chairman of the joint chiefs of staff advise?
just watched Charlic and the Chocolate Factory and can't 't get it out of my mind!
User:
SBERT Similarity: 0.75
Assistant: What an amazing movic! love it! User:
Dialig:
I have argument with my sister about the spoilt character.
User: who designed magic gathering
Assistant: Richard Garfield is the creator of the Magic: The
Assistant: That's the thing- - there so many characters in this movie that are not just black and white. They are complex churacters.
Gathering collectible card game.
User: who originally published the game
Assistant: Wizards of the Coast, subsidiary of Hasbro, Inc. is the original publisher of Magic: The Gathering
User: know the spoilt character boy but can you tell me who played him?
User: who is the current publisher
Question:
Who played the spoilt churacter Churlic and the
Q: who created magic the gathering?
Chocolate Factory? Answer: Julie Dawn Cole
Rg: who is the current publisher of the game Magic: The Gathering?
Dialog:
SBERT Similarity: 0.5
User: I was watching the movie 'Dolores Claiborne' yesterday. made me curious about the producer of the movie.
Dialog:
what year did the Miami Hurricanes win the National Championship
User:
Assistant: It the movie with Kathy Bates, right?
User: Yes, and I was wondering about the producer, do you know who his spouse is?
Assistant: The Miami Hurricanes won the national champi- onship in 2001.
Question: Who is the spouse of the producer of Dolores Clai- borne?
User: was the team undefcated
Assistant: The Hurricanes finished the 2001 senson 12-0 User: who did they win against the nationul championship
http://www.ooer
www.ooverleaf.co.. wttp:/oerleaa..oo//pjeeeeeeeeeee commmmomeee/s4222220027772222220
Answer: Helen Mirren
game
Q: who did the Miaimi Hurricanes win against in 1983
Dialog:
Hg: Who did the Miami Hurricanes play in the national championship game
User:
I have this friend that is totally into football.
Assistant: see.
User: He big fan the Al Janoub Stadium.
SBERT Similarity: 0.25
I am familiar with the Al Janoub Stadium. It is very modern stadium. and its design is very unique.
Assistant:
Dialog:
User: who the golden glove award the 2018 fifa world cup
User: I'm not surprised. I'm wondering what league includes the operating group of the stadium.
Assistant: Thibaut Courtois was named the winner of the Golden Glove at the 2018 FIFA World Cup.
Question: What league includes the operating group of Janouh Stadium? Answer: Qatar Stars Leugue
User: what team did he play for?
Q: who won the golden glove at the 2018 fifa world cup?
Rg: what team did thibaut courtois play for?
Dialog:
User: 1 um really into books these days. I just finished reading "The Final Testament of the Holy Bihle' and can't 't get out of my mind.
SBERT Similarity: 0
Dialog:
User: who inspired the book wonder by RJ Palacio
Assistant:
see how this book would be interesting. of people think about books like that.
Assistant: The book Wonder by RJ Palacio was inspired by
childhood story of a hoy with severe facial difference.
User: It is reully good book. think that this book was published by a great publisher.
User: what was the name of the boy
Assistant: The name of the boy was Julian.
Assistant: Yes, it is. is one of the best in the UK.
User: what was his facial difference
User: Do you know who founded the publisher of this book?
Assistant: The boy hud a cramiofiacial condition called
Questions: Who founded the publisher of The Final Testament of the Holy Bible?
Treacher Collins syndrome.
User: what is the condition
Answer: Larry Gagosian
Q: what inspired the book Wonder
Ro: what is Treacher Collins syndrome

---

Table MuSiQue Gene-Gene Dialogues Prompts
Write a dialog between un automated ussistant und user. and dialog should indirectly ask the initial question you received.
Table Prompts
QRECC NQ Auto-Generated Dialogues
Write dialog between an automated assistant and a user. and the dialog should indirectly initial question you received.
Question:
When was the institute that owned The Collegian
founded!?
Dialog:
User: have this homework need submit my history the journalism course. Can you help find some of the details?
Question: Why was the great wall built??
Dialig:
Assistant: Sure. am here help
User: where is the the great wall of china located
User: am working together with my friend Darren. We are looking into different newspapers, focusing on the powers that own them. I'm currently looking into The Collegian Assistant: can find out about The Collegian. is actually owned by educational institute. User: When the institute founded?
Assistant: The Great Wall of China is built the historical northern borders of China
User: how long is the wall
Assistant: The Great Wall is 21,196 km (13,171 mi).
User: why was the wall built
Question: Which U.S. states produce the most crued oil?
Question: What city the person who broadened the doctrine philosophy of language from?
Dialog:
User: What kind of oil is in North Dakota
Dialog:
Assistant: The oil-rich Bakken shale formation has mude North Dakota the second-largess crude-producing state behind only Texas.
User: am conducting some research in the area of doctrine of philosophy of language
Assistant: I see. It is a fascinating sub-field of linguistics. It developed quite an interesting process.
User: Why
Assistant: North Dakota generated 852 million barrels of Bukken crude, The Bakken and the Three Forks formation 94% of Dakota's 's current oil production. with about million
User: know that it was broadened by some important philoso- pher
Assistant: Indeed, a famous german philosopher broadened it by focusing language and its limits and ambiguitics
harrels a day.
User: how many U.S states we find the production Assistant: Crude oil is produced coastal waters.
User: What city was he from?
U.S. and U.S.
User: Which ones produce the most
Question: Who employs person who wrote book Ani- mal Liberation?
Dialog:
Question: Where is henry cavill from?
User:
just finished reading the book Animal Liberation. To be honest, can't get the book out of my mind! Assistant: Indeed, some people find animal liheration extremely impor- tant. Others care lot more about humans und not view this topic very important.
Dialog:
User: where is superman in the justice league movie Assistant: In the Justice League Movie, Superman has been dead for two years and mankind is in mourning him.
User: who plays the character
User: I wish could write such influential books one day, am not sure if my company would even let me do that. Assistant: Well, fortunately for the Australian writer of the book, he works for a compuny that allowed him to fulfill himself and publish the book
Assistant: Henry Cavill plays Superman in the Justice League movie
User: where is the actor from
User: Which company was that

---

Question
BlenderBot3 Predictions
What is the maximum airspeed (in km/h) of the third fastest bird?
Fastest bird
Who directed the highest grossing film?
Top grossing films
The husband of Lady Godiva was Earl of which Anglic kingdom?
Lady Godiva
Who founded the city where the founder of geometry lived?
Geometry
Who was the head of NASA during Apollo 11?
Apollo 11
When the longest bridge in the world opened?
Longest bridge
When the anime studio that made Sword Art Online founded?
Sword Art Online
What is the capital of the country where yoga originated?
Yoga origin
Who the father of the father of George Washington?
George Washington father
Who was the first king of the longest Chinese dynasty?
first king of the longest dynasty
Table Examples for the ground-trutt queries from the Bamboogle dialogues with the BlenderBot3 queries. The BlenderBot3 seems to be trained more on finding the topic than asking concrete questions.


Reviews:
[{"review_id": "ZlCa4mgpD0", "reviewer": "Reviewer_GsVR", "summary": "", "questions": "The dialogs are generated by foundation model, whether or not they may have limited assistance in improving the effectiveness of the original foundation model itself.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's positive comments and acknowledgement of our method as an effective and easy to implement method for enhancing foundation models with the ability to search independently..\n\n### Can Generated Data by a Foundation Model Improve the Original Foundation Model?\nThank you for highlighting this point. In our experiments, we primarily focused on demonstrating that the data generated by larger foundation models (e.g., PaLM 540B) can substantially benefit smaller models (e.g., T5-3B). Specifically, we showed that fine-tuning these smaller models on our generated data allows them to achieve performance that surpasses the performance of the original, larger models in query-generation tasks. While we did not explicitly test the improvement of the original foundation model (e.g., PaLM) using the generated data, it is an intriguing direction for future work. We will clarify this in the revised manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "(1) Automatic data generation is a critical challenge, especially in the era of foundation model;\n(2) It is essential to Enhance foundation models with the ability to independently search for relevant information to ground a given dialog response.\n(3) The proposed method is effective and easy to implement.", "reasons_to_reject": "NA", "questions_to_authors": "The dialogs are generated by foundation model, whether or not they may have limited assistance in improving the effectiveness of the original foundation model itself.", "paper_topic_and_main_contributions": "Given that manually-dataset-annotaion is usually time and resource consuming, this paper proposes an automatic data generation pipeline to generate information-seeking dialogs from questions.\n\nTo create a dialog, the pipeline starts from a search query and then generates an information-seeking dialog that corresponds to that query.\n\nExtensive experiments demonstrate that (1) the method is beneficial for training query generation and dialog generation; (2) the generated dialogs are natural, factual and correct.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "NA", "Typos_Grammar_Style_and_Presentation_Improvements": "NA"}, "extra_text": {}}, {"review_id": "GnbwY2ZHeO", "reviewer": "Reviewer_5Dsm", "summary": "", "questions": "Regarding table 2, I find it interesting that performance of the weaker baselines (first three rows) is generally lower on \"B\" vs. \"M\" - while for the stronger baselines (and the fine-tuned model proposed by the authors) that is not the case. It would be interesting to comment on this in the paper, in particular as the behavior of T5 itself changes after fine-tuning.\n\nI find the example shown in figure 3 somewhat unconvincing. The final turn in the dialog, while perhaps not filtered because it did not meet the filter criteria (e.g. >80% overlap with the original query), nevertheless seems substantially similar to the original query in terms of its complexity (or number of intermediate searches required). It would be interesting to comment on how often this happens in practice and how to capture this particular aspect of filtering, which should make for more human-like conversations being generated (or at least kept) in the final datasets used for later experiments.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their constructive feedback. We are happy the reviewer acknowledged the quality of our experimental setup, the enhanced similarity to human dialogs, and our contribution of publicly available code and datasets. \n\n### Abstract vs. Section 1 Information\nWe appreciate your concern about the discrepancy between the abstract and Section 1. To clarify, the first and second points in the abstract about \"query generation on the QReCC dataset\" and \"data generation in new domains like multi-hop MuSiQue and Bamboogle QA\" correspond directly to the second point in Section 1. Similarly, the third point in the abstract about \"thorough analysis\" aligns with the third point in Section 1. We'll review the document to make it clearer and easier to read.\n\n### Choice of T5 Model in Table 2 \nRegarding the selection of T5 for fine-tuning, the key takeaway from Table 2 is that even a relatively smaller model like T5-3B can achieve superior results when utilizing our data generation pipeline, outperforming much larger models like PaLM 540B and Flan-U-PaLM 540B. We are confident that fine-tuning bigger models would likely produce better results. **We are also open to executing it if requested for this review**. However, we believe that a strong testament for our data generation method's value is showing that it enables *the smallest model to surpass the largest model* when used. \n\n### Baseline Performance in Bamboogle vs. MuSiQue\nFor context, Bamboogle's dialogs are sourced from human annotations, while MuSiQue's are auto-generated. We hypothesize that the top three models may not be as adept at handling auto-generated dialogs, hence their struggles. On the contrary, models like PaLM 540B, which generated the auto-test set, and its fine-tuned counterpart Flan-U-PaLM 540B, are naturally more aligned with generated dialog styles. T5, after training on a MuSiQue-styled dataset (ensuring query and dialog uniqueness), better adapts to this distribution. \n\n### Figure 3 Example + Filter\nThe semantic similarity score between the queries you pointed out in Figure 3 is 0.5034, as calculated by the SentenceTransformer model we used. Importantly, the latter query doesn't reference the movie \"Gone with the Wind,\" which brings a different semantic nuance captured by the model. As stated in line 207, the filtered cases are ones that “include situations where no anaphora is required.”. For example, if the last turn was “BTW, about the female star in Gone with the wind. Any idea who is she married to?” This turn is still human-like, but does not require anaphora. It has a similarity of 0.8924, and therefore it will be filtered. The few-shot examples did not include such examples, and the overall percentage of these “non-anaphora requiring dialogs” did not surpass 5%.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper introduces a useful new method for producing dialogs from queries, which is proven in experiments to help improve dialog model performance compared to a number of meaningful baselines. The dialogs generated are found to be substantially similar to human dialogs when evaluated by human annotators. The experimental results are generally fairly convincing and the analysis is relatively detailed. Finally, the authors promise as a contribution to make publicly available both the code and the datasets.", "reasons_to_reject": "I have two reasons to argue against acceptance. \n\nOne is related to the overall clarity of the paper. I found the paper generally hard to read and sometimes information provided in one section did not align very well with information provided later. As an example, the abstract introduces a different set of contributions from those listed in section 1.\n\nSecond, while I think that the set of baselines included in experiments summarized in table 2 is reasonable, I am wondering whether the choice of adaptation only using T5 (whose performance out of those baselines is average) is the most convincing option. Applying the fine-tuning process to the other models used as baselines would give a clearer picture as to how resilient the approach is to fine-tuning different base models. There are other observations about table 2 results which are not discussed in the paper - I will include those as questions, below.", "questions_to_authors": "Regarding table 2, I find it interesting that performance of the weaker baselines (first three rows) is generally lower on \"B\" vs. \"M\" - while for the stronger baselines (and the fine-tuned model proposed by the authors) that is not the case. It would be interesting to comment on this in the paper, in particular as the behavior of T5 itself changes after fine-tuning.\n\nI find the example shown in figure 3 somewhat unconvincing. The final turn in the dialog, while perhaps not filtered because it did not meet the filter criteria (e.g. >80% overlap with the original query), nevertheless seems substantially similar to the original query in terms of its complexity (or number of intermediate searches required). It would be interesting to comment on how often this happens in practice and how to capture this particular aspect of filtering, which should make for more human-like conversations being generated (or at least kept) in the final datasets used for later experiments.", "paper_topic_and_main_contributions": "This paper focuses on the problem of generating dialogs from complex queries, with the goal of teaching conversational models how to search for information, e.g. on a user's behalf. There are a number of contributions made to the space in this paper, as follows:\n\n1) a method for generating dialogs from complex queries\n2) experiments showing that the generated dialogs are useful for training dialog models\n3) an analysis of the generated dialogs comparing them with human written dialogs", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "None that I saw."}, "extra_text": {}}, {"review_id": "lT0pzngJrB", "reviewer": "Reviewer_exvd", "summary": "", "questions": "1) (page 5 line 367) Is it not really clear how the author produces a partially decomposed query. The paper describes making 2 query generation calls to a decomposed query. How does the author know that the query can be decomposed into 2 calls? In this study, are there no query that can be broken down into 3 or more calls? \n2) (page 7 line 502) What does it mean by intent-preserving? The author cannot see the correlation between the term intent-preserving and full annotation instruction in the appendix. Maybe this can be clarified easily by explaining what is the intent-preserving in 1-2 short sentences.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for engaging with us to improve our work. We appreciate the constructive comments and your acknowledgment  that the proposed approach bridges the gap between LM and grounded knowledge.\n\n### Partially Decomposed Query\nWe appreciate your keen observation on query decomposition. To clarify, our paper emphasizes a single technique for handling dependent or \"multi-hop\" queries in dialogues. For instance, a dialog about the US president followed by \"How old is he?\" could lead a model trained in 2020 to generate a query like \"*How old is Donald Trump?*\", and we produce a query asking how old is the current US president without assuming their identity. By focusing on this method, we aim to avoid incorporating outdated or incorrect information from the model. This specific choice is outlined in line 382: \"*The fine-tuning technique described in this section uses the last approach to avoid making assumptions about the current president’s age or identity*.\". We acknowledge that this could be made clearer in the text, and we will revise accordingly.\n\n### Intent Preserving Query-Dialog Pairs\nThank you for noticing that. To clarify, the term refers to ensuring that the dialog and the query share the same core intent. We'll specify the meaning of \"intent-preserving\" at its first mention at line 186: \"*dialogs whose intent is different from the original query*\". The full explanation is available in the annotators' guidelines, as noted in footnote 6 and line 858: \"*Annotators should confirm that the dialog and query share the same intent, validated by comparable Google search results*.\"", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The author proposes a way to approach hallucinations in conversational LLM. The proposed approach bridges the gap between LM and grounded knowledge by deconstructing dialog to search query automatically. This is interesting and can provide insight to the Dialog/NLP researcher. Overall, this paper is complete and gives a full analysis on every experiment step.", "reasons_to_reject": "There is no strong reason to reject this paper, however there are some improvements that can be done to improve the clarity of the paper, it can be found below in the question for the authors section.", "questions_to_authors": "1) (page 5 line 367) Is it not really clear how the author produces a partially decomposed query. The paper describes making 2 query generation calls to a decomposed query. How does the author know that the query can be decomposed into 2 calls? In this study, are there no query that can be broken down into 3 or more calls? \n2) (page 7 line 502) What does it mean by intent-preserving? The author cannot see the correlation between the term intent-preserving and full annotation instruction in the appendix. Maybe this can be clarified easily by explaining what is the intent-preserving in 1-2 short sentences.", "paper_topic_and_main_contributions": "This paper turns a QA dataset into dialog conversation by prompting a PaLM. The mapping between dialog to query later on is exploited to train and evaluate query generation models. This query generation model is useful when combined with the search API, this is bridging the conversational dialog with the factual answer from the search API. Through evaluation, the author argues that the conversation produced by this approach to be natural, factual, and correct.", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Important data generation pipeline using LLMs. Intelligent use of search API to aid the generation of information-seeking dialog. It'd be valuable to community if the framework/pipeline/code is open-sourced for reproduction and further extension.
Decision: Accept-Main