q2d: Turning Questions into Dialogs to Teach Models How to Search (2023)

Abstract: One of the exciting capabilities of recent language models for dialog is their ability to independently search for relevant information to ground a given dialog response. However, obtaining training data to teach models how to issue search queries is time and resource consuming. In this work, we propose q2d : an automatic data generation pipeline that generates information-seeking dialogs from questions. We prompt a large language model (PaLM) to create conversational versions of question answering datasets, and use it to improve query generation models that communicate with external search APIs to ground dialog responses. Unlike previous approaches which relied on human written dialogs with search queries, our method allows to automatically generate query-based grounded dialogs with better control and scale. Our experiments demonstrate that: (1) For query generation on the QReCC dataset, models trained on our synthetically-generated data achieve 90%-97% of the performance of models trained on the human-generated data; (2) We can successfully generate data for training dialog models in new domains without any existing dialog data as demonstrated on the multi-hop MuSiQue and Bamboogle QA datasets. (3) We perform a thorough analysis of the generated dialogs showing that humans find them of high quality and struggle to distinguish them from human-written dialogs.

## introduction recent dialog generation models, such as lamda @xcite , blenderbot3 @xcite and sparrow @xcite use an external search api to generate grounded and factually accurate responses @xcite . this is important for providing reliable and consistent answers @xcite , especially when discussing entities and asking related questions with anaphora. to do this, these models use a query generation component that is trained on dialog-to-search-query datasets. when the model is triggered with a dialog turn that requires search, it generates a query that is used to obtain a search result, which is then used to generate a grounded response. this allows the model to provide relevant information about the world in its responses to user queries. for example, a model trained in 2021 should be able to provide a factual response to the question "how old is joe biden?" even in 2023. in a conversation, one might discuss an entity (e.g. "joe biden") and later ask a related question (e.g. "how old is he?") with anaphora. in order to provide reliable and consistent answers, it is necessary to generate a decontextualized query (e.g., "how old is joe biden") for a search engine. using apis also decouples language and reasoning from knowledge @xcite , which can help prevent errors caused by outdated information being stored in the model's parameters. for example, if a model trained at the end of 2021 is asked "how old is the current president?", it may produce the incorrect query "how old is donald trump" if its parameters are outdated or if it provides factually-inconsistent responses (a.k.a "hallucinations"). query generation datasets have been created using human annotators, limiting them in scale, control, and quality @xcite . as a result, when a new domain is introduced, a significant amount of human effort is required to create a new query generation dataset for that domain @xcite . the fact that language models often generate hallucinations @xcite @xcite , especially in new domains or dialogs that differ from the training data @xcite @xcite , highlights the need for more figure 1 : left: our q2d method starts from an existing query or question and prompt a few-shot language model to transform it into a dialog. we filter out cases where the intent of the generated dialogue differs from the intent of the initial query and apply additional filters. right: we take a question from the qrecc dataset (surrounded by a rectangle) and generate an informationseeking dialog with q2d. by starting with a query and generating a dialog, we create {@xmath0, which is used to train and evaluate query generation models, which communicate with an external search api to generate factual responses. in this work, we propose a data generation pipeline to improve grounded dialog models with access to search engines. to create a dialog-tosearch-queries dataset for training the query generation component in such models, we reverse the process, starting from a search query and generating an information-seeking dialog that corresponds to that query. our automatic pipeline, shown in figure 1 , begins with a search query or question, and prompts a large language model (palm; @xcite to generate a conversational dialog that conveys the information need implied by the given query. for example in figure 1 , we take the question "who played ardra on star trek the next generation?" from the natural questions dataset @xcite and generate a dialog with a similar intent: the correct answer to the original question ("marta dubois") is also a correct response to the generated dialog. this process allows us to leverage existing question-answering datasets, which are widely available for different domains, and extend them by generating dialogs that preserve the original information need while controlling the dialog domain and style. to assess whether the automatically generated dialogs can replace human-generated dialogs, we experiment with qrecc nq @xcite , a human-curated dialog dataset. we generate a training set that is the same size as the original dataset, but with synthetic dialogue, and use it to train a query generation model. the resulting model obtains 90%-95% of the performance of models trained on the human-generated training data, using the same metrics used to evaluate qrecc @xcite . other than training query generation models, our approach is also useful for training the dialog generation models themselves when no dialog data is available for a new domain. we demonstrate that on the domain of multi-hop question answering, where we first show that existing dialog models struggle to perform well on a domain-specific challenge set. we then generate synthetic dialog data from the musique @xcite multi-hop qa dataset, and show that training a dialog model on this data improves performance. we provide a thorough analysis of the quality of the generated datasets, demonstrating that they (a) looks natural, and humans struggle to distinguish the synthetic dialogs from natural; (b) factual: generated and human-annotated answers perform similarly in query generation; (c) correct: dataset labels are accurate, and strict filtering improves results. to conclude, our main contributions are: in this section, we describe our automatic method, called q2d, for generating dialogs from questions, and the properties of datasets produced by this algorithm 1 generate dialogues from questions input few-shot model m f s , qa dataset (q, a), examples queries s @xmath1, d i )} k @xmath2, examples dialogues s @xmath3, q i )} k @xmath4, instructions query i, instructions dialogue i r , execute dataset ← ∅ for (q, a) ∈ (q, a) do dialogue ← m (s q , i, q) q ′ ← m f s (s d , i r , dialogue) if filter(dialogue,q,q',a) then dataset.add((dialogue, q, a)) output query generation dataset: @xmath5, q i )} |q| @xmath6. our goal is to reduce the effort associated with generating a training dataset for training generation, and to improve query-generation-based dialog models with a high-quality training dataset. query generation can start by extracting queries from existing dialogs. however, our approach is unique in that it begins with factual queries or questions, allowing us to leverage existing resources. any question-answering dataset, queries dataset, or queries used in popular web search services or dialog model logs can be used with our algorithm. the algorithm is described in algorithm 1 and consists of three main steps: filtering. in this part we attempt to filter (dialog, query) samples that would not be beneficial for training or testing. we do it in three steps, elaborated below. we stress that there are many more filtering strategies possible, and exploring them is left for future work. first, we filter out dialogs whose intent is different from the original query by measuring the similarity between the query and its reversed version using sbert similarity (sim(q, q ′ )) and comparing it to a threshold (t query ). if the similarity is below the threshold, the generated query is considered to have a different intent and the dialog is filtered. appendix a, section a.2 shows several examples of dialogs, original and reversed query and sbert semantic similarity. second, we filter out cases where the answer is included in the dialog by measuring the n-gram overlap between the dialog and the answer using the rouge metric @xcite . if the overlap is above a threshold (t answer ), the answer is entailed in the dialog and the example is filtered. for example, if the final answer ("marta debois") would have been already written in the dialog for the role of playing ardra, the final question ("who played ardra") would not make sense. finally, we filter out cases where the last turn of the dialog is similar (>80%) to the original question using sbert similarity. these cases include situations where no anaphora is required. in this work, we use palm @xcite , a large language model with 540b parameters, as the few-shot language model for generating dialogs with a temperature of 0.6. we provide a fully working code with gpt-3 @xcite for reproducibility. the set of prompts and instructions can be found in appendix a, section a.3. for the similarity metric (sim), we use the allmpnet-base-v2 model from sentence transformers, with a threshold similarity of t @xmath7.999. this threshold is justified through human-evaluation and ablation studies for the filtering in section 5.3. ## replacing human-annotated with auto-generated data in this section, we evaluate the extent to which our automatically generated dataset can replace the human-annotated dataset. we use the qrecc nq dataset @xcite , which contains (dialog, query) pairs, and automatically generate a dialog from natural questions. this allows us to create an automatically generated train set of the same size, and compare it to the human-annotated dataset. an example of a human-generated dialog compared to an automatically generated dialog is shown in figure 2 . we use the version of the dataset where the intermediate questions are contextualized, rather than decontextualized. for example, the second and third user turns in the figure are contextualized versions of the decontextualized questions "who directed the film, the vikings?" and "was the film the vikings based on a novel?". dataset generation. to generate our dataset, we use our q2d method as described in section 2. for the few-shot examples of queries and dialogs (s q and s d ), we sample 15 examples from qrecc that fit within the maximum input sequence length. these examples are available in appendix a, section a.3. for the base questions (q), we use the natural questions @xcite dataset instead of the qrecc nq questions to reduce dependence on qrecc. importantly, all of the questions and dialogs in the natural and automatically generated datasets are disjoint. in total, we generate 13k samples, the same as the qrecc nq train set. full prompts, instructions and examples are available in appendix a, section a.1. ## conclusions we introduced q2d, a data generation pipeline that produces dialogs based on questions. we demonstrated that our method can replace humanannotated data to train query-generation models, and to create effective, natural, factual, and accurate evaluation and training data in new domains, even when no existing dialogue data is available. computational costs. the process of autogenerating data with large language models, although faster and more scalable than human annotations, still incurs significant computational costs. these costs, however, are expected to decrease as ai technologies advance.