A Synthetic Data Generation Framework for Grounded Dialogues (2023)

Abstract: Training grounded response generation models often requires a large collection of grounded dialogues. However, it is costly to build such dialogues. In this paper, we present a synthetic data generation framework (SynDG) for grounded dialogues. The generation process utilizes large pre-trained language models and freely available knowledge data (e.g., Wikipedia pages, persona profiles, etc.). The key idea of designing SynDG is to consider dialogue flow and coherence in the generation process. Specifically, given knowledge data, we first heuristically determine a dialogue flow, which is a series of knowledge pieces. Then, we employ T5 to incrementally turn the dialogue flow into a dialogue. To ensure coherence of both the dialogue flow and the synthetic dialogue, we design a two-level filtering strategy, at the flow-level and the utterance-level respectively. Experiments on two public benchmarks show that the synthetic grounded dialogue data produced by our framework is able to significantly boost model performance in both full training data and low-resource scenarios.

## introduction grounded dialogue systems are designed to engage in conversation with humans by incorporating external knowledge to provide relevant and informative responses @xcite @xcite . in recent years, various advanced techniques have been developed to train grounded dialogue models @xcite @xcite . despite the notable progress, training these models often requires large amounts of data. however, it is expensive and time-consuming to build a collection huskies are also today kept as pets, and groups work to f ind new pet homes for retired racing and adventure trekking dogs. dialogue dialogue flow i just got a husky puppy. it sounds cute! huskies are known amongst sled-dogs for their fast pulling style. i guess in the north they are working dogs huh? sled dogs, including huskies, are used for transportation in arctic areas. that is so cool and probably helpful but mine is just a pet. that's not uncommon! there are rescue groups that specialize in finding homes for retired sled dogs. figure 1: an illustrated example from the wizard of wikipedia dataset @xcite . this example shows the dialogue flow in knowledge-grounded dialogues, i.e., a sequence of knowledge pieces. as each agent response is grounded to a specific piece of knowledge, the dialogue flow implies the outline of the conversation. one solution is to generate grounded dialogue data from unstructured knowledge, by using large pre-trained language models (lms). previous work on this topic has explored synthetic dialogue data generation with reinforcement learning @xcite or user simulation @xcite . however, a key missing component in all these methods is the modeling of dialogue flow. dialogue flow can be viewed as the outline of a dialogue. the flow reflects the dialogue's content and trajectory, i.e., the topics discussed in each session and the topic shifts between sessions. we consider the dialogue flow of a grounded dialogue as the sequence of the grounded knowledge pieces. figure 1 shows an example dialogue along with 10866 its associated dialogue flow. in this example, the grounded knowledge is primarily from a wikipedia page about "husky" dogs. this dialogue follows a smooth knowledge flow, transitioning from "husky" to "sled dogs" and then to "huskies as pets". however, if we replace the second knowledge piece with "'esquimaux' or 'eskimo' was a common term for pre-columbian arctic inhabitants of north america.", which is also from the same wikipedia page, then the flow becomes less consistent. as the backbone guiding the dialogue generation process, a carefully planned dialogue flow is crucial for the coherence and smoothness of the resulting dialogue. to this end, we propose a novel framework named syndg, to synthetically generate coherent grounded dialogues. the generated dialogues are meant to be used as auxiliary training data. in syndg, we first determine the dialogue flow by task-specific heuristics, from the unstructured knowledge data (e.g., wikipedia pages, persona profiles, etc.). then, we employ t5 @xcite , a large pre-trained lm, to transform the dialogue flow into a synthetic dialogue, with sequential utterance generation, one at a time. to ensure the quality of the synthetic dialogue, we propose a two-level filtering strategy based on t5: flow-level filtering and utterance-level filtering. the flowlevel filtering is designed to select dialogue flows with higher consistency, whereas the utterancelevel filtering aims to eliminate the synthetic dialogues with poor coherence. we conduct experiments on two grounded dialogue benchmarks, in both full training data and low-resource scenarios. we use the synthetic grounded dialogue data produced by our framework as additional training data for commonly used grounded dialogue models. both the automatic and human evaluation results show that our synthetic data leads to significant improvement on model performance. further analysis also reveals that model performance increases along the increase in the number of synthetic dialogues. ## methodology our framework, named syndg, is illustrated in figure 2 . syndg first explicitly constructs a dialogue flow by task-specific heuristics, then realizes it into a synthetic dialogue with pre-trained models. further, a two-level filtering strategy is proposed to ensure the quality of synthetic dialogues. ## conclusion in this paper, we propose a framework, syndg, to automatically construct synthetic training data for the grounded dialogue task. we first construct dialogue flows based on unstructured knowledge, then transform them into synthetic dialogues by large lms, and finally filter and retain the generated dialogues with high quality. the experimental results demonstrate the effectiveness of our proposed framework in both full training data and lowresource scenarios. further analysis shows that the model performance tends to increase as the number of synthetic dialogues increases. for future work, we plan to investigate more efficient strategies for determining dialogue flows and take larger lms to produce synthetic dialogues with higher quality.