DFLOW : Diverse Dialogue Flow Simulation with Large Language Models (2025)

Abstract: Developing language model-based dialogue agents requires effective data to train models that can follow specific task logic. However, most existing data simulation methods focus on increasing diversity in language, topics, or dialogue acts at the utterance level, largely neglecting a critical aspect of task logic diversity at the dialogue level. This paper proposes a novel data simulation method designed to enhance the diversity of synthetic dialogues by focusing on task execution logic. Our method uses LLMs to generate decision tree-structured task plans, which enables the derivation of diverse dialogue trajectories for a given task. Each trajectory, referred to as a “dialog flow”, guides the generation of a multi-turn dialogue that follows a unique trajectory. We apply this method to generate a task-oriented dialogue dataset comprising 3,886 dialogue flows across 15 different domains. We validate the effectiveness of this dataset using the next action prediction task, where models fine-tuned on our dataset outperform strong baselines, including GPT-4. Upon acceptance of this paper, we plan to release the code and data publicly.

## introduction a task-oriented llm agent typically needs to manage user requests according to a designated "plan guide" which includes predefined task logics and policy constraints. this guideline could indicate different task trajectories or flows that lead to task completions. for example, when exploring a car rental service, the agent will first ask if the user has a preferred rental company, if yes, then the agent will further let the user to specify the preferred company from a constrained list. previous research @xcite @xcite has introduced dialogue flows to enable dialogue agents to follow task logic and constraints when addressing user requests. however, manually curating such dialogue flows is challenging due to * work performed during an internship at aws ai labs. the intricate task logic and alignment with policy constraints across various domains. consequently, existing task-oriented dialogue datasets often suffer from sparse flow annotations, and lack efficient and generalizable mechanisms for automated dialogue flow generation @xcite @xcite @xcite . in this work, we aim at designing an automatic data simulation method to generate task-oriented dialogues with structured dialogue flows. previous studies have utilized llms to generate taskoriented dialogues, focusing primarily on enhancing linguistic diversity @xcite , exploring varied topics @xcite , or proposing different combinations of dialogue acts at the utterance level @xcite . however, these efforts neglect the crucial need for maintaining task logic consistency at the dialogue level. our work proposes to use llms to generate diverse dialogues that consistently follow the task logic and constraints defined by dialogue flows. to achieve this goal, we propose a dialogue simulation framework that leverages the planning capabilities of llms @xcite to automatically construct diverse dialogue flows. starting with a task instruction, our framework employs an llm planner to generate a decision tree-structured task plan that outlines diverse trajectories. then depth-first-search is applied to parse all valid trajectories within this plan. each trajectory represents a dialogue flow, ensuring a coherent task logic to effectively solve the given task. subsequently, our framework uses the simulated dialogue flows to control an llm synthesizer to generate multi-turn dialogues that follow task logics at the dialogue level. as shown in figure 1 , the llm planner can simulate diverse dialogue flows, and the llm synthesizer can generate coherent dialogues based on different dialogue flows. to evaluate the effectiveness of the proposed framework, we construct a synthetic dataset dflow and conduct extensive empirical experiments. intrinsic evaluation on dflow shows that this synthetic dataset obtains high quality task plans, coherent flows and faithful dialogues across 130 tasks in 15 domains. extrinsic evaluation demonstrates that 7b language models fine-tuned on dflow dataset can outperform strong llms such as gpt-4 (openai, 2024) in the next action prediction task @xcite . in summary, this work introduces a novel data simulation method to synthesize diverse multi-turn dialogues that follow task logic and constraints. the simulated dataset enhances the dialogue understanding capabilities of small language models, enabling them to outperform more advanced llms such as gpt4 in the next action prediction task. task planning with llms. recent works enhance the planning ability of llms to solve complex tasks. @xcite proposes a tree-ofthought prompt to frame the problem solving as a search over a tree, and design search algorithms to obtain the best solution path to the problem. @xcite designs a plan-and-solve prompting method, which generates a plan to divide the entire task into smaller subtasks. @xcite introduce a logic-of-thought prompt that employs propositional logic to generate expanded logical information from input context, further enhancing the logical reasoning capability of llms. @xcite proposes an adaptive planning algorithm that explicitly plans and decomposes complex sub-tasks when the llm is unable to execute them. @xcite leverages a reward model to score action trajectories and provide heuristics for task planning. in this work, we leverage the task planning ability of llms to generate treestructured task plans. in contrast with enhancing the planning ability to find the optimal task trajectory, this work focuses on exploring diverse task trajectories that cover diverse task scenarios. ## dflow simulation framework figure 1 provides an overview of our framework that includes three steps: (1) generating task plan based on task instruction, (2) sampling dialogue flows from task plan, and (3) generating diverse and coherent dialogues based on sampled flows. step 1: task plan generation. given a task instruction and one in-context example, the llm planner is prompted to generate a decision treestructured task plan that outlines multiple trajectories, as shown in figure 5 . since we aim at prompting the diversity of task logic at the dialogue-level, we adopt the decision tree-structured task plan to cover diverse trajectories. each step in the plan has two components: (1) a system action which collects user information in order to fulfill user requests; (2) a set of values which guides or constrains the system when performing certain action. we categorize our system actions by viewing them as different types of nodes in the decision tree, in order to reflect different structures of generated task plans. here are the details of each system action: ## experiments this section introduces the dataset construction details in subsection 4.1, the intrinsic data diversity and quality evaluation in subsection 4.2, and the extrinsic data effectiveness evaluation for building llm agents in subsection 4.3. ## conclusions in this work, we propose a novel data simulation framework to synthesize diverse task-oriented dialogues that follow task logic and constraints for building llm agents. we leverage the proposed framework to construct the dflow dataset with 3.8k fine-grained dialogue flow annotations across 15 domains. empirical experiments show that the dflow dataset achieves comparable data quality with human annotations, and significantly improves 7b models' performance in the next action prediction task, outperforming strong llms such as gpt-4.