Turning Flowchart into Dialog: Augmenting Flowchart-grounded Troubleshooting Dialogs via Synthetic Data Generation (2023)

Abstract: Flowchart-grounded troubleshooting dialogue (FTD) systems, which follow the instructions of a flowchart to diagnose users’ problems in specific domains (e.g., vehicle, laptop), have been gaining research interest in recent years. However, collecting sufficient dialogues that are naturally grounded on flowcharts is costly, thus FTD systems are impeded by scarce training data. To mitigate the data sparsity issue, we propose a plan-based synthetic data generation (PlanSDG) approach that generates diverse synthetic dialog data at scale by transforming concise flowchart into dialogues. Specifically, its generative model employs a variational-base framework with a hierarchical planning strategy that includes global and local latent planning variables. Experiments on the FloDial dataset show that synthetic dialogue produced by PlanSDG improves the performance of downstream tasks, including flowchart path retrieval and response generation, in particular on the Out-of-Flowchart settings. In addition, further analysis demonstrate the quality of synthetic data generated by PlanSDG in paths that are covered by current sample dialogues and paths that are not covered.

## introduction flowchart-grounded troubleshooting dialogue (ftd) systems @xcite @xcite @xcite @xcite , which communicate with users to help them diagnose problems through the guidance of a flowchart, have been gaining interest in recent years. ftd systems face additional challenges to those faced by typical task-oriented dialogue systems @xcite , e.g., ftd systems must accurately follow the instructions of a flowchart, actively detect the root cause of issues, and provide users with reasonable solutions by following an action instruction along with the path in a flowchart (figure 1 ). collecting sufficiently large flowchart-related dialogue corpora for ftd is challenging, since it requires domain experts with relevant knowledge. this problem also applies to a crowd-sourced ftd corpus, such as flodial @xcite , whose collection still involved a great deal of human effort. despite this, the 1,789 dialogues in flodial ( § 3.1) cover only 65% of the paths in the underlying flowcharts on average (figure 2 ). an alternative approach to obtaining additional dialogues could involve crawling through websites. however, most of these data obtained in this manner focus on anecdotes and subjective opinions @xcite , and are thus unsuitable for ftd systems. in this paper, we propose plansdg: a plan-based synthetic data generation approach that generates synthetic dialogues from flowchart paths. specifically, plansdg takes as input a path extracted from an underlying flowchart, and generates a dialogue session consisting of dialogue acts and utterances. plansdg is formalised as a probabilistic generative model with structured planning latent variables, specifically global and local latent variables, that guide the generation process. the global latent variables are responsible for modeling the dialogue acts between the dialogue turns, providing a highlevel sketch. to be able to model these global variables, we manually labeled the dialogue acts for the utterances in the flodial dataset. the local latent variables control the diversity of generated synthetic dialogues during sentence realization. we conducted extrinsic and intrinsic evaluations of our approach on the flodial corpus, as well as follow-up ablation studies. our extrinsic evaluation shows that the retrieval and generative models trained on the synthetic dialogues produced by plansdg achieve better performance than other augmentation methods in terms of the downstream tasks: flowchart path retrieval and response generation, particularly on the out-of-flowchart settings. our intrinsic evaluation, which examines the quality of the synthetic dialogues, indicates that plansdg outperforms strong baseline models in term of diversity and faithfulness. our ablation studies demonstrate the effectiveness of our proposed global and local latent planning variables. further analysis demonstrate the quality of synthetic data generated by plansdg in uncovered paths that are included by flowchart but not in dialogues. ## conclusions in this paper, we explore the synthetic dialogue generation as a data augmentation approach with pre-trained model for flowchart-grounded troubleshooting dialogue systems. in further, in order to incorporate dialogue-specific features efficiently, we present a planning-based generative model plansdg for generating synthetic dialogues on troubleshooting dialogue task. the generated augmented dataset is then used to train an ftd systems. experiments on the flodial benchmark show the effectiveness of our proposed method. in the future, we plan to generalise our method to more complex dialogues, and apply it to other tasks.