Plan, Generate and Complicate: Improving Low-resource Dialogue State Tracking via Easy-to-Difficult Zero-shot Data Augmentation (2024)

Abstract: Data augmentation methods have been a promising direction to improve the performance of small models for low-resource dialogue state tracking. However, traditional methods rely on pre-defined user goals and neglect the importance of data complexity in this task. In this paper, we propose EDZ-DA, an Easy-to-Difficult Zero-shot Data Augmentation framework for low-resource dialogue state tracking that utilizes large language models to automatically catch the relationships of different domains and then generate the dialogue data. We also complicate the dialogues based on the domain relation to enhance the model’s capability for co-reference slot tracking. Furthermore, we permute slot values to mitigate the influence of output orders and the problem of incomplete value generation. Experimental results illustrate the superiority of our proposed method compared to previous strong data augmentation baselines on MultiWOZ.

## introduction the data scarcity challenge in dialogue state tracking (dst) is significant due to the incessant emergence of new domains in task-oriented dialogue systems (tods) and the high costs associated with data annotation. currently, large language models (llms) like chatgpt have shown promising results in zero-shot dst @xcite . however, although these models achieve superb performance, they have significant limitations such as closed source, request limitations, and deployment difficulties @xcite . therefore, a smaller, fine-tuned model is a more practical and cost-efficient choice for dst. nevertheless, developing such a powerful small model faces a big challenge in the absence of training data. ## methodology figure 2 illustrates the process of our data augmentation. first, we prompt the llm to judge whether it is reasonable for different combinations of domains to appear in one dialogue, and then generate the seed state, where the seed state describes the domains and co-reference information within a dialogue. second, we synthesize diverse dialogue states based on the seed states. finally, a series of tasks are proposed to generate the labeled dialogue for each synthetic dialogue state and then complicate them based on the co-reference information. dialogue state template user goal task 1: user goal generation dialogue template complicated dialogue task 4: dialogue complication co-reference information dialogue state template dialogue flow task 2: dialogue flow planning user goal db information dialogue flow template dialogue task 3: dialogue generation db information synthetic dialogue state construction dialogue state constructor ontology synthetic dialogue state seed states co-reference information db restaurant-area：hotel-area hotel-area: center, restaurant-food: chinese, … turn_1: i want a hotel in the center with wifi. … turn_n: i also need a restaurant in the center. turn_1: {hotel-area: center, hotel-internet: yes} … turn_n: {restaurant-area: center} turn_1: i want a hotel in the center with wifi. … turn_n: i also need a restaurant near the hotel. seed state generation template schema judgement template judgement seed states seed state 1: hotel-area hotel-type … restaurant-area restaurant-food hotel{area, type, parking, …}, taxi{departure, leave at, …}, … the combination of hotel and restaurant is reasonable hotel: name: xxx, address: xxx, phone: xxx, … … the user want to find a hotel, …, and a restaurant in the same area…. and 12 give an example of our dialogue generation method. ## conclusions and future work in this paper, we propose edz-da, an easy-todifficult zero-shot data augmentation framework for low-resource dst. we reveal three issues in constructing dst data and propose to first determine the logical relationship among domains and generate the user goal with the help of the llm's strong reasoning ability. in order to enhance the dst model's performance in tracking co-reference slots, we propose to complicate the dialogue content based on the domain relationship. moreover, we propose to permute slot values to mitigate the influence of output order and the incomplete generation problem. experimental results on the multi-woz dataset illustrate the superiority of edz-da over previous data augmentation approaches for low-resource dst. in future work, we will further study how to generate diverse natural dialogue flows.