Self-training with Two-phase Self-augmentation for Few-shot Dialogue Generation (2022)

Abstract: In task-oriented dialogue systems, response generation from meaning representations (MRs) often suffers from limited training examples, due to the high cost of annotating MR-to-Text pairs. Previous works on self-training leverage fine-tuned conversational models to automatically generate pseudo-labeled MR-to-Text pairs for further fine-tuning. However, some self-augmented data may be noisy or uninformative for the model to learn from. In this work, we propose a two-phase self-augmentation procedure to generate high-quality pseudo-labeled MR-to-Text pairs: the first phase selects the most informative MRs based on model’s prediction uncertainty; with the selected MRs, the second phase generates accurate responses by aggregating multiple perturbed latent representations from each MR. Empirical experiments on two benchmark datasets, FewShotWOZ and FewShotSGD, show that our method generally outperforms existing self-training methods on both automatic and human evaluations.

## introduction in task-oriented dialogue systems, a natural language generation (nlg) module is an essential component: it maps structured dialogue meaning representations (mrs) into natural language responses. the nlg module has a great impact on users' experience because it directly interacts with users using text responses @xcite @xcite . however, in real-world applications, developers often only have a few well-annotated data and confront a high data collection cost in specific domains. this real-world challenge makes building an nlg module in the low-data setting a valuable research problem @xcite @xcite . 1 please check the code, data, and evaluation scripts of this work at: https://github.com/wyu-du/ self-training-dialogue-generation self-augmented data e[p θ ] v ar[p θ ] 1 request (@xmath0: examples of our self-augmented data and data selection strategy. text is the input mr (e.g. request is the dialogue intent, and (@xmath1. the model p θ generates synthetic dialogue response conditioning on the text . for each self-augmented data, a low predictive mean e[p θ ] indicates that the model finds the augmented data "too noisy" (e.g. out-of-domain or invalid response), and a low predictive variance v ar[p θ ] indicates that the model finds the augmented data "too certain" (e.g. uninformative response). in this work, we propose to select examples with high e[p θ ] and high v ar[p θ ]. while language models have been widely adopted to build the nlg module in task-oriented dialogue systems, they usually require thousands of mr-to-text pairs for learning the domain-specific knowledge @xcite @xcite . to collect more training data under a feasible budget, previous works propose three general approaches: (1) designing handcraft rules to augment new data, which is hard to scale up @xcite ; ideally, the augmented data should help the model better learn the domain-specific knowledge. however, some augmented data can be "too noisy" that leads the model to learn irrelevant or inappropriate data patterns. this phenomenon is also described as negative transfer in other works @xcite @xcite . to address this challenge, some works leverage human judgements to filter out the "too noisy" augmented data, which are difficult to scale up across different domains and tasks @xcite . other works train task-specific discriminators to pick up the valid augmented data, which are likely to overfit in the low-data setting @xcite @xcite @xcite . in this work, we propose to address the issue of selecting high-quality self-augmented examples with a two-phase procedure, where each phase will take care of selecting inputs and generating outputs independently. as illustrated in figure 1 , the first phase evaluates input mrs with model's prediction uncertainty, aiming at selecting input examples that are informative to the current model. specifically, for each input mr, we let the current model generate a response, and then apply the monte carlo dropout method @xcite to estimate the predictive mean e[p θ ] and predictive variance v ar[p θ ] of the generated response. in uncertainty quantification @xcite , high predictive mean indicates that the model is familiar with this input (i.e. in-domain data) and high predictive variance reflects that the model is sensitive to this input (i.e. informative data). hence, we propose to select input mrs with high predictive mean and variance. note that our uncertainty-based data selection strategy neither requires training additional neural models to select the valid data @xcite @xcite , nor need to calculate the data statistics across all training epochs and re-train the model overall again @xcite . the second phase aims at further improving the quality of the selected data. we adopt an idea from contrastive representation learning @xcite and use the aggregation of randomly perturbed latent representations to help the model produce more accurate responses. the combination of these two phases guarantees the proposed method selects more informative mr inputs and generates less noisy responses for further model fine-tuning. in summary, the contributions of this work are as follows: ## related works task-oriented dialogue generation. previous nlg methods generate system responses by: (1) designing handcraft response templates and filling in slot-value pairs from system actions, or (2) building data-driven neural models, which encode systems actions into latent feature representations and decode natural language responses with more diversity in realization. however, both approaches cause high data collection costs. the template-based methods @xcite require collecting a comprehensive set of templates to cover all possible combinations of dialog acts and slot-value pairs, while data-driven methods @xcite @xcite require collecting thousands of system action and response pairs to ensure the neural model generating fluent responses. few-shot nlg. recent works on few-shot nlg mainly focus on developing or adapting pre-trained language models. @xcite presents the first few-shot nlg benchmark for task-oriented dialog systems, and develops a pre-trained language model which can be fine-tuned with only a few domain-specific labels to adapt to new domains. @xcite applies the switch mechanism to combine the information from both input data and pre-trained language models, which achieves good performance in table-to-text generation tasks. @xcite studies the training data selection strategies in few-shot nlg, and finds that clustering-based selection strategy consistently helps generative models get better performance than randomly sampling. ## proposed method in task-oriented dialogue systems, the nlg module translates a structured dialogue meaning representation a into a natural language response @xmath2, ..., x t }. one structured dialogue meaning representation a consists of k dialogue intents and a list of slot-value pairs for each intent: we define p θ (x | a) as the generation model that generates the response x in an auto-regressive way conditioning on a: ## experiments we conduct experiments to answer three research questions: (1) is sa 2 self-training algorithm a helpful method to deal with the few-shot dialogue generation problem? (2) can our data selection strategy effectively filter out the "too noisy" and "uninformative" augmented data? (3) can our response refinement method help improve the performance of the nlg model? unlabeled data. the two benchmark datasets are sampled and constructed based on the three datasets: rnnlg @xcite , multiwoz @xcite and sgd @xcite . to ensure the input mrs are within the same domain of the original training set d l , we collect all augmented mrs from the training set of rnnlg, multiwoz, and sgd. for few-shotwoz, we collect an average number of 9,080 unlabeled mrs per domain. for fewshotsgd, we collect an average number of 7,532 unlabeled mrs per domain. the detailed data statistics of each domain are demonstrated in appendix b. baselines. we compare our method with four baselines and describe the model configuration and training details in appendix c. (1) sc-gpt @xcite is the state-of-the-art pre-trained language model for nlg in task-oriented dialogue systems, which is further fine-tuned on each spe-cific domain using the original training data d l ; ## conclusions in this work, we present a two-phase selfaugmentation self-training algorithm to deal with the few-shot dialogue generation problem in taskoriented dialogue systems. we propose to select informative input mrs based on model's prediction uncertainty, and improve the pseudo response generation by aggregating randomly perturbed latent representations. empirical experiments on two fewshot nlg datasets show that our proposed method achieves the best performance among other baselines in both automatic and human evaluations.