D iact TOD : Learning Generalizable Latent Dialogue Acts for Controllable Task-Oriented Dialogue Systems (2023)

Abstract: Dialogue act annotations are important to improve response generation quality in task-oriented dialogue systems. However, it can be challenging to use dialogue acts to control response generation in a generalizable way because different datasets and tasks may have incompatible annotations. While alternative methods that utilize latent action spaces or reinforcement learning do not require explicit annotations, they may lack interpretability or face difficulties defining task-specific rewards. In this work, we present a novel end-to-end latent dialogue act model (DiactTOD) that represents dialogue acts in a latent space. DiactTOD, when pre-trained on a large corpus, is able to predict and control dialogue acts to generate controllable responses using these latent representations in a zero-shot fashion. Our approach demonstrates state-of-the-art performance across a wide range of experimental settings on the MultiWOZ dataset, including zero-shot, few-shot, and full data fine-tuning with both end-to-end and policy optimization configurations.

## introduction task-oriented dialogue systems have become increasingly prevalent in recent years, leading to a growth in research on related topics such as dialogue response generation. previous work @xcite found that incorporating dialogue act annotations, representing the illocutionary level of utterances, can enhance the quality of generated responses. despite the importance of dialogue act annotations, collecting them can be a time-consuming process that requires human effort. furthermore, existing annotations for dialogue acts are scattered across different datasets and may use different labeling schemes, making it difficult to generalize across tasks. as a result, * work performed during an internship at aws ai labs. ## related work response generation is an important task in taskoriented dialogue systems. there have been many previous approaches @xcite @xcite @xcite @xcite proposed to improve the task-oriented dialogue systems. one direction is the use of dialogue act annotations to improve the quality of responses in task-oriented dialogue systems. for example, simpletod (hosseini-asl et al., 2020) and ubar @xcite generate dialogue acts as part of the response generation process. pp-tod @xcite uses the context as a prompt and dialogue act generation for multi-task learning. recently, galaxy @xcite proposed a method that uses pre-training on a large corpus of dialogues with dialogue act annotations as an auxiliary objective to improve the quality of the generated responses. however, these methods are limited by the fact that different datasets may have incompatible or inconsistent dialogue act annotations for learning generalizable representations. to address this problem, previous work @xcite has attempted to define a new universal schema for dialogue acts. however, these approaches are either overly simplified or require additional human annotations, limiting their effectiveness and practicality. in addition to using explicit annotations of dialogue acts, researchers have also explored alternative methods to improve response generation, such as using latent action spaces and implementing reinforcement learning techniques. these approaches aim to improve the overall task success rate of generated responses. larl @xcite uses latent dialogue acts trained with reinforcement learning instead of surface-form dialogue acts to control response generation which results in the best task score. lava @xcite further improves over larl by utilizing a variational autoencoder (vae) to learn an informed and semantic prior when optimizing the latent action spaces, achieving state-of-the-art success and inform scores on multiwoz. krls @xcite is another recent approach that applies reinforcement learning to pre-trained language models. this approach utilizes a specifically designed objective function that focuses on learning the keywords in the input, with the goal of improving the overall performance of the language model. in our work, we adopt a sim-ilar approach but use dialogue act annotations to assign semantic meanings to the latent representations, allowing the model to learn generalizable and controllable latent dialogue acts, which improves the quality of generated response. pre-training with a large corpus of dialogues has been a widely adopted technique to enhance the response generation quality in dialogue systems @xcite . in the context of task-oriented dialogue systems, several recently proposed approaches have demonstrated the effectiveness of pre-training. galaxy @xcite pre-trains the model with a collection of dialogue datasets with dialogue act annotations. godel @xcite uses a larger dataset and model size, and it also incorporates the grounding of database results in the context. this allows it to achieve good performance under few-shot settings on the multiwoz dataset. in contrast, our work uses a smaller set of pre-training datasets but with more robust data processing techniques. we use the complete dialogue acts in each dataset without any simplification. we also train another model tanl @xcite to delexicalize the pre-training data to improve the model's zero-shot and few-shot capabilities. ## diacttod approach in this section, we first provide a brief overview of the traditional end-to-end task-oriented dialogue systems. then, we delve into the specifics of how our proposed latent dialogue act model operates, by providing details on both its training and inference processes, which offers a new approach to modeling dialogue acts. finally, we discuss how this model can be used to control response generation for a more efficient and accurate dialogue system. ## pre-training latent acts to ensure consistency across all datasets for pretraining, we pre-process the datasets with the same tokenization and truncation of dialogues when they exceed a certain length. additionally, we incorporate database search results as an input token to indicate the number of matches. a large portion of utterances in these datasets do not have dialogue act annotations. to effectively pre-train on those datasets, we utilize the system response as a proxy for the dialogue act. this allows the policy model to generalize to new and unseen dialogue acts. our experiments have shown this approach to be effective. in task-oriented response generation, system responses are typically in a delexicalized form, which means that specific values of certain variables are replaced by placeholders. to enable this automatic delexicalization during response generation, we use the model tanl (translation between augmented natural languages) @xcite . this model can extract slot spans from the input sentence. we fine-tune the tanl model with the sgd's predefined slot definitions. for downstream tasks and evaluation, we ensure compatibility by defining a one-to-one mapping of the sgd's slot definitions with the slots in the multiwoz dataset. ## experiment setup we initialize our model with t5-base and pretrain our model on the previously mentioned datasets. we evaluate our model on the multidomain task-oriented dialogue dataset multi-woz @xcite . it contains 8,438/1,000/1,000 dialogues for training, validation, and testing, respectively. there are seven different domains, including hotel, hospital, police, restaurant, train, and taxi. we use multiwoz 2.2 @xcite to be compatible with the standardized evaluation script @xcite . we evaluate our approach under different scenarios, such as zero-shot, few-shot, and finetuning with the full dataset, with both end-to-end and policy optimization configurations to evaluate the robustness and flexibility of our model. we use standardized evaluation metrics foot_0 with inform, success rates, and bleu scores. inform measures the extent to which the system provides sufficient and relevant information to fulfill the user's information needs. success evaluates the performance in completing the user's goal. also, we evaluate the model's zero-shot dialogue act prediction capabilities on an unseen dataset. to provide a comprehensive evaluation, we separately compare our model's performance against several strong baselines in both low-resource settings and full fine-tuning settings. in low-resource settings, we compare our model with dialogpt @xcite , t5 @xcite , and godel @xcite . godel and di-alogpt are trained with a much larger dialogue corpus. those models require a minimum of 50 training examples to adapt to multiwoz training data, while our work can perform zero-shot response generation without any fine-tuning. for the full dataset fine-tuning settings, we compare with models on the existing leaderboard of multiwoz. we evaluate both end-to-end and policy optimization settings. this includes ubar @xcite , pptod @xcite ## experiments in this section, we first show the experimental results under the low-resource and full fine-tuning settings. next, we analyze the model's zero-shot capability to predict dialogue acts. finally, we perform ablation studies for the proposed model to demonstrate the impact of dialogue act control and pre-training data. ## conclusion in this work, we present a novel end-to-end latent dialogue act model (diacttod) that represents dialogue acts in a latent space to improve the quality of response generation in task-oriented dialogue systems. diacttod addresses the challenge of utilizing generalized dialogue acts to control response generation across different datasets and tasks. the experimental results on the multiwoz dataset show that our approach outperforms previous state-of-the-art methods across a wide range of experimental settings, including zero-shot, fewshot, and full data fine-tuning with both end-to-end and policy optimization configurations. overall, this work demonstrates the effectiveness of diact-tod, making it possible to build more generalizable end-to-end dialogue systems.