Generalizable and Explainable Dialogue Generation via Explicit Action Learning (2020)

Abstract: Response generation for task-oriented dialogues implicitly optimizes two objectives at the same time: task completion and language quality. Conditioned response generation serves as an effective approach to separately and better optimize these two objectives. Such an approach relies on system action annotations which are expensive to obtain. To alleviate the need of action annotations, latent action learning is introduced to map each utterance to a latent representation. However, this approach is prone to over-dependence on the training data, and the generalization capability is thus restricted. To address this issue, we propose to learn natural language actions that represent utterances as a span of words. This explicit action representation promotes generalization via the compositional structure of language. It also enables an explainable generation process. Our proposed unsupervised approach learns a memory component to summarize system utterances into a short span of words. To further promote a compact action representation, we propose an auxiliary task that restores state annotations as the summarized dialogue context using the memory component. Our proposed approach outperforms latent action baselines on MultiWOZ, a benchmark multi-domain dataset.

## introduction task-oriented dialogue systems complete tasks for users, such as making a hotel reservation or finding train routes, in a multi-turn conversation @xcite @xcite . the generated system utterances should not only be naturally sound, but more importantly be informative, i.e., to proceed the dialogue towards task completion. to fulfill this requirement, conditioned response generation is widely adopted based on system actions * rui zhang is the corresponding author. ## preliminaries let {d i |1 ≤ @xmath0, and each dialogue contains n d turns: then, the surface realization model p r (x|a, c) is optimized from the (c t , a t , x t ) triples to maximize the likelihood of ground-truth responses furthermore, to achieve better task completion, reinforcement learning (rl) is adopted to boost the pre-trained supervised models @xcite . the rewards in terms of task completion (e.g., success rate) is usually computed based on the final generated response @xcite . to avoid divergence from fluent utterances, this fine-tuning stage focuses on the content planning model p l (a|c) and keeps the parameters of p r (x|a, c) fixed. the reward r t at each turn is back-propagated via policy gradients as: in order to enable conditioned response generation when action annotations are absent, latent action learning is introduced. given dialogues {(c t , x t )|1 ≤ @xmath1, latent action learning aims to map each utterance to a latent representation z d (x), e.g, one-hot @xcite , or continuous @xcite . based on the obtained (c t , z d (x t ), x t ) triples, conditioned response generation is run as mentioned above. existing latent action learning approaches mostly build on the idea of variational inference, where a latent space is found to reconstruct system utterances and thus encodes the main characteristics of utterances @xcite . the action representations learned from the latent space are, however, difficult to generalize due to the implicit nature and thus cause the sample inefficiency issue. ## experiments to show the effectiveness of the proposed approach, memory-augmented saliency with parallel corpus (masp), we experiment on two dialogue generation settings (sec. 4.1). we compare against stateof-the-art approaches in both settings (sec. 4.2). we analyze the effectiveness of masp components under different supervision ratios, and discuss how explainable generation is achieved (sec. 4.3). ## related work early studies of conditioned response generation focus on enriching the meaning representations in task-oriented dialogues, e.g., utilizing graph structures and hierarchies among actions @xcite , decomposing into our study is also related to attribution approaches, which aims to find features or regions of input that are important for tasks. different types of techniques, including gradient-based @xcite and post-hoc @xcite , are applied for reinforcement learning @xcite , computer vision @xcite , and text classification @xcite . while these works focus on interpreting model behaviors, we aim to find salient words beyond input and utilize them as action representations. ## conclusions we propose explicit action learning to achieve generalizable and interpretable dialogue generation. our proposed model masp learns unified and compact action representations. we propose a memory component that summarizes system utterances into natural language actions, i.e., spans of words from a unified vocabulary. we further introduce an auxiliary task to encourage natural language actions to only preserve task-relevant information. experimental results confirm that masp achieves better performance compared with the state-of-the-art in different settings, especially when supervision is limited. we plan to consider structural action representation learning that could convey more information as future work.