Learning Interpretable Latent Dialogue Actions With Less Supervision (2022)

Abstract: We present a novel architecture for explainable modeling of task-oriented dialogues with discrete latent variables to represent dialogue actions. Our model is based on variational recurrent neural networks (VRNN) and requires no explicit annotation of semantic information. Unlike previous works, our approach models the system and user turns separately and performs database query modeling, which makes the model applicable to task-oriented dialogues while producing easily interpretable action latent variables. We show that our model outperforms previous approaches with less supervision in terms of perplexity and BLEU on three datasets, and we propose a way to measure dialogue success without the need for expert annotation. Finally, we propose a novel way to explain semantics of the latent variables with respect to system actions.

## introduction while supervised neural dialogue modeling is a very active research topic @xcite @xcite , it requires a significant amount of work to obtain turn-level labels, usually with dialogue state annotation. we argue that in many real-world cases, it is very expensive to obtain the necessary labels or even to design an appropriate annotation schema. consider a call center with various dialogues that has a lot of transcripts available, including the corresponding api queries, but has no capacity to label them. this motivates our research of approaches that minimize the need for expert annotation. while most recent research focuses on pretrained language models (plms) and reaches state-of-theart performance in standard supervised @xcite or even few-shot @xcite settings, these models still require full supervision. furthermore, they lack the potential to interpret the model decisions. some recent works try to address plm interpretability with some success @xcite , but still face considerable difficulties due to plms' huge number of parameters and their structure. on the other hand, dialogue models using latent variables are able to infer interpretable attributes from unlabeled data @xcite . these models are mostly trained using variational autoencoders (vae; @xcite . improvements with discrete variables @xcite offer better interpretability, but the approaches are not directly applicable to task-oriented response generation as no distinction between the system and user roles is made, and database access or goal fulfillment are not considered; most research on unlabeled data only applies to a chit-chat setting. since interpretability and the ability to learn from unlabeled data are our primary goals, we choose working with rnn-based latent variable models over transformer-based plms in our work. unlike previous latent-variable approaches, we shift the focus towards task-oriented systems and take tracked entity values and database access into account. specifically, we base our approach on @xcite 's architecture. @xcite employ the vrnn model @xcite and experiment with conditioning the prior distribution. however, their focus is on uncovering dialogue structure, and they model user and system utterances together. in contrast, we fully take advantage of the vrnn model's generative capabilities and apply it for response generation. specifically, we train a specialized decoder for system response generation. furthermore, we extend the vrnn model so that the system and the user utterances are modeled separately. this modification brings the following major advantages: (1) we can model different behaviors on the side of the system and the user, which is expected in a task-oriented setting; (2) we can focus on modeling latent system actions in an explainable way; (3) we can predict the system response easily. task-oriented dialogue systems typically need to interact with an external database; otherwise, their responses cannot be grounded. therefore, we assume that database queries and results are known, but no dialogue state annotation is available. this allows a direct application of our model for dialogue response generation in a task-oriented setting while still keeping the amount of needed supervision very low. this scenario reflects the intended use case, i.e. automating a call center based on recordings of previous human-human dialogues. at some point of the dialogue, a database query is performed by the human agent and we know exactly when and with which parameters. our contributions in this paper are as follows: 1. we propose a novel modification of the vrnnbased model for minimally supervised taskoriented dialogue generation, with interpretable latent variables to represent system actions. we show that our model's latent variables explain most of our system's predicted responses and align well with gold-standard responses. our experimental code is released on github. foot_0 ## related work in the area of supervised dialogue systems, current leading research focuses on end-to-end sequenceto-sequence models @xcite . recent works make use of large pre-trained language models (plms) based on the transformer architecture @xcite such as gpt-2 @xcite or bert @xcite . for example, @xcite propose finetuning bert @xcite for task-oriented dialogue on multiple datasets; @xcite extended the gpt-2 plm to model open-domain chit-chat. however, we focus mainly on approaches that require less supervision. the hierarchical recurrent encoder-decoder (hred) by @xcite , where rnn hidden states represent the latent dialogue state, was among the first unsupervised neural dialogue models. however, the latent representations obtained from the vanilla autoencoder model trained with reconstruction loss suffer from poor generalization. for this purpose @xcite , the usage of variational autoencoders (vaes) (kingma and welling, 2014) was proposed. the vae training maximizes the variational lower bound of data log-likelihood. vae distributions are invariant in time, therefore it are not suitable for modeling sequences. chung et al. (2015) address this issue with the variational recurrent neural network model (vrnn). serban et al. ( 2017 ) then used vrnn's latent variables to represent dialogue state. recent works used modified transformer architectures with specific training tasks to obtain in-context representations of dialogue utterances @xcite . while both vaes and transformers improve generalization and consistency of the latent variables, they are not well interpretable. to obtain more interpretable latent states, generative models with discrete states such as hidden markov models were applied @xcite . @xcite used discrete latent variables to represent the state in a model trained using reinforcement learning. another proposed approach was the usage of quantization techniques by @xcite , who perform clustering on utterances and model the dialogue as a sequence of clusters to predict future responses. @xcite use vaes in combination with gumbel-softmax to model discrete latent variables representing the dialogue utterances. more recently, several works attempted to model latent system actions without any action-level annotation @xcite @xcite . however, they still rely on labeled data on different levels, such as turn-level dialogue state annotation. in a different line of research, @xcite aim to uncover the dialogue structure. they apply vrnns to estimate dialogue state transition probabilities. the same goal of uncovering and understanding semantic structure of the dialogue is explored by @xcite , who propose a vrnn-based model with structured attention to achieve this goal, or @xcite , who use an enhanced graph autoencoder. our proposed model combines the latter two approaches, but it is distinct from both. it models system actions using latent variables, but it does not rely on any turn-level labels for dialogue state or language understanding. moreover, our goal is not only to uncover the dialogue structure but rather to model system actions and generate responses. ## method we assume that each dialogue turn t consists of a user utterance x t u and a system utterance x t s . the context c t in turn t is a sequence of user and system utterances up to the previous turn t -1. we expect that conditioning the generation of x t s on a latent variable z t will allow the model to better incorporate context. ## experiments in this section, we focus on the quality of responses generated by our model as well as on model performance with respect to dialogue task success. we focus on theoretical modeling and feasibility at this stage, which we believe is sufficiently demonstrated by corpus-based evaluation complemented by manual checks. detailed interpretation of the learned representations follows in section 5. ## latent variable interpretation we believe that being able to explain and interpret the model behavior is crucial, especially in a setting without full supervision. therefore, we design a set of experiments to evaluate the model behavior and investigate whether the model captures salient dialogue features in the latent variables obtained during training on camrest676 and multiwoz. while it seems that the latent variables are mainly useful for interpretability or structure induction, they are likely also contributing to the performance as smaller latent spaces yield lower performance as we saw in preliminary experiments and show in appendix b. ## conclusion and future work we introduce a model for task-oriented dialogue with discrete latent variables that uses only minimal supervision and improves upon previous approaches @xcite . we also propose methods for task-based evaluation in this minimally supervised setting. our system is not yet ready for interactive evaluation on full dialogues, considering the clear performance gap with respect to with fully supervised approaches. however, we demonstrate that it learns meaningful representations from minimal supervision (in a realistic setup corresponding to pre-existing call center call logs) and compares favorably to previous weakly supervised approaches. a detailed analysis reveals that the learned representations capture relevant dialogue features and can be used to identify system actions. furthermore, the reason for choosing an action can be described in an explainable way. the results suggest that dialogue models with discrete latent variables can be successfully applied also in the task-oriented setting. the main limitations of our current model are its problems with providing the correct slot values in responses. we plan address this issue in future work by incorporating explicit copy mechanisms @xcite , i.e. the model will learn to copy slot values from the context and from database results. we also plan to experiment with incorporating transformer models into the variational autoencoder setup, following recent models such as the vae-transformer @xcite .