Knowing What You Know: Calibrating Dialogue Belief State Distributions via Ensembles (2020)

Abstract: The ability to accurately track what happens during a conversation is essential for the performance of a dialogue system. Current state-of-the-art multi-domain dialogue state trackers achieve just over 55% accuracy on the current go-to benchmark, which means that in almost every second dialogue turn they place full confidence in an incorrect dialogue state. Belief trackers, on the other hand, maintain a distribution over possible dialogue states. However, they lack in performance compared to dialogue state trackers, and do not produce well calibrated distributions. In this work we present state-of-the-art performance in calibration for multi-domain dialogue belief trackers using a calibrated ensemble of models. Our resulting dialogue belief tracker also outperforms previous dialogue belief tracking models in terms of accuracy.

## introduction task-oriented dialogue systems aim to act as assistants to their users, solving tasks such as finding a restaurant, booking a train, or providing information about a tourist attraction. they have become very popular with the introduction of virtual assistants such as siri and alexa. two tasks are fundamental to such a system. the first is the ability to track what happened in the conversation, referred to as tracking. based on the result of tracking, the system needs to conduct the conversation towards the fulfilment of the user goal, referred to as planning. the tracking component summarises the dialogue history, or the past, while the planning component manages the dialogue and concerns the future. in this work we focus on the first component. early approaches to statistical dialogue modelling view dialogue as a markov decision process @xcite and define a set of dialogue states that the conversation can be in at any given dialogue turn. the tracking component tracks the dialogue state. in recent years discriminative models achieve state-of-the-art dialogue state tracking (dst) results @xcite @xcite . still, in a multidomain setting such as @xcite , they achieve an accuracy of just over 55%. this means that in approximately 45% of cases they make a wrong prediction and, even worse, they have full confidence in that wrong prediction. in the wake of statistical dialogue modeling, the use of partially observable markov decision processes has been proposed to address this issue. the idea is to model the probability over all possible dialogue states in every dialogue turn @xcite . this probability distribution is referred to as the belief state. the advantages of belief tracking are probably best illustrated by an excerpt from a dialogue with a real user in @xcite : even though the dialogue state predicted with the highest probability is not the true one, the system is able to provide a valid response because the true dialogue state also has assigned a non-zero probability. a model is considered well calibrated if its confidence estimates are aligned with the empirical likelihood of its predictions @xcite . the belief state can be modelled by deep learning-based approaches such as the neural belief tracker @xcite , the multi-domain belief tracker @xcite , the globally conditioned encoder belief tracker @xcite and the slot utterance matching belief tracker (sumbt) @xcite models. none of these models however address the issue of calibrating the probability distribution that they provide, resulting in them being more confident than they should be. in a dialogue setting, overconfidence can lead to bad decisions and unsuccessful dialogues. in this work, we present methods for learning well-calibrated belief distributions. our contributions are the following: such a well-calibrated belief tracking model is essential for the planning component to successfully conduct dialogue. ## related work since no other belief tracking methods that we are aware of have achieved success in producing wellcalibrated confidence, we look towards methods used in other language tasks. natural language inference is a related task that also benefits from wellcalibrated confidence in predictions. @xcite introduce the use of post-processing techniques such as temperature scaling to produce better-calibrated confidence estimates. additionally, there have been recent advances in the construction of more adequate loss functions. these methods, including bayesian matching and prior networks, aim to learn well-calibrated models without the burden of requiring many extra parameters. these methods achieve good calibration in computer vision tasks such as cifar @xcite @xcite . when the limitations of a single model still inhibit us from producing more accurate and bettercalibrated models, a popular alternative is to use an ensemble of models. recently @xcite showed the success of using an ensemble of models for machine translation, and in particular utilising accurate confidence predictions for analysing translation quality. ## calibration techniques in this section we explain the details of three calibration techniques that we apply to dialogue belief tracking. ## experimental setup we seek to build a well-calibrated dialogue belief tracker. for our baseline belief tracker, we use the sumbt model architecture @xcite , which uses bert @xcite as a turn encoder and multi-head attention for slot candidate matching. we perform all experiments on the mul-tiwoz 2.1 dataset @xcite , the current standard dataset for multi-domain dialogue. when training using bayesian matching, we use a scaling coefficient of λ = 0.003, and for label smoothing, a smoothing coefficient of α = 0.05. for the ensemble belief tracker, we train 10 identical independent models, each with a sub-sample of 7500 dialogues. all hyper-parameters are obtained using a parameter search based on validation set performance. for all training, we use the bert-base-uncased model from pytorch transformers @xcite for turn embedding. we use a gated recurrent unit with a hidden dimension 300 for latent tracking and euclidean distance for value candidate scoring. during training, we use a learning rate of 5e -5 in combination with a linear learning rate scheduler, the warm-up proportion is set to 0.1. a dropout rate of 0.3 is used, and training is performed for 100 epochs. 1 ## conclusion we applied a number of calibration techniques to a baseline dialogue belief tracker. we showed that a label smoothed trained ensemble provides stateof-the-art calibration of the belief state distributions and has the best accuracy among the available belief trackers. although it does not compete with state trackers in terms of jga, when considering top 3 predictions it achieves 84.08% accuracy (top 3 jga), almost 30 percentage points above state-of-the art state trackers. we also find that our model has the best l2 norm performance, which suggests that the quality of predicted uncertainty is as important as the average jga. it is important to note that the proposed calibration methods can be applied to any neural dialogue belief tracking method. the uncertainty estimates predicted by this model could improve the success of dialogue systems because this model can provide the dialogue manager with a good measure of confidence. this could allow the system to ask questions in moments of confusion. in the appendix we include example dialogues to illustrate this. in future, we aim to combine the state-of-the-art dialogue state tracking and belief tracking methods to create a method that can achieve both states-of-theart joint goal accuracy and well-calibrated belief states.