Improving Dialogue State Tracking with Turn-based Loss Function and Sequential Data Augmentation (2021)

Abstract: While state-of-the-art Dialogue State Tracking (DST) models show promising results, all of them rely on a traditional cross-entropy loss function during the training process, which may not be optimal for improving the joint goal accuracy. Although several approaches recently propose augmenting the training set by copying user utterances and replacing the real slot values with other possible or even similar values, they are not effective at improving the performance of existing DST models. To address these challenges, we propose a Turn-based Loss Function (TLF) that penalises the model if it inaccurately predicts a slot value at the early turns more so than in later turns in order to improve joint goal accuracy. We also propose a simple but effective Sequential Data Augmentation (SDA) algorithm to generate more complex user utterances and system responses to effectively train existing DST models. Experimental results on two standard DST benchmark collections demonstrate that our proposed TLF and SDA techniques significantly improve the effectiveness of the state-of-the-art DST model by approximately 7-8% relative reduction in error and achieves a new state-of-the-art joint goal accuracy with 59.50 and 54.90 on MultiWOZ2.1 and MultiWOZ2.2, respectively.

## introduction task-based virtual personal assistants (vpas) interact with users in natural language to help complete tasks such as making hotel bookings and restaurant reservations. dialogue state tracking (dst) is an essential component for vpas that aims to track the dialogue state from the user's utterances at each turn @xcite . based on the current dialogue state, vpas decide the next action to perform. in general, existing dst models rely on an ontology that defines slots for a particular domain/task (e.g. hotel-name and taxidestination). to accomplish the tracking task, given turn 1 -usr: i'm looking for somewhere to stay in north cambridge. dialogue state prediction 1 hotel-area : none jga: 0 dialogue state prediction 2 hotel-area : north cambridge jga: 1 turn 2 -sys: would you like to try the lovell lodge? usr: yes, please. turn 3 -sys: the booking goes through [....] usr: can you book a taxi from the hotel to the ballare. dialogue state prediction1 hotel-area : none hotel-name : lovell lodge jga: 0 dialogue state prediction 2 hotel-area : north cambridge hotel-name : lovell lodge jga: 1 dialogue state prediction 1 hotel-area : none hotel-name : lovell lodge taxi-departure : lovell lodge taxi-destination : ballare jga: 1 dialogue state prediction 2 hotel-area : north cambridge hotel-name : lovell lodge taxi-departure : ballare taxi-destination : ballare jga: 0 the user's current utterance, a slot to track and dialogue history, the dst models need to 1) predict if the user has mentioned the given slot and 2) if so, predict/extract its value from the current utterance. joint goal accuracy (jga) is a widely used metric to evaluate the effectiveness of dst models @xcite @xcite . at each turn, the joint goal accuracy is 1.0 if and only if all domain-slot and value pairs are predicted correctly, otherwise 0. the existing dst models rely on the traditional cross-entropy loss function during the training process. we argue that this is not effective for optimizing joint goal accuracy. we illustrate this issue in figure 1 in dialogue state prediction 1 & 2, with the traditional cross-entropy loss function, if two models only make one mistake during the training process they will be penalised equally. however, the consequence that the first model incorrectly predicts the value at the first turn is worse than when the second model fails to predict the value at the forth turn (i.e. average jga across 3 turns is 0 and 0.66 for model 1 and model 2). training current dst models currently requires annotated dialogue datasets that cover a wide variety of diverse conversation flows. however, existing dialogue datasets (e.g. multiwoz2.1 @xcite ) are relatively small and do not provide coverage of all slot values for open-vocabulary slots (e.g. restaurant name and destination). to alleviate this problem, several recent data augmentation techniques propose @xcite @xcite replacing the ground-truth values of particular slots using additional information (e.g. restaurant and movie name corpus). although these augmented dialogues increase coverage of all slot values, the complexity of the dialogues remains the same foot_0 . to address the aforementioned challenges, we propose a novel turn-based loss function and sequential data augmentation algorithm that improves the effectiveness of dst models. our contributions are: to the best of our knowledge, this work is the first to incorporate turn dependence into the loss function of the dst models. zhang et al. (2019) propose an ontology-based dst model that leverages a pre-defined ontology to predict dialogue state based on the similarity between the encoded candidate values and encoded user utterance and slot description. recent work in dst focuses on the span-based approach to address the scalability and generalisation issues of previous ontologybased models. wu et al. (2019) proposed a scalable span-based dst model that encodes the whole dialogue context and decodes the value for every slot using a copy-augmented decoder. recently, several dst models @xcite incorporate the predicted dialogue state from previous turns when tracking the dialogue state at the current turn using a copy mechanism. data augmentation is widely used to improve the effectiveness of the existing dst models @xcite @xcite . @xcite use a sequence-tosequence model and delexicalisation to generate a variety of diverse utterances based on the original utterances. these generated augmented utterances help improve the language understanding of the dst models. in addition, the span-based dst models often encounter out-of-vocabulary words (e.g. unseen restaurant name) at inference time. as a result, these dst models are likely to fail to extract unseen words from the utterances. to address this problem, @xcite augment the training dataset by randomly replacing original slot values with other possible values obtained from external corpora (e.g. restaurant name corpus). similar to @xcite augment the training data by copying user utterances and replace the ground-truth slot values with randomly generated strings. recently, @xcite proposed to use the pre-trained utterance generator and counterfactual goal generator to create novel user utterances that are correlated with the original system response. their approach showed significant improvement on the dst performance. cls u t sep r t sep h t sep dialogue encoder e cls e 1 e 2 e 3 e max slot operation predictor slot value predictor e 4 e 5 e 6 e 7 e 8 ## neural models for dst we first formalise the task of dialogue state tracking and define key notations. then, we briefly describe the general architecture of existing neural network dst models (see figure 2 that consist of three main components: the dialogue encoder, the slot operation predictor, and the slot value predictor. ## proposed methods we now describe the proposed methods that improve the effectiveness of dst model. ## experimental setup we conduct experiments on the two most widely used multi-domain task-based dialogue state tracking datasets (multiwoz2.1 @xcite and multiwoz2.2 @xcite ). these two are the largest datasets which contain over 10,000 dialogues across seven domains: restaurant, taxi, attraction, hotel, train, hospital and police. following @xcite , we remove hospital and police domains in multiwoz2.1 and multiwoz2.2 because they only appear in the training dataset. this results in five domains with 30 domain-slot pairs. we use the standard training/validation/test splits provided in the original datasets. following previous literature @xcite , we evaluate all the dst models using the joint goal accuracy (jga) metric @xcite . at each turn jga is 1.0 if and only if all domain-slot and values pairs are correctly predicted, otherwise 0. the score is averaged across all turns in the test set. ## conclusion we propose two novel algorithms, tlf (turnbased loss function) and sda (sequential data augmentation), that improve the effectiveness of state-of-the-art dialogue state tracking models. tlf penalizes such models more heavily if they fail to predict slot values in the middle of the conversation. on the other hand, sda generates dialogues used to train existing dst models to generalize more effectively. our comprehensive experiments on multiple benchmark datasets demonstrate the combined utility of both tlf and sda to improve the effectiveness of the leading model in the literature. indeed, tlf and sda significantly improve the effectiveness of trippy by approximately 8.26% relative reduction in error on multiwoz2.2, which constitutes the new state-of-the-art result on this most recent benchmark dataset. for future work, we plan to extend both tlf and sda to incorporate additional information such as dialogue length and the number of active slots during the training process to be even more effective for long and complex dialogues. we also plan to investigate the effectiveness of tlf and sda on other existing dst models.