Out-of-Task Training for Dialog State Tracking Models (2020)

Abstract: Dialog state tracking (DST) suffers from severe data sparsity. While many natural language processing (NLP) tasks benefit from transfer learning and multi-task learning, in dialog these methods are limited by the amount of available data and by the specificity of dialog applications. In this work, we successfully utilize non-dialog data from unrelated NLP tasks to train dialog state trackers. This opens the door to the abundance of unrelated NLP corpora to mitigate the data sparsity issue inherent to DST.

## introduction the role of the dialog state tracker in a task-oriented dialog system is to summarise the history of the conversation so far and extract the user goal. dialog state tracking (dst) suffers extraordinarily from data sparsity. collecting data for dst is expensive and time consuming. typically, conversations are either staged or collected in a wizard-of-oz style setup and annotated by hand, severely inhibiting data collection. the enormous number of possible dialog states further exacerbates this. even if we combined all data that commercial assistants generate, there will still be realistic but unobserved dialog states. unsupervised learning, transfer learning and multi-task learning (mtl) @xcite in general help mitigate data sparsity. unsupervised learning relies on predicting inherent characteristics of the data. transfer learning exploits knowledge learned on related problems to generalize to new tasks. mtl optimizes towards solving multiple tasks at once for synergy effects. typically, utilized datasets have related domains and tasks share objectives. in other words, these strategies are typically used to address the problem of adaptation. recent approaches to adaptation in nlp tasks rely on contextual models. the methods above have been applied to improve generalization across related tasks and datasets. for instance, @xcite , @xcite and @xcite facilitate transfer learning by intermediate task fine-tuning (itft) on tasks that are related to the target task. @xcite and @xcite jointly optimize transformer based models @xcite towards multiple related tasks and/or domains. the latter apply mtl to pre-training, rather than fine-tuning. @xcite report improvements by continuing unsupervised pre-training for domain/task adaptation. @xcite and @xcite propose model architectures that handle diverse tasks with a unified mechanism. natural language understanding (nlu) and dialog state tracking (dst) benefit from joint modeling via multitask learning, as this utilizes dialog data more efficiently @xcite . recent approaches view dst as a generative problem @xcite or as a reading comprehension problem @xcite utilizing contextual models. in the latter, span prediction or sequence tagging extract relevant information from the input directly. these mechanisms utilize training data more efficiently than early approaches that relied on exhaustive classification given a list of known concepts @xcite @xcite . pre-training on multiple dialog datasets has been proposed to support subsequent fine-tuning towards specific dialog modeling tasks such as dst @xcite . synergies between dst subtasks can be exploited via roberta encoder 𝑥 1 [sep] [cls] 𝑥 |𝑋| dialog turn & history ... 𝑎 1 [sep] [cls] 𝑎 |𝐴| auxiliary task input ... 𝑟 𝑡 cls 𝑟 𝑡 1 𝑟 𝑡 max ... roberta encoder 𝑟 𝑡 cls 𝑟 𝑡 1 𝑟 𝑡 max ... initialize span prediction classifier span prediction slot gates ∨ ∨ (a) intermediate task fine-tuning (itft) 𝑥 1 [sep] [cls] 𝑥 |𝑋| dialog turn & history ... 𝑎 1 [sep] [cls] 𝑎 |𝐴| auxiliary task input ... mtl @xcite . better generalization across slots is attempted via knowledge transfer and zero-shot learning @xcite . all of the above approaches are suitable to better utilize available dialog data, but the general issue of data sparsity persists. there likely will never be enough task-specific data to train dialog models to their full potential. instead of resorting to the limited quantities of such data, we propose to utilize non-dialog data from unrelated tasks for the training of dst models. for this we explore two strategies: (1) in a sequential transfer learning approach, we first train a model to solve an unrelated task, followed by training towards solving dst; (2) we use mtl to jointly optimize towards dst and an unrelated task. we call our overall approach out-of-task training for dst. with our methods, we achieve new state-of-the-art performance on all four target datasets. we show that even small amounts of auxiliary task data are beneficial to support model training, especially with mtl, which particularly improves performance on difficult tasks. our positive experimental results open the door to the abundance of unrelated nlp corpora defined over a wide range of non-dialog tasks to mitigate the issue of data sparsity in dst. ## experiments datasets we conduct our evaluation on four dialog datasets. multiwoz 2.1 @xcite is the most challenging, containing over 10k dialogs defined over 5 domains with 30 domain-slot pairs. woz 2.0 @xcite ) is a single-domain benchmark. sim-m and sim-r @xcite are singledomain datasets that are challenging due to their high out-of-vocabulary (oov) rate in some slots. scoring we use joint goal accuracy (jga) on the evaluation sets as the primary measure to compare individual models. jga is the ratio of dialog turns for which all slots have been filled with the correct value according to the ground truth. we report average jga over 5 tests each with different seeds. training details as encoder we use roberta-base. training for the first phase of itft follows @xcite . 3 for target task training and mtl, the maximum input sequence length is 180 tokens after byte-pair encoding @xcite . we use adam optimizer (kingma and ba, 2014) with joint cross entropy and back-propagate through the entire network. the initial learning rate (lr) is 1e -4 . we conduct training with a warm-up proportion of 10% and linear lr decay. weight decay is set to 0.01. during training we use a dropout @xcite rate of 30% on the roberta output, and 10% on the classification heads. we use early stopping based on the jga of the development set. e @xmath0, and e @xmath1. we do not use slot value dropout @xcite except for sim-m. ## conclusion we investigated auxiliary out-of-task training for dst and found that model training benefits most from joint optimization, compared to sequential training. even though auxiliary tasks and target task are domain and task mismatched, our training schemes consistently improve target task performance, regardless of task types or data amounts. we reach state-of-the-art results with considerable improvements on all target datasets. we showed that out-of-task training is suitable to overcome data sparsity issues. in future work we pursue the direction of scaling up to do joint out-of-task training on multiple unrelated auxiliary tasks. we would also like to investigate iterative approaches to out-of-task training, where new data is added to the training during the course of a model's lifetime, rather than training from scratch. even though bert benefits less from the out-of-task training, the same tendencies as for roberta are observed. one notable difference is the poor performance of sst-2 for bert. trippy with bert uses segment id 0 for the current user utterance and segment id 1 for the system utterance plus dialog history, while trippy for roberta does not distinguish between multiple segments in the input @xcite . out-of-task training on sst-2 might negatively affect trippy with bert, because this data consists of single segments instead of pairs. however, the nature of the task (see