Zero-Shot Dialogue State Tracking via Cross-Task Transfer (2021)

Abstract: Zero-shot transfer learning for dialogue state tracking (DST) enables us to handle a variety of task-oriented dialogue domains without the expense of collecting in-domain data. In this work, we propose to transfer the cross-task knowledge from general question answering (QA) corpora for the zero-shot DST task. Specifically, we propose TransferQA, a transferable generative QA model that seamlessly combines extractive QA and multi-choice QA via a text-to-text transformer framework, and tracks both categorical slots and non-categorical slots in DST. In addition, we introduce two effective ways to construct unanswerable questions, namely, negative question sampling and context truncation, which enable our model to handle none value slots in the zero-shot DST setting. The extensive experiments show that our approaches substantially improve the existing zero-shot and few-shot results on MultiWoz. Moreover, compared to the fully trained baseline on the Schema-Guided Dialogue dataset, our approach shows better generalization ability in unseen domains.

## introduction virtual assistants are designed to help users perform daily activities, such as travel planning, online shopping and restaurant booking. dialogue state tracking (dst), as an essential component of these task-oriented dialogue systems, tracks users' requirements throughout multi-turn conversations as dialogue states, which are typically in the form of a list of slot-value pairs. training a dst model often requires extensive annotated dialogue data. these data are often collected via a wizard-of-oz (woz) @xcite setting, where two workers converse with each other and annotate the dialogue states of each utterance @xcite @xcite , or with a machines talking to machines * work done during internship at facebook (m2m) framework @xcite , where dialogues are synthesized via the system and user simulators @xcite @xcite . however, both of these approaches have inherent challenges when scaling to large datasets. for example, the data collection process in a woz setting incurs expensive and time-consuming manual annotations, while m2m requires exhaustive hand-crafted rules for covering various dialogue scenarios. in industrial applications, virtual assistants are required to add new services (domains) frequently based on user's needs, but collecting extensive data for every new domain is costly and inefficient. therefore, performing zero-shot prediction of dialogue states is becoming increasingly important since it does not require the expense of data acquisition. there are mainly two lines of work in the zero-shot transfer learning problem. the first is cross-domain transfer learning @xcite @xcite , where the models are first trained on several domains, then zero-shot to new domains. however, these methods rely on a considerable amount of dst data to cover a broad range of slot types, and it is still challenging for the models to handle new slot types in the unseen domain. the second line of work leverages machine reading question answering (qa) data to facilitate the low-resource dst (i.e., cross-task transfer) @xcite . however, the method of @xcite relies on two independent qa models, i.e., a span extraction model for non-categorical slots and a classification model for categorical slots, which hinders the knowledge sharing from the different types of qa datasets. furthermore, unanswerable questions are not considered during their qa training phase. therefore, in a zero-shot dst setting, the model proposed by @xcite is not able to handle "none" value slots (e.g., unmentioned slots) that present in the dialogue state. in this paper, to address the above challenges, we propose transferqa, a unified generative qa model that seamlessly combines extractive qa and multi-choice qa via a text-to-text transformer framework @xcite . such design not only allows the model to leverage both extractive and multi-choice qa datasets, but also provides a simple unified text-totext interface for tracking both categorical slots and non-categorical slots. to handle the "none" value slots in a zero-shot dst setting, we introduce two effective ways to construct unanswerable questions, namely, negative question sampling and context truncation, which simulate the out-of-domain slots and in-domain unmentioned slots in multi-domain dst. we evaluate our approach on two large multidomain dst datasets: multiwoz @xcite and schema-guided dialogue (sgd) @xcite . the experimental results suggest that our proposed model, without using any dst data, achieves a significantly higher joint goal accuracy compared to previous zero-shot dst approaches. our contributions are summarized as the following: ## conclusion in this paper, we present transferqa, a unified generative model that performs dst without using any dst training data. transferqa uses the textto-text transfer learning framework that seamlessly combines extractive qa and multi-choice qa for tracking both categorical slots and non-categorical slots. to enable our model to zero-shot "none" value slots, we introduce two effective ways to construct unanswerable questions, i.e., negative question sampling and context truncation. the experimental results on the multiwoz and sgd datasets demonstrate the effectiveness of our approach in both zero-shot and few-shot settings. we also show that improving the "none" value slot accuracy has