Leveraging Slot Descriptions for Zero-Shot Cross-Domain Dialogue S tate T racking (2021)

Abstract: Zero-shot cross-domain dialogue state tracking (DST) enables us to handle unseen domains without the expense of collecting in-domain data. In this paper, we propose a slot descriptions enhanced generative approach for zero-shot cross-domain DST. Specifically, our model first encodes a dialogue context and a slot with a pre-trained self-attentive encoder, and generates slot value in auto-regressive manner. In addition, we incorporate Slot Type Informed Descriptions that capture the shared information of different slots to facilitates the cross-domain knowledge transfer. Experimental results on MultiWOZ shows that our model significantly improve existing state-of-the-art results in zero-shot cross-domain setting.

## introduction task-oriented dialogue systems are designed to assist users in performing daily activities, such as restaurant booking, travel planning, and online shopping. these virtual assistants provide natural language interfaces to services and online apis @xcite . based on users' needs, these systems frequently require support for new domains. however, the current state-ofthe-art systems require a substantial amount of indomain data to properly model a new domain. the data-collection process is both expensive and timeconsuming, and thus it is very important to study methods that can build robust and scalable dialogue systems using little to no in-domain data. the dialogue state tracking (dst) is an essential component of task-oriented dialogue systems that tracks users' requirements over multi-turn conversations. a popular formulation of the dialogue state is in the form of a list of slot-value pairs. in dst, tracking unseen slots in a new domain, a.k.a. zeroshot domain adaptation, is a significant challenge, * work done during internship at facebook since the model has never seen in-domain training samples. there are two main lines of work to tackle this problem. the first proposes domain transferable models using copy mechanisms or ontology graph information @xcite . a limitation of such models is that they may not fully leverage pre-trained language models due to the specialized model architecture. the second line of work uses slot-descriptions as input to the model to facilitate the slot understanding @xcite . however, the provided slot descriptions are collected by crowd sourced human annotators and might be inconsistent among different domains. in general, the optimal approach for constructing slot descriptions in zero-shot settings remains unexplored. in this work, we tackle the challenge of zeroshot cross-domain dst via leveraging large scale pre-trained sequence-to-sequence (seq2seq) models and with effective encoding of slot descriptions. we first introduce a generative dst model called t5dst, which models the relation of a slot and its dialogue context with a self-attentive encoder, and generates the slot value with a decoder in an autoregressive manner. this simple design allows us to effectively incorporate a pre-trained seq2seq model (e.g., t5 @xcite ) without any task-specific modification. to further enhance the model's cross-domain transferability, we propose slot type informed descriptions that capture the shared information of different slots. experimental results on the multiwoz benchmark @xcite suggest that 1) our model achieves significantly higher joint goal accuracy compared to existing results in zero-shot cross domain dst; 2) models using the proposed slot description formulation substantially outperform those using other slot description variants. our contributions are summarized as the following: dialogue state tracking has been of broad interest to the dialogue research community @xcite @xcite @xcite . current state-of-the-art models @xcite @xcite @xcite trained with extensive annotated data have been shown promising performance in complex multi-domain conversations @xcite . however, collecting large amounts of data for every slot description has been shown to be a promising technique in cross domain semantic parsing @xcite @xcite . to encourage this line of research in dst as well, multiwoz2.1 @xcite provides a further annotation for slot descriptions. @xcite incorporated slot descriptions for facilitating cross domain dst, while @xcite formulated dst as a question answering problem by casting a slot name into questions. however, these works did not show the effectiveness of slot descriptions, by comparing the performance of models with and without them. there is no study on how to construct slot descriptions. in this paper, we aim to fill this research gap by providing an empirical study on the different slot description formulations. ## conclusion in this paper, we propose leveraging large scale pretrained models with an effective slot description formulation to tackle the zero-shot cross-domain dst challenge. specifically, we propose t5dst, a novel generative dst model based on the t5 language model, and incorporate slot type informed descriptions to facilitate cross-domain knowledge transfer. in the evaluation on the multiwoz dataset, our approach substantially improves existing results in both the zero-shot and few-shot settings.