Robust Dialogue State Tracking with Weak Supervision and Sparse Data (2022)

Abstract: Generalizing dialogue state tracking (DST) to new data is especially challenging due to the strong reliance on abundant and fine-grained supervision during training. Sample sparsity, distributional shift, and the occurrence of new concepts and topics frequently lead to severe performance degradation during inference. In this paper we propose a training strategy to build extractive DST models without the need for fine-grained manual span labels. Two novel input-level dropout methods mitigate the negative impact of sample sparsity. We propose a new model architecture with a unified encoder that supports value as well as slot independence by leveraging the attention mechanism. We combine the strengths of triple copy strategy DST and value matching to benefit from complementary predictions without violating the principle of ontology independence. Our experiments demonstrate that an extractive DST model can be trained without manual span labels. Our architecture and training strategies improve robustness towards sample sparsity, new concepts, and topics, leading to state-of-the-art performance on a range of benchmarks. We further highlight our model’s ability to effectively learn from non-dialogue data.

## introduction generalization and robustness are among the key requirements for naturalistic conversational abilities of task-oriented dialogue systems @xcite . in a dialogue system, dialogue state tracking (dst) solves the task of extracting meaning and intent from the user input, and keeps track of the user's goal over the continuation of a conversation as part of a dialogue state (ds) @xcite . a recommendation and booking system for places, for instance, needs to gather user preferences in terms of budget, location, and so forth. concepts like these are assembled in an ontology on levels of domain (e.g., restaurant or hotel), slot (e.g., price or location), and value (e.g., ''expensive'' or ''south''). accurate dst is vital to a robust dialogue system, as the system's future actions depend on the conversation's current estimated state. however, generalizing dst to new data and domains is especially challenging. the reason is the strong reliance on supervised training. virtually all top-performing dst methods either entirely or partially extract values directly from context @xcite . however, training these models robustly is a demanding task. extractive methods usually rely on fine-grained labels on word level indicating the precise locations of value mentions. given the richness of human language and the ability to express the same canonical value in many different ways, producing such labels is challenging and very costly, and it is no surprise that datasets of such kind are rare @xcite . reliance on detailed labels has another downside; datasets are usually severely limited in size. this in turn leads to the problem of sample sparsity, which increases the risk for models to over-fit to the training data, for instance, by memorizing values in their respective contexts. overfitting prevents a state tracker to generalize to new contexts and values, which is likely to break a dialogue system entirely @xcite . recently, domain-independent architectures have been encouraged to develop systems that may be built once and then applied to new scenarios with no or little additional training @xcite . however, training such flexible models robustly remains a challenge, and the evergrowing need for more training samples spurs creativity to leverage non-dialogue data @xcite . we propose novel strategies for extractive dst that address the following four issues of robustness and generalization. (1) we solve the problem of requiring fine-grained span labels with a self-supervised training scheme. specifically, we learn from random self-labeled samples how to locate occurrences of arbitrary values. all that is needed for training a full dst model is the dialogue state ground truth, which is undoubtedly much easier to obtain than fine-grained span labels. (2) we handle the sample sparsity problem by introducing two new forms of inputlevel dropout into training. our proposed dropout methods are easy to apply and provide a more economical alternative to data augmentation to prevent memorization and over-fitting to certain conversation styles or dialogue patterns. (3) we add a value matching mechanism on top of extraction to enhance robustness towards previously unseen concepts. our value matching is entirely optional and may be utilized if a set of candidate values is known during inference, for instance, from a schema or api. (4) we propose a new architecture that is entirely domain-agnostic to facilitate transfer to unseen slots and domains. for that, our model relies on the attention mechanism and conditioning on natural language slot descriptions. the established slot-independence enables zero-shot transfer. we will demonstrate that we can actively teach to track new domains by learning from non-dialogue data. this is nontrivial as the model must learn to interpret dialogue data from exposure to unstructured data. ## related work traditional ds trackers perform prediction over a fixed ontology @xcite @xcite and therefore have various limitations in more complex scenarios @xcite . the idea of fixed ontologies is not sustainable for real world applications, as new concepts become impossible to capture during test time. moreover, the demand for finely labeled data quickly grows with the ontology size, causing scalability issues. recent approaches to dst extract values directly from the dialogue context via span prediction @xcite @xcite , removing the need for fixed value candidate lists. an alternative to this mechanism is value generation via soft-gated pointergenerator copying @xcite @xcite . extractive methods have limitations as well, since many values may be expressed variably or implicitly. contextual models such as bert @xcite support generalization over value variations to some extent @xcite @xcite , and hybrid approaches try to mitigate the issue by resorting to picklists @xcite . trippy @xcite jointly addresses the issues of coreference, implicit choice, and value independence with a triple copy strategy. here, a transformer-based @xcite encoder projects each dialogue turn into a semantic embedding space. domain-slot specific slot gates then decide whether or not a slot-value is present in the current turn in order to update the dialogue state. in case of presence, the slot gates also decide which of the following three copy mechanisms to use for extraction. (1) span prediction extracts a value directly from input. for that, domain-slot specific span prediction heads predict per token whether it is the beginning or end of a slot-value. (2) informed value prediction copies a value from the list of values that the system informed about. this solves the implicit choice issue, where the user might positively but implicitly refer to information that the system provided. while contemporary approaches to dst leverage parameter sharing and transfer learning @xcite , the need for finely labeled training data is still high. sample sparsity often causes model biases in the form of memorization or other types of over-fitting. strategies to appease the hunger of larger models are the exploitation of out-of-domain dialogue data for transfer effects @xcite and data augmentation @xcite , 2020; @xcite . however, out-of-domain dialogue data is limited in quantity as well. data augmentation still requires high level knowledge about dialogue structures and an adequate data generation strategy. ultimately, more data also means longer training. we are aware of only one recent work that attempts dst with weak supervision. liang et al. ( 2021 ) take a few-shot learning approach using only a subset of fully labeled training samples-typically from the end of conversations-to train a soft-gated pointer-generator network. in contrast, with our approach to spanless training, we reduce the level of granularity needed for labels to train extractive models. note that these strategies are orthogonal. ## trippy-r: robust triple copy dst let {(u 1 , m 1 ), . . . , (u t , m t )} be the sequence of turns that form a dialogue. u t and m t are the token sequences of the user utterance and preceding system utterance at turn t. the task of dst is (1) to determine for every turn whether any of the domain-slot pairs in @xmath0, . . . , s n } is present, (2) to predict the values for each s n , and (3) to track the dialogue state ds t . our starting point is triple copy strategy dst @xcite , because it has already been designed for robustness towards unseen values. however, we propose a new architecture with considerable differences to the baseline regarding its design, training, and inference to overcome the drawbacks of previous approaches as laid out in section 2. we call our proposed framework trippy-r (pronounced ''trippier''), robust triple copy strategy dst foot_0 . figure 1 is a depiction of our proposed model. ## levels of robustness in dst we propose the following methods to improve robustness in dst on multiple levels. ## conclusion in this work we present methods to facilitate robust extractive dialogue state tracking with weak supervision and sparse data. our proposed architecture-trippy-r-utilizes a unified encoder, the attention mechanism, and conditioning on natural language descriptions of concepts to facilitate parameter sharing and zero-shot transfer. we leverage similarity based value matching as an optional step after value extraction, without violating the principle of ontology independence. we demonstrated the feasibility of training without manual span labels using a self-trained proto-dst model. learning from spanless labels enables us to leverage data with weaker supervision. we showed that token noising and history dropout mitigate issues of pattern memorization and train-test discrepancies. we achieved competitive zero-shot performance and demonstrated in a feasibility study that trippy-r can learn to track new domains by learning from non-dialogue data. we achieve either competitive or state-ofthe-art performance on all tested benchmarks. for future work we continue to investigate learning from non-dialogue data, potentially in a continuous fashion over the lifetime of a dialogue system.