Find or Classify? Dual Strategy for Slot-Value Predictions on Multi-Domain Dialog State Tracking (2020)

Abstract: Dialog state tracking (DST) is a core component in task-oriented dialog systems. Existing approaches for DST mainly fall into one of two categories, namely, ontology-based and ontology-free methods. An ontology-based method selects a value from a candidate-value list for each target slot, while an ontology-free method extracts spans from dialog contexts. Recent work introduced a BERT-based model to strike a balance between the two methods by pre-defining categorical and non-categorical slots. However, it is not clear enough which slots are better handled by either of the two slot types, and the way to use the pre-trained model has not been well investigated. In this paper, we propose a simple yet effective dual-strategy model for DST, by adapting a single BERT-style reading comprehension model to jointly handle both the categorical and non-categorical slots. Our experiments on the MultiWOZ datasets show that our method significantly outperforms the BERT-based counterpart, finding that the key is a deep interaction between the domain-slot and context information. When evaluated on noisy (MultiWOZ 2.0) and cleaner (MultiWOZ 2.1) settings, our method performs competitively and robustly across the two different settings. Our method sets the new state of the art in the noisy setting, while performing more robustly than the best model in the cleaner setting. We also conduct a comprehensive error analysis on the dataset, including the effects of the dual strategy for each slot, to facilitate future research.

## introduction i am looking for a place to stay that has cheap price range and it should be in a type of hotel. ## related work multi-domain dst, which tracks dialog states in complicated conversations across multiple domains with many slots, has been a hot research topic during the past few years, along with the development of dialogue state tracking challenges @xcite @xcite @xcite . traditional approaches usually rely on hand-crafted features or domain-specific lexicon @xcite , making them difficult to be adapted to new domains. in addition, these approaches require a pre-defined full ontology, in which the values of a slot are constrained by a set of candidate values @xcite @xcite @xcite . to tackle these issues, several methods have been proposed to extract slot values through span matching with start and end positions in the dialog context. for example, @xcite utilizes an attentionbased pointer network to copy values from the dialog context. @xcite poses dst as a reading comprehension problem and incorporates a slot carryover model to copy states from previous conversational turns. however, tracking states only from the dialog context is insufficient since many values in dst cannot be exactly found in the context due to annotation errors or diverse descriptions of slot values from users. on the other hand, pre-trained models such as bert @xcite and @xcite have shown promising performances in many downstream tasks. among them, dstreader @xcite uti-lizes bert as word embeddings for dialog contexts, sumbt @xcite employs bert to extract representations of candidate values, and bert-dst @xcite adopts bert to encode the inputs of the user turn as well as the previous system turn. different from these approaches where the dialog context and domain-slot pairs are usually separately encoded, we employ strong interactions to encode them. foot_0 . moreover, we investigate and provide insights to decide slot types and conduct a comprehensive analysis of the popular multiwoz datasets. another direction for multi-domain dst is based on generative approaches @xcite @xcite which generate slot values without relying on fixed vocabularies and spans. however, such generative methods suffer from generating ill-formatted strings (e.g., repeated words) upon long strings, which is common in dst. for example, the hotel address may be long and a small difference makes the whole dialog state tracking incorrect. by contrast, both the categorical (picklist-based) and non-categorical (span-based) methods can rely on existing strings rather than generating them. ## ds-dst: a dual strategy for dst let @xmath0, u usr 1 ), . . . , (u sys t , u usr t ) denote a set of pairs of a system utterance u sys t and a user utterance u usr t (1 ≤ @xmath1, given a dialogue context with t turns. each turn (u sys t , u usr t ) talks about a particular domain (e.g., hotel), and a certain number of slots (e.g., price range) are associated with the domain. we denote all the n possible domain-slot pairs as @xmath2, . . . , s n }, where each domain-slot pair consists of {s 1 , . . . , s n } tokens, e.g., hotel-price range includes three tokens. let x @xmath3, u usr 1 ), . . . , (u sys t , u usr t )} denote the dialogue context at the t th turn and x t has {x 1 , . . . , x m } tokens. our goal is to predict the values for all the domain-slot pairs in s. here we assume that m domain-slot pairs in s are treated as non-categorical slots, and the remaining n -m pairs as categorical slots. each categorical slot has l possible candidate values (picklist), i.e., {v 1 , . . . , v l }, where l is the size of the picklist, and each value has {v 1 , . . . , v c } tokens. bearing these notations in mind, we then pro-pose a dual strategy model with direct interactions between dialog context and domain-slot pairs for dst. fig. 2 shows an overview of the architecture of our proposed ds-dst model. we first utilize a pre-trained bert @xcite to encode information about the dialogue context x t along with each domain-slot pair in s, and obtain contextualized representations conditioned on the domain-slot information. we then design a slot gate to handle special types of values. in particular, for the non-categorical slots, we utilize a two-way linear mapping to find text spans. for the categorical slots, we select the most plausible values from the picklists based on the contextual representation. ## experimental results we evaluate all the models using the joint accuracy metric. at each turn, the joint accuracy is 1.0 if and only if @xmath4, slot, @xmath5, otherwise 0. the score is averaged across all the turns in the evaluation set. ## conclusion in this paper, we have proposed a dual strategy model with strong interactions between the dialog context and domain-slot pairs for the task of multi-domain dialog state tracking. in particular, we predict the slot value via selecting over a partial ontology for categorical slots or finding values from the dialog context for non-categorical slots. our models achieve the state of the art results on the multiwoz 2.0 and competitive results on the multiwoz 2.1. moreover, we conduct a comprehensive analysis on the dataset to facilitate future research.