Zero-shot Generalization in Dialog State Tracking through Generative Question Answering (2021)

Abstract: Dialog State Tracking (DST), an integral part of modern dialog systems, aims to track user preferences and constraints (slots) in task-oriented dialogs. In real-world settings with constantly changing services, DST systems must generalize to new domains and unseen slot types. Existing methods for DST do not generalize well to new slot names and many require known ontologies of slot types and values for inference. We introduce a novel ontology-free framework that supports natural language queries for unseen constraints and slots in multi-domain task-oriented dialogs. Our approach is based on generative question-answering using a conditional language model pre-trained on substantive English sentences. Our model improves joint goal accuracy in zero-shot domain adaptation settings by up to 9% (absolute) over the previous state-of-the-art on the MultiWOZ 2.1 dataset.

## introduction dialog agents are gaining increasing prominence in daily life. these systems aim to assist users via natural language conversations, taking the form of digital assistants who help accomplish everyday tasks by interfacing with connected devices and services. a key component to understanding and enabling these task-oriented dialogs is dialog state tracking (dst): extracting user intent and goals from conversations via filling in belief slots @xcite . assistive and recommendation use-cases for dialog agents in production settings are particularly challenging due to constantly changing services and applications with which they interface. traditional dst systems have achieved high accuracy when presented with a known ontology of slot types and valid values @xcite . in a real-world setting, however, a dst model must generalize to new slot values (e.g. new entities that are not present at training time) and new slot types (e.g. requirements regarding a new application). recent work has sought to address these issues by posing dst as a reading comprehension or question answering (qa) task @xcite -such models predict each slot value independently at any given turn and can theoretically be queried for new slots at inference time. some approaches toward dst as qa learn embedding vectors for each slot and/or domain word @xcite , but this is not robust to unseen slots whose specific names (e.g. 'internet access') may be totally unlike those in the training set. @xcite attempt to remedy this by posing a natural language question for each slot, but their hybrid span-extraction and classification-based system nonetheless requires access to the full ontology for unknown domains. we present an ontologyfree model using natural language questions to represent slots that builds on conditional language modeling techniques-taking advantage of the rise of powerful generative language models @xcite -to tackle dst as a generative qa task. our model can generalize to unseen domains, slot types, and values, and allows developers to query for arbitrary user requirements via simple questions. to summarize our main contributions: ## approach we follow @xcite in treating dialog state tracking as a reading comprehension problem: at each turn of dialog, our model reads the dialog history and answers a fixed set of queries about user requirements and preferences (slots), with predictions aggregated to form the belief state. in our framework (figure 1 ), we query for a given slot (e.g. hotel price range) by asking a natural language question @xcite -"what is the price range of the hotel the user prefers?". as our model's predictive ability is based on its general understanding of language and task-oriented conversation, we support zero-shot inference without the need to re-train the model or extend a formal ontology. for example, if a model has not been trained on data from the hotel domain, when presented with a hotel booking conversation we may nonetheless ask it a question like "in what area is the user looking for a hotel?" and received a prediction for that unseen requirement (hotel area). while we conduct our experiments on englishlanguage dst datasets, our approach is applicable to state tracking in any language, provided a conversation history is available. problem statement we consider a conversation with t turns of user u t and system utterances y t : @xmath0, u 1 , . . . y t , u t }. the belief state b t at turn t comprises many tuples of slots @xmath1,@xmath2, extracted from the conversation history c @xmath3, u 1 , . . . , y t , u t }. the set of possible values v s can be arbitrarily large (e.g. possible hotel names), so we represent these values as sequences of vocabulary tokens v s,@xmath4, w 2 , . . . , w k }, w @xmath5. at inference time we pose a natural language question @xmath6, . . . , w n } and our model predicts an answer (slot value v s,t ) based on its understanding of the dialog history c t . to predict the belief state b t , our model independently answers |s| different questions (figure 1 ). in zero-shot dst, the system must predict values for slots outside of the initial ontology-these slot queries correspond to arbitrary natural language questions s about entities and relationships in the conversation c t . generalizing to new domains and slots dialog state tracking systems in real-world settings must scale to new users and services, accommodating new slot values (e.g. a new movie release) as well as new domains and slot types (e.g. a service update, or a new connected api). existing methods require the developer to either write a complete ontology of slots and allowed values or modify their model architecture to add slot-specific prediction heads @xcite . span-based approaches @xcite can correctly predict values that appear verbatim in a conversation but fail when a user paraphrases or mis-phrases a value. they also fall back to treating open-valued slots as classification problems @xcite . we approach dst as an ontology-free generative question answering task, as generative methods @xcite have shown promise in few-shot and supervised dst settings. while some approaches toward dst as qa learn a set of embeddings for each slot and/or domain @xcite @xcite , this is not robust to unseen slots. we encode slots as natural language questions-manually formulating one question per slot-allowing us to share a pre-trained encoder for both dialog context and slot to leverage shared linguistic knowledge @xcite . thus, our model is also agnostic to ontologies and can answer arbitrary english questions about the dialog history. we treat dst via qa as a conditional language modeling task, and train our model to predict the conditional likelihood of question (slot s) and answer (value v s,t ) tokens given a dialog context c t at a given turn t: p (v s,t , s|c t ) = p (v s,t |s, c t ) * p (s|c t ) at inference time, the model is given the dialog context alongside a question-[c t ; s]-and asked to predict the value v s,t for that slot. ## model architecture for our conditional language model, we compared two common architectures: 1) an encoder-decoder model @xcite with a bi-directional encoder; and 2) a purely auto-regressive decoderonly model. we conducted preliminary experiments using both a transformer @xcite encoder-decoder language model pre-trained using a de-noising auto-encoder objective @xcite , as well as a transformer decoder pre-trained with next-token prediction on english web pages. we achieved 1% better supervised dst performance with the decoder-only model in half the training time. our model architecture thus comprises a transformer decoder language model that allows us to leverage pre-trained language models like gpt2 @xcite and common-sense world knowledge accrued through pre-training @xcite . we use a bpe @xcite tokenizer to convert input text into a sequence of tokens. these are embedded in r h and added to an r h sinusoidal positional embedding. this input embedding is processed by l transformer layers with hidden dimensionality h, each of which applies multi-headed attention with k heads followed by a feed-forward layer with a softmax nonlinearity. the final output hidden states are then projected into our vocabulary space of 50,257 sub-word tokens. we initialize our model weights with distil-gpt2 @xcite , gpt2 @xcite , or gpt2-medium with @xmath7, 768, 1024, @xmath8, 12, 24, and @xmath9, 12, 16 respectively. as seen in figure 2 , our input sequence consists of a concatenation of dialog context c t , slot query s, and slot value v s,t : [c t ; s; v s,t ]. we prepend each utterance with a speaker token [usr] or [sys] for a user or system speaker to allow our model to identify additional context about each utterance. we pre-pend the slot query and value with question: and answer: respectively to distinguish slot queries from user-posed questions in the conversation. at training time, we calculate a cross-entropy loss similar to encoder-decoder models by maximizing the log likelihood of the slot query and value conditioned on the dialog context: this allows for flexibility in learned representations for dialog context while regularizing slot query hidden states. ## data we perform our experiments on multiwoz @xcite , which contains over 10k single-and multi-domain task-oriented dialogs written by crowd-workers. we use the 2.1 version, with corrected and standardized annotations from @xcite . we follow @xcite in lower-casing all dialogs and removing dialogs from training-only domains (police and hospital). the final dataset contains 9,906 conversations from 5 domains (restaurant, hotel, attraction, train, taxi) covering 30 domain-slot pairs. each dialog contains an average of 7 user and system turns. we also experiment with augmenting our training dataset in zero-shot settings with observations drawn from the dstc8 @xcite dataset, 1 which contains 16,152 dialogs from 45 domains. dstc8 was created via template-based dialog models provided with service apis, and then edited by crowd-workers @xcite . we normalize domains and slots corresponding to the same domain (e.g. bus 1, bus 2) for a total of 19 domains and 124 slot types in dstc8. we further manually annotate each dataset with slot value types: open-valued (e.g. hotel name), numeric (e.g. restaurant guests), temporal (e.g. taxi leaveat), and categorical (e.g. attraction type). dataset statistics are shown in ## experiments we measure dst performance via joint goal accuracy (jga): the proportion of turns with all belief slots predicted correctly, including those not present. in section 5.1, we evaluate our model on fully supervised dst, in which all domains and slots are known at training time. in section 5.2, we investigate zero-shot domain adaptation in which the model is evaluated on conversations from an unseen domain with previously unseen slots. we then explore how our framework seamlessly accommodates teaching a model to predict slot carryover (section 5.3) and transfer learning with significantly more diverse domains and slot types (section 5.4). to measure zero-shot jga, we follow @xcite and only consider slots specific to the held-out domain. we focus our analysis on the zero-shot setting, as our goal is to build dst systems that can easily and effectively generalize to new domains and services. we train all models to convergence with a maximum of 10 epochs on nvidia v100 gpus, using the lamb optimizer @xcite with a base learning rate of 2e-5. all predictions are made using greedy decoding. ## qualitative analysis we manually reviewed 300 errors made by our gpt2-medium clmq model in the zero-shot setting-annotating 20 errors from each modality (spurious, ignored, wrong value) from each domain with the gold label quality and perceived cause of error totaling 300 annotated examples. as widely observed in recent dst work @xcite , a significant proportion of dst errors on multiwoz are unavoidablecaused by annotation errors. while version 2.1 corrected some of these, annotation errors and inconsistencies remain responsible for 30% of sampled errors-in particular, in 10% of errors the original annotator did not record reactive preferences while in 5% of errors the original annotator did. these inconsistencies can hurt our model's ability to infer reactive and implied requirements and preferences. we are also particularly interested in slot transfers-when our model mistakenly predicts one slot's value for a different slot, comprising 36% of our manually reviewed errors. in the taxi and hotel domains, our model transfers slots from the same domain over 75% of the time, with most swaps occurs between same-category slots (e.g. temporal slots like taxi 'leaveat' and 'arriveby'). slots in these domains are closely semantically related, with values that can fit any slot of that category (e.g. 13:10 vs. 15:15). while a human can easily infer that the earlier of two times must be departure and the later arrival, our model has no inherent understanding of temporal mechanics or numeracy @xcite . in future work, we will explore learning such knowledge directly via hierarchical softmax output distributions to distinguish between output modalities @xcite , and fine-tuning our model with contrastive losses to learn to rank numerals and times @xcite . for restaurant, attraction, and train, our model tends to swap slot values with those from other domains in the conversation. this is often due to semantically similar slots whose values, at first glance, may not be obviously identifiable as such (e.g. 'bridge' or 'the place'). @xcite similarly observe a particularly high incidence of slot transfers between different-domain 'name' slots. other such slots include price ranges and numbers of guests. we have seen that data augmentation with dstc8 can improve our model's ability to disambiguate such slots-this suggests that we could further improve our model by exposing it to in-domain, conversational reading comprehension data. while no such dataset currently exists, in future work we aim to explore using question generation @xcite and paraphrasing @xcite models to perform in-domain data augmen-tation, creating reading comprehension questions for task-oriented dialogs that targeting entities and relations not covered by an ontology. we also wish to explore methods for generating general reading comprehension questions for out-of-domain conversations @xcite to improve our model's domain adaptation ability. ## related works modern dialog state tracking seeks to capture evolving user intents in a structured belief state @xcite . traditional systems rely on hand-crafted features @xcite and classify slot values from a fixed ontology @xcite . @xcite and @xcite fill some slots via spans extracted from dialog history, although they treat non-numeric slots as categorical. generative methods @xcite can predict arbitrary unseen values, with hosseini-asl et al. (2020) achieving state-of-the-art supervised dst performance in multiwoz 2.1 although they cannot predict unseen slots. by posing dst as generative qa, our framework can leverage language models pre-trained on open-domain documents @xcite to understand unfamiliar queries. like @xcite , we seek to answer natural language questions about each slot. we contrast our approach to zero-shot dst-which never has access to slots or dialog from the target domain-and that of campagna et al. (2020), who expose their 'zero-shot' models to synthetic in-domain conversations that require access to the full ontology of the 'held-out' evaluation domain. we take inspiration from previous work that frames a wide selection of natural language understanding (nlu) tasks @xcite as qa @xcite and span extraction @xcite . while question-answering can be posed as a span extraction task @xcite , generative approaches have proven successful in answering questions about complex passages @xcite . we use a language modeling approach, taking cues from @xcite who demonstrate that a large language model trained on next-token prediction can learn to solve many different nlu tasks posed as text. recent work has also shown that large pre-trained language models can generalize to new nlu tasks with few or no examples (brown et al., 2020), and we leverage this alongside world knowledge acquired during the pre-training process @xcite to build a dst model that is robust to new domains and slot-value ontologies. ## conclusion this paper proposes a conditional language modeling approach to multi-domain dst posed as a generative question answering task. by leveraging natural language questions as state queries, our model can generalize to unseen domains, slots, and values via its understanding of language. our model achieves state-of-the-art zero-shot results on the multiwoz 2.1 dataset with average per-domain absolute improvements of 5.9% joint accuracy. we also demonstrate that our framework is easily extensible to support transfer learning and learning slot carry-over. in the future, it is worth exploring mechanisms for our model to better understand relative temporal values and general reading comprehension questions from conversations in order to disambiguate semantically similar dialog slots.