Dialogue State Tracking with a Language Model using Schema-Driven Prompting (2021)

Abstract: Task-oriented conversational systems often use dialogue state tracking to represent the user’s intentions, which involves filling in values of pre-defined slots. Many approaches have been proposed, often using task-specific architectures with special-purpose classifiers. Recently, good results have been obtained using more general architectures based on pretrained language models. Here, we introduce a new variation of the language modeling approach that uses schema-driven prompting to provide task-aware history encoding that is used for both categorical and non-categorical slots. We further improve performance by augmenting the prompting with schema descriptions, a naturally occurring source of in-domain knowledge. Our purely generative system achieves state-of-the-art performance on MultiWOZ 2.2 and achieves competitive performance on two other benchmarks: MultiWOZ 2.1 and M2M. The data and code will be available at https://github.com/chiahsuan156/DST-as-Prompting .

## introduction in task-oriented dialogues, systems communicate with users through natural language to accomplish a wide range of tasks, such as food ordering, tech support, restaurant/hotel/travel booking, etc. the backbone module of a typical system is dialogue state tracking (dst), where the user goal is inferred from the dialogue history @xcite @xcite . user goals are represented in terms of values of pre-defined slots associated with a schema determined by the information needed to execute task-specific queries to the backend. in other words, user goals are extracted progressively via slot filling based on the schema throughout the conversation. in this paper, we focus on multi-domain dst where the dialogue state is encoded as a list of triplets in the form of (domain, slot, value), e.g. ("restaurant", "area", "centre"). there are two broad paradigms of dst models, classification-based and generation-based models, where the major difference is how the slot value is inferred. in classification-based models @xcite , the prediction of a slot value is restricted to a fixed set for each slot, and non-categorical slots are constrained to values observed in the training data. in contrast, generationbased models @xcite decode slot values sequentially (token by token) based on the dialogue context, with the potential of recovering unseen values. recently, generationbased dst built on large-scale pretrained neural language models (lm) achieve strong results without relying on domain-specific modules. among them, the autoregressive model @xcite uses a uni-directional encoder whereas the sequence-to-sequence model @xcite represents the dialogue context using a bi-directional encoder. in this study, we follow a generation-based dst approach using a pre-trained sequence-to-sequence model, but with the new strategy of adding taskspecific prompts as input for sequence-to-sequence dst models, inspired by prompt-based fine-tuning @xcite . specifically, instead of generating domain and slot symbols in the decoder, we concatenate the dialogue context with domain and slot prompts as input to the encoder, where prompts are taken directly from the schema. we hypothesize that jointly encoding dialogue context and schema-specific textual information can further benefit a sequence-to-sequence dst model. this allows task-aware contextualization for more effectively guiding the decoder to generate slot values. although the domain and slot names typically have interpretable components, they often do not reflect standard written english, e.g. "arriveby" and "ref ". those custom meaning representations are typically abbreviated and/or under-specified, which creates a barrier for effectively utilizing the pretrained lms. to address this issue, we further incorporate natural language schema descriptions into prompting for dst, which include useful information to guide the decoder. for example, the description of "ref " is "reference number of the hotel booking"; the values of "has_internet" are "yes", "no", "free", and "don't care". in short, this work advances generation-based dst in two ways. first, candidate schema labels are jointly encoded with the dialogue context, providing a task-aware contextualization for initializing the decoder. second, natural language descriptions of schema categories associated with database documentation are incorporated in encoding as prompts to the language model, allowing uniform handling of categorical and non-categorical slots. when implemented using a strong pretrained text-to-text model, this simple approach achieves state-of-theart (sota) results on multiwoz 2.2, and performance is on par with sota on multiwoz 2.1 and m2m. in addition, our analyses provide empirical results that contribute towards understanding how schema description augmentation can effectively constrain the model prediction. ## prompting language model for dialogue state tracking in this section, we first set up the notations that are used throughout paper, and then review the generative dst with the sequence-to-sequence framework. ## conclusion in this work, we propose a simple but effective task-oriented dialogue system based on large-scale pretrained lm. we show that, by reformulating the dialogue state tracking task as prompting knowledge from lm, our model can benefit from the knowledge-rich sequence to sequence t5 model. based on our experiments, the proposed natural language augmented prompt-based dst model achieve sota on multiwoz 2.2 and comparable performance on multiwoz 2.1 and m2m to recent sota models. moreover, our analyses provide evidence that the natural language prompt is effectively utilized to constrain the model prediction. sim-m domain domain description slot slot description movie a go-to provider for finding movies, theatre_name the name of the theatre where the movie is playing searching for show times and booking tickets movie name of the movie date date of the show booking time time of the show booking num_people number of people to purchase tickets for sim-r domain domain description slot slot description restaurant find places to dine and whet your appetite price_range price budget for the restaurant location the location or area of the restaurant restaurant_name the name of the restaurant category the cuisine of the restaurant you are looking for num_people how many people for the restaurant reservation date date of the restaurant booking time time of the restaurant booking