Diverse Retrieval-Augmented In-Context Learning for Dialogue State Tracking (2023)

Abstract: There has been significant interest in zero and few-shot learning for dialogue state tracking (DST) due to the high cost of collecting and annotating task-oriented dialogues. Recent work has demonstrated that in-context learning requires very little data and zero parameter updates, and even outperforms trained methods in the few-shot setting. We propose RefPyDST, which advances the state of the art with three advancements to in-context learning for DST.First, we formulate DST as a Python programming task, explicitly modeling language coreference as variable reference in Python. Second, since in-context learning depends highly on the context examples, we propose a method to retrieve a diverse set of relevant examples to improve performance. Finally, we introduce a novel re-weighting method during decoding that takes into account probabilities of competing surface forms, and produces a more accurate dialogue state prediction. We evaluate our approach using MultiWOZ and achieve state-of-the-art multi-domain joint-goal accuracy in zero and few-shot settings.

## introduction dialogue state tracking (dst) is an important language understanding task required for supporting task-oriented conversational agents. for each turn in a dialogue, the goal of dst is to extract the intentions and arguments a user communicates into a meaning representation aligned with the capabilities of the system. often, this can be represented as a set of slot-value pairs, using slots defined in a system schema. for example, if a user asks a hotel booking agent for "a four-star hotel with somewhere to park", the agent could extract the state {(hotel-stars, 4), (hotel-parking, yes)}. annotating these turn-level dialogue states is challenging and time-intensive @xcite . further, as system capabilities evolve ## task definition a task-oriented dialogue consists of turns or paired utterances between a user and an agent which interfaces the user with a programmable system. at each turn t, the purpose of a dst module is to use the dialogue history up to that turn to predict a dialogue state y t , which represents the user's goal and progress in using the system. let a i be an agent utterance, u i be a user utterance, and c @xmath0, u 1 ), (a 2 , u 2 ), ...(a t , u t )] foot_0 be the dialogue history up to turn t. the task is to map the history c t to a state representation y t . in this work, we predict dialogue states y t which can be represented as slot-value pairs: we can equivalently define this task as predicting state changes, as proposed in @xcite . let x @xmath1, (a t , u t )] be a dialogue context consisting of the previous dialogue state prediction and utterances for the current turn. using this turn context x t , we predict a state change: for these reasons, we formulate our dst problem as mapping from the turn context x t to a state change ∆y t . for readability, we often use 'turn' to refer to this turn context x t , distinguishing it from the history c t or turn number t using notation. ## methods given a dialogue turn t, our method produces a state change ∆y t by (1) retrieving a set of incontext examples e k , (2) formatting these into a prompt f prompt (x t , e k ), (3) generating and scoring possible program solutions (lm completions) with openai codex @xcite , (4) executing the program to compute a state change ∆y t . given the state change, we compute the complete dialogue state y t by applying the difference to y t-1 . we describe our prompting function f prompt (x t , e k ), in § 3.1. in § 3.2, we describe our method for retrieving a diverse and representative set of examples e k . finally, we describe our method for scoring lm completions with a pointwise mutual information estimate in § 3.3. ## experiments we describe our zero and few-shot experimental setups, evaluation, and baselines. hyperparameter and implementation details can be found in appendix c. ## results few-shot dst on multiwoz we present fewshot and full-shot dialogue state tracking results on multiwoz 2.1 & 2.4 in ## analysis & ablations in this section, we further analyze the performance characteristics of our method. for full evaluation of all dialogues in the zero-shot setup, see ## related work dialogue state tracking there has been a recent increase in work on the zero and few-shot dst systems. many approaches fine-tune a pretrained language model by re-framing dst as some form of text-to-text or auto-regressive language modeling task @xcite @xcite @xcite @xcite @xcite . many of these methods often exhibit zero-shot transfer capabilities @xcite @xcite . however, these approaches still require re-training when a domain is added or changed, and zero-shot transfer performance is dependent on the relatedness of the new domain to existing ones. some recent works instead model dst as an incontext learning problem @xcite @xcite , bypassing the need for re-training when system definitions change. in particular, we build on the work of @xcite , which models dst by predicting dialogue state changes at each turn, relying on only a state summary and agent/user turn utterances for inference. their work models dst as a text-to-sql problem, whereas we model it as a python programming problem with novel methods for selecting in-context examples and scoring language model completions. in-context learning some recent works explore the properties of effective in-context examples. in classification settings, @xcite find random examples can significantly limit performance, and propose using a pre-trained embedding model to find examples semantically close to x, retrieving one per class. other works investigate the role of examples in icl performance in detail, finding that icl methods perform best when example inputs and test inputs are as close in distribution as possible, and when the distribution of exemplified labels closely matches the target distribution @xcite . paralleling this, a number of works across nlp tasks propose methods for retrieving relevant incontext examples. @xcite use an unsupervised embedding model to embed a test input x and all available examples, retrieving the k with highest embedding cosine similarity. other works use a similar dense retriever but in an embedding space learned with supervision. @xcite fine-tune an example retriever with contrastive learning in which positive examples maximize p lm (y|x, e i ). @xcite propose a contrastive learning objective specific to dst, finetuning an embedding model to embed turns with similar state changes in proximity to each other. rather than use a separate retrieval module, @xcite use the lm itself to select examples which are most likely when conditioned on x. given a test input x, each of these works scores the relevance of an individual example e i to a test input x and then selects the k most relevant ones to include in a prompt. in most cases, this yields a set of examples e k which are meaningfully similar to x. however, considering examples individually does not necessarily lead to adequate exemplification of the output space. in supervised settings that learn a relevance metric which approximates output similarity, this can lead to degenerate examples sets e k which all exemplify the same out-put. in contrast to this, we propose a novel method for using this score to construct e k with examples that are relevant to x while being distinct from each other. in concurrent work to our own, ye et al. (2022b) propose a method for decoding diverse examples of explanations from a retriever for use in reasoning problems, also based on maximum-marginalrelevance (mmr) @xcite . their work uses unsupervised measures of similarity between explanations, where ours uses a supervised retriever which approximates similarity of outputs. thus, diversity in our example sets correlates to diversity in exemplified outputs. in another concurrent work to our own @xcite propose a method for diverse example selection in a semantic parsing task, using the outputs of selected examples to incrementally cover more structures in e k . for tasks which can be re-framed as program synthesis, a number of works have also developed icl methods for use with lms pre-trained on code such as codex and codegen @xcite . @xcite use icl with codex to generate lisp-like programs in a dialogue semantic parsing task. @xcite evaluate such models capabilities in text-to-sql problems, and @xcite use a text-to-sql framing to use codex for dst. instead of sql queries, we generate python programs, allowing for intuitive modeling of phenomena like coreference. finally, recent works have considered adjusting how completion strings are scored with an lm. brown et al. (2020) normalize log-likelihoods by length before scoring completions. @xcite re-weigh lm probabilities by learning an affine transformation that yields uniform scores given 'content-free inputs'. @xcite propose p m i dc , a method for re-scoring completions using pointwise mutual information (pmi), which we adapt to our constrained generative setting. ## conclusion we propose refpydst, an in-context learning method for dst. our contributions address key challenges in dst and in retrieval-augmented icl, producing state-of-the-art results on multiwoz dst benchmarks for few-shot and zero-shot setups. future work could apply methods developed here to other in-context learning problems. while in-context learning methods for dst are promising in their data efficiency and flexibility to new domains, they typically require very large models to perform effectively. at 175 billion parameters, openai codex @xcite is much larger than some of the fine-tuned approaches to dst, though with better performance and ability to adapt to new domains without re-training. despite our advances, there are still significant errors when applying icl for dst. as such, icl may not necessarily be relied on in safety-critical settings.