Prompter: Zero-shot Adaptive Prefixes for Dialogue State Tracking Domain Adaptation (2023)

Abstract: A challenge in the Dialogue State Tracking (DST) field is adapting models to new domains without using any supervised data — zero-shot domain adaptation. Parameter-Efficient Transfer Learning (PETL) has the potential to address this problem due to its robustness. However, it has yet to be applied to the zero-shot scenarios, as it is not clear how to apply it unsupervisedly. Our method, Prompter, uses descriptions of target domain slots to generate dynamic prefixes that are concatenated to the key and values at each layer’s self-attention mechanism. This allows for the use of prefix-tuning in zero-shot. Prompter outperforms previous methods on both the MultiWOZ and SGD benchmarks. In generating prefixes, our analyses find that Prompter not only utilizes the semantics of slot descriptions but also how often the slots appear together in conversation. Moreover, Prompter’s gains are due to its improved ability to distinguish ”none”-valued dialogue slots, compared against baselines.

## introduction task-oriented dialogue (tod) systems serve users through several tasks, such as booking a table in a restaurant or suggesting tourist attractions. one crucial component of these systems, dialogue state tracking (dst), is responsible for extracting users' preferences (i.e. slot-values) over key attributes (i.e. slot-labels) of their service @xcite . dst has a significant role in tod systems as it ensures that both the action taken in the back-end and the responses returned to the users are aligned with the preferences that the users indicate. a challenging task in this field is to adapt an existing dst model to a new domain it has not seen before without using any supervised data, i.e. in the zero-shot scenario. this is important, as in many new scenarios, it is hard to collect data, let alone annotate it. yet it is still an essential need for a tod system to appropriately answer such queries in new contexts. the challenge arises from the differences in dialogue context, slot values, and slot labels among different domains. for example, a model could be trained on the 'taxi-booking' domain and thus capable of extracting the destination for a taxi; but when deployed to the 'train-booking' domain, the range of slot-values changes, resulting in a higher probability of a mistaken inference. we show an example (figure 1 ), where due to the superficial connections a baseline t5 model forms, it incorrectly predicts 'ashley hotel' as the train destination (bottom left). in many dialogue contexts, a large number of slots are unspecified. these are known as "none"-valued slots. in cases where the model is adapting to a new domain without any prior training, it often incorrectly predicts none values. this makes it even more important to address the problem of domain shift. @xcite proposed to address this domain shift challenge via the language model's intrinsic ability to reason over prompts. specifically, they concatenate the description of each slot as a hard prompt into the dialogue context and then generate the answers using the t5 model. while it does well for a naive baseline, it makes mistakes due to its superficial understanding of slot labels. meanwhile, another line of study has shown that parameter-efficient transfer learning (petl) methods are effective training methods to address domain shift. due to the small number of parameters it introduces per task/instance, it overcomes overfitting in few-shot scenarios, outperforming earlier baselines. there have been various attempts to use these methods for dst tasks within a fewshot, continual learning setting @xcite . however, a significant barrier to adopting petl is that such methods cannot be directly applied in zero-shot, as they all require some form of supervised training. in this study, we propose a new method to use prefix-tuning under a zero-shot scenario to benefit from the gains it brings for robustness, even without supervised data. rather than fine-tuning the prefixes during training, we add a new mechanism into the t5 architecture called prompter 1 . prompter simply takes the description of the slot and then generates the prefixes on the fly. we then append these prefixes at each layer of the encoder to represent the dialogue from the perspective of the subject slot label. this method makes minimal changes to lm parameters while generating unsupervised prefixes. this ensures both the preservation of general-purpose traits and extrapolation to new domains. we conduct experiments with the multiwoz 2.1 and sgd datasets. prompter improves average jga results across domains by 1.7 for multiwoz, and 9.1 points for the sgd dataset (considering 4 domains reported in prior studies) compared to the 1 implementation available at https://github.com/ cuthalionn/prompter strongest baseline. this shows that petl methods' robustness advantage is also favorable for unsupervised domain adaptation scenarios. to the best of our knowledge, these are the highest results achieved so far using a small language model. through further analysis, we have discovered that prompter not only considers the semantic similarities of slot descriptions but also the frequencies in which slots co-appear in the dialogue context. furthermore, prompter proves to be more effective in identifying slots that have no value within a conversation in comparison to previous methods. ## related work dialogue state tracking. dst has a long history of models working with a static, ontologybased problem definition (i.e. slot-values are fixed) @xcite . the static-ontology dst is a simplified classification problem where the model selects a value from each slot's value pool. @xcite @xcite . recently interest in dynamic ontologies have received attention, adding flexibility at inference time @xcite @xcite . ## method we propose to add a new mechanism into the t5 architecture @xcite , called prompter, to take advantage of prefix-tuning's extrapolation capabilities without requiring supervision. instead of fine-tuning the prefixes with source domain data, we generate them on the fly for each slot. however, we need a way to condition prompter for a new domain without any supervised data. taskoriented dialogue schemas provide a solution by annotating the slot descriptions for each slot-label. using these slot descriptions prompter can generate domain-specific prefixes which allow it to adapt to any domain without the need for supervised data. we can summarize the prompter pipeline in three key parts: (1) slot prompt generation, (2) prefix generation, and (3) multi-head self attention. slot prompt generation. is responsible for generating a prompt that is specific to each slot, using its unique description. previous approaches to this problem, such as simply concatenating the description to the input, result in only a superficial understanding of the slots in zero-shot settings @xcite . additionally, using slot embeddings as soft prompts can cause unstable training and hinder zero-shot adaptation due to changes in the descriptions. instead, we propose using a global prompt that is modified according to each slot's description. this modification is applied through a cross-attention mechanism that attends the global prompt to the slot description's embedding, c.f. figure 2a . this approach ensures that each slot prompt shares the same initialization addressing unstable training, and the modifications reflect changes in the slot-label addressing domain shift. it also has the advantage of making the final prompt's length fixed, regardless of the length of the description. the slot prompt is calculated as follows: prefix generation. for the dst task, the dialogue context can make up the majority of the language model input (i.e. 100-400 tokens long dialogue context compared to 10-15 tokens long slot description), this results in challenges with the prompt-tuning method because the prompt's impact can vanish easily before the decoding starts. this is why we opt for prefix-tuning because it ingests prompts at each layer and thus the generated value will have higher exposure to the prompt. so following the generation of slot prompts the next step is to generate key and value prefixes for each layer. for this step, we have tried several different architectural designs such as a simple mlp or a whole transformer block. we empirically observed that while the former lags behind due to the small number of parameters the latter results in overfitting. thus, inspired by @xcite we use a sequence of down and up projections separated by an activation function as prefix generators, c.f. figure2b. note that each transformer layer has a pair of dedicated prefix generators to generate key and value prefixes: multi-head self attention. after we get k i and v i for each layer i we split them to n h head vectors k j i and v j @xmath0, where d @xmath1. finally, we concatenate these key and value prefixes into the self-attention mechanism at each layer of the 4591 transformer encoder completing our modifications to the original t5 architecture, c.f. figure 2c . ## results and analysis in multiwoz ( we believe the lack of improvements in the hotel domain for prompter is due to it having many unique slots (i.e. 'hotel-internet', 'hotel-parking', 'hotel-type', etc.). this makes it harder to take advantage of earlier domains as they lack similar slots. this is also in line with the results from @xcite , as their cross-slot dependency design also lags behind for hotel domain results. ## conclusion parameter efficient transfer learning methods have been frequently used for their strong robust features under a low-resource setting. however, there is no straightforward way to take advantage of these features under a zero-shot setting because they require at least some supervised data during adaptation. the dialogue state tracking (dst) task, on the other hand, has just the right annotation for this scenario as it contains schema annotations with slot label descriptions. we propose prompter, which uses these descriptions to enable prefix-tuning, a well-known petl method, for use under a zero-shot domain adaptation setting. we show through experiments that this method improves the jga metric for the two most common dst benchmarks. we further explain through analyses and a case study that the reason behind the prompter's power is two-fold. (1) it has better capability to distinguish 'none' valued slots within the dialogue and (2) it can digest the frequency of slots co-occurrences within the dialogue context into the prefix generation process. we believe that this study shows petl's hidden potential for dst domain adaptation under a zero-shot setting.