Stabilized In-Context Learning with Pre-trained Language Models for Few Shot Dialogue State Tracking (2023)

Abstract: Prompt-based methods with large pre-trained language models (PLMs) have shown impressive unaided performance across many NLP tasks. These models improve even further with the addition of a few labeled in-context exemplars to guide output generation. However, for more complex tasks such as dialogue state tracking (DST), designing prompts that reliably convey the desired intent is nontrivial, leading to unstable results. Furthermore, building in-context exemplars for dialogue tasks is difficult because conversational contexts are long while model input lengths are relatively short. To overcome these issues we first adapt a meta-learning scheme to the dialogue domain which stabilizes the ability of the model to perform well under various prompts. We additionally design a novel training method to improve upon vanilla retrieval mechanisms to find ideal in-context examples. Finally, we introduce a saliency model to limit dialogue text length, allowing us to include more exemplars per query. In effect, we are able to achieve highly competitive results for few-shot DST on MultiWOZ.

## introduction tremendous gains have been made on dialogue state tracking (dst) using large pre-trained language models (plms) @xcite , fine-tuning such systems though require significant amounts of data, which in turn require substantial effort to collect. recently, prompting has emerged as a technique for achieving strong performance in a less resource intensive manner @xcite . even better performance is possible with in-context exemplars providing a pattern for the model to follow @xcite . ideally, we should be able to apply these concepts to complex tasks like dst, but results so far have been limited @xcite . exemplars prompt query exemplar context: i would like a train to cambridge arriving at 14:00 exemplar prompt: train destination is exemplar target: cambridge naive: area of the restaurant question: where is the desired restaurant location? statement: the preferred restaurant area is previous context: <@xmath0, what type of food would you prefer? <@xmath1. current turn: <@xmath2. do you have a specific area in mind? <@xmath3. dialogue query possible prompts t5 target value: in the @xmath4: north meta-training inference no gradient update! exemplar context: i would like a train to cambridge arriving at 14:00 exemplar prompt: train destination is exemplar target: cambridge exemplar context: i would like a train to cambridge arriving at 14:00. exemplar prompt: train destination is exemplar target: cambridge in-context exemplars one reason for the lack of progress comes from the difficulty of hand-crafting prompts (patterns) and targets (verbalizers), which are highly sensitive to exact phrasing @xcite . while manually designed prompts have been found to be brittle and unstable @xcite , automatically designed prompts @xcite cannot be easily applied to dst since many slots are non-enumerable @xcite . a second major hurdle is around dialogue sequence lengths, which are often much longer than those for other tasks @xcite preventing the inclusion of many exemplars for guidance. full conversations consist of long histories going back many turns, such that the context itself (sans prompt) is already capable of filling a model's entire input length. since state tracking requires carrying over previous dialogue states, naively truncating prior context effectively equates to random guessing @xcite . a third issue is selecting the exemplars themselves. prior work recommends choosing a representative example from each class @xcite , but this is not possible in many cases since most domain-slot-value label combinations simply do not appear in the dataset. moving to the fewshot scenario further exacerbates this sparsity. separately, recall that our main goal is to do well in few-shot dst because we purposefully operate in a practical, low-resource data setting. correspondingly, we aim to achieve good results with a similar low-resource model setting where training should be possible on a single publiclyavailable commodity server. this precludes the usage of gigantic models such as gpt-3, which are prohibitively expensive to train and bear high economic and environmental costs for inference alone @xcite . we directly tackle each of the three aforementioned issues to achieve state-of-the-art performance on multiwoz when restricted to models under 100 billion parameters. to minimize prompt issues, we introduce a meta in-context learning (icl) framework to stabilize training and reduce variance in prompt performance. to deal with long dialogues, we are inspired by summarizaton work to condense dialogue histories and filter out nonsalient sentences. our third contribution is designing a novel loss function to train a retrieval model that selects ideal exemplars for priming our downstream model. our analysis and ablations show that all components help improve our state tracking performance. finally, we show that unlike other models which only work on specialized lms, our proposed methods work on any sort of lm, and can be improved with additional training. ## our method this section describes our proposal of a stabilized dialogue state tracker, which leverages meta incontext learning, dialogue summarization and a novel multi-part training loss for fine-tuning a retrieval model, which we refer to as sm2 for short. ## experiments this section outlines our training implementation details as well as key experiments. ## results and analysis the goal of this work is to achieve strong results on dst without worrying about tedious promptengineering. consequently, we first analyze the ability of the best performing models and then discuss performance stability across different prompts. ## conclusion in this paper, we presented a method of performing few-shot dialogue state tracking by leveraging large pre-trained lms with prompts. our technique does not require any gradient-based training for the target task and instead relies on in-context learning to guide model generation. to enable success in this low-resource setting, we stabilize training across prompts with meta-icl, apply saliency filtering and context summarization to reduce dialogue length, and fine-tune a sentence embedder with a custom loss objective to improve exemplar retrieval. these techniques combined allow us to reach state-of-the-art results on multiwoz when limited to models under 100 billion parameters. exemplar retrieval dialog id target utterance exemplar score ssng0074.json i am looking for a restaurant in the moderate price range that serves bistro type food. e1: i would love to help. any particular food you'd like? no, i'd just like for it to be in the east and moderately priced. 0.738 e2: seventeen locations meet your criteria. would you prefer a guesthouse or a hotel? a hotel is fine whichever you recommend. -0.074 saliency filtering pmul0287.@xmath5: the phone number is 01223259988. <user>: perfect. can you help me with a reservation for 6 people at 14:30 this coming sunday? and please make sure i have a confirmation number to use. <agent>:our reservation is set! pmul1635.@xmath6: what day will you be staying? <user>: friday and can you book it for me and get a reference number ? <agent>:booking was successful. reference number is : bmukptg6. can i help you with anything else today? <user>: i am looking to book a train that is leaving from cambridge to bishops stortford on friday. moving forward, we plan to explore techniques that push model and data efficiency even further. distillation and pruning can lead to much fewer model parameters, while numerous data augmentation techniques seem promising in maximizing the advantage of limited labeled data. lastly, rather than meta-learning across different dialog domains, we also would like to explore meta-train model with different prompt styles. with the current framework, the prompt used in inference is required to be the same as the training. however, we might want to use flexible prompts in practice. consequently, we could meta-train across different prompt styles to allow the model to quickly learn a new prompt style during inference.