D emo NSF : A Multi-task Demonstration-based Generative Framework for Noisy Slot Filling Task (2023)

Abstract: Recently, prompt-based generative frameworks have shown impressive capabilities in sequence labeling tasks. However, in practical dialogue scenarios, relying solely on simplistic templates and traditional corpora presents a challenge for these methods in generalizing to unknown input perturbations. To address this gap, we propose a multi-task demonstration-based generative framework for noisy slot filling, named DemoNSF. Specifically, we introduce three noisy auxiliary tasks, namely noisy recovery (NR), random mask (RM), and hybrid discrimination (HD), to implicitly capture semantic structural information of input perturbations at different granularities. In the downstream main task, we design a noisy demonstration construction strategy for the generative framework, which explicitly incorporates task-specific information and perturbed distribution during training and inference. Experiments on two benchmarks demonstrate that DemoNSF outperforms all baseline methods and achieves strong generalization. Further analysis provides empirical guidance for the practical application of generative frameworks. Our code is released at https://github.com/dongguanting/Demo-NSF.

## introduction the slot filling (sf) task in the goal-oriented dialog system aims to identify task-related slot types in certain domains for understanding user utterances. recently, traditional discriminative and generative models @xcite @xcite @xcite @xcite have shown remarkable ability in slot filling. despite their powerful capabilities, the high performance of these models heavily depends on the consistency of data distribution between the training and test sets. when faced with the uncertainty and diversity of human language expression @xcite , these perturbations significantly impact the sf model's generalization ability, thereby hindering its application in practical dialogue scenarios. in real dialogue systems, models often encounter a wide range of input perturbations and errors made by humans. as illustrated in figure 1 , users may interact with the dialogue system in ways that deviate from the standard input format and even simplify their queries to convey the same intent, all due to diverse human expression habits. furthermore, errors originating from the upstream input system may introduce disturbances to the downstream model (e.g. typos from keyboard input, speech errors from asr systems). existing slot filling models are typically pre-trained and fine-tuned on perturbationfree datasets, leading to decreased performance when confronted with such situations. recently, existing studies @xcite @xcite have explored the issue of robustness. however, these methods are mainly designed for particular perturbations, limiting generalization ability to unknown perturbations. to capture the noisy semantic structure, pssat and cmda @xcite further introduce additional corpus and generative models. nevertheless, this approach carries the risk of introducing extra noise and increasing computing resource consumption. while large language models @xcite @xcite and prompt-based methods @xcite have achieved strong performance on information extraction, the exploration of these generative frameworks on diverse input perturbations remains a blank area, hindering their application in realistic task-oriented dialogue systems. to address this limitation, we propose a multitask demonstration-based generative framework for noisy slot filling tasks, named demonsf. specifically, we design three noisy auxiliary tasks, namely noisy recovery (nr), random mask (rm), and hybrid discrimination (hd), to boost the performance against input perturbations in different levels. nr aims to capture the mapping relationship between fine-grained noisy and clean data. rm implicitly learns the slot entity distribution of perturbed data during the process of mask infilling. hd assists generative models consider global information while implicitly capturing the semantic characteristics unique to perturbed data. in the downstream process, we formulate the sf task as a sequence-tosequence generation guided by noisy task demonstrations. in detail, demonsf selects a semantically similar example for each query from a noisy candidate pool, converts it into a natural demonstration sentence, and encodes the demonstration along with the input text by integrating noisy semantic information. with the boost of noisy auxiliary tasks and demonstrations, demonsf learns the semantic structure of perturbations from both explicit and implicit levels. our contributions are three-fold: ## method in this section, we introduce the overall framework of our proposed demonsf. we first briefly describe the problem definition against input perturbations in the slot filling task. next, we propose three distinctive noisy auxiliary tasks for diverse perturbations. finally, we present a novel noisy demonstration construction strategy. we will introduce these in the following subsections foot_0 . ## conclusion in this paper, we propose a unified multi-task demonstration-based generative framework for noisy slot filling tasks. specifically, we introduce three novel noisy auxiliary tasks and a noisy demonstration construction strategy for the generative framework, which aims to learn the semantic structure of perturbations from both explicit and implicit levels. experiments on two benchmarks show the effectiveness of demonsf, further analysis provides empirical guidance for the practical application of the generative framework.