M eta ASSIST : Robust Dialogue State Tracking with Meta Learning (2022)

Abstract: Existing dialogue datasets contain lots of noise in their state annotations. Such noise can hurt model training and ultimately lead to poor generalization performance. A general framework named ASSIST has recently been proposed to train robust dialogue state tracking (DST) models. It introduces an auxiliary model to generate pseudo labels for the noisy training set. These pseudo labels are combined with vanilla labels by a common fixed weighting parameter to train the primary DST model. Notwithstanding the improvements of ASSIST on DST, tuning the weighting parameter is challenging. Moreover, a single parameter shared by all slots and all instances may be suboptimal. To overcome these limitations, we propose a meta learning-based framework MetaASSIST to adaptively learn the weighting parameter. Specifically, we propose three schemes with varying degrees of flexibility, ranging from slot-wise to both slot-wise and instance-wise, to convert the weighting parameter into learnable functions. These functions are trained in a meta-learning manner by taking the validation set as meta data. Experimental results demonstrate that all three schemes can achieve competitive performance. Most impressively, we achieve a state-of-the-art joint goal accuracy of 80.10% on MultiWOZ 2.4.

## introduction task-oriented dialogue systems have recently become a hot research topic. they act as digital personal assistants, helping users with various tasks such as hotel bookings, restaurant reservations, and weather checks. dialogue state tracking (dst) is recognized as a core task of the dialogue manager. its goal is to keep track of users' intentions at each turn of the dialogue @xcite . tracking the dialogue state accurately is of significant importance, as the state information will be fed into the dialogue policy learning module to determine the next system action to perform @xcite . in general, the both frameworks utilize soft labels obtained by linearly combining pseudo labels (one-hot) and vanilla labels (one-hot) using a weighting parameter α to enhance the training process compared to standard training that only relies on vanilla noisy labels. assist adopts a single α that is shared by all slots and all training samples, while metaassist uses slot-wise (and instance-wise) αs. in order to robustly train dst models from noisy labels, @xcite proposed a general framework dubbed assist, which augments the standard model training procedure with a small clean dataset. as shown in figure 1 , assist first trains an auxiliary model on the small clean dataset and applies this model to generate pseudo labels for each sample in the noisy training set. then, it linearly combines the pseudo labels and vanilla labels to train the primary model. both theoretically and empirically, assist has been shown to be effective in reducing the impact of label noise. however, assist adopts a common weighting parameter to combine the pseudo labels and vanilla labels for all slots and all training samples, which is suboptimal. in reality, different slots tend to have different noise rates @xcite , indicating that the weighting parameter should be slot-wise. on the other hand, different training samples may also require different weighting parameters, since whether pseudo labels or vanilla labels should be preferred is highly dependent on specific training instances. furthermore, the weighting parameter is considered a hyperparameter and thus needs to be carefully tuned on each dataset. to address the aforementioned limitations of as-sist, we propose metaassist, a meta learningbased general framework that supports automatically learning slot-wise (and instance-wise) weighting parameters. specifically, our contributions are: ## preliminaries in task-oriented dialogue systems, the dst module transforms users' goals or intentions expressed in unstructured natural languages into structured state representations (e.g., a series of slot-value pairs). the state representations are continually updated in each round of the user-system interactions. ## metaassist: a meta learning-based version of assist equations ( 1 ) and (3) show that a single α is shared by all slots when combining the pseudo labels and vanilla labels. this is suboptimal, as the ratio of the noise rate of pseudo labels to that of vanilla labels tends to be different for different slots. when the vanilla labels have higher quality than the generated pseudo labels, α should be set to a small value; otherwise, a large α should be used. this implies that setting α to different values for different slots can help train the primary model more robustly. in the following, we first theoretically show that the combined labels obtained via slot-wise weighting parameters instead of a common one can better approximate the unknown true labels. then, we elaborate on the proposed framework metaassist. ## related work dst has been studied for more than one decade. traditional dst models rely on a separate language understanding module to extract relevant information @xcite . in recent years, designing dst models based on neural networks, especially pretrained language models, has become the mainstream and a large number of neural dst models have been proposed @xcite @xcite @xcite @xcite @xcite @xcite @xcite . although these neural dst models have demonstrated good performance, they fail to consider the effect of label noise. it has been shown that no matter how noisy the training data are, neural models can easily overfit the training data @xcite . as a result, the generalization performance of models trained on noisy data is usually unsatisfactory. recently, @xcite proposed a general framework, assist, to robustly train dst models on noisy data. their experimental results show that several existing dst models can achieve much higher performance when trained under this framework. however, as discussed earlier, assist contains a parameter that needs to be tuned on each dataset. besides, the parameter is shared among all slots and all training samples, which we have shown to be suboptimal. our proposed framework leverages meta learning to automatically learn slotwise (and instance-wise) parameters, overcoming the limitations of assist. the essence of meta learning is learning to learn @xcite , which makes it a natural fit for our task of automatically learning parameters. we found that several existing works @xcite @xcite have already applied meta learning to dst. these works directly adopt the maml @xcite algorithm or its variants and focus on improving the few-shot learning ability of dst models. while our focus is to improve the robustness of dst models. ## conclusion in this work, we proposed a meta learning-based general framework metaassist to robustly train dst models on noisy data. metaassist improves assist by automatically learning slot-wise (and instance-wise) weighting parameters that are used to combine pseudo labels and vanilla labels. our comprehensive experiments demonstrate the effectiveness of metaassist. for future work, we plan to extend the current framework to utilize pseudo labels generated by multiple auxiliary models.