Improving Limited Labeled Dialogue State Tracking with Self-Supervision (2020)

Abstract: Existing dialogue state tracking (DST) models require plenty of labeled data. However, collecting high-quality labels is costly, especially when the number of domains increases. In this paper, we address a practical DST problem that is rarely discussed, i.e., learning efficiently with limited labeled data. We present and investigate two self-supervised objectives: preserving latent consistency and modeling conversational behavior. We encourage a DST model to have consistent latent distributions given a perturbed input, making it more robust to an unseen scenario. We also add an auxiliary utterance generation task, modeling a potential correlation between conversational behavior and dialogue states. The experimental results show that our proposed self-supervised signals can improve joint goal accuracy by 8.95% when only 1% labeled data is used on the MultiWOZ dataset. We can achieve an additional 1.76% improvement if some unlabeled data is jointly trained as semi-supervised learning. We analyze and visualize how our proposed self-supervised signals help the DST task and hope to stimulate future data-efficient DST research.

## introduction dialogue state tracking is an essential component in task-oriented dialogue systems designed to extract user goals/intentions expressed during a conversation. accurate dst performance can facilitate downstream applications such as dialogue management. however, collecting dialogue state labels is very expensive and timeconsuming @xcite , requiring dialogue experts or trained turkers to indicate all (domain, slot, value) information for each turn in dialogues. this problem becomes important from single-domain to multi-domain scenarios. it will be more severe for a massive-multi-domain setting, making dst models less scalable to a new domain. existing dst models require plenty of state labels, especially those ontology-based dst approaches @xcite @xcite . they assume a predefined ontology that lists all possible values is available, but an ontology requires complete state annotation and is hard to get in real scenario @xcite . they also cannot track unseen slot values that are not predefined. ontology-free approaches @xcite , on the other hand, are proposed to generate slot values from dialogue history directly. they achieve good performance on multi-domain dst by copyattention mechanism but still observe a significant performance drop under limited labeled data scenario @xcite . in this paper, we approach the dst problem using copy-augmented ontology-free models from a rarely discussed perspective, assuming that only a few dialogues in a dataset have annotated state labels. we present two self-supervised learning (ssl) solutions: 1) preserving latent consistency: we encourage a dst model to have similar latent distributions (e.g., attention weights and hidden states) for a set of slightly perturbed inputs. this assumption is known as consistency assumption @xcite @xcite in semi-supervised learning, making distributions sufficiently smooth for the intrinsic structure collectively. 2) modeling conversational behavior: we train a dst model to generate user utterances and system responses, hoping that this auxiliary generation task can capture intrinsic dialogue structure information and benefit the dst performance. this training only needs dialogue transcripts and does not require any further annotation. we hypothesize that modeling this potential correlation between utterances and states is helpful for generalization, making a dst model more usr can you help me find a nightclub in south cambridge? ## background let us define x 1:@xmath0, r 1 ), . . . , (u t , r t )} as the set of user utterance and system response pairs in t turns of a dialogue, and @xmath1, . . . , b t } are the annotated dialogue states. each b t contains a set of (domain, slot, value) tuples accumulated from turn 1 to turn t, therefore, the number of tuples usually grows with turn t. note that it is possible to have multiple domains triggered in the same state b t . a dialogue example and its labeled states are shown in we briefly introduce a common approach for ontology-free dst in the following. as shown in figure 1 , a context encoder encodes dialogue history x 1:t , and a state generator decodes slot values v ij for each (domain, slot) pair {(d i , s j )}, where i denotes the domain index and j is the slot index. the context encoder and the state generator can be either a pre-trained language model or a simple recurrent neural network. during the decoding stage for each v ij , a copy-attention mechanism such as text span extraction @xcite or pointer generator @xcite approach is added to the state generator and strengthen its value generation process. moreover, many ontology-free dst models are also equipped with a slot gate mechanism @xcite @xcite , which is a classifier that predicts whether a (domain, slot) pair is mentioned, not mentioned, or a user does not care about it. in this pipeline setting, they can add additional supervision to their models and ignore the not mentioned pairs' prediction. more specifically, the (domain, slot) pair {(d i , s j )} obtains its context vector c ij to predict a slot gate distribution g ij . the context vector c ij is the weighted-sum of encoder hidden states using the attention distribution a ij , and g ij is a threeway classification distribution mapping from the context vector: such model is usually trained end-to-end with two loss functions, one for slot values generation and the other for slot gate prediction. the overall supervised learning objective from the annotated state labels is ## self-supervised approaches this section introduces how to leverage dialogue history x, which is easy to collect, to boost dst performance without annotated dialogue state labels implicitly. we first show how we preserve latent consistency using stochastic word dropout, and we discuss our design for utterance generation. ## analysis and visualization we would interpret how self-supervised signals help to learn better dst performance. the first interesting observation is that the key improvement comes from the slot-dependent context vectors c ij . if we remove the context vector c ij from eq ( 6) , the performance of 1% labeled data setting drops from 18.31% to 11.07%. the next question is: what do these contextual vectors influence? first, context vectors are the weighted-sum of encoder hidden states, which means they correlate with the learned attention distribution. also, context vectors are used to predict slot gates, which is essential to be able to trigger the state generator. therefore, using self-supervision to align contextual slot vectors may help get better attention distributions and better slot gate prediction. slot gate as shown in we randomly pick up 2,000 dialogue turns on the test set to compute the correlation between latent dialogue history 100% data ; hi ; hello , i am trying to find a train that goes from cambridge to london kings cross . can you help me book a ticket ? ; i can help with that . can you tell me what day you will be traveling ? ; i need to leave on saturday after 18:45 . ; the soonest departure time would be at 19:00 on saturday , is that okay ? ; yes , that s perfect . can you book that for 8 people ? ; you are all booked with reference number 144vdbrm . the cost of 151.04 gbp will be payable at the station . can i be of further assistance today ? ; i am looking for an expensive place to eat in the centre , what is there that fits that criteria ? ; there 33 place -s that fit your criteria . do you have a particular cuisine type in mind so that i can narrow the results down ? ; it does not matter what kind of food . what would you recommend for a large group of 8 people ? ; how about don pasquale pizzeria ? ; that sounds great . please book it for 8 on saturday at 14:15 and get a reference number . ; unfortunately , the restaurant does not have a table for that time . can you do it earlier or later ? ; how about 13:15 ? ; great . that was successful . your reference number is q0ij8u6u . ; thank you , you've been a great help . ; is there anything else that i could help you with today ? ; no thank you , that s all for now ! ; 1% data w/o self-supervision ; hi ; hello , i am trying to find a train that goes from cambridge to london kings cross . can you help me book a ticket ? ; i can help with that . can you tell me what day you will be traveling ? ; i need to leave on saturday after 18:45 . ; the soonest departure time would be at 19:00 on saturday , is that okay ? ; yes , that s perfect . can you book that for 8 people ? ; you are all booked with reference number 144vdbrm . the cost of 151.04 gbp will be payable at the station . can i be of further assistance today ? ; i am looking for an expensive place to eat in the centre , what is there that fits that criteria ? ; there 33 place -s that fit your criteria . do you have a particular cuisine type in mind so that i can narrow the results down ? ; it does not matter what kind of food . what would you recommend for a large group of 8 people ? ; how about don pasquale pizzeria ? ; that sounds great . please book it for 8 on saturday at 14:15 and get a reference number . ; unfortunately , the restaurant does not have a table for that time . can you do it earlier or later ? ; how about 13:15 ? ; great . that was successful . your reference number is q0ij8u6u . ; thank you , you've been a great help . ; is there anything else that i could help you with today ? ; no thank you , that s all for now ! ; 1% data w/ self-supervision ; hi ; hello , i am trying to find a train that goes from cambridge to london kings cross . can you help me book a ticket ? ; i can help with that . can you tell me what day you will be traveling ? ; i need to leave on saturday after 18:45 . ; the soonest departure time would be at 19:00 on saturday , is that okay ? ; yes , that s perfect . can you book that for 8 people ? ; you are all booked with reference number 144vdbrm . the cost of 151.04 gbp will be payable at the station . can i be of further assistance today ? ; i am looking for an expensive place to eat in the centre , what is there that fits that criteria ? ; there 33 place -s that fit your criteria . do you have a particular cuisine type in mind so that i can narrow the results down ? ; it does not matter what kind of food . what would you recommend for a large group of 8 people ? ; how about don pasquale pizzeria ? ; that sounds great . please book it for 8 on saturday at 14:15 and get a reference number . ; unfortunately , the restaurant does not have a table for that time . can you do it earlier or later ? ; how about 13:15 ? ; great . that was successful . your reference number is q0ij8u6u . ; thank you , you've been a great help . ; is there anything else that i could help you with today ? ; no thank you , that s all for now ! ; the darker color means higher attention weight. the 1% labeled data model with self-supervision learns attention distribution more similar to the one using 100% labeled data. learned states (h init ) of 1% labeled data and the true gating status (g) of the (domain, slot) pairs. as shown in figure 3 , the x-axis is the cosine similarity score between two latent dialogue states the model learned, and the y-axis is the cosine similarity score of their true gating status. ideally, when the slot gate status is similar, then the learned representations should also have a high similarity score. we find the model trained with self-supervision (right) has a higher pearson correlation coefficient than the one without (left), increasing from 0.4315 to 0.7035, implying that with self-supervision, models can learn better state representations. copy attention we also visualize the attention distributions of a dialogue history in figure 4 . the darker red color means the higher attention weight and the higher copy probability. we sum attention distributions of a ij for all (domain, slot) pairs and normalize it. the 1% labeled data model with selfsupervision has an attention distribution similar to the one using 100% labeled data. for example, both of them focus on some useful slot information such as "cambridge", "london", "saturday", and "18:45". the results of attention distribution are crucial, especially in our limited labeled setting. the higher the attention weight, the higher the probability that such word will be copied from the dialogue history to the output slot values. more attention visualizations are shown in the appendix. ## conclusion we investigate the potential of using selfsupervised approaches for label-efficient dst in task-oriented dialogue systems. we strengthen latent consistency by augmenting data with stochastic word dropout and label guessing. we model conversational behavior by the next response generation and turn utterance generation tasks. ex-perimental results show that we can significantly boost the joint goal accuracy with limited labeled data by exploiting self-supervision. we conduct comprehensive result analysis to cast light on and stimulate label-efficient dst.