SAS : Dialogue State Tracking via Slot Attention and Slot Information Sharing (2020)

Abstract: Dialogue state tracker is responsible for inferring user intentions through dialogue history. Previous methods have difficulties in handling dialogues with long interaction context, due to the excessive information. We propose a Dialogue State Tracker with Slot Attention and Slot Information Sharing (SAS) to reduce redundant information’s interference and improve long dialogue context tracking. Specially, we first apply a Slot Attention to learn a set of slot-specific features from the original dialogue and then integrate them using a slot information sharing module. Our model yields a significantly improved performance compared to previous state-of the-art models on the MultiWOZ dataset.

## introduction the recent global adoption of personal assistants such as alexa and siri made dialogue system a more popular topic in research. the major difference between dialogue systems and questionanswering is that dialogue systems need to track dialogue history effectively. so, we normally use a dialogue state tracking component to track user's intention throughout the conversation. a dialogue state is typically composed as a set of slot value pairs in a task-oriented dialogue, such as "hotelinternet-yes". it means the slot "hotel-internet" has a value of "yes". early dialogue state tracking model needs a predefined ontology which means the values of every slot are enumerated in advance @xcite @xcite . such practice is inefficient and costly. the large number of possible slot-value pairs makes deploying these models in the real-life * *corresponding author. moreover, it is a common sense that there is some kind of relevance between two slots involving the same domain or the same attribute. for example, people tend to have a meal near the attraction they visit, so slot "attraction-area" and slot "restaurant-area" have the same value at most times. for these slots with a common or related value, if a slot never or seldom appears in the training set, sharing the learned feature of data-sufficient slot may benefit the model's tracking ability on these rare or unknown slots. so we propose sas, a new multi-domain dialogue state tracking model to resolve this issue to some extent. to be specific, we use an slot attention to localize the key features from the original information-excessive dialogue and a slot infor-mation sharing to improve the models ability to deduce value from related slots. the processed information provided by the slot attention and the sharing module makes the generator more sensitive to the location of the values in the dialogue history and thus generates correct slot values. experiments on the multi-domain multiwoz dataset @xcite shows sas can achieve 51.03% joint goal accuracy and outperform previous state-of-the-art model by 2.41%. on the single domain dataset which only contains the restaurant domain, we achieve 67.34% joint goal accuracy, outperforming prior best by 1.99%. in addition, we conduct an analysis of the experimental results to evaluate the quality of values generated by our model. ## related work the early research of dst focused on the pipelined approach which involves a special module named spoken language understanding (slu) before the dst module @xcite @xcite . but obviously, it was not reasonable to train slu and dst respectively since the accumulated error in slu may be passed to the dst. in order to alleviates this problem, later study focuses on the joint training methods @xcite @xcite . although the higher performance shows the effectiveness of models without slu, there still remains some shortcomings. for example, these models typically rely on semantic dictionaries which list the potential rephrasings for all slots and values in advance. make such a list is costly. fortunately, the recent development of deep learning and representation learning helps the dst to get rid of this problem. @xcite proposed a novel neural belief tracking (nbt) framework which was able to learn distributed representations of dialogue context over pre-trained word vectors, while @xcite described a novel tracking method which used elaborate string matching and coreference resolution to detect values explicitly presented in the utterance. these models greatly improve the performance of dst, but they are not good at handling rare and unknown slot value pairs which seldom or never appear in the training set. there were many efforts to exploit general features between rare slot value pairs and common ones. @xcite proposed glad, a model which built global modules to share parameters between estimators for different slots and local modules to learn slot-specific features. @xcite improved glad by reducing the latency in training and inference time, while preserving its powerful performance of state tracking. but as the dialogues become increasingly complex, the performance of these models on multi-domain is not as satisfying as on single domain. because of the dependency on the dialogue ontology, they have difficulty in scaling up with domains. once the number of domains increases, the amount of slot value pairs will boom. with the copy mechanism, the sequence-to-sequence model trade @xcite successfully got rid of any predefined slot value pairs and generated dialogue states from conversation utterances. but we find there still remain several crucial limitations which have not been well solved on multidomain dialogues. first, these models rely on the long dialogue history to identify the values which belong to various domains and slots. sometimes the information contained in the dialogue history is too rich for these models to efficiently utilize and the redundant information tends to interfere with their value identification or value generation. second, the related information among similar slots is wasted. to alleviate these problems, a slot attention and a slot information sharing module are suggested. the former can isolate the most valuable information for each slot, while the latter integrates information kept by its all similar slots and improve the models ability to deduce value from related slots. ## task definition the dialogue state tracking models take the interaction context as input and extract slot value pairs explicitly or implicitly presented in conversations. the combinations of these slot value pairs are the representations of the user's goal. in this paper, we denote @xmath0, r 1 ), • • • , (u t , r t )} as the dialogue history, where u 1 , • • • , u t and r 1 , • • • , r t are respectively the set of user utterances and the set of system responses in t turns. the dialogue state of turn t is marked as st @xmath1: s j , value: y value j ). here, s j indicates the j-th slot, while y value j means the ground turth value sequence for this slot. all the slots in ontology are obtained by preprocessing the original multiwoz dataset with the delexicalization. moreover, we extend the definition of the slot to include the domain name for convenience. for instance, a slot in this paper will be "hotel-star", rather than "star". our primary goal is to learn a generative dialogue state tracker model m : x × @xmath2. and unlike most of the previous models, the ontology o mentioned in this paper only contains the predefined slots and excludes their values. ## our proposed model figure 1 shows the architecture of sas. sas is a sequence-to-sequence model augmented with slot attention and slot information sharing. slot attention enables better feature representation and slot information sharing helps understanding less-seen slots. we describe the details of every component in sas as follows: ## experiments in this section, we first introduce the dataset and the evaluation metrics. we then describe our model's implementation details. finally, we show our baseline models. ## results in this section, we first show the result of our model on multiwoz dataset, then on multi-woz(restaurant) and multiwoz (except hotel) dataset. after conducting the ablation experiment, we also display the improvement the slot attention and slot information sharing brings. our model achieves the best performance in the most important metric, joint goal accuracy. our model outperformed the previous state-of-the-art model, trade by 2.41% absolute score on joint goal accuracy. we only observe slight increase of slot accuracy compared to trade. we suspect it is because trade was already achieving nearly 97% accuracy, which is close to the up-bound of the slot accuracy in this task. after carefully checking the error cases, we found these errors mainly come from the difficulty of generating name phrases. to test sas's ability on single domain dialogue tasks, we also evaluate our model on the a subset of multiwoz which contains only the restaurant search task. as displayed in to investigate whether the slot similarity matrices used by the two sharing methods really reflect the similarity among slots, we also compare them with a human constructed similarity matrix. we invite three volunteers to carefully rate (1 or 0) the relationship between every two slots and obtain the slot similarity matrix used in the human evaluated method. as shown in ## error analysis to better understand why our model improves the performance, we investigated some dialogue examples and shown them in in the first dialogue, by asking "could you also find me a hotel with a moderate price that offers internet?", the user has briefly informed the agent that he/she is looking for a hotel "with internet". the previous model missed the "hotel-internet" in the tracked slots. because the model is mislead by the long interaction history. our model learns to focus on important information using the slot attention to track the correct internet slot. in the second dialogue, although the previous model manages to capture the value "21:30". it still confused "arriveby" with "leaveat". while sas can distinguish them. we suspect this is because our model can learn the differences between these slots by training on isolated key features per slot without seeing any irrelevant information. in the third example, the user agrees to visit an attraction named "christ's college" from many college-type choices the agent suggests. previous model fetches a wrong message and fills the slot "attraction-name" with "clare college". in contrast, sas captures the correct attraction name and also deduces that the attraction type is college. similar to the first dialogue, the slot attention helps model gain more clean information to detect slot values more accurately. and by sharing the information fetched from slot "attraction-name" with the slot "attraction-type", our model is more sensitive with the value "college". we also investigate the limitation of our model by analyzing the state tracking errors. we noticed two types of errors. first, sas can not effectively identify value "dontcare" for most slots. for example, when the agent asks the user about his/her requirement on the hotel rating, though he/she answers "that is not really important for me", the model fails to fill "dontcare" into the slot "hotelstar". we believe this is due to the fact that the meaning of "dontcare" has plenty of expressions, it is much harder for the model to learn the semantic mation passed to each slot by integrating the information from similar slots. the sharing allows sas to generalize on rare slot-value pairs with few training data. our model reaches the state-of-the-art performance compared with previous models. we believe that sas provides promising potential extensions, such as adapting our model on other tasks where are troubled by excessive information. besides that, we also notice that it is hard for sas to correctly extract names of hotel or attraction which have rich variations. designing a new model to address these problems may be our future work.