Multi-Domain Dialogue State Tracking with Top-K Slot Self Attention (2022)

Abstract: As an important component of task-oriented dialogue systems, dialogue state tracking is designed to track the dialogue state through the conversations between users and systems. Multi-domain dialogue state tracking is a challenging task, in which the correlation among different domains and slots needs to consider. Recently, slot self-attention is proposed to provide a data-driven manner to handle it. However, a full-support slot self-attention may involve redundant information interchange. In this paper, we propose a top-k attention-based slot self-attention for multi-domain dialogue state tracking. In the slot self-attention layers, we force each slot to involve information from the other k prominent slots and mask the rest out. The experimental results on two mainstream multi-domain task-oriented dialogue datasets, MultiWOZ 2.0 and MultiWOZ 2.4, present that our proposed approach is effective to improve the performance of multi-domain dialogue state tracking. We also find that the best result is obtained when each slot interchanges information with only a few slots.

## introduction as a crucial component of task-oriented dialogue systems, dialogue state tracking (dst) is designed to track the dialogue states through the conversations between users and systems @xcite , which is generally expressed as a list of {(domain, slot, value)}. in recent years, dialogue state tracking has drawn more and more attention, and numerous methods are proposed @xcite @xcite . despite many progresses have been achieved, these approaches track dialogue states for each slot separately without considering the correlation among slots @xcite @xcite @xcite . spoken language is not formal, in which ellip-user: hi, i'm looking for a cheap restaurant in the centre of the city. sys: nutnut is a steal and popular there sis and cross-reference phenomena make multidomain dialogue state tracking problematic as shown in several researchers have paid attention to modeling the correlations to some certain degrees @xcite @xcite . in these works, the correlation between the slot names is taken into consideration @xcite or a strong prior knowledge is involved, i.e., the similarity coefficient is set to one manually if two slots are regarded to be relevant by human @xcite . but it may overlook the dependencies of some slots with the approach only considering the slot names. to address it, @xcite proposed a slot self-attentive attention extracting slot-specific information for each slot from the dialogue context by utilizing a stacked slot selfattention module to learn the correlations among slots in a fully data-driven way without any human efforts or prior knowledge. however, it may involve some redundant information for some specific slots from other slots and result in incorrect prediction. in this paper, we propose a dialogue state tracking with top-k slot self-attention. here we have a premise of this work: for each slot, not all of the others play a positive role in the value prediction for it. the more redundant information is involved, the worse would be the performance. more specifically, in our work, in the layer where the slots interchange their information, we force each slot to pay attention to the other k slots with the highest scores and mask the rest out rather than considering all of them. we conduct experiments on multiwoz 2.0 and multiwoz 2.4 datasets and present that our proposed model works better than the methods handling the correlations with fully slot self attention. the contributions of this paper are as follows: (1) we propose a top-k attention-based slot selfattention method for multi-domain dialogue state tracking; (2) the experimental results verify the effectiveness of our approach, and we find that the best result is obtained when each slot interchanges information with only a few slots. ## approach figure 1 shows the overview of the proposed model. it consists of a dialogue encoder, slot attention, topk slot self-attention, and slot value matching. ## conclusion in this work, to address the correlation among different slots, we propose multi-domain dialogue state tracking with top-k slot self-attention, in which, each slots is forced to interchange information with the k slots with highest scores than all of them. we conduct experiments on multiwoz 2.0 and multiwoz 2.4 datasets and present that our model works better than existing methods that consider the correlations. the best results can be obtained when each slot interchanges information with only a few other slots.