Dialogue State Tracking with Multi-Level Fusion of Predicted Dialogue States and Conversations (2021)

Abstract: Most recently proposed approaches in dialogue state tracking (DST) leverage the context and the last dialogue states to track current dialogue states, which are often slot-value pairs. Although the context contains the complete dialogue information, the information is usually indirect and even requires reasoning to obtain. The information in the lastly predicted dialogue states is direct, but when there is a prediction error, the dialogue information from this source will be incomplete or erroneous. In this paper, we propose the Dialogue State Tracking with Multi-Level Fusion of Predicted Dialogue States and Conversations network (FPDSC). This model extracts information of each dialogue turn by modeling interactions among each turn utterance, the corresponding last dialogue states, and dialogue slots. Then the representation of each dialogue turn is aggregated by a hierarchical structure to form the passage information, which is utilized in the current turn of DST. Experimental results validate the effectiveness of the fusion network with 55.03% and 59.07% joint accuracy on MultiWOZ 2.0 and MultiWOZ 2.1 datasets, which reaches the state-of-the-art performance. Furthermore, we conduct the deleted-value and related-slot experiments on MultiWOZ 2.1 to evaluate our model.

## introduction dialogue state tracking (dst) is utilized by the dialogue system to track dialogue-related constraints and user's requests in the dialogue context. traditional dialogue state tracking models combine semantics extracted by language understanding modules to estimate the current dialogue states @xcite @xcite , or to jointly learn speech understanding (henderson et al., 2014; zilka and jurcicek, 2015; wen * corresponding author et al., 2017). they rely on hand-crafted features and complex domain-specific lexicons, which are vulnerable to linguistic variations and difficult to scale. recently proposed approaches attempt to automatically learn features from the dialogue context and the previous dialogue states. most of them utilize only the context @xcite or encode the concatenation of context and dialogue states @xcite or utilize a simple attention mechanism to merge the information from the above two sources @xcite . these methods do not fully exploit the nature of the information in the context and the predicted dialogue states. the information nature of the context is complete and may be indirect. while the nature of the predicted dialogue states is direct and may be incomplete or erroneous. our fpdsc model exploits the interaction among the turn utterance, the corresponding last dialogue states, and dialogue slots at each turn. a fusion gate (the turn-level fusion gate) is trained to balance the keep-proportion of the slot-related information from the turn utterance and the corresponding last dialogue states at each turn. then it applies a hierarchical structure to keep the complete information of all dialogue turns. on top of the model, we employ another fusion gate (the passage-level fusion gate) to strengthen the impact of the last dialogue states. @xcite shows that such strengthening is vital to solve the related-slot problem. the problem is explained in u1: i need a place to dine in the centre. state: restaurant-@xmath0:i recommend the rice house. would you like me to reserve a table? u2:yes, please book me a table for 9. state: restaurant-@xmath1; restaurant-book @xmath2; restaurant-@xmath3:unfortunately, i could not book the rice house for that amount of people. u3: please find another restaurant for that amount of people at that time. state: restaurant-@xmath4; restaurant-book @xmath5: how about tang restaurant ? u4: yes, please make me a reservation. i also need a taxi. state: restaurant-@xmath6; restaurant-book @xmath7; s5: what is your destination ? u5: to the restaurant. state: restaurant-@xmath8; restaurant-book @xmath9; taxi-@xmath10: an example of dialogue contains (1) the deleted-value problem at the 3 rd turn, which changes restaurant-name from rice house to none, and (2) the related-slot phenomenon at the 5 th turn, which carries over the value from restaurant-name to taxi-destination. ## model figure 3 shows the overall structure of fpdsc (dual-level). the followings are important notations for our model. inputs: the context @xmath11, r 1 ), . . . , (u t , r t )} where u t and r t represent utterance for user and system at the t-th dialogue turn; the previous dialogue states @xmath12, . . . , b t-1 } where b @xmath13, v t ), @xmath14, s is slot set, s is one of the slot names, v t is the corresponding slot value at the t-th turn; @xmath15, @xmath16. turn-level information: the slot-related information for each dialogue turn in figure 2 is the turnlevel information. in figure 3 , the turn-level information is denoted as {m s,tl 1 , . . . , m s,tl t-1 , m s,tl t }, which is the fusion (the turn-level fusion gate) result of the slot-utterance attention results {c s l , . . . , c s t-1 , c s t } and the slot-dialogue-states attention results {l s l , . . . , l s t-1 , l s t }. the weights {g s,tl 1 , . . . , g s,tl t-1 , g s,tl t } are from the same fusion gate, which is utilized to allocate the keepproportion from the conversations and previous dialogue states. the turn-level information of a slot is fed to a transformer encoder to form the mutual interaction information {h s,tl t,1 , • • • , h s,tl t,t-1 , h s,tl t,t }. passage-level information: the attention {attention 3} result of the mutual interaction information and a slot is the passage-level information {m s,pl t } of a slot. core feature: the weight {g s,pl t } are applied to balance the turn-level information of the current dialogue turn {m s,tl t } and the passage-level information {m s,pl t } of a slot. we employ the attention {attention 4} mechanism between the turn/passage-level balanced information {f s,pl t } and the last dialogue states {h l t } to strengthen the impact of the last dialogue states. another weight {g s,pl t } (from the passage-level fusion gate) merge the turn/passage-level balanced information {f s,pl t } and the strengthened information {f s,pl t } to form the core feature {f s t }, which is utilized in the downstream tasks. ## conclusion in this paper, we combine the entire history of the predicted dialogue state and the contextual representation of dialogue for dst. we use a hierarchical fusion network to merge the turn/passage-level information. both levels of information is useful to solve the deleted-value and related-slot problem. besides, our models reach state-of-the-art performance on multiwoz 2.0 and multiwoz 2.1. the turn-level fusion gate is sensitive to signal tokens from the current turn utterance. the passagelevel fusion gate is relatively stable. uniform scheduled sampling training is crucial to improve the performance. the entire history of dialogue states helps at extracting information in each dialogue utterance. although error exists in the predicted dialogue states, the errors work as the data noise in the training to enhance the model's robustness. although our approach is based on predefined ontology, the strategy for information extraction is universal. besides, the core feature f s t can be introduced to a decoder to generate the slot state, which suits the open-domain dst.