Dialogue State Tracking with Explicit Slot Connection Modeling (2020)

Abstract: Recent proposed approaches have made promising progress in dialogue state tracking (DST). However, in multi-domain scenarios, ellipsis and reference are frequently adopted by users to express values that have been mentioned by slots from other domains. To handle these phenomena, we propose a Dialogue State Tracking with Slot Connections (DST-SC) model to explicitly consider slot correlations across different domains. Given a target slot, the slot connecting mechanism in DST-SC can infer its source slot and copy the source slot value directly, thus significantly reducing the difficulty of learning and reasoning. Experimental results verify the benefits of explicit slot connection modeling, and our model achieves state-of-the-art performance on MultiWOZ 2.0 and MultiWOZ 2.1 datasets.

## introduction task-oriented dialogue systems assist users to achieve their certain goals, such as making a restaurant reservation or booking a taxi. to fulfill users' goals, dialogue state tracking (dst) is employed to estimate dialogue states at each turn. dialogue states consist of constraints and requests conveyed by user utterances, typically are represented by a set of predefined slots and their corresponding values. for instance, the user utterance "i am looking for a korean restaurant in the centre" mentions two slots, food and area, whose values are korean and centre respectively. numerous methods are proposed to tackle the challenge of dst recently, and these methods can be mainly categorized into two types: fixed vocabulary and open vocabulary @xcite . fixed vocabulary models are designed in the paradigm of multi-class classification, relying on a predefined turns target slot source slot u0: i am looking for a korean restaurant in the centre. s0: i have 1 restaurant name little seoul in the expensive price range. restaurant-food restaurant-area . . . --. . . u2: are there any places to go in the same area as the restaurant? s2: there are dozens of places to go in city centre. what type of attraction are you interested in today? attraction-area restaurant-food restaurant-area . . . restaurant-area --. . . u5: i also need a taxi to commute and need it to arrive at the restaurant. s5: i have booked a cab to take you to the restaurant when you leave all saint's church. the booked car type is a yellow volkswagen. taxi-departure taxi-destination attraction-area restaurant-food restaurant-area . . . attraction-name restaurant-name restaurant-area --. . . despite significant improvements achieved by those open vocabulary models, they always suffer from understanding enormous ellipsis and reference expressions in multi-domain scenarios. as shown in in this paper, we formally consider the above challenge as related-slot problem and propose a novel model dst-sc (dialogue state tracking with slot connections) to address it. we follow previous work to build a copy-augmented encoderdecoder model. specially, dst-sc is designed with a slot connecting mechanism to establish the connection between the target slot and its source slot explicitly. thus it can take advantage of the source slot value directly instead of reasoning from preceding turns. the contributions of this work are two-fold: ## model in this section, we will describe dst-sc model in detail. dst-sc is an open vocabulary model based on the encoder-decoder architecture. as shown in figure 1 , there are three components that contribute to obtain the target slot value: (1) word generation from the vocabulary; (2) word copying from the dialogue history; (3) value copying from the source slot. to reduce the burden on the decoder, dst-sc also equips with a slot gate @xcite to predict for slot values of none and dontcare. ## related work traditional approaches for dialogue state tracking @xcite @xcite rely on manually constructed semantic dictionaries to extract features from input text, known as delexicalisation. these methods are vulnerable to linguistic variations and difficult to scale. to overcome these problems, @xcite propose the first data-driven model for dst, the employed deep learning approaches provide stronger representa- ## conclusion in this paper, we highlight a regularly appeared yet rarely discussed problem in multi-domain dst, namely the related-slot problem. we propose a novel dialogue state tracking model dst-sc, which equips with the slot connecting mechanism to build slot connections across domains. our model achieves significant improvements on two public datasets and shows effectiveness on relatedslot problem tests. annotations complement for multiwoz dataset in the future might enable dst-sc to handle the related-slot problem more effectively and further improve the joint accuracy. shuyang gao, abhishek sethi, sanchit agarwal, tagyoung chung, and dilek hakkani-tur. 2019. dialog state tracking: a neural reading comprehension approach. in proceedings of the 20th annual sigdial meeting on discourse and dialogue, pages 264-273, stockholm, sweden. association for computational linguistics. rahul goel, shachi paul, and dilek hakkani-tür. 2019. hyst: a hybrid approach for flexible and accurate dialogue state tracking. arxiv preprint arxiv:1907.00883. kazuma hashimoto, caiming xiong, yoshimasa tsuruoka, and richard socher. 2017. a joint many-task model: growing a neural network for multiple nlp tasks. in proceedings of the 2017 conference on empirical methods in natural language processing, pages 1923-1933, copenhagen, denmark. association for computational linguistics. matthew henderson, blaise thomson, and steve young. 2014a. word-based dialog state tracking with recurrent neural networks. in proceedings of the 15th annual meeting of the special interest group on discourse and dialogue (sigdial), pages 292-299. matthew henderson, blaise thomson, and steve young. 2014b. word-based dialog state tracking with recurrent neural networks. in proceedings of the 15th annual meeting of the special interest group on discourse and dialogue (sigdial), pages 292-299, philadelphia, pa, u.s.a. association for computational linguistics. diederik p. kingma and jimmy ba. 2015. adam: a method for stochastic optimization. in 3rd international conference on learning representations, iclr 2015, san diego, ca, usa, may 7-9, 2015, conference track proceedings. wenqiang lei, xisen jin, min-yen kan, zhaochun ren, xiangnan he, and dawei yin. 2018. sequicity: simplifying task-oriented dialogue systems with single sequence-to-sequence architectures. in proceedings of the 56th annual meeting of the association for computational linguistics (volume 1: long papers), pages 1437-1447, melbourne, australia. association for computational linguistics. nikola mrkšić, diarmuid ó séaghdha, blaise thomson, milica gašić, pei-hao su, david vandyke, tsung-hsien wen, and steve young. 2015. multidomain dialog state tracking using recurrent neural networks. in proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (volume 2: short papers), pages 794-799, beijing, china. association for computational linguistics. nikola mrkšić, diarmuid ó séaghdha, tsung-hsien wen, blaise thomson, and steve young. 2017. neural belief tracker: data-driven dialogue state tracking. in proceedings of the 55th annual meeting of the association for computational linguistics (volume 1: long papers), pages 1777-1788, vancouver, canada. association for computational linguistics. elnaz nouri and ehsan hosseini-asl. 2018. toward scalable neural dialogue state tracking model. corr, abs/1812.00899. jeffrey pennington, richard socher, and christopher manning. 2014. glove: global vectors for word representation. in proceedings of the 2014 conference on empirical methods in natural language processing (emnlp), pages 1532-1543, doha, qatar. association for computational linguistics. liliang ren, jianmo ni, and julian mcauley. 2019. scalable and accurate dialogue state tracking via hierarchical sequence generation. in proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (emnlp-ijcnlp), pages 1876-1885, hong kong, china. association for computational linguistics. liliang ren, kaige xie, lu chen, and kai yu. 2018. towards universal dialogue state tracking. in proceedings of the 2018 conference on empirical methods in natural language processing, pages 2780-2786, brussels, belgium. association for computational linguistics. kai sun, lu chen, su zhu, and kai yu. 2014. the sjtu system for dialog state tracking challenge 2. in proceedings of the 15th annual meeting of the special interest group on discourse and dialogue (sigdial), pages 318-326, philadelphia, pa, u.s.a. association for computational linguistics. chien-sheng wu, andrea madotto, ehsan hosseini-asl, caiming xiong, richard socher, and pascale fung. 2019. transferable multi-domain state generator for task-oriented dialogue systems. in proceedings of the 57th annual meeting of the association for computational linguistics, pages 808-819, florence, italy. association for computational linguistics. puyang xu and qi hu. 2018. an end-to-end approach for handling unknown slot values in dialogue state tracking. in proceedings of the 56th annual meeting of the association for computational linguistics (volume 1: long papers), pages 1448-1457, melbourne, australia. association for computational linguistics. victor zhong, caiming xiong, and richard socher. 2018. global-locally self-attentive encoder for dialogue state tracking. in proceedings of the 56th annual meeting of the association for computational linguistics (volume 1: long papers), pages 1458-1467, melbourne, australia. association for computational linguistics. lukás zilka and filip jurcícek. 2015. incremental lstmbased dialog state tracker. 2015 ieee workshop on