Parallel Interactive Networks for Multi-Domain Dialogue State Generation (2020)

Abstract: The dependencies between system and user utterances in the same turn and across different turns are not fully considered in existing multidomain dialogue state tracking (MDST) models. In this study, we argue that the incorporation of these dependencies is crucial for the design of MDST and propose Parallel Interactive Networks (PIN) to model these dependencies. Specifically, we integrate an interactive encoder to jointly model the in-turn dependencies and cross-turn dependencies. The slot-level context is introduced to extract more expressive features for different slots. And a distributed copy mechanism is utilized to selectively copy words from historical system utterances or historical user utterances. Empirical studies demonstrated the superiority of the proposed PIN model.

## introduction spoken dialogue system (sds) is an application that can help users complete their goals efficiently. an sds usually has a logic engine, called dialogue manager, which involves two main sub-tasks for determining how the system will respond to the users: dialogue state tracking and dialogue policy learning. the task we discuss in this paper is dialogue state tracking, which allows the system maintaining an internal representation of the state of the dialogue as the dialogue progress @xcite . dialogue state tracking involving a single domain has been extensively studied and achieved much progress. as a more challenging task, multidomain dialogue state tracking (mdst) has been introduced in @xcite and attracts figure 1 : the dependencies between the system utterances and user utterances in a multi-domain dialogue. the red lines imply cross-turn dependencies and the blue lines imply in-turn dependencies. several models have been proposed for mdst task and proven to be successful @xcite @xcite @xcite . among these models, trade @xcite achieves the state-of-the-art on the multiwoz 2.0 dataset (one of the standard mdst datasets) by encoding the entire dialogue history using a bidirectional gru and incorporating soft-gated copy mechanism to generate the values. inspired by trade, we purpose to build a more accurate and robust state generator pin. the motivations of proposing pin is in two aspects. one aspect is considering the interactive nature of the dialogues. the interaction of the user and the system is often organized by a question-answering style. it is common in dialogue state tracking that a domain or slot being specified by one of the user or system, then the value being answered by the other. for example, in the dialogue in figure 1 , the user specifies a restaurant domain, and the system answers a restaurant name curry garden. as is shown in figure 1 , there are two types of dependencies, in-turn dependencies and cross-turn dependencies, both contribute to discovering slot-value pairs. it is worth noting that some hard cases involving inference actually rely on cross-turn dependencies (e.g., the dependency between utterance s2 and u3 in figure 1 ). thus a correctly modeling of these dependencies can improve slot-value extraction and cross-turn inference. in this work, we build an interactive encoder which completely accords with the dependencies expressed in figure 1 to jointly model the in-turn dependencies and cross-turn dependencies. the interactive nature of dialogues also implies that the value for a slot tends to be specified frequently either by a system or by a user. for example, the values for slots involving names, such as restaurant-name and hotel-name are likely to be provided by the system. and the values for the slots like hotel-stay (the days to stay) and hotelpeople (the number of people booking for) are usually provided by the user. this observation inspires our designing of the distributed copy mechanism, which allows the state generator choosing to copy words from either the historical system utterances or the historical user utterances. the other aspect is the slot overlapping problem in mdst. unlike single-domain dst, slot overlapping is common in mdst, and these overlapping slots share similar values. for example, both the restaurant and hotel domain have a slot price range that shares the same values. under this condition, a generator without considering slot-specific features may mistakenly extract the value of one slot as the value of some other slot. to overcome the slot overlapping problem, we introduce a slotlevel context in the state generator. in summary, we propose a generation-based mdst model which takes into consideration of the interactive nature of dialogues and slot overlapping problem in mdst. the contributions of this work are as follows. ## problem statement in multi-domain dialogue state tracking, the state is usually expressed as a set of (domain, slot, value) triplets. the domain refers to the topics of the dialogue, such as the restaurant domain, which indicates that the dialogue involves restaurant booking. the slot is an aspect of the user's goals, such as food, area and pricerange in the restaurant-booking dialogues. and the value is the user's specific interests, such as chinese value for food slot that indicates the user is interested in chinese food. the dialogue state is maintained so as to track the progress of the dialogue. at each turn, the system generates a system utterance in natural language, and the user responds to the system with some sentences, referred to as user utterance. the objective of multi-domain dialogue state tracking is to predict the value of each (domain, slot) pair at each turn given the historical system utterances and user utterances. in this paper, the multi-domain dialogue state tracking is treated as a sequence generation task, where each word of a value is generated from a state generator. ## methodology in this section, we introduce the proposed pin model. the model consists of four components: interactive encoder, slot-level context, value generator and slot gate. we next describe each component in detail. ## conclusion this paper studies the problem of state generation for multi-domain dialogues. existing generationbased models fail to model the dialogue dependencies and ignore the slot-overlapping problem in mdst. to overcome the limitation of existing models, we present novel parallel interactive networks (pin) for more accurate and robust dialogue state generation. the design of the pin model is inspired by the interactive nature of the dialogues and the overlapping slots in the ontology. the interactive encoder characterizes the cross-turn dependencies and the in-turn dependencies. the slot-overlapping problem is solved by introducing the slot-level context. furthermore, a distributed copy mechanism is introduced to perform a selective copy from either the historical system utterances or the historical user utterances. empirical studies on two benchmark datasets demonstrate the effectiveness of the pin model.