Generation and Extraction Combined Dialogue State Tracking with Hierarchical Ontology Integration (2021)

Abstract: Recently, the focus of dialogue state tracking has expanded from single domain to multiple domains. The task is characterized by the shared slots between domains. As the scenario gets more complex, the out-of-vocabulary problem also becomes severer. Current models are not satisfactory for solving the challenges of ontology integration between domains and out-of-vocabulary problems. To address the problem, we explore the hierarchical semantic of ontology and enhance the interrelation between slots with masked hierarchical attention. In state value decoding stage, we solve the out-of-vocabulary problem by combining generation method and extraction method together. We evaluate the performance of our model on two representative datasets, MultiWOZ in English and CrossWOZ in Chinese. The results show that our model yields a significant performance gain over current state-of-the-art state tracking model and it is more robust to out-of-vocabulary problem compared with other methods.

## introduction dialogue state tracking (dst) is in charge of updating the belief state in task-oriented dialogue system @xcite . traditional discriminative dst models assume that the task ontology is well defined in advance, that is to say, all states and their values are known to the model. they usually rely on hand-crafted features or taskspecific lexicon @xcite ). an inconvenience is that they are time-consuming and hard to expand to new tasks. to overcome it, the open vocabulary-based models are proposed to decode the state value according to the dialogue context @xcite @xcite . in recent years, the research frontier for taskoriented dialogue systems has expanded from single domain to multiple domains (budzianowski et al., 2018; @xcite . there come new challenges demanding prompt solution. firstly, current model does not sufficiently consider the interrelation between slots in multi-domain scenario. for example, user asks "i also want to find an attraction near the restaurant", which implies that the hotel need to have the same area with the restaurant. the implicit relation between the area slot of hotel and restaurant is the key to exactly track user's intent @xcite . prior work simply used the summed @xcite or concatenated @xcite embedding of the domain and slot as the states representation for the decoder. secondly, out-of-vocabulary (oov) problem gets more severe since the user asking question with wider entities and more diverse words. in this paper, we propose the generation and extraction combined method with hierarchical ontology integration, named geex, for dialogue state tracking. first, we explore the hierarchical semantics of the ontology to enhance the representation of slots in multiple domains. inspired by @xcite , we adopt the directed acyclic graph to represent the ontology and enhance the slots interaction between domains with masked hierarchical attention. we use the ontology of multiwoz @xcite to illustrate this mechanism. as shown in figure 1 , the ontology has four hierarchies, i.e., domain, intent, slot and value. the states can be expressed as the combination of domain, intent and slot and the goal of dst is to decode the value for the state mentioned in the dialogue context. the hierarchically represented ontology is efficient and effective in two aspects. first, it enhances the interrelation between slots in multiple domains. second, the compact structure is efficient for state representation which is appropriate to domain expansion since new domain often shares slots with old one @xcite . to address the oov problem, we leverage generation and extraction by combining the two methods together. we first predict the state operation policy to select the suitable decoding strategy. then, we enter into the corresponding decoder for value decoding according to the predicted policy. the contributions of this paper are summarized as follows: (i) we adopt the masked hierarchical attention to represent the ontology to enhance the slots interrelation between domains. (ii) we combine generation and extraction to handle oov problem in dialogue state tracking. (iii) experiment results demonstrate that geex outperforms state-of-the-art baseline on two representative datasets. furthermore, geex also shows robustness in oov testing. ## architecture we use a four-stage model for state tracking, figure 2 illustrates the architecture. ## conclusion in the paper, we explore the hierarchical structure of ontology and combine generation and extraction together for state value decoding. with the domain expanding, supervised learning is not satisfactory for rapidly increasing requirements. in future work, few-shot learning and knowledge fusion can be applied to further improve domain transferring performance.