A Contextual Hierarchical Attention Network with Adaptive Objective for Dialogue State Tracking (2020)

Abstract: Recent studies in dialogue state tracking (DST) leverage historical information to determine states which are generally represented as slot-value pairs. However, most of them have limitations to efficiently exploit relevant context due to the lack of a powerful mechanism for modeling interactions between the slot and the dialogue history. Besides, existing methods usually ignore the slot imbalance problem and treat all slots indiscriminately, which limits the learning of hard slots and eventually hurts overall performance. In this paper, we propose to enhance the DST through employing a contextual hierarchical attention network to not only discern relevant information at both word level and turn level but also learn contextual representations. We further propose an adaptive objective to alleviate the slot imbalance problem by dynamically adjust weights of different slots during training. Experimental results show that our approach reaches 52.68% and 58.55% joint accuracy on MultiWOZ 2.0 and MultiWOZ 2.1 datasets respectively and achieves new state-of-the-art performance with considerable improvements (+1.24% and +5.98%).

## introduction recently, task-oriented dialogue systems have attracted increasing attention in both industry and academia due to their broad application for helping users accomplish tasks through spoken interactions @xcite @xcite . dialogue state tracking (dst) is an essential part of dialogue management in task-oriented dialogue systems. given current utterances and dialogue history, dst aims to determine the set of as ## approach as shown in figure 1 , the proposed model consists of three components: 1) the contextual hierarchical attention network (chan); 2) the state transition prediction module; 3) the adaptive objective. we share all the model parameters for each slot to keep our model universal for all slots. ## conclusion we introduce an effective model that consists of a contextual hierarchical attention network to fully exploit relevant context from dialogue history and an adaptive objective to alleviate the slot imbalance problem in dialogue state tracking. experimental results show that our model achieves state-of-theart performance of 52.68% and 58.55% joint accuracy with considerable improvements (+1.24% and +5.98%) over previous best results on multiwoz 2.0 and multiwoz2.1 datasets, respectively. although our model is based on predefined ontology, it is universal and scalable to unseen domains, slots and values. the main contributions of our model, chan and adaptive objective, can also be applied to open-vocabulary models. we will explore it in the future.