Comprehensive Study: How the Context Information of Different Granularity Affects Dialogue State Tracking? (2021)

Abstract: Dialogue state tracking (DST) plays a key role in task-oriented dialogue systems to monitor the user’s goal. In general, there are two strategies to track a dialogue state: predicting it from scratch and updating it from previous state. The scratch-based strategy obtains each slot value by inquiring all the dialogue history, and the previous-based strategy relies on the current turn dialogue to update the previous dialogue state. However, it is hard for the scratch-based strategy to correctly track short-dependency dialogue state because of noise; meanwhile, the previous-based strategy is not very useful for long-dependency dialogue state tracking. Obviously, it plays different roles for the context information of different granularity to track different kinds of dialogue states. Thus, in this paper, we will study and discuss how the context information of different granularity affects dialogue state tracking. First, we explore how greatly different granularities affect dialogue state tracking. Then, we further discuss how to combine multiple granularities for dialogue state tracking. Finally, we apply the findings about context granularity to few-shot learning scenario. Besides, we have publicly released all codes.

## introduction currently, task-oriented dialogue systems have attracted great attention in academia and industry @xcite , which aim to assist the user to complete certain tasks, such as buying products, booking a restaurant, etc. as a key component of task-oriented dialogue system, dialogue state tracking plays a important role in understanding the natural language given by the user and expressing it as a certain dialogue state @xcite @xcite . the dialogue in general, there are two strategies to track a dialogue state: predicting it from scratch and updating it from previous state. the scratch-based strategy obtains each slot value in dialogue state by inquiring all the dialogue history @xcite @xcite @xcite @xcite , the advantage of this strategy is to ensure the integrity of the dialogue information. the previous-based strategy relies on the current turn dialogue to update the previous dialogue state @xcite @xcite @xcite , the main character of this strategy is to greatly improve the efficiency of dialogue state prediction and avoid the computational cost of encoding all dialogue history. however, both kinds of strategies above have great defects because of their own characters. for the scratch-based strategy, it is hard to correctly track short-dependency dialogue state because of the noise associated with encoding all dialogue history. for example, the dialogue history of turn 1 to 3 in figure 1 (a) does not contribute to the prediction of slot values in the restaurant domain. for the previous-based strategy, it is difficult to solve the problem of long-dependency dialogue state tracking because it utilizes only limited dialogue information from the current turn dialogue and the previous state. as in figure 1 (b), the slot taxi -departure cannot be predicted due to the absence of corresponding dialogue history content. obviously, it plays different roles for the context information of different granularity to track different kinds of dialogue states. intuitively, less context information is needed for short-dependency dialogue state, while more context information must be taken into account for long-dependency dialogue state tracking. for example, the dialogue state in figure 1 (c ) is tracked from turn 2, which utilizes context information of granularity 4 (turn 3 to 6), providing evidence for the prediction of all slots while bringing as little noise as possible. thus, in this paper, we will study and discuss how the context information of different granularity affects dialogue state tracking. the contribution of this paper is that it is, to the best of our knowledge, the first detailed investigation of the impact of context granularity in dialogue state tracking and promotes the research on dialogue state tracking strategy. our investigation mainly focuses on three points foot_0 : the rest of paper is organized as follows: the relevant definitions and formulas in the dialogue state tracking strategy are introduced in section 2. section 3 lists the detailed experimental settings. section 4 presents the survey report and results, followed by conclusions in section 5. ## preliminary to describe the dialogue state tracking strategy, let's introduce the formula definitions used in this paper: dialogue content: @xmath0, t 2 , ..., t n ) is defined as the dialogue of length n , where t @xmath1, u i ) is the dialogue content of i-th turn, which includes the system utterance s i and the user utterance u i . ## experimental settings in order to investigate how the context information of different granularity affects dialogue state tracking, we analyze the performance of several different types of dialogue state tracking models on different datasets. for a clearer illustration, the detailed settings are introduced in this section. ## experimental analysis this section presents our detailed investigation of how the context information of different granularity affects dialogue state tracking, focusing on the impact of granularity on dialogue state tracking, the combination of multiple granularities, and the application of context granularity in few-shot learning scenario. for simplicity, in all experimental results, the maximum granularity is expressed as 0, the maximum granularity minus 1 is expressed as -1, and so on. ## conclusion in the paper, we analyze the defects of two existing traditional dialogue state tracking strategies when dealing with context of different granularity and make a comprehensive study on how the context information of different granularity affects dialogue state tracking. extensive experimental results and analysis show that: (1) different granularities have their own advantages in different situations of dialogue state tracking; (2) the multi-granularity combination can effectively improve the dialogue state tracking; (3) the application of multi-granularity combination in few-shot learning can bring significant effects. in future work, dynamic context granularity can be used in training and inference to further improve dialogue state tracking.