Hi- DST : A Hierarchical Approach for Scalable and Extensible Dialogue State Tracking (2021)

Abstract: Dialogue State Tracking (DST) is a sub-task of task-based dialogue systems where the user intention is tracked through a set of (domain, slot, slot-value) triplets. Existing DST models can be difficult to extend for new datasets with larger domains/slots mainly due to either of the two reasons- i) prediction of domain-slot as a pair, and ii) dependency of model parameters on the number of slots and domains. In this work, we propose to address these issues using a Hierarchical DST (Hi-DST) model. At a given turn, the model first detects a change in domain followed by domain prediction if required. Then it decides suitable action for each slot in the predicted domains and finds their value accordingly. The model parameters of Hi-DST are independent of the number of domains/slots. Due to the hierarchical modeling, it achieves O(|M|+|N|) belief state prediction for a single turn where M and N are the set of unique domains and slots respectively. We argue that the hierarchical structure helps in the model explainability and makes it easily extensible to new datasets. Experiments on the MultiWOZ dataset show that our proposed model achieves comparable joint accuracy performance to state-of-the-art DST models.

## introduction in a goal-oriented or task-oriented dialogue system, dialogue state tracking (dst) refers to the problem of extracting the goal or intention shown by the user at each turn. the user's goals are captured through a set of dialogue states which are the system's internal representation of the ongoing conversation. dst is essential because it not only helps to understand the user's requirement but also impacts the next dialogue generation. in this era of immersive ai, task-based dialogue systems are gaining popularity day by day. as a result, dealing with a large number of domains and slots will soon we now briefly define dst with an illustration shown in fig 1 . let u t and s t be the user and system utterance respectively at turn t. then a task-based conversation is generally expressed as @xmath0, (s 1 , u 1 ), • • • , (s n , u n )}. let belief state b t be the ground-truth dialogue state for turn t. b t represents the set of (domain, slot, slot-value) triplets that have been extracted so far till turn t. the task of dst is to predict b t given the dialogue history till turn t. the solution framework for the dst model can be broadly categorized into three classes -i) picklist-based, ii) generation-based, and iii) endto-end modeling. the first two methods approach the dst problem explicitly, whereas the third class solves it as a part of end-to-end modeling of the task-based dialogue system. picklist-based models @xcite @xcite find the value of a given domain-slot pair from a pre-defined candidate set. this is why these methods need access to the complete ontology of the dataset. this type of modeling can be used only when the candidate set is limited. but in reality, there are many slots (e.g. name, time, etc.) where the range of values can be indefinitely large. generation-based approaches @xcite @xcite solve this problem by generating the slot-value directly from the dialogue history. these methods usually formulate the slot-value prediction as a reading comprehension @xcite or text summarization @xcite task. there are hybrid models @xcite which take the advantages of both picklist and generation-based methods by choosing the slotvalue prediction strategy based on the type of slot. on the other hand, end-to-end models @xcite @xcite aim to unify multiple sub-tasks of a task-oriented dialogue system using a single model. they have the advantage of being fully generative and are usually trained as a conditional or causal language model to generate the next system utterance. although recent progress in generation-based and end-to-end approaches has shown significant performance gain in dst, there are still some scalability and extensibility issues that need to be addressed. these issues mainly occur due to two properties -i) predicting domain and slot as a pair, ii) dependency of model parameters on number domains and slots. all the existing dst solutions hold either of these properties and in most cases both. the first property leads to o(|s|) belief state prediction time for each turn where s is the set of all possible domain-slot pairs in a given dataset. in the worst case, |s| = |m | × |n | where m and n are the sets of unique domains and slots respectively. since task-based chatbots are designed to work in real-time, reducing time complexity is of critical need. @xcite tackles this issue by predicting domain and slot sequentially and thereby reducing the time complexity to o(n ) using their o(1) domain prediction strategy. however, their domain prediction depends on the ordering of domains which can be hard to maintain in a real setup. they also satisfy the second property due to the inclusion of the previous belief state as input. even though this kind of auxiliary feature has been helpful in improving the joint accuracy @xcite , it makes the model difficult to extend. the end-to-end models also possess the second property because they encode the previous belief state along with dialogue history to represent a complete turn @xcite . with the growing popularity of task-based conversational systems, we can anticipate larger datasets with lots of domains and slots to be used in the future for the training and development of such systems. since these datasets will contain a large set of unique domains and slots, scalability and extensibility will become an issue for the existing models. in this paper, we propose a hierarchical dst (hi-dst) model to tackle the issues discussed above. we break the dst task into a hierarchy of four generic sub-tasks -domain change prediction, domain prediction, slot action prediction, and slotvalue prediction. we adopt the triple copy strategy @xcite for slot-value prediction and use the neural span-based question-answering method to extract the slot values from the utterances directly. in contrast to others, we reduce the problem of slot-value prediction to squad @xcite to leverage transfer learning. we keep our model parameters independent of the number of domains/slots. this is why we refrain from using any kind of auxiliary features that depend on the domain/slot set. contributions of our work can be summarized as followsfoot_0 the core idea behind our approach is to decouple the prediction of domain-slot pairs to achieve belief state prediction in o(|m | + |n |) time. we also keep our model free from any kind of dependency on the number of domains and slots to make it easily extensible. we propose hi-dst that comprises of four generic components: domain change prediction (section 2.1), domain prediction (section 2.2), slot-action prediction (section 2.3), and slotvalue prediction (section 2.4). during prediction (section 2.5), we first detect any change in domain. if there is a change in domain predicted, we run domain prediction and update the set of current domain(s) that keeps track of the active domains for a given turn. we next predict the appropriate actions necessary for relevant domain-slot pairs. finally, we extract the slot values using span-based method @xcite when required. we incrementally update our predicted dialogue states at each turn to get the desired belief state. fig. 2 shows the workflow of our proposed approach. ## yes slot-value is "yes" parking is "yes". ## no slot-value is "no" in "i don't need internet or free parking", slot-value for internet and parking is "no". a given domain-slot pair (d, c) can be expressed as @xmath1; glove(d)])) where @xmath2. the rest of the modeling remains the same as the domain prediction model except for the final classification head. the dimension of the final linear layer becomes (2d 2 × k) where k is the number of slot actions. glove embedding of the domains or slots is not updated during training just like our domain prediction model. ## conclusion in this work, we propose hierarchical-dst (hi-dst), a scalable and extensible solution framework for dst. we split the task of dst into four generic modules that not only make hi-dst scalable and extensible for larger datasets but also improve its explainability. hi-dst takes o(|m | + |n |) time belief state prediction per user turn and achieves comparable performance to existing dst models. we discuss the performance trade-off due to the enforcement of scalability and extensibility. as future work, we want to continue our experimentation in zero-shot and few-shot scenarios and investigate the efficiency of hi-dst in complex datasets like sgd @xcite . we would also like to explore the possibility of including additional information or auxiliary features without impacting the desirable properties of hi-dst such as scalability, explainability etc.