Improving Dialogue State Tracking by Joint Slot Modeling (2021)

Abstract: Dialogue state tracking models play an important role in a task-oriented dialogue system. However, most of them model the slot types conditionally independently given the input. We discover that it may cause the model to be confused by slot types that share the same data type. To mitigate this issue, we propose TripPy-MRF and TripPy-LSTM that models the slots jointly. Our results show that they are able to alleviate the confusion mentioned above, and they push the state-of-the-art on dataset MultiWoz 2.1 from 58.7 to 61.3.

## introduction multi-domain dialogue state tracking (dst) has been an important challenge of conversational artificial intelligence @xcite . compared to single-domain dst, the belief state of a multidomain dst contains slot types across multiple domains. it causes difficulty to the traditional dst models that assume the complete ontology is available because a complete ontology becomes hard to obtain @xcite when there are many domains, to tackle these challenges, many of the mainstream approaches for dst formulate this as a span prediction task @xcite @xcite . instead of predicting a predefined value in the ontology, the span-based dst models predict slot values by extracting a span from the conversation between the user and the system. by directly extracting spans as slot values, the dst models are able to handle unseen slot values and are potentially transferable to different domains. however, the span-based dst models can only deal with the slot values that are explicitly expressed as a sequence of tokens and fall short in dealing with coreference ("i'd like a restaurant in the same area") and implicit choice ("any of those is ok") the current state-of-the-art dst model trippy @xcite is designed to deal with these problems. in addition to extracting values directly from the user utterance, trippy maintains two additional memories on the fly and uses them to deal with the coreference and implicit choice challenges. the design details of trippy will be further illustrated in section 2.2 despite the success of these span-based models, one concern is that they often consider the slots conditionally independently. for example, in trippy, the conversation is first encoded by bert @xcite , and then the extracted features are used by n modules for the n slot types independently. that implies, conditioning the features extracted by bert, the slots are predicted independently. this may increase the difficulty of this task because these slot types are indeed not independent of each other. if the value of one slot type in a domain is present in an utterance, then it is more likely that the value of another slot type of the same domain is present. in particular, in this work, we focus on the cases where modeling the joint probability of the slots could possibly be helpful. we notice that there are some slot types that share the same data type. the data types of both the slot type "hotel-people" and the slot type "restaurant-book people" are both integers. therefore, when an integer is present in an utterance, a dst model needs to predict which slot(s) this integer is for. given the fact that some slots co-occur more often than others, modeling the slot types jointly may be helpful when making this kind of prediction. based on this intuition, we group the slot types into 4 groups by their data type: time, place name, integer. we then analyze the current state-of-the-art model trippy @xcite . our analysis shows that trippy is indeed often confused with slot types in the same group. we then investigate whether modeling the relation between slot types explicitly can alleviate this problem. by modeling the slot types jointly, information from the slots that the model is more confident with may reduce the confusion for the other slots. specifically, to achieve this goal, we propose to model the joint probability of slots with mrf and lstm models. they can be plugged into any dst model that models the slots conditionally independently. in this work, we choose the current state-of-the-art dst model, trippy pretrained with dialoglue @xcite . we experiment on the most widely used multi-domain dst dataset multiwoz 2.1 @xcite . the results show that both the mrf and the lstm models are able to mitigate the confusion, and can improve the performance consistently. we push the state-of-the-art performance on multiwoz 2.1 from 58.7 to 61.3. ## proposed method we conjecture some mistakes made by trippy maybe because it models the slots conditionally independently. based on if c 1 , c 2 , • • • , c n are the correct classes, modeling the joint probability does not require p (c (t) for all s and all incorrect assignments {c s } n @xmath0. instead, to predict the belief state correctly, the model is only required to ensure p (c (t) ## results and discussion our main results are shown in as a more fine-grained analysis, we plot the frequency of incorrect prediction as described in section 3.2. compared to figure 1 , figure 2 and figure 3 show that both trippy+mrf and trippy+lstm can relieve the confusion between slot types that have the same data type. this is coherent to our intuition for modeling the slot types jointly. the results also show that trippy+lstm performs better than trippy+mrf. it may be because trippy+lstm models all the assignment {c s } n @xmath1, while trippy+mrf only models {c s } n @xmath2. also, trippy+lstm has the potential to model the joint distribution across the domains. however, this also sacrifices efficiency. since trippy+lstm predicts the slots sequentially, it cannot predict the slots in parallel. trippy+mrf could be a better choice when latency is an important issue. early approaches for dst rely on a list candidate slot values @xcite @xcite . some recent approaches formulate it as a sequence generation task. trade @xcite and ma-dst @xcite generates slot values with a pointer generator with a copy mechanism. som-dst @xcite ) uses a bert model to encode the dialogue. to improve the efficiency, the state from the previous turn instead of the whole dialogue history is fed into the model. sumbt @xcite utilizes a bert model @xcite to encode both the dialogue history and candidate slot values. in @xcite they formulate dst as a question answering task. ds-dst @xcite adopts a dual strategy: they predict slot values from candidates but also predict slot values by copying from the dialogue. trippy @xcite predicts slot values by copying text span from three different sources. in @xcite , they improve the performance of trippy by pre-training the bert with multiple dialogue tasks. @xcite and soloist @xcite utilize gpt-2 for endto-end slot value and response generation. in @xcite , they investigate the generalization capability of dst models for unseen slot values. mintl @xcite augments data by substituting slot values in the utterances and modifying the syntax structure. ## conclusion in this work, we explore the relation between slot types. our contributions are two-fold. 1) we identify a problem with the current state-of-the-art model trippy. we observe that it is often confused by slots that share the same type of value. 2) we propose trippy+mrf and trippy+lstm that aim at mitigating this problem. one can choose one of them based on the trade-off between efficiency and accuracy. our preliminary results show that both of them are able to alleviate the confusion. our findings in this work lead future researchers in a promising direction to improve the performance of multi-domain dst.