Value type: the bridge to a better DST model (2023)

Abstract: Value type of the slots can provide lots of useful information for DST tasks. However, it has been ignored in most previous works. In this paper, we propose a new framework for DST task based on these value types. Firstly, we extract the type of token from each turn. Specifically, we divide the slots in the dataset into 9 categories according to the type of slot value, and then train a Ner model to extract the corresponding type-entity from each turn of conversation according to the token. Secondly, we improve the attention mode which is integrated into value type information between the slot and the conversation history to help each slot pay more attention to the turns that contain the same value type. Meanwhile, we introduce a sampling strategy to integrate these types into the attention formula, which decrease the error of Ner model. Finally, we conduct a comprehensive experiment on two multi-domain task-oriented conversation datasets, MultiWOZ 2.1 and MultiWOZ 2.4. The ablation experimental results show that our method is effective on both datasets, which verify the necessity of considering the type of slot value.

## introduction task-oriented dialogue systems have become more and more important as people's demand for life increases(booking flights or restaurants), which have become increasingly important in the field of nlp(nature language process). @xcite @xcite traditionally, the task-oriented dialogue system consists of four modules @xcite : natural language understanding(nlu), dialogue state tracking(dst), dialogue manager(dm) and natural language generation(nlg). this module directly affects the decision-making behavior of the dialogue system, and plays an extremely important ⇤ the first two authors contribute equally. weiran xu is the corresponding author. role in the task-based dialogue system. @xcite the recent methods in dst work are mainly divided into two categories. the first category is based on ontology which means the candidate slot value is assumed to be known eg @xcite @xcite . the second is the way without ontology. these studies have completely abandoned ontology, and they assume that the slot value is unknown. eg @xcite @xcite . however, most of their work is based on dialog state, dialog and slot modeling, ignoring that the value type of each slot may be different. if these slots are modeled uniformly, then there is a lack of a specific feature of each slot. in this work, we propose a new dst framework named svt-dst, which uses the slot-value type as the bridge to increrase the model performance. with this method, each slot has specificity for the attention of the conversation history to better identify the slot value. specifically, we first classify all the slots in the dataset according to their slot value types. as shown in figure 1 , adjectives, time and numbers correspond to pricerange, arrive-time and book-people respectively. we train a sequence annotation model with dialogue training which is used to extarct entities and corresponding entitytypes in each on the turn. we hope that the attention between the dialogue and slots can be higher when the turn is near to current turn with the same slotvalue type. in order to achieve the goal, we use monotonically decreasing functions to integrate the attention weights, which will be described in detail in the method. we use monotonically decreasing functions to integrate these types into the attention operation. our main contributions are as follows: 1) we classify the slot according to the slot-value type, then train the ner model to extract these types to improve the attention formula. 2)we design a sampling strategy to integrate these types into the attention formula, which decrease the error of ner model. 3)we have achieved competitive results on multiwoz 2.1 and 2.4. we analyze the results and point out the future work. ## method figure 2 shows the structure of our dst model, including encoder, attention module and slot value processing module. in this section, we will introduce each module of this method in detail. a t-turn conversation can be expressed as c @xmath0, r 1 ), ..., (r t 1 , u t )}, where r t represents system discourse and u t represents user discourse. we define the dialogue state of the t-th turn as b @xmath1, v t j )| 1 <= @xmath2, where v t j represents the value of the j-th slot s j in the t-th turn. j represents the number of predefined slots. follow @xcite , we express the slot as a "domain slot" pair, such as 'restaurant-price range'. ## conclusion in this paper, we propose an effective method to integrate slot-types into the dst model. specifically, we propose the svt-dst. this framework incorporates the slot-types information into the attention operation to help model pay more attention to these turns that include the type of one slot. further, we design a sampling strategy to integrate these types into the attention formula to decrease the error of ner model. results on multiwoz dataset show that our method has significant improvement on this task.