Dialogue State Tracking with Incremental Reasoning (2021)

Abstract: Tracking dialogue states to better interpret user goals and feed downstream policy learning is a bottleneck in dialogue management. Common practice has been to treat it as a problem of classifying dialogue content into a set of pre-defined slot-value pairs, or generating values for different slots given the dialogue history. Both have limitations on considering dependencies that occur on dialogues, and are lacking of reasoning capabilities. This paper proposes to track dialogue states gradually with reasoning over dialogue turns with the help of the back-end data. Empirical results demonstrate that our method outperforms the state-of-the-art methods in terms of joint belief accuracy for MultiWOZ 2.1, a large-scale human–human dialogue dataset across multiple domains.

## introduction dialogue state tracking (dst) usually works as a core component to monitor the user's intentional states (or belief states) and is crucial for appropriate dialogue management. a state in dst typically consists of a set of dialogue acts and slot value pairs. consider the task of restaurant reservation as shown in figure 1 . in each turn, the user may inform the agent of particular goals (e.g. single one as inform(@xmath0,@xmath1. such goals given during a turn are referred as turn belief. the joint belief is the set of accumulated turn goals updated until the current turn, which summarizes the information needed to successfully maintain and finish the dialogue. traditionally, dialogue system is supported by a domain ontology, which defines a collection of slots and the values that each slot can take. the aim of dst is to identify good features or patterns, and map to entries such as specific slotvalue pairs in the ontology. it is often treated as a classification problem. therefore, most efforts center on (1) finding salient features: from handcrafted features @xcite , semantic dictionaries @xcite , to neural network extracted features @xcite ; or (2) investigating effective mappings: from rule-based models @xcite , generative models @xcite , to discriminative ones @xcite @xcite . on the other hand, some researchers attack these methods' over-dependence on domain ontology. they perform dst in the absence of a comprehensive domain ontology and handle unknown slot values by generating words from dialogue history or knowledge source @xcite @xcite . however, the critical problem of modeling the dependencies and reasoning over dialogue history is not well researched. many existing methods work on turn level only, which takes in the current turn utterance and outputs the corresponding turn belief @xcite @xcite . compared to joint belief, the resulting turn belief only reflects single turn information, and thus is of less practical use. therefore, more recent efforts target at the joint belief that summarizes the dialogue history. generally speaking, they accumulate turn beliefs by rules ( @xcite ; @xcite or model information across turns via various recurrent neural networks (rnns) @xcite . although these rnn based methods model dialogue in turn by turn style, they usually feed the whole turn utterance directly to the rnn, which contains a large portion of noise, and result in unsatisfactory performance @xcite . more recently, there are works that directly merge fixed window of past turns @xcite as new input and achieve state-of-the-art performance @xcite . nonetheless, their capability of modeling long-range dependencies and doing reasoning in the interactive dialogue process is rather limited. for example, @xcite performs gated copy to generate slot values from dialogue history. although certain turns of utterances are exposed to the model, since the interactive signals are lost when concatenating turns together, it fails to do in-depth reasoning over turns. very recently, there is research starting to work in turn-by-turn style with pre-trained models. generally speaking, such methods take the previous turn's belief state and the current turn utterances as input to generate new dialogue state @xcite @xcite . however, there exists a long ignored fact that as an agent's central component, the state tracker not only receives dialogue history but also observes the back-end database or knowledge base. such an information source provides valuable hints for it to reason about user goals and update belief states. it is therefore natural to construct a bipartite graph based on the database where the entities and entity attributes are the two groups of nodes; with edges connecting them to express attribute belonging relation. as the example in figure 1 , the database does not contain restaurant entity serving jamaican food and located in center area. thus there would be no two-hop path between these two nodes. existing methods like @xcite have to understand it via system utterances, while a dst reasoning over database would easily obtain such clues explicitly. in this paper, we propose to do reasoning over turns and reasoning over database in dialogue state tracking (redst) for task-oriented systems. for reasoning over turns, we model dialogue state tracking as a recursive process in which the current joint belief relies on the generated current turn belief and last joint belief. motivated by the limited length of single turn utterance and the good performance of pre-trained bert @xcite , we formalize the turn belief prediction as a token and sequence classification problem. it follows a multitask learning setting with augmented utterance inputs. to integrate the last turn belief results, an incremental inference module is applied for more robust belief updates. for reasoning over a database, we abstract the back-end database as a bipartite graph, and propagate extracted beliefs over the graph to obtain more realistic dialogue states. contributions are summarized as: ## redst model the proposed redst model in figure 2 consists of three components: a turn belief generator, a bipartite graph belief propagator, and an incremental belief generator. instead of predicting the joint belief directly from dialogue history, we perform two-stage inference: it first obtains turn belief from augmented turn utterance via transformer models. then, it reasons over turn belief and last joint belief with the help of the bipartite graph propagation results. based on this, it incrementally infers the final joint belief. to facilitate the model description in detail, we first introduce our mathematical notations here. we define @xmath2, r 1 ), • • • (u t , r t )} as the set of user utterance and system response pairs in t turns of dialogue, and @xmath3, • • • , b t } as the joint belief states at each turn. while b t summarizes the dialogue history up to the current turn t, we also model the turn belief q t that corresponds to the belief state of a specific turn (u t , r t ), and denote d t as the domain of this specific turn. following @xcite , we design our state tracker to handle multiple tasks. thus, each b t or q t consists of tuples like (domain, slot, value). suppose there are k different (domain, slot) pairs in total, we denote y k as the true slot value for the k-th (domain, slot) pair. ## conclusion we rethink dst from the angle of agent and point out the urgent need for in-depth reasoning other than being obsessed with generating values from history text as a whole. we demonstrated the importance of doing reasoning over turns and over the database. in detail, we fine-tuned pre-trained bert for more accurate turn level belief generation while doing belief propagation in bipartite graph to harvest more clues. experiments on a large-scale multi-domain dataset demonstrate the superior performance of the proposed method. in the future, we will explore more advanced algorithms for performing reasoning over turns and on graphs for generating more accurate summarization of user intention.