“Do you follow me?”: A Survey of Recent Approaches in Dialogue State Tracking (2022)

Abstract: While communicating with a user, a task-oriented dialogue system has to track the user’s needs at each turn according to the conversation history. This process called dialogue state tracking (DST) is crucial because it directly informs the downstream dialogue policy. DST has received a lot of interest in recent years with the text-to-text paradigm emerging as the favored approach. In this review paper, we first present the task and its associated datasets. Then, considering a large number of recent publications, we identify highlights and advances of research in 2021-2022. Although neural approaches have enabled significant progress, we argue that some critical aspects of dialogue systems such as generalizability are still underexplored. To motivate future studies, we propose several research avenues.

## introduction since human conversation is inherently complex and ambiguous, creating an open-domain conversational agent capable of performing arbitrary tasks is an open problem. therefore, the practice has focused on building task-oriented dialogue (tod) systems limited to specific domains such as flight booking. these systems are typically implemented through a modular architecture that provides more control and allows interaction with a database, desirable features for a commercial application. among other components, dialogue state tracking (dst) seeks to update a representation of the user's needs at each turn, taking into account the dialogue history. it is a key component of the system: the downstream dialogue policy uses this representation to predict the next action to be performed (e.g. asking for clarification). a response is then generated based on this action. beyond processing isolated turns, dst must be able to accurately accumulate information during a conversation and adjust its prediction according to the observations to provide a summary of the dialogue so far. several papers have surveyed dst from the perspective of a specific paradigm or period. @xcite give an overview of pomdp-based statistical dialogue systems. these generative approaches model the dialogue in the form of a dynamic bayesian network to track a belief state. they made it possible to account for the uncertainty of the input related to speech recognition errors and offered an alternative to conventional deterministic systems which were expensive to implement and often brittle in operation. however, pomdp-based systems presuppose certain independencies to be tractable and were gradually abandoned in favor of discriminative approaches that directly model the dialogue state distribution. these methods, surveyed by @xcite , rely on annotated dialogue corpora such as those from the dialogue state tracking challenge, the first standardized benchmark for this task @xcite . the past years have been marked by the use of neural models that have allowed significant advances, covered by @xcite . since then, many works dealing with key issues such as adaptability have been published. the first contribution of this paper is a synthesis of recent advances in order to identify the major achievements in the field. the recent development of neural models has addressed fundamental issues in dialogue systems. dst models can now be decoupled from a given domain and shared across similar domains @xcite . they even achieve promising results in zero-shot scenarios @xcite . despite these advances, modern approaches are still limited to a specific scenario as dialogues in most dst datasets consist of filling in a form: the system asks for constraints until it can query a database and returns the results to the user. moreover, these datadriven approaches are rigid and do not correspond to the need for fine-grained control of conversational agents in an application context. in a realworld scenario, access to annotated data is limited but recent dst models appear to have poor generalization capacities @xcite . these limitations show that there remain major challenges to the development of versatile conversational agents that can be adopted by the public. the second contribution of this paper is to propose several research directions to address these challenges. ## dialogue state tracking a successful tod system makes it possible to automate the execution of a given task in interaction with a user. the last few years have seen an increase in research in this field, which goes hand in hand with the growing interest of companies in implementing solutions based on tod systems to reduce their customer support costs. a typical modular architecture for these systems is composed of the following components: natural language understanding (nlu), dst, dialogue policy, and natural language generation (nlg). nlu consists of two main subtasks, namely, intent detection which consists in identifying the user's intent, such as booking a hotel, and slot filling which consists in identifying relevant semantic concepts, such as price and location. dst aims to update the dialogue state, a representation of the user's needs expressed during the conversation. the dialogue policy predicts the system's next action based on the current state. along with dst, it forms the dialogue manager which interfaces with the ontology, a structured representation of the back-end database containing the information needed to perform the task. finally, nlg converts the system action into natural language. in the case of a spoken dialogue system, automatic speech recognition (asr) and speech synthe-sis components are integrated to move from speech to text and back. nlu then operates on the asr hypotheses and is denoted slu for spoken language understanding. 1 traditionally, dst operates on the output of slu to update the dialogue state by processing noise from asr and slu. for example, slu may provide a list of possible semantic representations based on the top asr hypotheses. dst manages all these uncertainties to update the dialogue state. however, recent dst datasets are collected in text format with no consideration for noisy speech inputs, which has caused slot filling and dst to be studied separately. note that such modular architecture is one possible approach to conversational ai and others have been considered. another approach is end-to-end systems which generate a response from the user's utterance using a continuous representation. such systems have been studied extensively for opendomain dialogue @xcite and have also been applied to tod @xcite . ## recent advances in recent years, several important problems have been addressed, notably through the use of pretrained language models (plms) trained on large amounts of unannotated text. this section summarizes the advances in 2021-2022. ## challenges and future directions despite recent advances, there are still many challenges in developing dst models that can accurately capture user needs dynamically and in a variety of scenarios. there are many interesting paths, this section articulates the needs around three axes: generalization, robustness and relevance of models. ## conclusion dialogue state tracking is a crucial component of a conversational agent to identify the user's needs at each turn of the conversation. a growing body of work is addressing this task and we have outlined the latest developments. after giving an overview of the task and the different datasets available, we have categorized modern neural approaches according to the inference of the dialogue state. despite encouraging results on benchmarks such as mul-tiwoz, these systems lack flexibility and robustness, which are critical skills for a dialogue system. in recent years, many works have sought to address these limitations and we have summarized the advances. however, there are still significant challenges to be addressed in the future. there are many interesting avenues and we have proposed three key features of dst models to guide future research: generalizability, robustness and relevance.