Multi-Domain Dialogue State Tracking By Neural-Retrieval Augmentation (2022)

Abstract: Dialogue State Tracking (DST) is a very complex task that requires precise understanding and information tracking of multi-domain conversations between users and dialogue systems. Many task-oriented dialogue systems use dialogue state tracking technology to infer users’ goals from the history of the conversation. Existing approaches for DST are usually conditioned on previous dialogue states. However, the dependency on previous dialogues makes it very challenging to prevent error propagation to subsequent turns of a dialogue. In this paper, we propose Neural Retrieval Augmentation to alleviate this problem by creating a Neural Index based on dialogue context. Our NRA-DST framework efficiently retrieves dialogue context from the index built using a combination of unstructured dialogue state and structured user/system utterances. We explore a simple pipeline resulting in a retrieval-guided generation approach for training a DST model. Experiments on different retrieval methods for augmentation show that neural retrieval augmentation is the best performing retrieval method for DST. Our evaluations on the large-scale MultiWOZ dataset show that our model outperforms the baseline approaches.

## introduction dialogue state tracking (dst) involves analyzing the user's dialogue and previous turn state expressed during the conversation, extracting the user's goal/intent, and representing it in the form of a well-defined set of slots and values @xcite @xcite . the release of a large-scale multi-domain conversational data set @xcite prompted advances in cross-domain dialogue systems. figure 1 shows an example from the dataset where the user starts the conversation about reserving a hotel, then requests for booking a taxi, and finally, changes the original hotel reservation. the dialogue state here is defined figure 1 : an example from the large-scale multi-domain wizard-of-oz (mutliwoz) dataset where the user is booking a hotel and a train ticket. the dialogue state is represented as @xcite followed by a list @xmath0. one turn refers to a single user utterance and a single system response. the dialogue state is updated based on the previous dialogue state, the current user utterance and the previous oneturn context. as list @xmath1.g., ([hotel] people 2 stay 5 days) , @xcite departure hotel santa)). recent works approach this either by classifying each slot over pre-defined slot-values that are selected from an ontology based on training data @xcite or first classifying a slot and then detecting the span of text in the original context as value for that slot @xcite . however, these models are highly dependant on the values in the dataset and the ontology. another approach to dst is generating the value of a slot or both slot and value using a sequence-to-sequence model @xcite . papers using large pre-trained models such as gpt2 @xcite have shown promising results @xcite . a single generative model can also be used to manage entire dialogue by generating dialogue state, system action, and user response altogether @xcite . but these models are more prone to error propagation as explained below. dialogue state can be considered as a representation of the entire conversation and is used by subsequent modules in resolving system's action and response. error in the dialogue state propagates not only to these other modules but also to dialogue states of subsequent turns. to analyze this issue, we perform a simple analysis similar to @xcite , by replacing the previous dialogue state with ground truth on the state-of-the-art mintl @xcite model. as shown in large pre-trained models have shown to be very efficient in retrieval-based approaches compared to sparse representations based on tf/idf, or bm25 @xcite @xcite . several works in open domain question answering have augmented retrieval-based results for better response generation @xcite . however, this is generally done on natural text such as a question or a passage. in @xcite , the retrieval is done using an unstructured dialogue state, but the index is created only from structured paragraph text data. in this work, we aim to improve dst by leveraging neural retrieval-augmentation on a combination of unstructured dialogue state and structured user/system utterances. the contributions of our work are as follows: ## background in this section, we briefly explain the notations used in further sections. let us denote the dialogue with t turns as, @xmath2, r 1 ), (u 2 , r 2 ), ...(u t , r t )}, where u i represents user utterance at i th turn and r i represents system response at i th turn. over the course of a dialogue, the goal of dst is to keep track of a dialogue state, @xmath3, (s 1 , v 1 ), (s 2 , v 2 ), ..), (d k , (s 1 , v 1 )..)} where d k is the domain, s i is a slot from the domain, d k and v i is the value of s i . the dialogue context at turn t is defined as, c @xmath4, u t-1 , r t-1 , u t ). in this paper, we formulate dialogue context using only the last turn but this can be extended to multiple previous turns. we formulate the original dst task as predicting the dialogue state from the dialogue context, dst @xmath5. the concept of belief span @xcite allows dialogue states to be represented as a span of text, enabling the conversion of a classification problem into a generation problem. lin et al. (2020) builds upon belief spans and defines levenshtein belief span (lev t ) as a minimal editing from previous dialogue state dst t-1 to current dialogue state dst t . for example, dst t-1 ← [restaurant] food french, price cheap, day sunday dst t ← [restaurant] food thai, day sunday, area centre lev t ← dst t -dst t-1 lev @xmath6, price null, area centre we extend the belief spans by creating a neural index and guiding the model with possible levenshtein spans from the retrieved result. the retrieved topk result contains possible dst updates, lev 1..k . the dst task is now updated as predicting the dialogue state from a combination of retrieved results and dialogue context as in eq 1. figure 2 describes the architecture of nra-dst. ## methods given a training dataset d @xmath7, d 2 , .., d m }, we create a neural index, d index such that we can query the index based on neural representation (latent space representation) of dialogue context c t , which is a combination of previous dialogue state dst t-1 and user/system utterances. section 3.1 explains the d index creation method in detail. the contents of the d index can be represented as (e(c t ), lev t ), where the key, e(c t ) is the neural representation of dialogue context and the value, lev t represents the corresponding dialogue state updates. the key idea is that given a dialogue context, we retrieve domains and slots detected in another dialogue with a similar context. figure 2 shows an example of similar contexts, c i and c + i . the previous dialogue states of both contexts contain the slots named ["area" and "stars"], from the domain named ["hotel"] and the utterances are also similar. ## ablation analysis we analyze the influence of different changes on the results with the following experiments. we try to analyze the importance of previous dialogue state information and delexicalization while creating the neural index and conditioning retrieved results at the encoder or the decoder of our dst model. ## conclusions in this work, we demonstrated that neural retrieval augmentation increases the performance of generation-based dst. we explore a simple pipeline resulting in a retrieval-guided generation approach for dst. moreover, our experiments and ablation studies indicate that neural retrieval can efficiently retrieve a combination of unstructured data (dialogue state) and structured data (user/system utterances). as a result, we improve the performance of the baseline approach on a large-scale multi-domain dataset, multiwoz 2.0. in future work, we will investigate the end-to-end training of our nra-dst framework.