DKAF : KB Arbitration for Learning Task-Oriented Dialog Systems with Dialog- KB Inconsistencies (2023)

Abstract: Task-oriented dialog (TOD) agents often ground their responses on external knowledge bases (KBs). These KBs can be dynamic and may be updated frequently. Existing approaches for learning TOD agents assume the KB snapshot contemporary to each individual dialog is available during training. However, in real-world scenarios, only the latest KB snapshot is available during training and as a result, the train dialogs may contain facts conflicting with the latest KB. These dialog-KB inconsistencies in the training data may potentially confuse the TOD agent learning algorithm. In this work, we define the novel problem of learning a TOD agent with dialog-KB inconsistencies in the training data. We propose a Dialog-KB Arbitration Framework (DKAF) which reduces the dialog-KB inconsistencies by predicting the contemporary KB snapshot for each train dialog. These predicted KB snapshots are then used for training downstream TOD agents. As there are no existing datasets with dialog-KB inconsistencies, we systematically introduce inconsistencies in two publicly available dialog datasets. We show that TOD agents trained with DKAF perform better than existing baselines on both these datasets.

## introduction a task-oriented dialog (tod) system often requires information from a knowledge base (kb) to complete user goals like restaurant reservations, flight bookings, and calendar enquiry. this paper follows the recent line of research in end-to-end approaches @xcite @xcite , where dialog agents are trained using just the training dialogs and an associated kb, without any expensive dialog state annotation. the kb contents typically change to reflect the transactions that happened during the user-agent dialogs. for example, in figure 1 , the kb snapshot k 1 can transform into k 2 when la margherita and prezzo become unavailable due to reservations, and bangkok city becomes available due to a cancellation. due to this evolving nature of the kb, two dialogs which started with the same user goal can result in two different outcomes. for example, consider the dialogs d 1 and d 2 in figure 1 . in d 1 , the agent makes two recommendations from k 1 , whereas, in d 2 , no recommendation is feasible as k 2 has no restaurants that fit the user's need. existing approaches for learning tod agents assume the kb snapshot contemporary to each dialog is available during training. such an assumption is limiting due to two reasons. first, kb snapshots are usually created at periodic intervals not after each kb transaction due to storage constraints. second, dialogs used for training tod models are often collected from messaging applications where human agents and users interact. human agents often access the associated kb using a different application and so the kb queries fired during the dialog do not get logged with the dialogs @xcite . without these kb query logs, it is difficult to reconstruct the contemporary kb. as the contemporary kb snapshots are unavailable, a single kb snapshot (generally, the latest) is made available during training. when the latest kb snapshot gets associated with the train dialogs, the dialogs and the kb may portray diverging information resulting in dialog-kb inconsistencies. in the running example, k t denotes the latest kb snapshot. dialog d 1 disagrees with k t , as la margherita is missing from k t . dialog d 2 also disagrees with k t , since k t contains an italian restaurant, contradicting agent response. dialog-kb inconsistencies hinder the learning of tod agents. these inconsistencies can force the tod agent to either learn spurious patterns (e.g., using d 2 and k t may force the agent to ignore prezzo) or memorizes responses (using d 1 and k t , will force the agent to generate la margherita) leading to poor generalization. to overcome these challenges, we define the novel problem of endto-end learning of tod systems with dialog-kb inconsistencies in training data. we also propose dkaf, whose goal is to reduce the dialog-kb inconsistencies by predicting the contemporary kb for each dialog in the training corpus. these predicted kb snapshots and the associated dialogs can then be used to train any existing end-to-end tod learning approaches. given a dialog, inconsistencies can be removed by inserting a new row in the kb based on the entities and relationships present in the dialog (e.g., adding la margherita to k t can make d 1 consistent with k t ). inconsistencies can also be removed by deleting rows (e.g., removing prezzo from k t can make d 2 consistent). as dialogs offer weak supervision to reduce dialog-kb inconsistencies, we use distant supervision and reinforcement learning to train dkaf. we construct two datasets by systematically infusing dialog-kb inconsistencies on babi @xcite , and bitod (english) @xcite datasets and refer to them as inc-babi and inc-bitod respectively. our experiments show that dkaf reduces the dialog-kb inconsistencies and the overall tod system trained with the kb predicted by dkaf outperforms existing state-of-the-art models on both the datasets. in summary, 1. we introduce the novel problem of training task-oriented dialog systems over data with dialog-kb inconsistencies. we release all resources for future research foot_0 . ## related work traditionally, dialog systems are modular @xcite @xcite with different modules for natural language understanding, dialog state tracking, and natural language generation. these models require hand-crafting of dialog states and require expensive intermediate annotations for training each component. on the other hand, end-to-end tod models @xcite @xcite @xcite ) that directly predict system response given dialog history and the kb are becoming increasingly popular as they alleviate the need for expensive annotations. dkaf approach proposed in this work focuses on learning end-to-end tod system when training data has dialog-kb inconsistencies. recent works on inconsistency in dialog generation by @xcite ; @xcite study problem of detecting inconsistent dialog responses with respect to dialog history, user intent, the kb. @xcite explores a similar problem but in domain of persona-based dialog systems. @xcite studies the topology of annotation inconsistencies in crowd-sourced data for slot-filling models. dkaf differs from these works in two key ways: (1) its objective is learning a tod model when training data includes dialogs inconsistent with the kb and, (2) it explicitly resolves dialog-kb inconsistencies via a novel kb arbitration procedure. ## problem definition we first describe the task of learning an end-to-end tod system. we denote a dialog between user u and agent a as we now consider the setting where training dialogs are grounded in an evolving kb. here, a training dialog d j is consistent with its contemporary kb snapshot, k j . however, at training time, a single kb snapshot k t is available which gets associated with all training dialogs resulting in dialog-kb inconsistencies. so, we propose the task of learning end-to-end tod system using {d j , k t } n @xmath0. ## dkaf to solve dialog-kb inconsistencies, we propose dkaf that updates k t based on d j such that the resultant kb snapshot kj resembles with k j . a tod system is then trained using {d j , kj } n @xmath1. dkaf's updates to k t happen through a cascade of three models -row insertion, row deletion, and row completion. each model takes the kbs resulting from the preceding model and performs modifications to them based on the training dialogs. figure 2 highlights this process. we now describe each model in detail. ## results we answer the following research questions in our experiments: ## conclusions we define the novel task of end-to-end training of task-oriented dialog agents, when training data may have inconsistencies between dialog and accompanying kb. this scenario arises, when kb evolves over time, but only one final kb is attached with the data, instead of saving kb snapshots associated with each training dialog. we also contribute two datasets, curated by systematically modifying babi and bitod datasets, for our task. existing state-of-the-art tod models, when trained on our datasets, can get quite confused. our proposed solution, dkaf, hypothesizes corrections to kb for each dialog so that the kb becomes dialog-consistent. since no explicit annotation is available, the modules for kb correction are trained via distant supervision and reinforcement learning. when trained on such corrected data, dkaf-based tod models outperform vanilla tod models in almost all settings. we release our code and data for further research on the topic.