Dual-Feedback Knowledge Retrieval for Task-Oriented Dialogue Systems (2023)

Abstract: Efficient knowledge retrieval plays a pivotal role in ensuring the success of end-to-end task-oriented dialogue systems by facilitating the selection of relevant information necessary to fulfill user requests. However, current approaches generally integrate knowledge retrieval and response generation, which poses scalability challenges when dealing with extensive knowledge bases. Taking inspiration from open-domain question answering, we propose a retriever-generator architecture that harnesses a retriever to retrieve pertinent knowledge and a generator to generate system responses. Due to the lack of retriever training labels, we propose relying on feedback from the generator as pseudo-labels to train the retriever. To achieve this, we introduce a dual-feedback mechanism that generates both positive and negative feedback based on the output of the generator. Our method demonstrates superior performance in task-oriented dialogue tasks, as evidenced by experimental results on three benchmark datasets.

## introduction task-oriented dialogue (tod) systems @xcite are designed to fulfill specific tasks, such as hotel bookings, through natural language conversations with users. these systems can be integrated into applications such as chatbots and voice assistants, serving various industries like hospitality, e-commerce, and customer service. to generate informative system responses, tod systems typically rely on an external knowledge base (kb) to retrieve relevant entity information. while large language models (llms) like chatgpt have demonstrated impressive capabilities in understanding multi-turn dialogues and generating fluent responses, there are still cases where they require access to localized kbs to handle specific tasks. therefore, knowledge retrieval remains a critical component that necessitates long-term research in dialogue systems. traditional pipeline approaches in tod systems involve multiple modules such as dialogue state tracking and dialogue policy learning, which heavily rely on annotated belief states for knowledge retrieval @xcite . in contrast, end-to-end task-oriented dialogue (e2e-tod) systems aim to generate responses in a single step without the need for intermediate retrieval annotations, thereby highlighting the importance of external knowledge retrieval. existing e2e-tod systems can be classified into three categories based on their utilization of external knowledge. firstly, memory networks are employed to store the knowledge, and multi-hop interactions are designed to aggregate relevant information @xcite @xcite . secondly, pre-trained language models encode linearized kb records, which are then used as input for the response generator @xcite @xcite . thirdly, the knowledge base can be embedded into model parameters through data augmentation, enabling implicit knowledge retrieval @xcite . these approaches typically integrate the processes of knowledge retrieval and response generation and train them under the supervision of reference responses. however, this approach suffers from two notable shortcomings. firstly, the system response often comprises both pure language tokens and kb-related tokens (e.g., hotel names and addresses), making it challenging to train an effective retriever using weak supervision signals from reference responses. secondly, the efficiency of these systems may decrease as the knowledge base expands in size. unlike the aforementioned works, we employ a retriever-generator architecture that explicitly separates the retrieval process from response generation. the retriever is responsible for identifying relevant information from the kb, while the generator utilizes the dialogue context and retrieved information to generate the response. although this architecture seems straightforward, constructing an effective retriever presents significant challenges, particularly in two key aspects. firstly, tod systems inherently possess ground-truth responses to train the generator, but lack annotations for training the retriever. thus, the retriever can only be trained using weak supervision signals derived from the response generator. secondly, within a specific domain, different entities often exhibit structural and content similarities, making it challenging for the generator to learn which entities are truly relevant. consequently, the weak supervision signals from the generator may not be reliable. figure 1 provides a visual illustration of these challenges. to tackle these challenges, we propose a dualfeedback mechanism for the retriever, consisting of positive feedback and negative feedback. positive feedback is constructed based on the conditional generation probabilities of responses corresponding to different retrieved entities. intuitively, if the relevance between an entity and the response is higher, the conditional generation probability of the response corresponding to this entity will also be higher. utilizing positive feedback allows us to train the retriever based on the knowledge learned by the generator from reference responses. in order to prevent the retriever from being misled by inaccurate information, we contend that calibration is necessary. calibration involves the initial identification and resolution of errors, which we accomplish by sampling negative samples derived from the generator's hypothesis responses. then, we construct negative feedback based on these negative samples to calibrate the positive feedback. we conducted evaluations of our system on three well-established tod datasets: multi-woz 2.1 (mwoz) @xcite , stanford multi-domain dataset (smd) @xcite , and camrest @xcite . the experimental results demonstrate that our model outperforms the baseline methods, particularly in scenarios involving large-scale kb. additionally, through extensive analysis, we have made several notable findings. firstly, our retriever exhibits clear advantages over the baseline methods as the size of kb increases. secondly, our dual-feedback mechanism effectively mitigates the problem of incorrect knowledge learned solely with positive feedback from the generator. lastly, our method exhibits good compatibility with llms like chatgpt. secondly, the entire linearized kb is encoded by pre-trained language models and taken as input to a generator to generate the final system response. for instance, unifiedskg @xcite uses a unified text-to-text framework, while q-tod @xcite rewrites the dialogue context into a natural language query for knowledge retrieval. maker @xcite introduces a multi-grained retrival with both entity and attribute selection. thirdly, knowledge bases are stored in model parameters for implicit retrieval during response generation. gpt-ke @xcite embeds the kb into pre-trained model parameters through data augmentation. following gpt-ke, eco @xcite first generates the most relevant entity with trie constraint before response generation to ensure entity consistency in the response. ## methods as illustrated in figure 2 , our system comprises a knowledge retriever and a response generator. the retriever first retrieves top-k relevant entities from a knowledge base. these retrieved entities, along with the dialogue context, are then fed into the generator model to generate the response. ## experiments in this section, we provide a detailed description of the experiment and present the main results. ## analysis and discussion to further investigate our method, we conduct a comprehensive analysis that includes an ablation study, an exploration of different retriever training methods, an examination of different negative sample selecting methods, and an assessment of ## conclusion in this paper, we propose a novel dual-feedback knowledge retriever for e2e-tod systems. our approach separates the knowledge retrieval process from response generation, and leverages the knowledge learned by the generator to create synthetic positive and negative feedback for retriever training, eliminating the need for retrieval annotations. through empirical evaluations, we demonstrate that our system achieves state-of-the-art performance, regardless of whether a small or largescale kb is used in each dialogue. furthermore, ablation studies indicate that our dual-feedback mechanism effectively mitigates the problem of incorrect knowledge learned solely from positive feedback generated by the generator. consequently, this improvement in retrieval performance directly translates to enhanced performance in e2e-tod systems. lastly, our method exhibits good compatibility with llms like chatgpt. yunyi yang, yunhao li, and xiaojun quan. 2021. ubar: towards fully end-to-end task-oriented dialog system with gpt-2. proceedings of the aaai conference on artificial intelligence, 35(16):14230-14238. wen-tau yih, kristina toutanova, john c. platt, and christopher meek. 2011. learning discriminative projections for text similarity measures. in proceedings of the fifteenth conference on computational natural language learning, pages 247-256, portland, oregon, usa. association for computational linguistics.