Building a Knowledge-Based Dialogue System with Text Infilling (2022)

Abstract: In recent years, generation-based dialogue systems using state-of-the-art (SoTA) transformer-based models have demonstrated impressive performance in simulating human-like conversations. To improve the coherence and knowledge utilization capabilities of dialogue systems, knowledge-based dialogue systems integrate retrieved graph knowledge into transformer-based models. However, knowledge-based dialog systems sometimes generate responses without using the retrieved knowledge. In this work, we propose a method in which the knowledge-based dialogue system can constantly utilize the retrieved knowledge using text infilling . Text infilling is the task of predicting missing spans of a sentence or paragraph. We utilize this text infilling to enable dialog systems to fill incomplete responses with the retrieved knowledge. Our proposed dialogue system has been proven to generate significantly more correct responses than baseline dialogue systems.

## introduction building open-domain dialog systems that generate human-like response is a challenging area for natural language processing. in recent years, generation-based dialogue systems, such as microsoft's dialogpt @xcite ) and google's meena @xcite , have demonstrated impressive performance in simulating human-like conversations. however, when the human asks "what time is it?", the generationbased system will develop a conversation based on the old information contained in the training data. it has been reported that the "illusion problem" generates responses that are not based on the latest facts @xcite . to address this, research on knowledge-based dialogue systems utilizing external knowledge has attracted attention as a dialogue system that can retrieve appropriate external knowledge. alternatively, many knowledge-based dialogue systems @xcite learn to generate target response sentences by inputting retrieved knowledge and dialogue history in a concatenated form to a language model during the learning phase. however, it has been reported that in the inference phase, the response sentences are generated based only on the input dialogue history, despite the input of retrieved knowledge @xcite . in this work, we propose a knowledge-based dialogue system with text infilling, which enables the dialogue system to constantly generate responses that include retrieved knowledge. specifically, the system first inserts blank tokens before and after the retrieved knowledge. the inserted text is the incomplete response. next, the proposed dialogue system takes the incomplete response as input and generates text. finally, it replaces the blank tokens in the incomplete response with this text and outputs the completed response. ## building the dialogue system in this section, we introduce our proposed knowledge-based dialogue system that includes text infilling. we will introduce the training phase and the inference phase of the proposed dialogue system. ## experiments we conducted experiments on the opendialkg dataset @xcite which contains 15,000 dialogues. the dataset was collected in a wizardof-oz setup, by connecting two human participants who were tasked to have an engaging dialogue about a given topic. ## conclusion we proposed a knowledge-based dialogue system based on the text infilling method, aiming to improve the problem that the knowledge-based dialogue system generates responses without using retrieved knowledge. the proposed dialogue system can constantly incorporate external knowledge. in our experiments, the proposed dialogue system generated significantly more correct responses than baseline approaches.