PLATO - KAG : Unsupervised Knowledge-Grounded Conversation via Joint Modeling (2021)

Abstract: Large-scale conversation models are turning to leveraging external knowledge to improve the factual accuracy in response generation. Considering the infeasibility to annotate the external knowledge for large-scale dialogue corpora, it is desirable to learn the knowledge selection and response generation in an unsupervised manner. In this paper, we propose PLATO-KAG (Knowledge-Augmented Generation), an unsupervised learning approach for end-to-end knowledge-grounded conversation modeling. For each dialogue context, the top-k relevant knowledge elements are selected and then employed in knowledge-grounded response generation. The two components of knowledge selection and response generation are optimized jointly and effectively under a balanced objective. Experimental results on two publicly available datasets validate the superiority of PLATO-KAG.

## introduction recently, the capability of large-scale pre-trained models has been verified in open-domain dialogue generation, including meena @xcite , blender @xcite , and plato-2 @xcite . without introducing explicit knowledge in learning process, substantive knowledge is implicitly embedded into parameters from the training corpus. however, these models are found to suffer from knowledge hallucinations @xcite , producing plausible statements with factual errors. to boost the generation accuracy, there is a trend to leverage external knowledge in addition to the parameters of large-scale pre-trained models @xcite . in knowledge-grounded conversation, several datasets have been collected through crowdsourcing @xcite @xcite . given that manual annotation is expensive and time-consuming, it is not feasible to annotate the corresponding knowledge for each response on a large scale. therefore, it is desirable to develop knowledge-grounded dialogue generation models without reliance on explicit knowledge labels. some attempts have been made to learn the unsupervised retrieval of external knowledge based on semantic similarity @xcite . whereas, there exists the oneto-many phenomenon in knowledge-grounded conversation @xcite , where multiple knowledge elements can be appropriate to reply a given context. the prior top-1 knowledge selection employed by these approaches @xcite has difficulties to hit the knowledge contained in the target response, deteriorating the learning of knowledge utilization. as an improvement, postks @xcite and knowledgpt @xcite rely on the target response to identify the grounded knowledge. however, involving the posterior knowledge selection will inevitably cause discrepancy between the training and inference stages @xcite . in this paper, we propose an unsupervised approach for end-to-end knowledge-grounded conversation modeling, namely plato-kag (knowledge-augmented generation). as shown in figure 1 , given each dialogue context, the top-k relevant knowledge elements are selected for the subsequent response generation. then, the model learns to generate the target response grounded on each of the selected knowledge. the generation probability can in turn provide backpropagating signal for the precedent knowledge selection. these two components of knowledge selection and response generation are optimized jointly. two essential ingredients contribute to the performance of plato-kag: top-k knowledge selection and balanced joint training. firstly, in comparison to the conventional top-1 selection, top-k knowledge selection 𝑝 ! (𝑧|𝑐) knowledge grounded response generation 𝑝 " (𝑟|𝑐, 𝑧) transformer blocks context transformer blocks 𝐸(𝑐) 𝐸(𝑧 # ) ## methodology there are two main components in plato-kag: knowledge selection and knowledge-grounded response generation. ## related work knowledge-grounded conversation is becoming a more important and popular topic, with several datasets @xcite @xcite @xcite collected to study it. besides interactive dialogues, some of these datasets have annotated the corresponding knowledge for each response, aiming to ease the learning difficulty of knowledge-grounded conversation. however, given that manual annotation is expensive and time-consuming, it is not feasible to carry out the knowledge labelling on a large scale. unsupervised approaches have been introduced to model knowledge-grounded conversation. some of these such as @xcite al. (2019); yavuz et al. (2019); lin et al. (2020) perform implicit soft fusion over provided knowledge elements and do not select knowledge explicitly. some attempts have been made to learn the unsupervised selection of external knowledge based on semantic similarity @xcite . due to the one-to-many problem in knowledgegrounded conversation @xcite , the prior top-1 knowledge selection employed by these approaches has difficulties to hit the knowledge contained in the target response, and deteriorates the learning of knowledge utilization. our top-k selection improves the robustness of prior knowledge selection. some other works @xcite @xcite employ the target response to identify the grounded knowledge. since the posterior knowledge selection is involved, it will inevitably cause discrepancy between the training and inference stages @xcite . with end-to-end modeling and optimization, plato-kag gets exempt from this discrepancy. kif @xcite explicitly selects external knowledge through a retrieval module, and fuses into one integrated representation to assist dialogue generation. while some knowledge details might be obscured with this fusion. as comparison, the knowledge keeps its independence and integrity in our response generation, which helps reduce the hallucination. more recently, @xcite attempts to utilize the pre-trained retriever dpr @xcite . dpr has been trained on wikipedia which includes the knowledge sets of wow and holl-e. due to the concern of potential data contamination, we choosed to initialize our knowledge selection module with a general dialogue model which is pre-trained on reddit. thus, we facilitated an unbiased setting for our experiments and the analysis of framework generalization. ## conclusion in this paper, an unsupervised approach is proposed for end-to-end knowledge grounded conversation modeling. there are two main components in our method: knowledge selection and response generation. given a dialogue context, top-k relevant knowledge elements are selected and utilized for response generation. the generation probability can in turn provide training signal for the precedent knowledge selection. joint balanced training is further introduced for the effective optimization of these two components. comprehensive experiments have been carried out on wow and holl-e, verifying the effectiveness and superiority of the proposed method.