# Knowledge Interpolated Conditional Variational Auto-Encoder for Knowledge Grounded Dialogues
## 1 Introduction  
End-to-end neural conversation models have shown significant promise, demonstrating remarkable advancements from both academia and industry perspectives [1; 2; 3; 4; 5; 6]. Nonetheless, these conventional conversation models often grapple with generating informative and engaging responses due to their limited capacity to retain and leverage background knowledge [7; 8]. To surmount this knowledge-absence issue prevalent in existing conversation models, Knowledge Grounded Dialogue (KGD) generation is recently proposed for generating responses by simultaneously referring to both the background knowledge and the dialogue context [7; 9; 10; 11; 12; 13; 14; 15; 16; 17]. The objective is to enhance dialogue response generation to facilitate engaging and in-depth conversations, while avoiding the inclusion of non-factual information. Existing methodologies in KGD can be broadly classified into extractive and generative models. Extractive models typically view KGD as a reading comprehension task [18] or a document question-answering challenge [19], employing established models such as BiDAF [20] to address this issue. Nevertheless, these models, primarily focusing on extracting knowledge snippets, need to provide more engaging and natural responses akin to human conversation. To mitigate these limitations, attention has turned towards integrating external factoid knowledge into generative dialogue models [21; 22; 23; 24]. Most of these works focus on selecting appropriate knowledge and combining the chosen knowledge during the response generation process via copy mechanisms [10; 25] or memory network [26]. However, it is crucial to note that responses exhibit varying degrees of knowledge specificity. Some responses may rely heavily on external knowledge, whileothers may predominantly depend on the query due to differing dialogue contexts. Despite this, existing generative models do not explicitly model the instance-variety of knowledge specificity in KGD. The Conditional Variational Auto-Encoder (CVAE) has emerged as an effective model for integrating information from multiple sources within a latent space, showing promise in the domain of response generation [27]. Previous research has utilized CVAE by incorporating a latent variable that is conditioned on the concatenation of dialogue context and additional knowledge information such as dialog acts [28], persona sentences [29] or even images [30]. However, existing CVAE models encounter the issue of representation entanglement [31]. This issue arises due to the single latent variable in CVAE, making it challenging for the model to learn informative and interpretable representations for context and knowledge simultaneously. This issue can lead to inconsistent and unnatural knowledge-grounded responses by the CVAE models. In this work, we build upon the strengths of CVAE to address the challenges of KGD. To enable the explicit capture of knowledge specificity by CVAE and to overcome its representation entanglement problem, we present a novel Knowledge Interpolated conditional Variational auto-encoder (KIV). This innovative approach introduces two separate latent variables to model dialogue context and external knowledge independently. Unlike prior models that concatenate multiple latent information sources, our approach uses linear interpolation to seamlessly integrate the latent variables associated with context and knowledge, where the interpolation weight corresponds to the level of knowledge specificity. In addition to proposing a deterministic method for obtaining the interpolation weight, we explore a probabilistic interpolation method. This approach views the interpolation weight as a latent variable and models it using a Logistic-Normal distribution. Our proposed probabilistic interpolation method empowers the model to robustly emulate the process of adaptively leveraging background knowledge in response generation. We evaluate the effectiveness of our proposed model on the Wizard-of-Wikipedia [7] and Holl-E [10] datasets.