C o LV : A Collaborative Latent Variable Model for Knowledge-Grounded Dialogue Generation (2021)

Abstract: Knowledge-grounded dialogue generation has achieved promising performance with the engagement of external knowledge sources. Typical approaches towards this task usually perform relatively independent two sub-tasks, i.e., knowledge selection and knowledge-aware response generation. In this paper, in order to improve the diversity of both knowledge selection and knowledge-aware response generation, we propose a collaborative latent variable (CoLV) model to integrate these two aspects simultaneously in separate yet collaborative latent spaces, so as to capture the inherent correlation between knowledge selection and response generation. During generation, our proposed model firstly draws knowledge candidate from the latent space conditioned on the dialogue context, and then samples a response from another collaborative latent space conditioned on both the context and the selected knowledge. Experimental results on two widely-used knowledge-grounded dialogue datasets show that our model outperforms previous methods on both knowledge selection and response generation.

## introduction knowledge-grounded dialogue generation @xcite @xcite , which utilizes external knowledge to enhance conversation backgrounds, has achieved promising performance. to exploit external knowledge efficiently for conversations, typical approaches @xcite @xcite @xcite @xcite tend to decompose this task into two streamlined sub-tasks: knowledge selection and knowledgeaware response generation. besides, some other work @xcite also tries to @xcite what is your favorite number? → i love the number 7. what do you think about that? 1. anyone who dares to kill cain "will suffer vengeance seven times over". 2. seven is the natural number following six and preceding eight. knowledge candidates 3. islam first came to the western coast when arab traders as early as the 7th century ce. 4. the number 7 has been associated with a great deal of symbolism in religion. in western culture, it is often considered lucky. ...... n. this genre has been popular throughout the history of culture. ## related work our work is mainly related to two research branches: knowledge-grounded dialogue generation and variational auto-encoder learning. knowledge-grounded dialogue generation has raised broad interest and also has been greatly advanced by many new datasets @xcite @xcite @xcite . existing methods on this task mainly focus on resolving two research problems: knowledge selection (ks) and knowledgeaware response generation. @xcite proposed a memory network to retrieve knowledge and combined it with a transformer-based model to generate response. external knowledge base was also utilized to facilitate the utterance understanding and knowledge selection @xcite . @xcite used memory network and copy mechanism to keep deep interaction between knowledge and utterances. @xcite employed a dual learning paradigm to enhance knowledge interaction. @xcite proposed to augment the dialogue generation by utilizing external non-conversational text, which is effective but also introduce noise. @xcite and @xcite proposed to employ pretraining methods on the structured/unstructured knowledge representation and fine-tune the model using the limited knowledge-grounded training examples. other work took efforts on utilizing future information. @xcite firstly employed posterior network, while @xcite further utilized sequential characteristics of knowledge. besides, to further bridge the gap between prior and posterior network, @xcite and @xcite devised specific posterior information prediction modules. hereby, our proposed colv model differs from previous work by utilizing collaborative latent variables to model the distributions of knowledge and response simultaneously. vae learning (kingma and welling, 2014) is widely used in a variety of natural language processing tasks, including machine translation @xcite , question answering (lee et al., 2020), and conversations @xcite @xcite . the core idea of variational auto-encoder is to utilize the advantage of posterior information or external information during training phase, and optimize the objectives by minimizing the kl divergence @xcite . unlike previous work that applied vaes on dialogue generation @xcite @xcite , we aim at using collaborative latent variables to connect the external knowledge, dialogue context, and response, which will further enhance the correlation between knowledge selection and response generation. to the best of our knowledge, our method takes the first attempt to collaboratively model these two different distribu-tions for knowledge-grounded conversations. ## conclusion in this paper, we propose a novel collaborative latent variable (colv) model to simultaneously learning to select knowledge and generate responses in knowledge-grounded dialogue generation. colv model helps improve the diversity not only on knowledge selection but also help generate diverse response while given a specific knowledge. besides, the colv model uses two collaborative latent variables for coupling the knowledge and dialogue. extensive experiments on two benchmark datasets show that colv achieves satisfied performance, indicating that colv can select more diverse knowledge and further generate more coherent and diverse responses than baseline models.