Efficient Latent Variable Modeling for Knowledge-Grounded Dialogue Generation (2023)

Abstract: Knowledge-grounded dialogue generation requires first retrieving appropriate external knowledge based on a conversational context and then generating a response grounded on the retrieved knowledge. In general, these two sequential modules, a knowledge retriever and a response generator, have been separately trained in a supervised manner. However, obtaining intermediate labels of the ground-truth knowledge is expensive, especially in open-domain conversations. Latent variable modeling avoids this need for the labels. In this paper, we propose an efficient algorithm for this latent variable modeling that is able to leverage a large amount of dialogue data. Rather than directly training the complex retriever, we adapt a query generator with an off-the-shelf retriever, and the query generator and response generator are simultaneously trained over the latent variable of query. Moreover, we employ lower bound of the evidence as a training objective and modify it to robustly perform the joint training. Experimental results on diverse knowledge-grounded dialogue datasets show that the proposed algorithm significantly outperforms the supervised learning algorithm even without the use of the annotated knowledge while maintaining efficiency and scalability.

## introduction recently, knowledge-grounded dialogue generation has drawn increasing attention especially as a key ingredient for open-domain conversational agents @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite ; anantha * work done at kakao brain † corresponding author @xcite . it usually obtains knowledge from external resources such as wikipedia-based databases and web search engines since internal knowledge even in large-scale parametric language models @xcite @xcite is incomplete or outdated, and moreover it can provide hallucinated information. in order for a dialogue response to be grounded on such external knowledge, the conversational agent generally consists of a knowledge retriever, which retrieves knowledge corresponding to a given dialogue context, followed by a response generator that produces an informative response based on the dialogue context and the retrieved knowledge. in many previous methods, supervised learning has often been applied to independently optimize each module using the ground-truth or gold knowledge @xcite @xcite @xcite @xcite . however, human annotation of knowledge information is cumbersome, expensive, and often incomplete due to the existence of multiple possible knowledge candidates and the overlooking of the target response. in addition, existing automatic annotations are generally limited to a simple extractive question answering @xcite . this difficulty in obtaining annotations of knowledge information could be severe under the open-domain conversation and hinders the use of large-scale dialogue data. therefore, there have been a number of recent approaches to learn the knowledge retriever and the response generator without the knowledge supervision. in specific, they have treated the retrieved knowledge, document, or passage as an unobserved latent variable and adapt latent variable modeling based on approximated marginalization (e.g. top-k) @xcite @xcite , reinforcement learning @xcite @xcite or variational methods @xcite @xcite @xcite . however, joint training of the retriever along with the generator under this latent variable modeling has some restrictions in utilizing the retriever. for example, a retriever needs to produce a differentiable prior probability for the gradient propagation through the marginalization or to be intermittently updated to rebuild the whole passage index during training @xcite @xcite . this leads to limitation in facilitating complicated or large-scale retrievers and further in leveraging large-scale data. in addition, the posterior probability used in the previous variational methods has been modeled by a separated network, which grows the training complexity and also incurs the discrepancy between the trainingtime and test-time knowledge samplings. to overcome such restrictions, in this paper, we propose an efficient latent variable modeling, named elvm, for knowledge-grounded dialogue generation. in particular, we reduce the burden for training a whole retriever by employing a query generator followed by an off-the-shelf retriever that is fixed during training. more precisely, in tandem with the response generator, the query generator rather than the retriever is jointly optimized by latent variable modeling, in which a query is taken as a latent variable, on paired observations of dialogue contexts and responses. to this end, we exploit the variational method using the evidence lower bound (elbo) @xcite and approximate the expected conditional likelihood in the elbo by subset sampling from the prior distribution, which acts as the training objective for the response and the query generators. this approximation gets rid of an extra modeling of a surrogate posterior distribution or online posterior inference such as markov chain monte carlo (mcmc) and also reduces the training-inference discrepancy in knowledge retrieval. moreover, we further modify the kullback-leibler (kl) regularization of the elbo by constructing the approximated posterior distribution from the conditional likelihood and prior distribution and set this posterior distribution as a teacher for a distillation objective to further learn the query generator. experimental results show that the proposed elvm allows to efficiently and robustly perform training without (1) the use of the annotated knowledge, (2) an explicit training of the knowledge retrieval, and (3) a complex posterior sampling. especially, it significantly outperforms previous state-of-the-art methods for knowledge-grounded dialogue generation. in addition, the proposed posterior distillation improves the performances over the baseline that solely maximizes the expectation of the conditional likelihood. our main contributions can be summarized as: ## conclusion in this paper, we propose elvm as an efficient latent variable modeling for knowledge-intensive dialogue generation. in elvm, the knowledge retrieval is realized by the query generation followed by document retrieval using the off-the-shelf retriever for ease of joint training of both the retriever and the response generator, without the knowledge supervision. furthermore, the training scheme of elvm modifies the elbo for the marginal likelihood to effectively perform the joint training without the use of complex posterior sampling, which also eliminates the discrepancy between the training-time and test-time knowledge samplings. elvm empirically demonstrates that it significant outperforms over previous latent variable methods with in-depth analysis on diverse dialogue datasets.