Teach Me What to Say and I Will Learn What to Pick: Unsupervised Knowledge Selection Through Response Generation with Pretrained Generative Models (2021)

Abstract: Knowledge Grounded Conversation Models are usually based on a selection/retrieval module and a generation module, trained separately or simultaneously, with or without having access to a ‘gold’ knowledge option. With the introduction of large pre-trained generative models, the selection and generation part have become more and more entangled, shifting the focus towards enhancing knowledge incorporation (from multiple sources) instead of trying to pick the best knowledge option. These approaches however depend on knowledge labels and/or a separate dense retriever for their best performance. In this work we study the unsupervised selection abilities of pre-trained generative models (e.g. BART) and show that by adding a score-and-aggregate module between encoder and decoder, they are capable of learning to pick the proper knowledge through minimising the language modelling loss (i.e. without having access to knowledge labels). Trained as such, our model - K-Mine - shows competitive selection and generation performance against models that benefit from knowledge labels and/or separate dense retriever.

## introduction the ability to properly ground conversations in structured and unstructured data, has become an increasingly important feature in designing conversational agents. by generating more informative and specific responses, such models can establish human-machine interactions that are more engaging and less prone to producing bland and common responses. the task of modelling knowledgegrounded conversations is traditionally decomposed into two sub-tasks: 1) knowledge selection (ks), i.e. picking the proper knowledge piece(s) from a provided pool based on dialogue history, and 2) response generation (rg), i.e. producing a response to the user's utterance conditioned on both dialogue history and selected knowledge piece(s). therefore and because of this sequential dependency, the generation performance is directly affected by model's selection/retrieval ability and the way this knowledge is being incorporated in the generation process. early examples of knowledge grounded conversation models mainly tried to diffuse the external knowledge as an extra hidden state into the decoder part of a recurrent seq-to-seq architecture @xcite . with the release of large knowledge grounded conversational datasets like wizard of wikipedia @xcite , topical-chat @xcite and holl-e @xcite , the field witnessed numerous studies aimed to best coordinate the ks and rg sub-tasks to improve the overall performance of models. as an early standard baseline @xcite proposed variations of transformer memnet, a generative model trained to do ks and rg using a memory network for selecting the most relevant knowledge piece. attempts to improve on these benchmarks can mostly be divided into two categories, based on their point of focus. selection oriented methods focus on enhancing the ks task, usually by introducing additional learning signals like the priorposterior discrepancy @xcite or long-term structural traits of conversations like flow and initiative changes @xcite @xcite @xcite . generation oriented methods on the other hand, try to mitigate the selection bottleneck by employing more powerful methods to incorporate knowledge in the generation process, thus reformulating the ks problem as an adaptive fine-grained selection to be dealt with in decoding @xcite . this was especially encouraged with the introduction of large pretrained generative models like gpt2 @xcite , bart @xcite and t5 @xcite which allow leveraging their ability in conditional text generation @xcite . more recently rag @xcite and fid (izacard and grave, 2021) models have been proposed (primarily in the qa context) to ease the computational costs and limitations of these big models, especially if the supporting passage(s) needs to be retrieved from a huge unstructured corpus. since these models integrate the ks and rg tasks, they do not need labeled knowledge for training, although a knowledge pool of manageable size is provided often via a parametric or non-parametric retrieval module. in this study we propose k-mine (knowledge mixing in encoder); a model that bridges between the two aforementioned paradigms by doing unsupervised knowledge selection with and within pretrained generative models (e.g. bart). using a simple score-and-aggregate module between the encoder and decoder of such a model, k-mine learns to (soft-) select the most relevant passage without needing knowledge labels or a separate retriever, while maintaining the generative skills of the original pretrained model. our experiments show very competitive performances on two knowledge grounded conversation datasets and state of the art results in integrated unsupervised knowledge selection. our work has similarities with both fid and rag in the sense of learning knowledge grounded generation with pretrained models and without the need to have labeled knowledge. it is however different in some key aspects. the retrieval/selection part in k-mine is truly and completely integrated inside the pretrained model, so unlike rag, it does not require a separate parametric retriever, and unlike fid, it is not totally disentangled from the retriever. moreover, unlike both fid and rag, k-mine aggregates over encoded knowledge options before passing them to the decoder. we will discuss the advantages and disadvantages of these choices at the end of the paper. ## results and discussion tables 2 and 3 show the performance of k-mine in knowledge selection and response generation for the wow (seen and unseen) and holl-e test sets against baselines. as one can see, k-mine shows very competitive results, especially in knowledge selection accuracy (r@1) although it does not benefit from knowledge labels in training, or a separate retrieval module. adding ks supervision (k-mine-full) enhances the knowledge selection performance by 3% although it negatively affects the rg performance. regarding the data, the ks performance boosts by more than 2% when only the 'w/kn' turns (turns that use knowledge to generate response) are used for training and testing, which shows that in the absence of explicit (ks) labels, the model prefers to select something than nothing foot_1 . to have a better understanding of the way knowledge selection happens under the hood, we introduce a localization metric to measure the level by which the knowledge distribution vector p deviates from the uniform distribution u towards a one-hot distribution o. defined as: loc varies between 0 (@xmath0; zero certainty) and 1 (@xmath1; absolute certainty) and provides a good way to track the average knowledge distribution. figure 3 shows how this metric changes during training for the standard k-mine model along with k-mine-max and k-mine-max-full. we can see that for k-mine-max the localization stays close to zero (~1e-4) whereas when knowledge labels are provided, it surges rapidly in less than 100 iterations. k-mine shows a delayed (starting around 500 iterations) but more extreme surge. this can 3 comparing roberta and bart in this manner is not trivial but it is also not unsound considering that the number of parameters are close (139m vs. 125m) and the selection task in k-mine is 'mainly' done via the bart encoder. be attributed to the weighted fusion which forces the model to localize the distribution as soon as possible -and thus at the cost of a less accurate distribution-in order to pass a less noisy representation to the decoder. the model seems to partly re-evaluate and improve this process subsequently as can be seen in the slightly descending behaviour of loc in later steps, eventually converging to the ascending supervised values. ## conclusion and future work in this work we introduced k-mine (knowledge mixing in encoder), which provides a simple way to train knowledge grounded conversation models based on pretrained generative models without knowledge labels or a separate retriever. k-mine uses a weighted aggregation method to fuse different encoded knowledge-context pairs into one sequence of the same length before passing it to the decoder, which has its advantages and disadvantages in comparison to models like rag and fid. it significantly reduces the computational cost in the decoder (at least by a factor of @xmath2. this was partly confirmed by the relatively low kf1 values of k-mine compared with the fully supervised version in another interesting topic is the relationship between the ks and rg performance in k-mine. in our experiments we saw that in later iterations, the ks performance often keeps improving while the rg performance (according to automatic metrics) starts to suffer. a more comprehensive study can determine whether it is possible to reconcile the two, or k-mine actually serves better as an auxiliary retriever or re-ranker module. either way, the potentials and limitations of this approach (i.e. selection via generation) in similar tasks and problems is worth exploring.