Triplet-Free Knowledge-Guided Response Generation (2023)

Abstract: Generating vivid and informative responses (e.g., comments for social posts and utterances for dialogues) is challenging without giving relevant knowledge. Prior works focus on constructing the ”latent” knowledge first and then learning how to ”ground” it based on pseudo (context, knowledge, response) triplets. However, the retrieval between real responses and their latent knowledge is difficult in nature. In this paper, instead of focusing on how to ground knowledge given the responses, we take a different perspective to optimize the final responses for given guided knowledge directly. This allows us to re-formulate the entire problem in a simplified yet more scalable way. Specifically, we pretrain a response language model (LM) to measure the relevance and consistency between any context and response, then use search engines to collect the top-ranked passages to serve as the guiding knowledge without explicitly optimizing the ‘‘best” latent knowledge that corresponds to a given response. The final response generation model is trained through reinforcement learning by taking both the response LM prior and knowledge-injection rate as rewards. For better evaluations, we construct a new Chinese benchmark, ”IceKC”, using fresh multimodal online social posts. Both automatic evaluations and human evaluations show our zero-resource approach performs significantly better than prior works.

## introduction response generation, including dialogue utterances and post comments, is a testbed for machine intelligence and has many applications. however, previous ai models tend to output generic and bland responses as shown in @xcite , triplet-based approach is highlighted in blue, which retrieves the most likely knowledge for training. our approach is highlighted in orange, which aims to generate knowledge-guided responses directly. 2017), which led to a few recent works that leverage external knowledge to improve the generation quality from both diversity and informativeness perspectives @xcite @xcite @xcite @xcite . although impressive progress has been made, major hurdles still exist. constructing a context-knowledge-response triplet dataset via crowd-sourcing (i.e., @xcite ; @xcite ) for training purposes might be too expensive to scale up, which also violates the mainstream paradigm of large-scale self-supervised pretraining @xcite . as expected, prior works @xcite @xcite show that models trained on such manually-constructed triplets cannot be generalized to other languages and unseen domains. more recently, zero-resource methods @xcite @xcite are proposed where knowledge-aware generation is learned on such triplets with either matched or inferred knowledge. however, one critical challenge is that given the response, their corresponding "knowledge" is extremely difficult to retrieve from a vast knowledge space, especially when retrieving from internet search results. as shown in figure 1 , such constructed triplets are very unreliable because the corresponding "knowledge" is scattered over the internet and the retrieval results are noisy where irrelevant sentences can have high overlap with the response. forcing the generator to associate the response with such pseudo knowledge is insensible. this raises the question: can we relax the requirement of constructing such triplets for training purposes? to mitigate those challenges, we introduce a new training methodology that does not require constructing any triplet. our key idea is inspired by a relaxed casual graphical model where the conditional probability distribution (p(r|c, k)) of response (r) given both context (c) and knowledge (k) can be approximated by its lower-bound which only contains two prior, p(r|c) and p(r|k). more specifically, given a context, we use search engines to sample a knowledge passage without explicitly inferring or optimizing, then we use reinforcement learning to optimize the response generation model by jointly considering both prior models as critics to provide reward signals. this allows us to steer the focus from knowledge selection to knowledge-guided generation, which therefore pivots the optimization to explore how to generate informative and engaging responses given both context and knowledge. without loss of generality, to validate our idea, we leverage a pretrained response language model for p(r|c) and a non-parametric lm for p(r|k) in the current experiments, leaving more advanced prior models for future work. to further encourage more investigation from the community, we construct a benchmark on chinese multi-modal knowledge-grounded social post commenting, called "icekc", that facilitates more faithful evaluations. extensive experiments show that our approach performs significantly better than previous work under a zero-resource setting. our contributions are three-fold: (1) we propose a novel zero-resource training strategy for knowledge-guided response generation without the need to build triplets, which is model-agnostic as well as more flexible and easy to scale. (2) we construct a new benchmark from social media posts, which can be used to more faithfully evaluate knowledge-aware multimodal commenting systems. (3) experiments show that our approach generates significantly better responses than other strong baselines. ## approach prior works all focus on constructing the "latent" knowledge (k) first given both the context (c) and response (r). however, as we argued before, such triplet-based approaches are challenging due to the difficulty in knowledge retrieval, which results in noisy triplets (c, k, r) and thus hurts the training. in this paper, we take a triplet-free approach by directly feeding sampled knowledge (i.e., search engine) into the response generation model p(r|c, k; θ), for any given context. as for a fixed context and sampled knowledge pair (c, k), there is no corresponding "ground-truth" response (r) available to apply any supervised learning method. we resolve such unpaired training by leveraging a pretrained response language model that provides distribution-level reward signals. ## icekc to evaluate models on chinese multimodal social post commenting, we construct a new benchmark testset for evaluation purposes only. to ensure that external knowledge is necessary to generate the comment, we collect testcases from weibo trending posts 2 starting from february 2022 so that the topics are fresh and not likely to overlap with training data collected previously. annotators are instructed to choose a post in one of the trending topics, use search engines to search for an appropriate knowledge sentence, and write a sensible comment to the original post based on the information in the knowledge sentence. although we focus on knowledge-aware comments generation in this work, this benchmark testset can also be used to evaluate other tasks, including query generation and knowledge selection, as shown in appendix a. ## related work generating vivid responses, such as comments @xcite @xcite and dialogue utterances @xcite , is known to be difficult for neural models. to address the problem, previous work attempts to use additional key phrases @xcite , user profiles @xcite , and images @xcite to serve as an external knowledge to be grounded on. more recently, unstructured documents @xcite @xcite , such as those retrieved from the internet @xcite , are prevalent, especially for open-domain contexts. generally, knowledge-grounded generation is decomposed into two tasks, knowledge selection @xcite and knowledge-aware generation @xcite , and additional search query generation for internet-retrieved knowledge @xcite . given knowledge-grounded datasets, supervised approaches @xcite have been proposed for knowledge-aware generation. several unsupervised approaches @xcite @xcite @xcite have also been proposed. they rely on retrieving a most likely knowledge sentence for a specific response by either n-gram matching or model inference and leverage such weak triplets to train models. while triplets can be relatively reliable on human-annotated datasets such as wow @xcite , it is generally not feasible in internet-retrieved documents @xcite and leads to poor generalization @xcite to different domains. although some approaches @xcite build on unreliable automatically constructed triplets and are free from ill generation, they still struggle to incorporate knowledge since knowledge sentences are loosely associated with responses in such triplets. ## conclusion in our study, we examine the effectiveness of previous approaches in utilizing unstructured text for knowledge-grounded response generation in a zeroresource setting. additionally, we propose a novel approach to tackle this challenging task. instead of adopting a triplet-based approach, which focuses on identifying the specific knowledge underlying a response, we employ a triplet-free approach that aims to generate coherent and knowledgeable responses given appropriate knowledge. furthermore, we develop the first benchmark specifi-cally designed for chinese multimodal knowledgegrounded commenting to evaluate the effectiveness of our approach. experimental results demonstrate that optimizing the lower bound for p(r|c, k) without relying on triplets as learning signals can achieve superior performance compared to existing triplet-based approaches.