Achieving Conversational Goals with Unsupervised Post-hoc Knowledge Injection (2022)

Abstract: A limitation of current neural dialog models is that they tend to suffer from a lack of specificity and informativeness in generated responses, primarily due to dependence on training data that covers a limited variety of scenarios and conveys limited knowledge. One way to alleviate this issue is to extract relevant knowledge from external sources at decoding time and incorporate it into the dialog response. In this paper, we propose a post-hoc knowledge-injection technique where we first retrieve a diverse set of relevant knowledge snippets conditioned on both the dialog history and an initial response from an existing dialog model. We construct multiple candidate responses, individually injecting each retrieved snippet into the initial response using a gradient-based decoding method, and then select the final response with an unsupervised ranking step. Our experiments in goal-oriented and knowledge-grounded dialog settings demonstrate that human annotators judge the outputs from the proposed method to be more engaging and informative compared to responses from prior dialog systems. We further show that knowledge-augmentation promotes success in achieving conversational goals in both experimental settings.

## introduction generic responses which lack specificity have been a major issue in existing dialog models @xcite . the issue in part stems from bottlenecks in dialog models due to a limited scope of scenarios and access to limited knowledge available during training. on the other hand, encoding all possible world knowledge at training time is not feasible, and even undesirable in cases where knowledge sources are dynamically varying @xcite @xcite @xcite . one possible approach is to incorporate there are plenty of museums to visit around cambridge. if you love hiking, you can enjoy the trails alongside the river. some of my friends like to go the centre of the town and catch a movie. find me something fun to do around cambridge area in daytime! you can go for a movie. is there anything else that your prefer? many prefer to visit museums. you can do hiking around the river if you love nature. or you can watch a movie. which one do you prefer? retrieved knowledge initial response final response dialog context 🤖 🤖 relevant knowledge at decoding-time. for example, in figure 1 , the user is seeking options for a fun activity around cambridge. while the initial dialog response suggests watching a movie as an option, it does not provide any information behind that choice. we propose and evaluate an approach for unsupervised knowledge injection into a dialog model's response at decoding time foot_0 -not addressed in any previous work. we first sample a response from the model (trained on dialog data) conditioned on the dialog context. next, we utilize the dialog context and the sampled response to query external knowledge sources. finally, the retrieved knowledge is used to construct a more informative and engaging response (figure 1 ). a major advantage of such post-hoc knowledge injection is its flexibility in adding newer knowledge sources especially where the success of achieving conversational goals relies upon the availability of relevant knowledge. post-hoc injection also promotes efficiency in nlp applications @xcite : it mitigates the need to retrain dialog models to accommodate dynamically evolving knowledge. we experiment with two types of knowledge sources: language models, which we treat as parametric knowledge bases @xcite ; brown et al., 2020); and user review datasets such as yelp reviews @xcite as nonparametric knowledge sources ( § 2). since it is possible to gather a large amount of related knowledge given a query, we select a relevant and diverse (estimated via information-theoretic measures) subset of knowledge snippets using an unsupervised method ( §3.1). then, a gradient-based inference approach is used to construct an updated response that incorporates the selected knowledge ( § 3.2). note that our framework does not require retraining the existing dialog model-it only relies upon updating the model's output hidden states at decoding time for unsupervised knowledge injection. we experiment with two scenarios: goaloriented and knowledge-grounded dialog where the training data covers only a fraction of the needed knowledge. automatic evaluation reveals that our method is capable of generating highly diverse responses in both settings. in some cases, the generated response shows high overlap with the original target response showing that our unsupervised method bridges the knowledge gap between available knowledge and human-written responses present in the existing dialog corpus. an extensive human evaluation confirms that generated responses are indeed engaging, interesting, and human-like without any loss in fluency. to pinpoint the usefulness of knowledge injection in the above settings, we design a real-time study ( §5.3) where users interact with our system to reach a conversational goal (e.g. planning a holiday or knowing more about the solar system). we find that external knowledge enables users to achieve their goals more efficiently. additionally, we observe that the our approach of sub-selecting relevant but diverse knowledge leads to responses that promote success in achieving conversational goals. ## post-hoc knowledge for dialog our goal is to construct a dialog response by injecting knowledge (from external textual sources) at decoding time, without having to retrain the models. consider a dialog model m from which we can sample a dialog response x d given a dialog history h. we shall refer to the response x d sampled from such a model without any decoding time knowledge injection as the initial response. however, as motivated earlier, samples from such a dialog model often lack detail. to improve such responses, we retrieve and incorporate relevant external knowledge k into the initial response. to achieve our goal, we construct a query using both dialog history h and the initial response x d , and gather a relevant knowledge candidate k from a knowledge source k. the retrieved snippet can provide useful information to the end-user to achieve the conversational goal (see §5.3). we explore both parametric (e.g querying a language model) and non-parametric (e.g. deterministic retrieval using word-overlap) ways to obtain post-hoc knowledge. ## unsupervised knowledge injection in generated dialog effectively utilizing the retrieved knowledge snippets to construct an enriched dialog response encompasses two major challenges. firstly, it is not practical to use potentially hundreds of knowledge snippets obtained from the retrieval step for a single response generation. thus, we need to find a relevant but diverse subset of the snippets. secondly, the dialog model m is trained to condition only on the dialog context, and not on the external knowledge. hence, to leverage the knowledge snippets, we need a decoding strategy to rewrite the initial response x d such that the resulting final response x f should closely follow the knowledge snippet to be injected without a loss in the fluency and consistency. thus, our method requires no additional training and only assumes a language model trained on dialog context (i.e. m). we refer to our proposed framework (figure 2 ) as poki (post-hoc knowledge injection in generated dialog). ## related work knowledge grounded dialog datasets such as wizard-of-wikipedia @xcite and topical chat @xcite typically consist of dialog responses paired with relevant knowledge available as collected annotations. hence, models trained on such datasets are restricted to the knowledge sources they were exposed to at training time. past work @xcite @xcite @xcite @xcite @xcite has looked into injecting extra knowledge sources at training time in a bid to add knowledge not available originally as paired to dialog responses. however, such approaches require re-training the model if some new knowledge source were to be used. moreover, while previous work focuses on just improving specificity of dialog response using external knowledge, we also study the effect of additional knowledge in achieving conversational goals. improving the diversity of dialog responses by using diversity-promoting sampling has been explored in past work @xcite . we use a gradient-based decoding method, building on past work in this direction @xcite @xcite . however, we propose new objectives to inject post-hoc knowledge obtained based on already generated dialogan unsupervised knowledge injection method that has not been explored so far. ## conclusion we propose a framework for unsupervised knowledge injection into dialog responses. we show that knowledge can be obtained post-hoc from any knowledge sources that can improve users' ability to reach their conversational goal more effectively. in future, our idea can be generalized to setups where external knowledge can justify model's predictions such as conversational recommendation.