Investigating Content Planning for Navigating Trade-offs in Knowledge-Grounded Dialogue (2024)

Abstract: Knowledge-grounded dialogue generation is a challenging task because it requires satisfying two fundamental, yet often competing constraints: being responsive in a manner that is specific to what the conversation partner has said while also being attributable to an underlying source document. In this work, we bring this trade-off between these two objectives (specificity and attribution) to light, and ask the question: Can explicit content planning before the response generation help the model to address this challenge? To answer this question, we design a framework called PLEDGE, which allows us to experiment with various plan variables explored in prior work supporting both metric-agnostic and metric-aware approaches. While content planning shows promise, our results on whether it can actually help to navigate this trade-off are mixed – planning mechanisms that are metric-aware (use automatic metrics during training) are better at automatic evaluations but underperform in human judgment compared to metric-agnostic mechanisms. We discuss how this may be caused by over-fitting to automatic metrics, and the need for future work to better calibrate these metrics towards human judgment. We hope the observations from our analysis will inform future work that aims to apply content planning in this context.

## introduction a knowledge-grounded dialogue system that aims to address a user's information needs must meet two fundamental requirements. first, the knowledge shared by the system must be credible. a common formulation for this constraint is that the system must share information that is faithful or attributable to the retrieved document (what we refer to as attribution). more importantly, we argue that for the information to be useful to the user, * work done during an internship at google this credibility (as captured by attribution) is insufficient -the generated response must also make sense in the context of the conversation. it must be specific, in the sense that it must fit within the flow of the dialogue (what we refer to as specificity). this fundamental requirement is what differentiates research in this space from single-turn interactions of a user with a typical search engine. one major open challenge in knowledgegrounded dialogue research is that the model must balance these two objectives, which unfortunately, as we discuss later, can be at odds with each other. for instance, we show in figure 1 how responses can fail along either of these dimensions independently of each other. there is a scarcity of research explicitly investigating how to navigate the trade-off between these objectives. for example, @xcite investigated using control tokens for improving attribution, but their results showed that this often came at the expense of the specificity of the response to the conversation. in this work, we present a discussion of the challenges in optimizing for both specificity and attribution in knowledge-grounded dialogue. in section 2, we discuss automatic metrics that can serve as a proxy for these dimensions, demonstrating trivial means to increase either quality at the expense of the other. drawing from other nlg tasks, we pose the following question: can explicit content planning help to address this trade-off? content planning approaches add an intermediate step of generating the desirable features in the response, referred to as a plan, before actually generating the final surface realization conditioned on this plan. prior work showed that splitting the generation into guided steps could be effective in indirectly encouraging the model to be more grounded to commonsense @xcite and source documents @xcite @xcite , or to be more coherent @xcite @xcite . hence, it is only natural to hypothesize that content planning can also help to handle the trade-off between these two objectives as well. to enable a thorough investigation based on various plan variables explored in prior work, we design a framework called pledge. figure 2 provides an intuitive overview of the general methodology followed in pledge. this framework allows us to explore the utility of planning in navigating this trade-off, as well as the effects of structural vs keyword-based plans for this task. while content planning shows promise in general, our results on whether it can actually help to navigate this tradeoff are mixed. we observe that planning mechanisms that use automatic metrics during training are better at automatic evaluations but underperform in human judgments compared to mechanisms that do not rely on these metrics explicitly. we discuss how metrics that are better calibrated towards human judgment might help to address this misalignment. we provide insights from our analysis with the hope of informing future work that aims to apply content planning in this context. we now summarize our contributions: i. we present a computational discussion of the trade-offs between specificity and attribution in knowledgegrounded dialogue (section 2), ii. we present a novel framework pledge (section 3) that automates some of the heuristic approaches in prior work to analyze whether content planning can help to handle this trade-off, and iii. we present our analysis based on both automated metrics and human evaluation and discuss our insights about the utility of content planning in this context. ## evaluation metrics for grounded dialogue response generation in the task of knowledge-grounded dialogue, a system m q is given a sequence of previous conversation turns (@xmath0...x nx ) and an evidence span (@xmath1...e ne ) selected from a knowledge corpus foot_0 , and must generate a response ŷ = m q (x, e) such that the response quality q(ŷ, x, e) is maximized. a good response must be: (1) conversationally appropriate in the context of the rest of the dialogue and (2) accurately representing the information from the knowledge evidence. as mentioned earlier, these two are fundamental to any practically useful knowledge-grounded dialogue system. hence, we now discuss automated metrics to capture these requirements. ## pledge: plan-edit-generate pledge consists of two modules: a response generation model g (section 4.1) and an editor e q (section 4.2). g is our underlying sequence-tosequence model trained to perform plan-based response generation. the editing model e q is tasked with modifying the candidate plans generated by g, for better alignment with the quality estimator q. keeping the two modules separate provides the flexibility to train them independently with different datasets and training objectives. three-stage inference: once g and e q are trained, the final response is generated in three stages during inference (top diagram in figure 4 ). first, the generation model g takes in the conversation history x and the evidence e to generate a candidate plan ĉ = g(x, e). next, the editor e q iteratively modifies this plan to better satisfy the quality constraints defined by q, generating ĉ = e q (ĉ, x, e). finally, ĉ is fed back to g to generate the output response ŷ = g(ĉ , x, e). we first describe the general plan format used by our models and then describe the design of the two modules. plan format: in order to investigate rq 3, we investigate two different types of plan formats for defining content plans ĉ. we take inspiration from prior work that used content plans constructed from different kinds of attributes, including dialogue acts, emotion labels, and topic words @xcite , along with phrase outlines @xcite @xcite , and entity chains @xcite . first, we investigate using structural features -we use a set of variables that describe desired response qualities, such as the level of objectiveness, the proximity to the prior utterance, the proximity to the evidence, dialogue act, and conveyed emotion. we provide a complete list of these variables along with how they were computed in appendix a. we encode each variable using special tokens that we add to the model vocabulary. second, we investigate a keyword-based plan consisting of an ordered list of the salient words that should appear in the model output (the salient words are selected via tf-idf counts following the keyword-based plan construction procedure proposed by @xcite ). in our experiments, a plan consists of concatenated structural features (struct), a keyword list (kw), or both concatenated with a delimiter (full). at training time, the plan is extracted automatically from the gold response, and at inference time, they are generated by the generation model. we include a shortened plan example in figure 2 with more detailed examples in ## experiments we evaluate our models on the wizard of wikipedia (wow) dataset @xcite to answer the four rqs from section 3. wow is a popular dataset consisting of dialogues between an 'apprentice', who seeks information, and a 'wizard', who has access to relevant documents extracted from wikipedia. along with submitting a grounded response, in each turn, the 'wizard' also labels the knowledge sentences used for formulating the utterance. we use these labeled sentences as input evidence for the pledge framework. wow contains 73, 571 instances for training, while 3905 and 3842 for validation and testing respectively. 2 baselines: we compare to the standard t5 model. we also compare to @xcite , which used t5 with control codes (labeled as control-codes in tables) for encouraging attribution but didn't control for specificity. we also include the baselines (e2e and dodeca) from that paper. training details: for all of the models, we use beam-search to be aligned with baselines @xcite . 3 for all variants of planning and controllable models, we used t5-base @xcite as the model architecture for consistency. 4 for training the masker model, we used automatically constructed plans from wow and two different dialogue tasks (top-icalchat @xcite and cmu-dog @xcite ). we provide more details in appendix e. ## conclusion we investigated the trade-off between attribution and specificity for knowledge-grounded dialogue, analyzing whether content planning prior to final output generation can help to navigate this tradeoff. we find that although content planning shows promise in general, we observe differences in the trends in automated and human evaluations. hence, whether content planning can help to handle the trade-off remains an open question and more efforts are needed to answer it, with automated metrics that are potentially better calibrated with human judgment. we hope that the insights gained in this work inform future efforts on exploiting content planning in similar contexts.