Transferable Persona-Grounded Dialogues via Grounded Minimal Edits (2021)

Abstract: Grounded dialogue models generate responses that are grounded on certain concepts. Limited by the distribution of grounded dialogue data, models trained on such data face the transferability challenges in terms of the data distribution and the type of grounded concepts. To address the challenges, we propose the grounded minimal editing framework, which minimally edits existing responses to be grounded on the given concept. Focusing on personas, we propose Grounded Minimal Editor (GME), which learns to edit by disentangling and recombining persona-related and persona-agnostic parts of the response. To evaluate persona-grounded minimal editing, we present the PersonaMi-nEdit dataset, and experimental results show that GME outperforms competitive baselines by a large margin. To evaluate the transferability, we experiment on the test set of BlendedSkillTalk and show that GME can edit dialogue models’ responses to largely improve their persona consistency while preserving the use of knowledge and empathy.

## introduction grounding dialogue agents on external information is important for building engaging conversational ai systems @xcite . along this track, various datasets and models have been proposed to ground dialogues on personas @xcite , knowledge @xcite , emotions @xcite , and images @xcite . generally, grounded dialogue modeling trains a dialogue model on a dataset d that consists of triples (c, r, g), where c is the dialogue history, r is the response, and g is the grounded concept. the model is generally optimized using maximum likelihood estimate (mle), i.e., arg max θ e (c,r,g)∼d log p θ (r|c, g). in this paper, we explore persona-grounded minimal editing, as demonstrated in figure 1 . we propose grounded minimal editor (gme), which is trained on persona-grounded dialogue data. specifically, response templates are sampled by corrupting persona-related spans and sentences based on gradient-based attribution and word overlap. by denoising the templates, gme disentangles and recombines persona-related and persona-agnostic expressions. since the personas of original responses are not observed at inference, we train a classifier for template generation at inference. two research questions are investigated in this paper: q1) is the proposed gme model effective for grounded minimal editing? q2) does our framework address the transferability challenges (more specifically, the distributional gap and the grounding gap)? for q1, we build personaminedit, a new dataset derived from personachat with multiple human references for the edited response. automatic and human evaluations show that gme outperforms competitive baselines and has the most similar behavior to humans references. for q2, we evaluate gme on the test set of blendedskil-ltalk @xcite , whose data distribution and grounded concepts are different from per-sonachat, which requires gme to be transferable. we observe that gme improves the persona consistency of responses generated by pretrained blender-90m models @xcite , while preserving the use of knowledge and empathy. results also show that gme-edited responses largely outperforms transfertransfo @xcite , which is trained in the canonical way as in eq. ( 1 ). ## related work recent work leveraged grounded information in dialogue agents to chat engagingly, e.g., using knowledge @xcite , emotions @xcite , personas @xcite , and images @xcite . for persona grounding @xcite , transfer learning methods @xcite @xcite and latent variable models @xcite have shown promising results. further, the persona consistency issue @xcite and persona-augmented empathetic agents @xcite have also been explored. as discussed in section 1, existing methods generally adopt the mle objective in eq. ( 1 ) and suffer from two transferability challenges, i.e., the distributional gap and the grounding gap, which are addressed by the proposed grounded minimal editing framework. the idea of editing existing responses has been explored, e.g., the deliberation network @xcite , two-pass response generation @xcite , and retrieval-augmented dialogue modeling @xcite @xcite @xcite . this paper is essentially different from these works from two perspectives. 1) regarding the formulation, we emphasize minimal editing, while previous works do not. as analyzed in section 1, minimal editing is an important component to address the transferability challenges; 2) regarding the training algorithm, previous works derive templates from self-generated or retrieved texts, while our model derives templates from the observed responses. our work is also related to controlled text editing without parallel data, e.g., unsupervised text style transfer @xcite ; rao and tetreault, 2018; @xcite , semisupervised contextual text style transfer @xcite , syntax-controlled paraphrasing @xcite , contrastive model explanation @xcite , counterfactual story generation @xcite , and sentence-level editing for empathetic dialogues @xcite . some of these studies also utilize masked templates @xcite @xcite @xcite . however, these previous works only focus on categorical conditions in a small label space, while the personas in our study are embedded in much larger spaces. in the large persona space, the persona sentences at test time are never seen during training. further, when generating masked templates, the personas of the original responses are unobserved in our study. ## formulation we provide a formulation of the proposed framework. grounded dialogue modeling uses a dataset d that consists of triples (c, r, g), where c, r, and g are the dialogue history, the response, and the grounded concept, which are shown in grey in the left part of figure 2 . to formulate the term "minimal", we need to add unobserved variables into the graphical model, denoted as u in figure 2 , which cover all unobserved variables. the graph states that @xmath0, g, u). as shown in the right part of figure 2 , we observe (c, r o , g e ) at inference time, where r o and g e stand for the original response and the grounded concept for editing. the graph states that the original response r @xmath1, g o , u), where g o represents the concept the original response is grounded on, and that both g o and u are unobserved. the edited response is defined as r @xmath2, g e , u), which replaces g o as g e , and keeps c and u intact. our formulation follows the idea of counterfactual reasoning @xcite , and it guarantees that 1) the content irrelevant to the grounded concept is preserved, and that 2) the edited response is coherent with the dialogue history. since it is costly to collect paired (r o , r e ) for training, the grounded minimal editor should be trained on the grounded dialogue data (c, r, g) ∼ d as in eq. ( 1 ). as the first attempt toward the proposed framework, we focus on persona-grounded minimal editing in the experiments. thus, in the remaining part of this paper, we set the grounded concept g, g o , g e as the persona p, p o , p e . ## experiment on personaminedit we use personaminedit to evaluate personagrounded minimal editing (q1 in section 1). ## transferability experiment we evaluate the transferability of gme to minimally edit existing responses on the test split of blendedskilltalk @xcite . we also evaluate whether grounded minimal editing addresses the transferability challenges of grounded dialogue modeling (q2 in section 1). ## discussion as mentioned in section 2, the term "minimal" distinguishes our work from two-pass generation @xcite and retrieval-augmented dialogue models @xcite . generally, their objective can be formulated as p (r|c, r ) where r is a response either generated by the model itself or retrieved from the dataset. however, these works do not require r and r to be a minimal editing pair. by contrast, we formulate r e and r o to be a minimal editing pair. to encourage minimal editing, we construct response templates from the observed responses themselves, while these works derive templates from the r defined above. gme itself is also trained on a grounded dialogue dataset that has biased distribution. thus, as we mentioned at the beginning of section 7, we also need to evaluate the transferability of gme. section 7 shows that gme editing only slightly changes the distribution of the responses generated by the blender-90m variants, while the distribution of transfertransfo's responses is further away from the human references. this observation suggests that minimally editing out-of-domain responses is easier than generating them. while we focus on the persona, other types of grounding, e.g., knowledge and image, remain to be explored. many of gme's failure cases (see appendix e) contain grammatical errors or fail to correct contradictions, which could be addressed by improving the quality of response templates or incorporating stronger language model priors. ## conclusions we propose a framework named grounded minimal editing to address the transferability challenges of grounded dialogue modeling, which include the distributional gap and the grounding gap. our grounded minimal editor (gme) model achieves minimal editing by disentangling and recombining persona-related and persona-irrelevant expressions. for evaluation, we present the personaminedit dataset with multiple human references. experimental results show the effectiveness of gme for persona-grounded minimal editing. gme is also transferable to edit responses generated by pretrained dialogue models and improve their persona consistency while preserving their use of knowledge and empathy.