Stylized Knowledge-Grounded Dialogue Generation via Disentangled Template Rewriting (2022)

Abstract: Current Knowledge-Grounded Dialogue Generation (KDG) models specialize in producing rational and factual responses. However, to establish long-term relationships with users, the KDG model needs the capability to generate responses in a desired style or attribute. Thus, we study a new problem: Stylized Knowledge-Grounded Dialogue Generation (SKDG). It presents two challenges: (1) How to train a SKDG model where no <context, knowledge, stylized response> triples are available. (2) How to cohere with context and preserve the knowledge when generating a stylized response. In this paper, we propose a novel disentangled template rewriting (DTR) method which generates responses via combing disentangled style templates (from monolingual stylized corpus) and content templates (from KDG corpus). The entire framework is end-to-end differentiable and learned without supervision. Extensive experiments on two benchmarks indicate that DTR achieves a significant improvement on all evaluation metrics compared with previous state-of-the-art stylized dialogue generation methods. Besides, DTR achieves comparable performance with the state-of-the-art KDG methods in standard KDG evaluation setting.

## introduction every good conversational agent needs the ability to generate good responses, which are not only knowledgeable and coherent with contexts but also have abundant and desirable styles and sentiments @xcite @xcite . such an agent can deliver depth dialogues on various topics and yield more engaging and vivacious conversations to attract more users. in other words, rational and perceptual thought are all necessary for a perfect dialogue agent. nevertheless, most existing knowledge-grounded dialogue generation (kdg) methods @xcite figure 1 : the kdg models only produce a pedantic response, which lacks emotion and attraction compared with the responses with polite style, positive and negative sentiments. @xcite pay more attention to the former and ignore the latter. specifically, let's claim our motivation: the previous kdg works mainly focus on selecting knowledge and expressing knowledge in response accurately. however, the excessive emphasis on knowledge makes the kdg models tend to mechanically copy large sections from the unstructured knowledge (e.g., wikipedia). as a result, the responses from the kdg models reflect a "pedantic" style (i.e., use very technical terms and language), making the conversation less engaging and less natural. in this paper, we are aiming to have the first attempt to incorporate stylized-text-generation into kdg to tackle the above challenge. as shown in figure 1 , the kdg model takes the context and related document as input and outputs a knowledgeable but pedantic response corresponding to the polite one, which makes people feel respected and comfortable. in the meanwhile, the polite, positive responses all show bright and lively styles which not only are able to condense the core meaning of the response, but also sound appealing to the users for more exposure and memorableness. specifically, we formulate a new problem: styliron man's perseverance and technology are incredible i dislike cruel walter white in horrible breaking bad. what makes us fall in love with a toy? iron man's incredible perseverance is admirable. i fall in love with breaking bad and incredible walter white. negative fragments positive fragments knowledge fragments kdg response: skdg response: positive corpus figure 3: our model combines the style-related fragments from the style corpus and the knowledge fragments from the kdg response to generate a response in a desired sentiment or style. to achieve this goal, we propose a new paradigm: generate-disentangle-rewrite. firstly, given a dialogue context and the associated external knowledge, a kdg model is adopted to generate a response. then as shown in figure 2 and 3, we leverage a sequential style disentangler to delete stylerelated fragments from the kdg response to form a style-agnostic template. then the rewriter rewrites the entire template token-by-token, injecting stylerelated fragments in the process, to generate a vivid and informative response in the desired style. as there is no supervision on the style disentangler and the style rewriter, we propose a reinforcement learning-based method to train the style disentangling and style rewriting in an end-to-end manner using a style intensity reward and a semantic similarity reward. the huge joint action space of the two modules fragile the training, thus we propose a novel weakly supervised stylistic template disentangle method to initialize both the disentangler and the rewriter. as a result, our method successfully produces the knowledgeable response in the desired style without any paired training data. we name our model dtr standing for "disentangled template rewriting". we demonstrate this approach using knowledge-grounded dialogues from wizard of wikipedia @xcite and topical chat @xcite with three sets of sentences with distinct sentiments (positive, negative) and styles (polite). automatic and human evaluations show that our method significantly outperforms competitive baselines with a large margin in generating coherent and knowledgeable dialogue responses while rendering stronger stylistic features. our contributions are three-fold: (1) to the best of our knowledge, it is the first work on the generation of stylized knowledge-grounded responses without any labeled paired data for style-specific context-knowledge-response. ( 2 ) we proposed a stylized knowledge-grounded dialogue generation method via disentangled template rewriting. to optimize the model, we propose a reinforcement learning approach with a novel weakly supervised method to guide the learning of both disentangler and rewriter. (3) extensive experiments on two benchmarks indicate that dtr significantly outperforms previous state-of-the-art sdg methods on all evaluation metrics. besides, dtr achieves comparable performance with the state-of-the-art kdg methods in the standard kdg evaluation setting. our source code will be released at https:// github.com/victorsungo/skdg-dtr . zheng et al. (2020) captures stylistic features embedded in unpaired texts, su et al. (2020b) uses the pointwise mutual information (pmi) to determine stylistic word, ze et al. (2020) adopts pre-trained models to tackle the opendomain stylized response generation. we propose a novel disentangled template rewriting approach as the first attempt to study stylized knowledge-grounded dialogue generation without any supervised style-specific context-knowledgeresponse triples data. ## approach heading for learning an effective disentangled template rewriting model for skdg task, we need to deal with several challenges: (1) how to distinguish the style-related fragments from a given sentence without any supervision;(2) how to retain the stylerelated fragments in knowledge section to defend the completeness;(3) how to rewrite the disentangled template holistically instead of inserting a few style words, thus to enhance fluency and diversity. our dtr model is made up of a knowledgegrounded response generator g g , a sequential style disentangler f and a style rewriter g r . given a dialogue context u and its associated knowledge k, we first use g g to generate a response y . figure 2 illustrates the cooperation of f and g r . the former reads y and disentangles the style-related content from y to form a style-agnostic template sequence ỹ , which is further provided as input to g r to generate the transferred response ŷ in a target style. since ỹ is discrete, the major hinder of learning f lies in the gradient is not differentiable. to cope with the challenge, we exploit a reinforcement learning approach to optimize f leveraging the signals from g r . so why do we need a disentangler + rewriter architecture? the previous sdg methods fuse the knowledge and style into mixed representation and decode a response. due to the difficulty of mixing knowledge and style implicitly under the unsupervised setting, it is possible to lose knowledge or style in the decoding stage. motivated by this, we propose to decouple the response generation into two relatively independent processes: 1) knowledge fragments generation 2) style fragments generation. the knowledge fragments and style fragments are explicitly composited into the response in the final stage. such a method ensure the knowledge is successfully presented in the final output. the disentangler plays a central role in decoupling and composition. in the following, we will elaborate details of each component. ## experiments we conduct experiments on wizard of wikipedia (wizard) and topical chat with positive and negative sentiments, and polite style. ## conclustion we explore stylized knowledge-grounded dialogue generation by proposing bridging the knowledgegrounded response generation with the stylized rewriting via sharing a disentangled template. evaluation results on benchmarks of the task indicate that our model can achieve state-of-the-art performance and exhibits a superior generation ability over different knowledge domains and styles.