Generate, Delete and Rewrite: A Three-Stage Framework for Improving Persona Consistency of Dialogue Generation (2020)

Abstract: Maintaining a consistent personality in conversations is quite natural for human beings, but is still a non-trivial task for machines. The persona-based dialogue generation task is thus introduced to tackle the personality-inconsistent problem by incorporating explicit persona text into dialogue generation models. Despite the success of existing persona-based models on generating human-like responses, their one-stage decoding framework can hardly avoid the generation of inconsistent persona words. In this work, we introduce a three-stage framework that employs a generate-delete-rewrite mechanism to delete inconsistent words from a generated response prototype and further rewrite it to a personality-consistent one. We carry out evaluations by both human and automatic metrics. Experiments on the Persona-Chat dataset show that our approach achieves good performance.

## introduction in an open-domain conversation scenario, two speakers conduct open-ended chit-chat from the initial greetings and usually come to focus on their characteristics, such as hobbies, pets, and occupations, etc., in the course of the conversation. for humans, they can easily carry out conversations according to their personalities @xcite , but fulfilling this task is still a challenge for recent neural dialogue models @xcite . one main issue is that these models are typically trained over millions of dialogues from different speakers, and the neural dialogue models have a propensity to mimic the response with the maximum likelihood in the training corpus @xcite , which results in the frequent inconsistency in responses @xcite . another issue * this work was done when the first author was an intern at tencent ai lab. wei-nan zhang is the corresponding author. figure 1 : a common problem for persona-based dialogue models is that they can hardly avoid the generation of inconsistent persona words. although the model generates a response which looks good, it is an inconsistent one. with further rewriting, the model can focus more on improving persona consistency. to alleviate the above issues, @xcite introduced the persona-chat dataset to build more consistent dialogue models. different from conventional dialogue corpora, this dataset endows dialogue models with predefined personas, which is in the form of textually described profile (as shown in the first line of figure 1 ). the persona-based dialogue models also adopt an encoder-decoder architecture and are enhanced with persona encoding components, such as memory network @xcite and latent variable @xcite . these models turn out to produce more consistent responses than the persona-free ones @xcite . despite the successful application of the encoderdecoder framework in persona-based dialogue models, one concern is that they lack extra attention to the key persona information. the model will learn to minimize the overall loss of every decoded word, but this may lead to the neglect of the key personas: change of one persona-related word may not significantly affect the overall loss, but could turn a good response into a totally inconsistent one. as shown in stage 1 of figure 1 , only one improper word "colorado" leads the response to be inconsistent. a desirable solution should be able to capture personas and automatically learn to avoid and refine inconsistent words before the response. in this paper, we present a generate-delete-rewrite framework, gdr, to mitigate the generation of inconsistent personas. we design three stages specifically for the goal of generating persona consistent dialogues: the first generate stage adopts a transformer-based generator to produce a personabased response prototype. the second delete stage employs a consistency matching model to identify inconsistencies and delete (by masking) the inconsistent words from the prototype. finally, in the rewrite stage, a rewriter polishes the masked prototype to be more persona consistent. to examine the effectiveness of our gdr model, we carried out experiments on the public available persona-chat dataset @xcite . we summarize the main contributions as follows: ## related work end-to-end dialogue generation approaches are a class of models for building open-domain dialogue systems, which have seen growing interests in recent years @xcite @xcite @xcite . these dialogue models adopted recurrent units in a sequence to sequence (seq2seq) fashion @xcite . since the transformer has been shown to be on par with or superior to the recurrent units @xcite , some dialogue models began to take advantage of this architecture for better dialogue modeling @xcite . besides the advancements in dialogue models, the emergence of new dialogue corpus has also contributed to the research field. @xcite introduced the persona-chat dataset, with explicit persona texts to each dialogue. based on seq2seq model and memory network, they further proposed a model named generative profile memory network for this dataset. following this line, @xcite designed the deepcopy model, which leverages copy mechanism to incorporate persona texts. @xcite integrated persona texts into the per-cvae model for generating diverse responses. however, the persona-based models still face the inconsistency issue @xcite . to model the persona consistency, @xcite annotated the persona-chat dataset and introduced the dialogue natural language inference (dnli) dataset. this dataset converts the detection of dialogue consistency into a natural language inference task @xcite . personalized dialogue generation is an active research field @xcite @xcite @xcite . in parallel with this work, @xcite leveraged adversarial training to enhance the quality of personalized responses. @xcite incorporated mutual persona perception to build a more explainable @xcite dialogue agent. other relevant work lies in the area of multi-stage dialogue models @xcite . some retrieval-guided dialogue models @xcite @xcite ) also adopted a multi-stage framework, but the difference from our work is obvious: we generate the prototype rather than retrieve one. a high-quality retrieved response is not always available, especially under the persona-based setting. ## conclusion and future work in this paper, we presented a three-stage framework, generate-delete-rewrite, for persona consistent dialogue generation. our method adopts transformer architecture and integrates a matching model to delete the inconsistent words. experiments are carried out on public-available datasets. both human evaluations and automatic metrics show that our method achieves remarkably good performance. in the future, we plan to extend our approach to improve the consistency of multi-turn dialogues.