Sketch-Fill-A- R : A Persona-Grounded Chit-Chat Generation Framework (2020)

Abstract: Human-like chit-chat conversation requires agents to generate responses that are fluent, engaging and consistent. We propose Sketch- Fill-A-R, a framework that uses a persona-memory to generate chit-chat responses in three phases. First, it generates dynamic sketch responses with open slots. Second, it generates candidate responses by filling slots with parts of its stored persona traits. Lastly, it ranks and selects the final response via a language model score. Sketch-Fill-A-R outperforms a state-of-the-art baseline both quantitatively (10-point lower perplexity) and qualitatively (preferred by 55% in head-to-head single-turn studies and 20% higher in consistency in multi-turn user studies) on the Persona-Chat dataset. Finally, we extensively analyze Sketch-Fill-A-R’s responses and human feedback, and show it is more consistent and engaging by using more relevant responses and questions.

## introduction chit-chat is a rich domain that challenges machine learning models to express fluent natural language and to successfully interact with other agents. chitchat stands in contrast to goal-oriented dialogue, such as when a customer has the explicit goal of booking a flight ticket. when agents communicate, they each have internal state (e.g., their knowledge, intent) and typically have limited knowledge of the state of other agents @xcite . as a result, human-like chit-chat requires agents to be fluent, engaging and consistent with what has been said and their persona @xcite . these requirements make learning generative chit-chat models a complex task. first, given an existing conversation history, there may be a large number of valid responses @xcite . hence, supervised learning of chit-chat models that cover a large number of topics and styles requires a significant amount of data @xcite . second, as conversations progress and more opportunities for contradiction arise, maintaining consistency becomes more difficult @xcite . third, engaging chit-chat responses follow conversational structures that are not captured well by perplexity @xcite . indeed, our human user studies show that both consistency and engagingness are only weakly correlated with perplexity, and fluency is not at all. we propose sketch-fill-a-r, a dialogue agent framework that can learn to generate fluent, consistent and engaging chit-chat responses. our key motivation is the hypothesis that human-like chit-chat responses often 1) follow common conversational patterns with insertions of agent-specific traits, and 2) condition explicitly on those persona traits. sketch-fill-a-r decomposes response generation into three phases: sketching, filling and ranking, see figure 1 . first, sketch-fill-a-r dynamically generates a sketch response with slots, which enables it to learn response patterns that are compatible with many specific persona traits. second, it generates candidate responses by filling in slots with words stored in memory. this enables sketch-fill-a-r's responses to adhere to its persona. third, the candidate responses are ranked by perplexity under a pre-trained language model (lm), which encourages the final response (with lowest lm perplexity) to be fluent. in sum, our contributions are as follows: ## related work chit-chat dialogue dialogue agents such as amazon alexa, apple siri, and google home are commonplace today, and are mainly task-oriented: they help users achieve specific tasks. on the other hand, microsoft xiaoice @xcite is an example of an undirected chit-chat dialogue agent. historically task-oriented dialogue systems are composed via components such as dialogue state tracking and natural language generation (jurafsky and @xcite . even now, the natural language generation component often uses handcrafted templates and rules defined by domain experts that are filled via heuristics @xcite . more recently task-oriented dialogue systems have been trained end-to-end @xcite , but these systems have specific user intents they aim to fulfill, and so represent a more constrained task. early conversational dialogue systems such as eliza @xcite and alice @xcite were also based on hand-crafted rules and thus brittle. to alleviate this rigidity, more recent neural seq2seq models @xcite are trained end-to-end @xcite @xcite . to help guide conversation @xcite @xcite incorporated knowledge-grounded datasets, while @xcite created the persona-chat dataset used in this work. sketch-fill-a-r dynamically generates slot sketches and bears resemblance to @xcite which assumed data are structured domain-specific triplets and contexts follow templates. however, sketch-fill-a-r does not assume the personas and responses have rigid syntactic structure, and introduces a ranking procedure. converse to our sketch-and-fill procedure, @xcite train a model to select a persona trait and decode around the trait. finally, @xcite also re-rank by scoring utterances with natural language inference to improve consistency. neural sequence models sketch-fill-a-r extends a neural encoder-decoder structure @xcite but is agnostic to the chosen form of encoder-decoder. in this work we use recurrent models and attention @xcite , which auto-regressively embed and generate sequences, but our framework is general, allowing non-recurrent encoders and decoders like transformer networks with non-recurrent self-attention @xcite to be substituted for the recurrent encoder and decoder. sketch-fill-a-r uses a simple memory module to store words from personas, which act as context for generation. @xcite ; @xcite introduced learned key-value memory networks, while @xcite introduced dynamic memory nets for question-answering via an iterative attention over memory. also, sketch-fill-a-r decodes responses using a re-ranking strategy based on language model scores, which complements strategies in @xcite . ## sketch-fill-a-r our key motivation is to generate human-like chitchat responses that are conditioned on personarelevant information. sketch-fill-a-r generates chit-chat using a persona-memory to dynamically generate sketches that capture conversational patterns, and inserting persona-relevant information. to set notation: capitals w, v, . . . denote matrices, i, j, k are vector-matrix indices and x, y, . formally, we aim to learn a response generation model that predicts words y u using a probability distribution p (y 0:u |x 0:t ; θ) over sequences of t words and n persona traits with r rare words. here u is the output sequence length and θ are the model weights. we use deep neural networks, a model class that has recently seen success in language generation tasks @xcite . ## empirical validation to validate sketch-fill-a-r, we first show that it achieves better supervised learning performance than baselines on a chit-chat dialogue dataset. persona-chat dataset we trained sketch-fill-a-r to generate single-turn agent responses on the persona-chat dataset @xcite , which contains 10,907 dialogues. here, a dialogue consists of multiple turns: a single turn contains the utterance of a single agent. we processed this dataset into training examples that each consist of the conversation history x c t , set of persona traits x p t of the model, and the ground truth sketch response y u . this process yielded 131,438 training examples. rare words were identified by removing all punctuation and stop words from the set of persona traits (see appendix for more information). ground truth sketch responses were then constructed by replacing all rare word instances in ground truth responses with @persona tags. language model pre-training sketch-fill-a-r uses a transformer-based gpt @xcite pre-trained on the books text corpus @xcite to rank candidate responses with filled @persona slots according to their lm-perplexity scores. for model details, see the appendix. ## user study and qualitative analysis although sketch-fill models perform well quantitatively, a crucial test is to evaluate how well they perform when judged by human users on conversational quality, which is not explicitly captured by perplexity. we performed single and multi-turn dialogue user studies to assess the quality of sketch-fill-a-r, rated along several dimensions: our definition of engagingness includes relevance, defined in pragmatics and relevance theory @xcite as a statement leading to positive cognitive effect. however an engaging statement may be ironic @xcite , humorous, or specific to individuals. we also explore which qualities of sketch-fill-a-r's outputs are correlated with human ratings and perplexity scores. our results suggest that: ## appendix 7.1 ethical implications during experiments, we identified a number of ethical implications for future work. the persona-chat dataset was noted by some raters to contain potentially inappropriate statements (e.g., "my wife spends all my money") and is based in us culture (e.g., food, music, cars, names). it also lacked content to fail gracefully when it didn't have an appropriate response (e.g., "i'm sorry i don't understand," "i don't know"). as such, learned model responses were occasionally insensitive and confusing to human users.