Think Before You Speak: Learning to Generate Implicit Knowledge for Response Generation by Self-Talk (2021)

Abstract: Humans make appropriate responses not only based on previous dialogue utterances but also on implicit background knowledge such as common sense. Although neural response generation models seem to produce human-like responses, they are mostly end-to-end and not generating intermediate grounds between a dialogue history and responses. This work aims to study if and how we can train an RG model that talks with itself to generate implicit knowledge before making responses. We further investigate can such models identify when to generate implicit background knowledge and when it is not necessary. Experimental results show that compared with models that directly generate responses given a dialogue history, self-talk models produce better-quality responses according to human evaluation on grammaticality, coherence, and engagingness. And models that are trained to identify when to self-talk further improves the response quality. Analysis on generated implicit knowledge shows that models mostly use the knowledge appropriately in the responses.

## introduction end-to-end response generation (rg) models based on pre-trained transformer-based @xcite language models (lm) have shown to produce human-like responses @xcite @xcite . humans, however, not only "just utter the right sentence", but also contribute to the common ground consisting of mutual beliefs and common knowledge "whose truth is taken for granted as part of the background of the conversation" @xcite @xcite . for example, consider the utterance "i need to buy some flowers for my wife", a potential appropriate response is "perhaps you'd be interested in red roses". to produce this response, the participant needs to understand relevant background knowledge such as "rose is a type of flower". note that this background knowledge is implicit in the dialogue turns, meaning that an end-to-end rg system that takes utterances as input and outputs responses will omit this intermediate process that is crucial for humans in communications. to fill the gap between current model's rg process and how humans use implicit background knowledge in conversations, we focus on commonsense inference as a type of background knowledge and aim to train rg models that first generate implicit commonsense inferences before producing a response given previous utterances. inspired by inquiry-based discovery learning @xcite and the self-talk procedure @xcite , we encourage the rg model to talk with itself to elicit implicit knowledge before making a response. collecting training data is challenging for our purpose since commonsense knowledge is by definition often omitted in conversations. we use con-ceptnet triples @xcite as the knowl- * work done while pei zhou was an intern at amazon alexa ai edge schema to represent knowledge and align common sense-focused dialogues @xcite with knowledge to train rg models. we conduct extensive human evaluation on different variants of our training procedure and find that models that generate implicit background knowledge before responding produce more grammatical, coherent, and engaging responses compared to rg models that directly generate responses. further analysis shows that models can sometimes even learn to distinguish when it is necessary to self-talk to generate implicit knowledge, i.e., be aware of potential knowledge gaps in dialogues. we hope our findings encourage more future studies on making rg models better emulate human communication process and produce better-quality responses. ## task formulation and setup given a dialogue history (multi-turn or single-turn) u , the task is to generate implicit background knowledge i and then produce a response r given both u and i. extending most neural rg models that treat this as a conditional language modeling problem, we aim to learn the conditional probability distribution p (i, r|u ) by training on human dialogues. ## preliminary results for evaluation, we randomly sample 300 instances from unseen test dialogues for human evaluation. we provide a dialogue history and two model responses and ask three workers on amazon mechanical turk (amt) platform to select which one is better or not sure. we consider three criteria following most previous rg evaluation: which response is more grammatical, coherent, and engaging. we generally find trained models follow the template of first generating implicity knowledge and then respond. some example outputs to the utterance "i need to buy some flowers for my wife" are "(buy is related to expensive) can you afford the expensive flowers?" and "(buy is related to money) how much money do you want to spend?". self-talk models produce higher quality responses than vanilla rg models