Regularizing Dialogue Generation by Imitating Implicit Scenarios (2020)

Abstract: Human dialogues are scenario-based and appropriate responses generally relate to the latent context knowledge entailed by the specific scenario. To enable responses that are more meaningful and context-specific, we propose to improve generative dialogue systems from the scenario perspective, where both dialogue history and future conversation are taken into account to implicitly reconstruct the scenario knowledge. More importantly, the conversation scenarios are further internalized using imitation learning framework, where the conventional dialogue model that has no access to future conversations is effectively regularized by transferring the scenario knowledge contained in hierarchical supervising signals from the scenario-based dialogue model, so that the future conversation is not required in actual inference. Extensive evaluations show that our approach significantly outperforms state-of-the-art baselines on diversity and relevance, and expresses scenario-specific knowledge.

## introduction neural dialogue generation has drawn increasing attention due to its vast commercial values and practical demands. typically, given the dialogue history, neural dialogue models, such as plain seq2seq model @xcite and transformer @xcite , learn to predict responses via maximum likelihood estimation @xcite . different from other sequence generation tasks, such as machine translation and paraphrase generation, the dialogue generation task can be regarded as a loose-coupling task, which has much freedom in the semantic and the linguistic aspects of the generated responses. however, it is often hard for the existing models to handle such freedom, compared to the fact that humans have no problem in i'm glad these cheap and cheerful batteries are on sale. i'm sorry. these batteries are not on sale. but the ad said they will be sold for a week. i'm glad these cheap and cheerful batteries are on sale. unfortunately, all types of batteries are costing more these days. could you cut the price a little, please? ## proposed approach in this section, we first introduce the scenariobased dialogue model, then describe the imitation learning framework shown in figure 2 , and finally present the training objective. ## related work diversified dialogue generation recently, various researches have focused on neural dialogue models to generate diverse, informative, and relevant responses. one line of research attempts to extract relevant contexts from redundant dialogue history accurately @xcite @xcite . another line of research tries to explicitly incorporate a latent variable to inject the variability of response in the decoding process @xcite . @xcite ; @xcite ; @xcite further enriched the latent variable approach. also, some works redesigned the objective function or automatically learned it by adversarial learning @xcite @xcite , which improves diversity but brings a fragile training process. finally, some researchers have adapted external knowledge, such as topic information @xcite , persona @xcite , knowledge base @xcite . unlike the above models to pre-dict responses given a dialogue history, our method combines the future conversation with the dialogue history as the implicit conversation scenario, which contains comprehensive background information to guide the response generation. imitation learning imitation learning, acquiring skills from observing demonstrations, has proven to be promising in structured prediction, such as alleviating the exposure bias problem @xcite , transferring knowledge to guide non-autoregressive translation model @xcite , and automatically learning the reward of the dialogue system @xcite . in our work, the conventional dialogue model as a student mimics the scenariobased dialogue model on both the output layer and intermediate layers. ## conclusion in this work, we introduce the future conversation with the corresponding dialogue history to learn the implicit conversation scenario, which entails latent context knowledge and specifies how people interact in the real world. to incorporate such scenario knowledge without requiring future conversation in inference, we propose an imitation learning framework. the scenario-based teacher model first learns to generate responses with access to both the future conversation and the dialogue history and then a conventional student model is trained to imitate the teacher by hierarchical supervisory signals. as a result, the student is effectively regularized to reach a robust local minimum that represents better generalization performance. evaluation on four datasets demonstrates the effectiveness and the scalability of our approach, compared to the state-of-the-art baselines. the proposed framework enables the generation of responses that pertain more closely to the scenario indicated by the given dialogue history. moreover, detailed analyses illustrate how imitating implicit scenarios regularizes the student model. for future work, we will incorporate pre-trained models into our framework (e.g., bert as a teacher and gpt as a student) to further unlock the performance improvement and explore how to balance diverse prior knowledge from multiple teachers.