A T hree- S tage L earning F ramework for L ow- R esource K nowledge- G rounded D ialogue G eneration (2021)

Abstract: Neural conversation models have shown great potentials towards generating fluent and informative responses by introducing external background knowledge. Nevertheless, it is laborious to construct such knowledge-grounded dialogues, and existing models usually perform poorly when transfer to new domains with limited training samples. Therefore, building a knowledge-grounded dialogue system under the low-resource setting is a still crucial issue. In this paper, we propose a novel three-stage learning framework based on weakly supervised learning which benefits from large scale ungrounded dialogues and unstructured knowledge base. To better cooperate with this framework, we devise a variant of Transformer with decoupled decoder which facilitates the disentangled learning of response generation and knowledge incorporation. Evaluation results on two benchmarks indicate that our approach can outperform other state-of-the-art methods with less training data, and even in zero-resource scenario, our approach still performs well.

## introduction neural dialogue systems have made rapid progress in recent years thanks to the advances in sequence generation technology @xcite . though such models in neural architectures are able to reply with plausible responses regarding to dialogue history, people can still feel a clear gap when they converse with the chatbots, compared with the conversation with humans. to bridge the gap and generate fluent and informative responses, a number of approaches have been proposed by leveraging external knowledge. knowledge-grounded dialogue is a task of generating an informative response based on both dialogue history and a collection of external knowledge @xcite . the forms of knowledge are diverse, and in this work, we only focus on knowledge in the form of unstructured documents. generally, it is difficult to construct large scale conversations that are naturally grounded on the documents for learning of a response generation model @xcite , and most of the previous methods @xcite @xcite perform poorly when transfer into a new domain with limited training samples. so there are growing appeals for lowresource dialogue response generation, which aims to leverage past experience to improve the performance with limited labeled training examples of target corpus. to address this issue, we envisage to absorb useful information from other easily accessible heterogeneous datasets to enhance the performance of the knowledge-based dialogue model under lowresource setting. based on this assumption, we propose a novel three-stage learning framework (tslf). tslf attempts to divide the parameters of a model into dialogue-related and knowledge integration-related. in the first stage, we use supervised learning to pre-train dialogue-related parameters on general dialogues (e.g., online forum comments), and perform domain-adaptive pre-training @xcite to initialize knowledgerelated parameters on unlabeled knowledge base (e.g., items in wikipedia). in the second stage, inspired by the distant supervision in the relation extraction @xcite , we match a set of pseudo-knowledge for each ungrounded dialogue to construct a lower quality knowledge-grounded dialogue dataset, and further co-pretrain the above two groups of parameters on this dataset. in the third stage, the trained model will be fine-tuned on the target low-resource dataset. the flow of tslf is shown in figure 1 . in order to better cooperate with the disentangled learning mechanism in tslf, we devise knowledge-aware transformer (kat), a variant of vanilla transformer @xcite whose parameters are decoupled that facilitates the separate learning of dialogue generation and knowledge incorporation. as shown in figure 2 , besides dialogue history, kat also accepts a set of knowledge as additional input. kat has a knowledgeaware decoder which could obtains information from the dialogue context and background documents through cross-attention and integrates them through a controller. we conduct experiments on two knowledgegrounded dialogue generation benchmarks including wizard-of-wikipedia @xcite and cmu_dog @xcite . evaluation results in terms of both automatic metrics and human judgment indicate that using only about 1/4 of the training data on wizard (1/16 on cmu_dog), the performance of our approach outperforms the competitive baselines which are learned from full crowdsourced training corpora. even without using any training data of the target dataset, our method still performs well. the contributions in this work are summarized as follows: (1) we propose a novel three-stage learning framework that leverages weakly supervised learning to help build a low-resource knowledgegrounded dialogue generation model; (2) we devise knowledge-aware transformer, a knowledgegrounded neural conversation model with a novel dynamic knowledge selection mechanism, which can fully exploits the external knowledge to generate fluent and informative dialogue responses; (3) our kat-tslf achieves surprising performance under the scenarios of full data, low-resource and even zero-resource. the source code is available at https:// github.com/neukg/kat-tslf . ## approach low-resource knowledge-grounded dialogue generation is task that requires a method to learn from experience e, which consists of direct experience e d containing limited monolingual contextknowledge-response triples and indirect experience e i , to improve the performance in response generation measured by the evaluation metric p . the direct experience e d refers to the training samples of target corpus ## related work open domain end-to-end dialogue response generation is inspired by the success of applying neural sequence to sequence models on machine translation @xcite @xcite . very recently, in order to generate fluent, coherent and informative response, many approaches have been proposed by introducing external background documents @xcite @xcite . besides documents @xcite , the are many forms of knowledge such as images @xcite and triples in knowledge graph @xcite . @xcite presents to divide knowledgegrounded dialogue into two steps: knowledge selection and dialogue generation. postks @xcite , skt @xcite , pipm @xcite and skt-kg @xcite use the prior and posterior distribution of knowledge to improve the accuracy of knowledge selection. @xcite devise a reinforcement learning method to train a knowledge selector without ground-truth knowledge label. deepcopy @xcite , itdd @xcite and kic @xcite have improved the structure of the decoder so that it can better integrate knowledge. since knowledgeguided dialogue corpora need to be constructed through crowdsourcing, the size of datasets such as wizard-of-wikipedia @xcite are relatively small. @xcite and @xcite proposed to conduct the knowledge-grounded conversation under the low-resource and zero-resource settings respectively. we do not compare with @xcite ; @xcite since they did not release their entire source codes. our three-stage learning framework is inspired by @xcite , which uses ungrounded dialogues and unstructured documents to train a knowledge-grounded dialogue model that can work in low-resource situations. in addition, the design of stage ii is inspired by distant supervision technology in relation extraction task @xcite . the idea of kat is also encouraged by disentangled decoder @xcite and the recent breakthrough in variants of transformer @xcite @xcite . ## conclusion we study knowledge-grounded dialogue generation under a low-resource setting by proposing a threestage learning framework and a knowledge-aware transformer. evaluation results on two benchmarks indicate that our model achieves the stateof-the-art performance with less training data. besides, kat-tslf exhibits a good generalization ability on zero-resource scenario.