Syntactically Diverse Adversarial Network for Knowledge-Grounded Conversation Generation (2021)

Abstract: Generative conversation systems tend to produce meaningless and generic responses, which significantly reduce the user experience. In order to generate informative and diverse responses, recent studies proposed to fuse knowledge to improve informativeness and adopt latent variables to enhance the diversity. However, utilizing latent variables will lead to the inaccuracy of knowledge in the responses, and the dissemination of wrong knowledge will mislead the communicators. To address this problem, we propose a Syntactically Diverse Adversarial Network (SDAN) for knowledge-grounded conversation model. SDAN contains an adversarial hierarchical semantic network to keep the semantic coherence, a knowledge-aware network to attend more related knowledge for improving the informativeness and a syntactic latent variable network to generate syntactically diverse responses. Additionally, in order to increase the controllability of syntax, we adopt adversarial learning to decouple semantic and syntactic representations. Experimental results show that our model can not only generate syntactically diverse and knowledge-accurate responses but also significantly achieve the balance between improving the syntactic diversity and maintaining the knowledge accuracy.

## introduction nowadays, conversation generation has become a research hotspot because of its wide application, such as voice assistant, customer service assistant and chat robot @xcite . the goal of conversation model is to generate diverse and informative responses like human. although the existing models have achieved promising performance, they still suffer from generating general and meaningless responses @xcite , which significantly disrupt the user experience. consequently, it is * jinan xu is the corresponding author. to generate high-quality responses, many researches have been proposed to improve informativeness or diversity of responses. for informative responses, some early studies utilize context information to the decoding process @xcite . after that, researchers extract topic information from context @xcite or add external topic to the decoder @xcite . lately, researchers focus on fusing knowledge into conversation model @xcite @xcite @xcite . although the knowledge-grounded model can generate informative responses with accurate knowledge, which may generate responses that lack diversity. for diverse responses, previous studies generally adopt beam search algorithm @xcite and its variants to improve diversity @xcite . in recent year, latent variables are widely used in conversation model, and can significantly enhance the diversity @xcite @xcite @xcite , and generative adversarial networks (gan) @xcite and reinforcement learning (rl) @xcite are also adopted to generate diverse responses. although the introduction of hidden variables can increase diversity while maintaining semantic consistency, it may lead to inaccuracy in decoding specific knowledge, because the latent variables may generate semantically similar responses with a certain probability. for example, as shown in to tackle this challenge, we propose a syntactically diverse adversarial network (sdan) for knowledge-grounded conversation generation. first, we utilize a hierarchical network to model the semantic information of context and an adversarial network to prevent semantic information from affecting syntactic information. next, we adopt a knowledge-aware network to represent the knowledge related to the query, which takes attention mechanism to capture more important knowledge. then, we design a syntax encoder to model syntax information and use a latent variable to keep the syntactic diversity. finally, the encoded knowledge, syntax and context are concatenated together to initialize the decoder. additionally, we employ adversarial network to keep the separation of syntax and semantics to prevent their mutual influence. the results of experiments on kdconv datasets show that our model can achieve better trade-off between improving diversity and maintaining knowledge accuracy than baselines. our main contributions are as follows: ## related work sequence-to-sequence (seq2seq) model @xcite with attention @xcite has been widely used in the conversation generation. however, models tend to generate meaningless and generic responses @xcite . to alleviate this issue, researchers have utilized context @xcite , topic information @xcite @xcite or knowledge @xcite @xcite to enhance response quality. the studies of knowledge-grounded conversation generation mainly focus on the method of knowledge retrieval @xcite or knowledge fusion @xcite @xcite with copy mechanism. the knowledge-grounded models can improve the ac-curacy of knowledge, but the responses generated by some of them may lack the diversity, which is also a significant reason for generating generic responses. recently, to tackle the lack of diversity, researchers have begun to introduce the beam search algorithm @xcite to decoder or latent variables @xcite @xcite . adopting latent variables can significantly improve the diversity of responses, but it will lead to the inaccuracy of knowledge. to the best of our knowledge, this problem has not been investigated in conversation generation so far. different from all the models mentioned above, our approach introduces syntax to conversation generation. we propose a syntactically diverse adversarial network, which utilizes latent variables to control the syntactic diversity. additionally, we utilize adversarial learning to preserve the disentanglement of syntax and semantics for preventing them from influencing each other. our model can not only generate sentences with diverse syntax but also keep the accuracy of knowledge. ## conclusion in this paper, we propose a syntactically diverse adversarial network for knowledge-grounded conversation model, which utilizes adversarial hierarchical semantic network, knowledge-aware network and syntactical latent variable network to model the semantics, knowledge and diverse syntax information. moreover, our model adopts adversarial learning to enhance the controllability of syntax. according to automatic and manual evaluation, our model competitively improves the quality of generated responses, and obtains better trade-off between improving the diversity and preserving the knowledge accuracy.