Learning to Express in Knowledge-Grounded Conversation

Grounding dialogue generation by extra knowledge has shown great potentials1 towards building a system capable of replying with knowledgeable and engaging2 responses . Existing studies focus on how to synthesize a response with proper3 knowledge , yet neglect that the same knowledge could be expressed differently4 by speakers even under the same context . In this work , we mainly consider5 two aspects of knowledge expression , namely the structure of the response and6 style of the content in each part . We therefore introduce two sequential latent7 variables to represent the structure and the content style respectively . We propose8 a segmentation-based generation model and optimize the model by a variational9 approach to discover the underlying pattern of knowledge expression in a response.10 Evaluation results on two benchmarks indicate that our model can learn the structure11 style defined by a few examples and generate responses in desired content style.12 1 Introduction13 Human-machine conversation is a long-standing goal of artificial intelligence ( AI ) . In the past14 few years , with advances in deep learning [ 28 , 5 , 31 ] and availability of huge amount of human15 conversations on social media [ 1 ] , building an open domain dialogue system with data-driven16 approaches has attracted increasing attention from the community of AI and NLP . By synthesizing a17 response with text generation techniques [ 32 ] , current natural models are able to naturally reply to18 user prompts . Despite the impressive progress , existing generation models are notorious for replying19 with generic and bland responses , resulting in meaningless and boring conversations [ 13 ] . Such20 deficiency is particularly severe when human participants attempt to dive into specific topics in21 conversation [ 3 ] .22 To bridge the gap , some researchers resort to ground dialogue generation by extra knowledge such as23 unstructured documents [ 49 , 3 ] . By this means , the documents ( e.g. , wiki articles ) serve as content24 sources and make a dialogue system knowledgeable regarding various concepts in a discussion.25 However , existing studies focus on how to synthesize a response with proper knowledge [ 3 , 11 , 45 ] ,26 but pay little attention to the fact that the same knowledge could be expressed differently even under27 the same context . These models usually employ a regular decoder to generate the response in an28 auto-regressive manner given the contextual representations of knowledge and dialogue context,29 which makes the generation process less explainable and controllable.30 In general , we break down the expression style of a response into two components : the structure of31 the response and the style of the content in each part . First , the knowledge expression in response32 varies in structure , including but not limited to the position and the length of knowledge expression.33 As the example shown in Table 1 , knowledge-related phrases and clauses could be long , like “ And I ’ d34 give credit to three different voice actors for anna. ” , or short , like “ 74 in Metacritics ” . Besides , they35 may appear at the beginning of the sentence , or at the end . For the sake of description , we decompose36 a response into a sequence of non-overlapping segments , each is either related to certain background37 Submitted to 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Do not distribute . knowledge and diverse in content style , or almost irrelevant to the knowledge but simply playing the38 role of stitching the context and carrying on the conversation . We therefore define the structure style39 as the distribution and number of two kinds of segments . Structure style itself is far from dominant40 in the sentence expression , since different speakers could convey converse attitude even the context41 and the knowledge are exactly the same , as shown in Table 1 . So it is necessary to introduce the42 content style as the expression fashion within each knowledge-related segment . We further introduce43 two latent variables to facilitate end-to-end training , one for predicting the start and end positions44 of a segment , the other for deciding the category of each segment . Since the human annotations for45 sentence segmentation are absent and enumerating over all possibilities to maximize the likelihood46 of the response is time-consuming , we propose a variational framework for segmentation-based47 generation and induce an evidence lower bound of the likelihood.48 Formally , our model is on the basis of encoder-decoder architecture . The encoder is to obtain the49 contextual representation of conversational context and knowledge in a regular way . The decoder50 consists of three types of modules : ( 1 ) a context module , for response only based on context without51 knowledge ; ( 2 ) a plain-knowledge module , for response referring knowledges but without particular52 style ; and ( 3 ) one or more stylized-knowledge module , for response referring knowledges and with a53 specific style . The context module is the only module not relying on knowledge , but simply paying54 attention to contextual information . Compared with plain-knowledge module , stylized-knowledge55 module has unique adapters , which is their primary discrepancy . When decoding , the decoder first56 predicts the segmentation of the response and then makes a choice in three kinds of modules to57 generate a single segment . Both the segmentation and the module selection are instructed under58 sequential latent variables.59 We train our model on the Reddit Corpus published by [ 15 ] and evaluate our model on two bench-60 marks of knowledge-grounded conversation : Wizard of Wikipedia ( Wizard ) [ 3 ] and CMU Document61 Grounded Conversation ( CMU_DoG ) [ 49 ] . Evaluation results indicate that our model can significantly62 outperform state-of-the-art methods in the zero-resource setting ( i.e. , only trained on the Reddit63 Corpus ) . In addition , the performance of our model improves significantly on Wizard and CMU_DoG64 with the presence of only 10 % training data and the segment distributions after fine-tuning are consis-65 tent with our prior knowledge about the two datasets , indicating that our model can learn the structure66 style with little cost . Finally , our model outperforms previous state-of-the-art models on the accuracy67 of performing sentiment classification using generated responses . It is worth noting that our model68 achieves 10 % + accuracy improvement on Wizard Seen , 12 % + accuracy improvement on Wizard69 Unseen , and 12 % + accuracy improvement on CMU_DoG than the present state-of-the-art model,70 which indicates that the model can be controlled to express knowledge with the desired content style.71 Contributions in this work are three-fold : ( 1 ) exploration the knowledge expression in knowledge-72 grounded conversation ; ( 2 ) proposal of a variational segmentation-based generation model to discover73 the underlying expression style in a response ; ( 3 ) empirical verification of the effectiveness of the74 proposed model on two benchmarks of knowledge-grounded conversation.75 2 Related Work76 Early research on end-to-end open-domain dialogue generation is inspired by the successful applica-77 tion of neural networks on machine translation [ 20 , 24 , 32 ] . On the vanilla encoder-decoder architec-78 ture , various extensions have been made to model the structure of dialogue contexts [ 22 , 23 , 37 , 39 ] ; to79 improve diversity of responses [ 13 , 36 , 43 , 29 ] ; to control attributes of responses [ 38 , 46 , 40 , 34 , 21 ] ; 80 and to bias responses to some specific personas [ 14 , 41 ] . Recently , grounding dialogue generation81 by extra knowledge has seemed promising to bridge the gap between conversation with existing82 systems and conversation with humans , and the knowledge could be obtained from knowledge graphs83 [ 48 , 18 , 30 ] , retrieved from unstructured documents [ 3 , 16 , 44 , 11 , 45 , 15 ] , or extracted from visual84 background [ 19 , 26 , 9 ] . In this work , we study document-grounded dialogue generation . Rather than85 selecting knowledge relevant to dialogue context and directly exploiting pre-trained language models86 to generate the response , we focus on expressing knowledge in this task.87 The idea of sequence modeling via segmentation [ 33 ] has attracted widespread attention in several88 natural language processing tasks . In the field of text segmentation , [ 33 ] propose a probabilistic89 model for sequence modeling via their segmentation and a “ Sleep-WAke Network ” ( SWAN ) method.90 In machine translation , [ 8 ] propose a neural phrase-based machine translation system that models91 phrase structures in the target language using SWAN . In data-to-text generation , [ 35 ] develop a neural,92 template-like generation model based on an HSMM decoder , which can be learned tractably by93 backpropagating through a dynamic program ; to tackle the problem of weak Markov assumption for94 the segment transition probability , [ 25 ] propose to explicitly segment target text into fragments and95 align them with their data correspondences , and jointly learn the segmentation and correspondence via96 dynamic programming . Though quite a few methods have been proposed to reduce the computational97 complexity [ 33 , 25 ] , using dynamic programming to calculate likelihood is still expensive . This98 work introduces two sequential latent variables to model the knowledge expression and proposes a99 variational segmentation-based generation framework , which enjoys less computation cost.100 3 Approach101 3.1 Problem Formalization and Motivation102 Suppose that we have a dataset D = { ( Ui , Ki , Ri ) } Ni=1 , where ∀i ∈ { 1 , . . . , N } , Ki serves as103 background knowledge of the dialogue ( Ui , Ri ) with Ki , j the j-th sentence , Ui is the context of the104 dialogue with Ui , j the j-th utterance , and Ri is the response . To bias the expression to a specific105 structure style , we further assume that there are a few examplesDsty = { ( Ui , Ki , Ri ) } Mi=1 provided by106 users depicting the required style for knowledge expression . Note that we have N ≫M , since corpus107 in a specific expression style is rare and difficult to acquire . The goal is to learn a generation model108 pθ ( R∣U , K ) ( θ denotes the parameters of the model ) from D , to generate a response R following109 pθ ( R∣U , K ) given a new dialogue context U and the associated knowledge K. Besides , one can110 either ( 1 ) bias the structure style of Pθ ( R∣U , K ) to Dsty with little cost ; or ( 2 ) switch the content111 style of knowledge expression in R.112 As mentioned above , the response can be decomposed into a sequence of segments , each is other113 knowledge-related , various in expression , or knowledge-irrelevant . Therefore manipulating the114 expression style of a response could be split into two subproblems . One is to control the structure115 style , in other word , distribution and number of two kinds of segments . The other subproblem is the116 content style , or generating every knowledge-related segment in desired content style , such as positive117 style or negative style or other customized styles defined by users . To solve the two subproblems,118 we propose a segmentation-based generation model , which could automatically detect and predict119 the segmental structure of the response and then generate segments of a response one by one . Each120 segment is either knowledge-irrelevant or knowledge-related , and knowledge-related segments could121 be expressed in arbitrary style , defined and manipulated by users . Both the segmentation and the122 choice are modeled by a latent variable , so as to facilitate end-to-end training . Furthermore , to123 guarantee the efficiency and practicality of our model , we propose a variational approach to optimize124 the evidence lower bound ( ELBO ) of the likelihood of response to circumvent directly marginalizing125 over all possible combinations of segmentation and action choice , which is time-consuming in both126 training and test stages.127 3.2 Model Architecture128 Figure 1 gives an overview of the proposed model , which is based on the encoder-decoder architecture.129 The encoder generates the contextual representations of the dialogue and knowledge , while the130 decoder generates the segments one after another . hNt encodes the dialogue context up to timestep131 t − 1 with N denoting the number of decoder layers . Given R = ( r1 , ⋯ , rt , ⋯ , rlr ) with rt referring132 the t-th token of R whose length is supposed to be lr , the variable Z = { zt } lrt=1 is utilized to control133 the choice of module of each segment ( Module Indicator ) , and its historical information is encoded134 by { ct } lrt=0 . M = { mt } lr t=1 is a sequence of binary variables and used to determine the boundary135 of each segment ( Boundary Indicator ) . Specifically , mt = 1 indicates that the current segment is136 already completed and a new segment should be created at the next timestep . Otherwise mt = 0 and137 the current segment remains unfinished . The generative process is disassembled into two steps : ( 1 ) 138 determine the type of a new segment based on previously generated text and previous segment types ; 139 ( 2 ) generate within the current segment until the binary variable mt = 1.140 Context and Knowledge Encoding . We exploit BART [ 12 ] as the backbone of our architecture,141 which is pre-trained using a variety of denoising objectives and achieves state-of-the-art results on a142 range of text generation tasks . Given the dialogue context U = ( U1 , ⋯ , Un ) , we simply concatenate143 them as ( u1 , ⋯ , ulu ) . Similarly , we concatenate the associated knowledge K = ( K1 , ⋯ , Km ) as144 ( k1 , ⋯ , klk ) . lu and lk are the length of dialogue context and background knowledge respectively.145 The input of the encoder is then defined as:146 I = [ BOS ] k1 . . . klk [ EOS ] u1 . . . ulu [ EOS ] . ( 1 ) The input I then passes through the stacked self-attention layers and results in a knowledge-aware147 context representation C , and a context-aware knowledge representation K. Specifically , the context-148 aware knowledge representation is defined as K = [ henc1 , ⋯ , henclk+1 ] where h enc t is the last layer149 of BART encoder at time t. Similarly , the knowledge-aware context representation is defined as150 C = [ henclk+2 , ⋯ , h enc lk+lu+2 ] .151 Prior of Module Indicator . We use the sequential discrete latent variable Z = { zt } lrt=1 to decide152 which module to invoke at each timestep . The transition of zt occurs only when a segment is153 completed , which is decided by the binary boundary variable M . The prior quantifies the distribution154 of zt before we observe the segment , and it is reasonable to assume that the prior of zt depends155 on previous module choices z < t and previously generated text . As a result , the transition of Z is156 implemented as follows:157 pθz ( zt∣r < t , z < t , mt−1 ) =mt−1 ⋅ p̃ ( zt∣ct ) + ( 1 −mt−1 ) ⋅ δ ( zt = zt−1 ) , ( 2 ) where ct encodes all previous latent state z < t and generated text r < t as follows:158 ct =mt−1 ⋅ fz−rnn ( z̃t−1 , ct−1 ) + ( 1 −mt−1 ) ⋅ ct−1 . ( 3 ) z̃t−1 = [ et−1 ; hN , dect−1 ] with et−1 the embedding of zt−1 and h N , dec t−1 the representation of last generated159 token . Specifically , mt−1 = 0 means that the next timestep t is still in the same segment as the160 previous timestep t− 1 and thus the latent variable zt should not be updated . Otherwise , it means that161 current segment is completed and zt is updated with the transition function p̃ ( zt∣ct ) . Because we only162 have Nsty + 2 options when choosing a module , where Nsty is the number of different user-defined163 styles , in addition with 2 default styles , so in this model , the latent variable zt ranges in natural integer164 to denote corresponding style type . Specifically , zt = 0 denotes choosing the context expression165 module to generate a knowledge-irrelevant segment ; zt = 1 tells the model to choose the knowledge166 expression module without specially customized style ; we leave the zt ≥ 2 to be user-defined so as to167 select the knowledge expression module combined with customized style . The transition function168 p̃ ( zt∣ct ) is then implemented as a multinomial distribution parameterized by Softmax ( fz−mlp ( ct ) ) .169 Prior of Boundary Indicator . The boundary indicator M = { mt } lrt=1 depicts the segmental struc-170 ture of the response , with mt = 1 indicates that a new segment will start at time t + 1 . Presumably,171 the prior of mt could be inferred from r≤t and zt . We model the distribution pθm ( mt∣r≤t , zt ) by a172 Bernoulli distribution parameterized by σ ( fm−mlp ( [ et−1 ; hN , deczt−1 , t−1 ] ) ) , where σ denotes the sigmoid173 function and fm−mlp is a multi-layer perceptron network.174 Stylized Generation As mentioned above , the generation process involves scheduling different175 modules according to zt . Here we give a systematic description of the generation process . The176 decoder accepts the token generated last timestep rt−1 as input , performs transformation inN decoder177 layers , finally obtains a dense representation.178 We use hlt to denote the hidden state after the l-th layer at timestep t , which is a shorthand for h l , dec t179 for brevity . Specially , h0t is the output of the embedding layer . When zt = 0 , it implies that knowledge180 encoding is unnecessary for current segment so hlt is defined as:181 hlt = DecoderLayer ( hl−1t , Hl−1t−1 , C ) , ( 4 ) where Hlt−1 = [ hl1 , ⋯ , hlt−1 ] is a sequence of decoder hidden states in previous timestep , and182 C is the context representation mentioned above . The implementation of DecoderLayer ( ⋅ , ⋅ , ⋅ ) is183 identical to the vanilla Transformer [ 31 ] where hl−1t first plays self-attention on H l−1 t−1 then performs184 cross-attention on C. The probability p ( rt∣r < t , zt = 0 ) is defined as a multinomial distribution185 parameterized by Softmax ( fr−mlp ( hNt ) ) , where hNt encodes the generated tokens up to timestep186 t − 1 . When zt = 1 , the implementation of decoder layer is analogous to the zt = 0 case except that187 we replace C with K , since knowledge is needed:188 hlt = DecoderLayer ( hl−1t , Hl−1t−1 , K ) . ( 5 ) To generate a segment with a particular customized style when zt ≥ 2 , we introduce some adapters189 [ 7 ] to bias the generation . Specifically , the hidden state hlt is defined as:190 hlt = DecoderLayeradp ( hl−1t , Hl−1t−1 , K ) , ( 6 ) where DecoderLayeradp ( ⋅ , ⋅ , ⋅ ) denotes the transformer decoder layer with adapters inserted . Note191 that we need to introduce a separate set of adapters for each style . To make the style fine-grained and192 adjustable , each style has a unique set of adapters . Different styles have no adapter in common . In193 addition , our model has the ability to learn to express in any type of style , as long as a discriminator194 for the desired style is provided.195 3.3 Learning Details196 We introduce auxiliary distributions qφm ( M ∣R ) = ∏lrt=1 qφm ( mt∣R ) and qφz ( Z ∣M , R ) =197 ∏lrt=1 qφz ( zt∣M , R ) , which serves as an approximation to the intractable posterior of the bound-198 ary indicator M and the module indicator Z . We then apply variational approximation which gives199 the following evidence lower bound objective 1 ( ELBO ) [ 6 ] :200 log pθ ( R∣U , K ) ≥ Eqφm ( M ∣R ) ( Eqφz ( Z∣M , R ) lr ∑ t=1 log pθ ( rt∣r < t , zt ) − lr ∑ t=1 mt−1 ⋅DKL ( qφz ( zt∣M , R ) ∥pθz ( zt ) ) ) − lr ∑ t=1 DKL ( qφm ( mt∣R ) ∥pθm ( mt ) ) , ( 7 ) where pθz ( zt ) and pθm ( mt ) stand for pθz ( zt∣r < t , z < t , mt−1 ) and pθm ( mt∣r≤t , zt ) respectively , and201 DKL ( ⋅∥⋅ ) ) refers to Kullback–Leibler divergence . Detailed derivations are presented in supplementary202 material.203 Base on the intuition that the response provides hints about the segmentation , we construct the204 posterior distribution qφm ( mt∣R ) as a Bernoulli distribution parameterized by σ ( f ′m−mlp ( ψt ) ) . ψt is205 a feature extracted from a bi-directional LSTM ψ ( R ) . Since the module indicator keeps unchanged206 within a segment , the posterior distribution qφz ( zt∣M , R ) is conditioned on the boundary indicator207 mt−1 and defined as:208 qφz ( zt∣M , R ) =mt−1 ⋅ q̃ ( zt∣ψt ) + ( 1 −mt−1 ) ⋅ δ ( zt = zt−1 ) , ( 8 ) where the transition function q̃ ( zt∣ψt ) is implemented as a multinomial distribution parameterized by209 Softmax ( f ′z−mlp ( ψt ) ) . Once we have the posterior distribution , we apply Gumbel-Softmax [ 10 ] with210 straight-through estimators [ 2 ] to take samples of mt and zt.211 Weak Supervision on M and Z . We first use StanfordNLP toolkit [ 17 ] to parse every response in212 the training set as a sequence of segments , and use M̃ = { m̃t } lrt=1 to denote the results of segmentation213 labeling . The pseudo label of module choice Z̃ = { zt } lrt=1 is tagged in a similar way to multiclass214 classification , determined by ( 1 ) the similarity between each segment and knowledge and ( 2 ) the215 classification confidence of the style discriminator . More details about the construction of Z̃ and M̃216 are provided in the supplementary material.217 With Z̃ and M̃ , the loss function of weak supervision is defined as:218 Lm = − lr ∑ t=1 log pθm ( m̃t∣r≤t , z̃t ) , Lz = − lr ∑ t=1 m̃t−1 ⋅ log pθz ( z̃t∣r < t , z̃ < t , m̃t−1 ) . ( 9 ) The learning algorithm is summarized in the supplementary material.219 4 Experiments220 4.1 Datasets221 We test our model on benchmarks of knowledge-grounded dialogue generation , including Wizard of222 Wikipedia ( Wizard ) and CMU Document Grounded Conversations ( CMU_DoG ) [ 49 ] . Both datasets223 are split into training sets , validation sets , and test sets by the data owners . Topics in Wizard cover a224 wide range ( 1,365 in total ) , and each conversation happens between a wizard who has access to the225 knowledge about a specific topic and an apprentice who is just eager to learn from the wizard about226 the topic . The test set is split into two subsets : Test Seen and Test Unseen . Test Seen only contains227 dialogues with topics that have already appeared in the training set , while topics in Test Unseen never228 appear in the training set and the validation set . We follow [ 3 ] and conduct the pre-processing with229 the code published on ParlAI2 . Different from Wizard , CMU_DoG focuses on movie domain , and230 1We always have m0 = 1 2 https : //github.com/facebookresearch/ParlAI/blob/master/projects/wizard_of_wikipedia besides wizard-apprentice conversations , the data also contain conversations between two workers231 who know the document and try to discuss the content in depth . In both datasets , only the turns where232 knowledge is accessible are considered in response generation . More details are described in the233 supplementary material.234 We choose the Reddit Corpus published by [ 15 ] as D. The data contains 842,521 context-knowledge-235 response triples for training and 2,737 context-knowledge-response triples for validation . On average,236 each dialogue contains 3.1 utterances in both sets , and the average length of the utterance is 16.0 in237 training and is 16.1 in validation . The dataset enjoys a great diversity of expression styles thanks to238 the large scale of corpus and little restriction on expression . We use part of the training data of Wizard239 and CMU_DoG as Dsty respectively , for these two datasets are distinctive in expression style and240 differ from each other . The dialogues in CMU_DoG tend to be causal and short , with most utterances241 irrelevant to knowledge . While the responses in Wizard are usually long and knowledgeable , as some242 phrases are directly extracted from wiki articles.243 4.2 Experimental Setup244 In this paper , we mainly consider two experimental setups , corresponding to the two subproblems245 mentioned in Sec 3.1 . To explore how our model can be used to control the distribution of different246 kinds of segments ( knowledge-related and knowledge-irrelevant ) , we first train the model on the247 Reddit Corpus and then fine-tune it on a small amount of examples in Wizard and CMU_DoG,248 respectively . To verify whether our model can generate the knowledge-related segments in the desired249 style , we still train the model on the Reddit Corpus , and use a style tag to control the generation250 process . In this experimental setup , we are primarily concerned with generating with two kinds251 of styles , positive and negative , where zt = 2 ⋅min ( 1 , zt ) tells the model to generate a response in252 positive sentiment and zt = 3 ⋅min ( 1 , zt ) is for response in negative sentiment.253 Evaluation Metrics . We choose distinct and unigram F1 [ 3 ] as metrics , where the F1 metric is cal-254 culated with the code published at https : //github.com/facebookresearch/ParlAI/255 blob/master/parlai/core/metrics.py . Distinct-1 ( D-1 ) and Distinct-2 ( D-2 ) are cal-256 culated as ratios of distinct unigrams and bigrams in responses , respectively . We also employ257 classification accuracy as the evaluation metrics for style control experiments . Specifically , we exploit258 Roberta trained on the SST-2 training set [ 27 ] as the evaluator , which is more accurate than that from259 the classifiers in [ 46 ] .260 Baselines . For the exploration of the first subproblem , we select the following models as baselines:261 ( 1 ) BART [ 12 ] : a model that achieves state-of-the-art performance on various text generation tasks.262 Note that our model degrades into BART once we remove the module indicator Z and the boundary263 indicator M ; ( 2 ) Zero-resource Knowledge-grounded Conversation ( ZRKGC ) [ 15 ] : 3 a model264 that is based on UniLM [ 4 ] and optimized with Generalized EM method . The model is trained265 on the Reddit Corpus and achieves comparable performance with state-of-the-art methods that266 rely on knowledge-grounded dialogues for training . For the second subproblem , we consider the267 following models as baselines : ( 1 ) Emotional Chatting Machine ( ECM ) [ 47 ] : 4 a model which can268 generate appropriate responses not only content-relevant but also emotional consistent ; ( 2 ) variant269 of DialoGPT [ 42 ] : DialoGPT is a model that is pre-trained on large-scale conversation corpus and270 attains a performance close to human in single-turn dialogues . As DialoGPT is not designed for271 sentiment control , we add a sentiment indicating token at the first of the sequence and explore272 whether such simple heuristics works for controlling knowledge expression . Comparisons with more273 state-of-the-art models are provided in the supplementary material.274 4.3 Results on Learning Structure Style275 In this section , we demonstrate the effectiveness of our segmentation-based generation framework in276 both low-resource setting and zero-resource setting and empirically verify that our model can learn277 structure style with a few annotated examples . In zero-resource setting , we trained our model on278 the Reddit Corpus published by [ 15 ] and tested on Wizard and CMU_DoG respectively . Automatic279 evaluation results are shown in Table 2 . It could be observed that : ( 1 ) our model significantly280 3 https : //github.com/nlpxucan/ZRKGC 4 https : //github.com/thu-coai/ecm outperforms ZRKGC and BART on most metrics and achieves the new state-of-the-art performance281 on Wizard . It is impressive that our model exceeds BART in CMU_DoG especially since the proposed282 model degrades into BART without two sequential latent variables Z and M. The result serves as283 strong evidence for the effect of two latent variables , which enable the model to learn complex284 expression style in Reddit Corpus to handle flexible expression in CMU_DoG . By contrast , BART is285 far from satisfying with only a regular decoder . ( 2 ) our model exceeds ZRKGC significantly in terms286 of Distinct metrics , for ZRKGC mainly focuses on leverage external knowledge sources for response287 generation , but falls short on expression diversity . In low-resource setting , after training our model288 on the Reddit Corpus , we then fine-tune it with only 10 % training size of Wizard and CMU_DoG289 respectively ( i.e. , Dsty in Sec 3.1 ) to adjust p ( zt ) and p ( mt ) to a new structure style . When provided290 with only 10 % training data , our model gets obvious improvement ( ∼ 1 % increase in F1 ) in contrast291 with BART ( ∼ 0.5 % increase in F1 ) and ZRKGC ( ∼ 0.2 % increase in F1 ) , proving that the proposed292 model can learn more sophisticated structure style through quickly adjustment on a specific dataset293 with little cost . Furthermore , we are interested in its potential in learning with less annotated data.294 We also want to investigate how our model is adjusted to different annotated data . Exploration of295 these two topics is as follows.296 Fine-tune with less annotated data . We first train the model on the Reddit Corpus and then fine-297 tune it with the amount of annotated data ( e.g. , Wizard and CMU_DoG ) gradually increasing from298 2 % to 10 % . To have a more intuitive understanding of the effects of latent variables Z and M , we299 compare the proposed model with BART , which generates the response with a single decoder . The300 evaluation results are shown in Figure 2 . It can be concluded from the result that : ( 1 ) our model can301 learn the expression style of a particular dataset more efficiently . As the training data increase , our302 model has a more significant improvement in terms of the F1 metric ; ( 2 ) our model performs better in303 meager resources since there is a considerable gap between our model and BART when the training304 data is close to 0 % ; ( 3 ) the expression style of CMU_DoG can be learned with less data because the305 model has a significant change in performance after using 2 % CMU_DoG training data.306 Refashioning of knowledge-related segments . To know307 how our model adjusts to different datasets , we compare the308 knowledge-related segments before and after trained with an-309 notated data from two aspects : ( 1 ) the average proportion of310 knowledge-related segments ( pklg ) in a sentence ; ( 2 ) the aver-311 age proportion of words belonging to knowledge-related seg-312 ments ( lklg ) . Figure 3 reports the results . The results indicate313 that our model could learn the underlying structure style of both314 datasets , with the great difference of pklg and lklg before and315 after fine-tuning as evidence . After fine-tuned with Wizard data,316 pklg drops to 0.26 while the lklg grows up a bit , indicating that the knowledge-related segments317 generated by our model are fewer and longer , which tallies with the fact that the responses in Wizard318 are probably directly copied from background knowledge . However , after CMU_DoG data is fed to319 the model , both pklg and lklg shrinks drastically , which agrees with the fact that crowd-sourcing320 workers converse more liberally online and the responses are less relevant to background knowledge.321 4.4 Results on Learning Content Style322 We further investigate whether the proposed model could express knowledge with the desired323 sentiment . Specifically , we introduce two sets of style adapters to endow knowledge expression324 in two different sentiments , namely positive and negative . So in this scenario , it is required that325 responses are not only coherent with context but also limited in positive or negative sentiment . To326 apply ECM on knowledge-grounded conversation , we label the sentiment category for each response327 with a classifier pre-trained on the SST [ 27 ] training set . For DialoGPT , we similarly annotate each328 response with a sentiment category and append the sentiment token before the context tokens . The329 evaluation results is shown in Table 3 . We can conclude that : ( 1 ) The proposed model outperforms330 all baseline models in terms of all metrics , which indicates that our model can control the sentiment331 of knowledge expression and guarantee high quality of the generated responses ; ( 2 ) Simply adding a332 sentiment indicating token at the beginning of the sequence can not effectively control the style of333 knowledge expression , as the performance of DialoGPT on sentiment control is poor ; ( 3 ) Although334 ECM is designed for sentiment control , it still fails to perform well in this task , proving that sentiment335 control in the knowledge-grounded conversation is rather difficult . Besides , ECM can only control336 the sentiment of the whole response but is helpless to manage every knowledge-related segment at a337 more refined level.338 5 Conclusions339 We explore knowledge expression in knowledge-grounded conversation and break down the ex-340 pression style of a response into the structure of the response ( structure style ) and the style of the341 content in each part ( content style ) . We propose a variational segmentation-based generation model to342 discover the underlying expression style in response . Specifically , we introduce two latent variables343 to model these two aspects of expression style respectively and induce an evidence lower bound344 of the likelihood . Evaluation results on two benchmarks of the task indicate that our model can345 learn the structure style with little cost and generate responses in desired content style without any346 human-annotated data.347 Broader Impact348 Enabling an open-domain dialogue system to automatically detect and discover the underlying349 structural pattern of a sentence is of great significance . This process is destined to be hailed as a350 milestone on the way to thoroughly reveal the essential nature of open-domain dialogue . Capable of351 handling different expression styles , positive or negative , casual or serious , our work implies that we352 are now much closer to the final destination of constructing an artificial intelligent dialogue system353 that could communicate freely with human being , which is beyond the wildest dream of most AI and354 NLP researchers . In the future , we heartily look forward to seeing advanced methods or ideas based355 on our work , and we expect the appearance of related industrial projects and applications to benefit356 the people and the public.357 References358 [ 1 ] D. Adiwardana , M.-T. Luong , D. R. So , J . Hall , N. Fiedel , R. Thoppilan , Z. Yang , A. Kulshreshtha,359 G. Nemade , Y. Lu , et al . Towards a human-like open-domain chatbot . arXiv preprint arXiv:2001.09977,360 2020.361 [ 2 ] Y. Bengio , N. Léonard , and A. Courville . Estimating or propagating gradients through stochastic neurons362 for conditional computation . arXiv preprint arXiv:1308.3432 , 2013.363 [ 3 ] E. Dinan , S. Roller , K. Shuster , A . Fan , M. Auli , and J. Weston . Wizard of wikipedia : Knowledge-powered364 conversational agents . In ICLR , 2019.365 [ 4 ] L. Dong , N. Yang , W. Wang , F. Wei , X. Liu , Y. Wang , J. Gao , M. Zhou , and H.-W. Hon . Unified language366 model pre-training for natural language understanding and generation . In Advances in Neural Information367 Processing Systems , pages 13042–13054 , 2019.368 [ 5 ] J. Gehring , M. Auli , D. Grangier , D. Yarats , and Y. N. Dauphin . Convolutional sequence to sequence369 learning . In Proceedings of the 34th International Conference on Machine Learning-Volume 70 , pages370 1243–1252 . JMLR . org , 2017.371 [ 6 ] M. D. Hoffman , D. M. Blei , C. Wang , and J. Paisley . Stochastic variational inference . Journal of Machine372 Learning Research , 14 ( 5 ) , 2013.373 [ 7 ] N. Houlsby , A. Giurgiu , S. Jastrzebski , B. Morrone , Q . De Laroussilhe , A. Gesmundo , M. Attariyan , and374 S. Gelly . Parameter-efficient transfer learning for nlp . In International Conference on Machine Learning,375 pages 2790–2799 . PMLR , 2019.376 [ 8 ] P.-S. Huang , C. Wang , S. Huang , D. Zhou , and L. Deng . Towards neural phrase-based machine translation.377 arXiv preprint arXiv:1706.05565 , 2017.378 [ 9 ] B. Huber , D. McDuff , C. Brockett , M. Galley , and B. Dolan . Emotional dialogue generation using379 image-grounded language models . In CHI , page 277 . ACM , 2018.380 [ 10 ] E. Jang , S. Gu , and B. Poole . Categorical reparameterization with gumbel-softmax . arXiv preprint381 arXiv:1611.01144 , 2016.382 [ 11 ] B. Kim , J. Ahn , and G. Kim . Sequential latent knowledge selection for knowledge-grounded dialogue.383 arXiv preprint arXiv:2002.07510 , 2020.384 [ 12 ] M. Lewis , Y. Liu , N. Goyal , M. Ghazvininejad , A. Mohamed , O . Levy , V. Stoyanov , and L. Zettlemoyer.385 Bart : Denoising sequence-to-sequence pre-training for natural language generation , translation , and com-386 prehension . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics,387 pages 7871–7880 , 2020.388 [ 13 ] J. Li , M. Galley , C. Brockett , J. Gao , and B. Dolan . A diversity-promoting objective function for neural389 conversation models . NAACL , pages 110–119 , 2015.390 [ 14 ] J. Li , M. Galley , C. Brockett , G. Spithourakis , J. Gao , and B. Dolan . A persona-based neural conversation391 model . In ACL , pages 994–1003 , 2016.392 [ 15 ] L. Li , C. Xu , W. Wu , Y. Zhao , X. Zhao , and C. Tao . Zero-resource knowledge-grounded dialogue393 generation . arXiv preprint arXiv:2008.12918 , 2020.394 [ 16 ] R. Lian , M. Xie , F. Wang , J. Peng , and H. Wu . Learning to select knowledge for response generation in395 dialog systems . arXiv preprint arXiv:1902.04911 , 2019.396 [ 17 ] C. D. Manning , M. Surdeanu , J. Bauer , J. R. Finkel , S. Bethard , and D. McClosky . The stanford corenlp397 natural language processing toolkit . In Proceedings of 52nd annual meeting of the association for398 computational linguistics : system demonstrations , pages 55–60 , 2014.399 [ 18 ] S. Moon , P. Shah , A. Kumar , and R. Subba . Opendialkg : Explainable conversational reasoning with400 attention-based walks over knowledge graphs . In Proceedings of the 57th Annual Meeting of the Association401 for Computational Linguistics , pages 845–854 , 2019.402 [ 19 ] N. Mostafazadeh , C. Brockett , B. Dolan , M. Galley , J. Gao , G. Spithourakis , and L. Vanderwende . Image-403 grounded conversations : Multimodal context for natural question and response generation . In Proceedings404 of the Eighth International Joint Conference on Natural Language Processing ( Volume 1 : Long Papers ) ,405 pages 462–472 , 2017.406 [ 20 ] A. Ritter , C. Cherry , and W. B. Dolan . Data-driven response generation in social media . In Proceedings of407 the 2011 Conference on Empirical Methods in Natural Language Processing , pages 583–593 , 2011.408 [ 21 ] A . See , S. Roller , D. Kiela , and J. Weston . What makes a good conversation ? how controllable attributes409 affect human judgments . arXiv preprint arXiv:1902.08654 , 2019.410 [ 22 ] I. V. Serban , A. Sordoni , Y. Bengio , A. C. Courville , and J. Pineau . Building end-to-end dialogue systems411 using generative hierarchical neural network models . In AAAI , volume 16 , pages 3776–3784 , 2016.412 [ 23 ] I. V. Serban , A. Sordoni , R. Lowe , L. Charlin , J. Pineau , A. C. Courville , and Y. Bengio . A hierarchical413 latent variable encoder-decoder model for generating dialogues . In AAAI , pages 3295–3301 , 2017.414 [ 24 ] L. Shang , Z. Lu , and H. Li . Neural responding machine for short-text conversation . In ACL , pages415 1577–1586 , 2015.416 [ 25 ] X. Shen , E. Chang , H. Su , C. Niu , and D. Klakow . Neural data-to-text generation via jointly learning417 the segmentation and correspondence . In Proceedings of the 58th Annual Meeting of the Association for418 Computational Linguistics , pages 7155–7165 , 2020.419 [ 26 ] K. Shuster , S. Humeau , A. Bordes , and J. Weston . Engaging image chat : Modeling personality in grounded420 dialogue . arXiv preprint arXiv:1811.00945 , 2018.421 [ 27 ] R. Socher , A. Perelygin , J. Wu , J. Chuang , C. D. Manning , A. Y. Ng , and C. Potts . Recursive deep422 models for semantic compositionality over a sentiment treebank . In Proceedings of the 2013 conference on423 empirical methods in natural language processing , pages 1631–1642 , 2013.424 [ 28 ] I. Sutskever , O. Vinyals , and Q. V. Le . Sequence to sequence learning with neural networks . In Advances425 in neural information processing systems , pages 3104–3112 , 2014.426 [ 29 ] C. Tao , S. Gao , M. Shang , W. Wu , D. Zhao , and R. Yan . Get the point of my utterance ! learning towards427 effective responses with multi-head attention mechanism . In IJCAI , pages 4418–4424 , 2018.428 [ 30 ] Y.-L. Tuan , Y.-N. Chen , and H.-y . Lee . Dykgchat : Benchmarking dialogue generation grounding on429 dynamic knowledge graphs . In Proceedings of the 2019 Conference on Empirical Methods in Natural430 Language Processing and the 9th International Joint Conference on Natural Language Processing ( EMNLP-431 IJCNLP ) , pages 1855–1865 , 2019.432 [ 31 ] A. Vaswani , N. Shazeer , N. Parmar , J. Uszkoreit , L. Jones , A. N. Gomez , Ł. Kaiser , and I. Polosukhin.433 Attention is all you need . In NIPS , pages 5998–6008 , 2017.434 [ 32 ] O. Vinyals and Q . Le . A neural conversational model . arXiv preprint arXiv:1506.05869 , 2015.435 [ 33 ] C. Wang , Y. Wang , P.-S. Huang , A. Mohamed , D. Zhou , and L. Deng . Sequence modeling via segmentations.436 In International Conference on Machine Learning , pages 3674–3683 . PMLR , 2017.437 [ 34 ] Y. Wang , C. Liu , M. Huang , and L. Nie . Learning to ask questions in open-domain conversational systems438 with typed decoders . In Proceedings of the 56th Annual Meeting of the Association for Computational439 Linguistics ( Volume 1 : Long Papers ) , pages 2193–2203 , 2018.440 [ 35 ] S. Wiseman , S. M. Shieber , and A. M. Rush . Learning neural templates for text generation . In Proceedings441 of the 2018 Conference on Empirical Methods in Natural Language Processing , pages 3174–3187 , 2018.442 [ 36 ] C. Xing , W. Wu , J. Liu , Y. Huang , M. Zhou , and W.-Y . Ma . Topic aware neural response generation . In443 AAAI , pages 3351–3357 , 2017.444 [ 37 ] C. Xing , W. Wu , Y. Wu , M. Zhou , Y. Huang , and W.-Y . Ma . Hierarchical recurrent attention network for445 response generation . arXiv preprint arXiv:1701.07149 , 2017.446 [ 38 ] C. Xu , W. Wu , C. Tao , H. Hu , M. Schuerman , and Y. Wang . Neural response generation with meta-words.447 arXiv preprint arXiv:1906.06050 , 2019.448 [ 39 ] H. Zhang , Y. Lan , L. Pang , J. Guo , and X. Cheng . Recosa : Detecting the relevant contexts with self-449 attention for multi-turn dialogue generation . In Proceedings of the 57th Annual Meeting of the Association450 for Computational Linguistics , pages 3721–3730 , 2019.451 [ 40 ] R. Zhang , J. Guo , Y . Fan , Y. Lan , J. Xu , and X. Cheng . Learning to control the specificity in neural452 response generation . In Proceedings of the 56th Annual Meeting of the Association for Computational453 Linguistics ( Volume 1 : Long Papers ) , pages 1108–1117 , 2018.454 [ 41 ] S. Zhang , E. Dinan , J. Urbanek , A. Szlam , D. Kiela , and J. Weston . Personalizing dialogue agents : I have455 a dog , do you have pets too ? arXiv preprint arXiv:1801.07243 , 2018.456 [ 42 ] Y. Zhang , S. Sun , M. Galley , Y.-C. Chen , C. Brockett , X. Gao , J. Gao , J. Liu , and B. Dolan . Dialogpt : Large-457 scale generative pre-training for conversational response generation . arXiv preprint arXiv:1911.00536,458 2019.459 [ 43 ] T. Zhao , R. Zhao , and M. Eskenazi . Learning discourse-level diversity for neural dialog models using460 conditional variational autoencoders . In ACL , pages 654–664 , 2017.461 [ 44 ] X. Zhao , W. Wu , C. Tao , C. Xu , D. Zhao , and R. Yan . Low-resource knowledge-grounded dialogue462 generation . arXiv preprint arXiv:2002.10348 , 2020.463 [ 45 ] X. Zhao , W. Wu , C. Xu , C. Tao , D. Zhao , and R. Yan . Knowledge-grounded dialogue generation with464 pre-trained language models . In Proceedings of the 2020 Conference on Empirical Methods in Natural465 Language Processing ( EMNLP ) , pages 3377–3390 , 2020.466 [ 46 ] H. Zhou , M. Huang , T. Zhang , X. Zhu , and B. Liu . Emotional chatting machine : Emotional conversation467 generation with internal and external memory . arXiv preprint arXiv:1704.01074 , 2017.468 [ 47 ] H. Zhou , M. Huang , T. Zhang , X. Zhu , and B. Liu . Emotional chatting machine : Emotional conversation469 generation with internal and external memory . In Proceedings of the AAAI Conference on Artificial470 Intelligence , volume 32 , 2018.471 [ 48 ] H. Zhou , T. Young , M. Huang , H. Zhao , J. Xu , and X. Zhu . Commonsense knowledge aware conversation472 generation with graph attention . In IJCAI , pages 4623–4629 , 2018.473 [ 49 ] K. Zhou , S. Prabhumoye , and A. W. Black . A dataset for document grounded conversations . arXiv preprint474 arXiv:1809.07358 , 2018.475 Checklist476 1 . For all authors ... 477 ( a ) Do the main claims made in the abstract and introduction accurately reflect the paper ’ s478 contributions and scope ? [ Yes ] 479 ( b ) Did you describe the limitations of your work ? [ No ] 480 ( c ) Did you discuss any potential negative societal impacts of your work ? [ No ] 481 ( d ) Have you read the ethics review guidelines and ensured that your paper conforms to482 them ? [ Yes ] 483 2 . If you are including theoretical results ... 484 ( a ) Did you state the full set of assumptions of all theoretical results ? [ Yes ] 485 ( b ) Did you include complete proofs of all theoretical results ? [ Yes ] 486 3 . If you ran experiments ... 487 ( a ) Did you include the code , data , and instructions needed to reproduce the main experi-488 mental results ( either in the supplemental material or as a URL ) ? [ Yes ] 489 ( b ) Did you specify all the training details ( e.g. , data splits , hyperparameters , how they490 were chosen ) ? [ Yes ] 491 ( c ) Did you report error bars ( e.g. , with respect to the random seed after running experi-492 ments multiple times ) ? [ No ] 493 ( d ) Did you include the total amount of compute and the type of resources used ( e.g. , type494 of GPUs , internal cluster , or cloud provider ) ? [ Yes ] 495 4 . If you are using existing assets ( e.g. , code , data , models ) or curating/releasing new assets ... 496 ( a ) If your work uses existing assets , did you cite the creators ? [ Yes ] 497 ( b ) Did you mention the license of the assets ? [ No ] 498 ( c ) Did you include any new assets either in the supplemental material or as a URL ? [ Yes ] 499 ( d ) Did you discuss whether and how consent was obtained from people whose data you ’ re500 using/curating ? [ Yes ] 501 ( e ) Did you discuss whether the data you are using/curating contains personally identifiable502 information or offensive content ? [ N/A ] 503 5 . If you used crowdsourcing or conducted research with human subjects ... 504 ( a ) Did you include the full text of instructions given to participants and screenshots , if505 applicable ? [ Yes ] 506 ( b ) Did you describe any potential participant risks , with links to Institutional Review507 Board ( IRB ) approvals , if applicable ? [ No ] 508 ( c ) Did you include the estimated hourly wage paid to participants and the total amount509 spent on participant compensation ? [ No ] 510

Summary: The authors study the problem of knowledge-grounded conversation, and propose a segment generation process to better “express” knowledge in the conversation. Specifically, the authors identify that expressing a response can be broken down into two components: the structure of the response and the content style; the former marked by either “knowledge-related” or “knowledge-irrelevant” segments, while the latter is characterized by positive, negative, or other styles of formulating a response. The generation model is based on a normal encoder/decoder architecture, but the decoder is influenced by various introduced latent variables when determining how to generate the next token. A “boundary” indicator informs the decoder when it has reached the boundary of a segment (and must therefore transition), and a “module” indicator determines which structure/segment type to use when generating; these indicators inform the decoder which encoded representations to attend to when generating and are learned via weak supervision. The authors conduct experiments on two knowledge-grounded datasets and show that their model outperforms strong baselines in both zero-resource and low-resource settings, and additionally show that their method is more adaptable to training data (as it can learn more in a low-resource setting than the other models). The authors additionally show that the model is more controllable in a sentiment-control setting, such that sentiment classifiers indicate the controlled responses more effectively model a positive or negative sentiment depending on the setup.