Title: CONVERSATION GENERATION WITH CONCEPT FLOW Anonymous authors Paper under double-blind review

Abstract: Human conversations naturally evolve around related entities and connected concepts, while may also shift from topic to topic. This paper presents ConceptFlow, which leverages commonsense knowledge graphs to explicitly model such conversation flows for better conversation response generation. ConceptFlow grounds the conversation inputs to the latent concept space and represents the potential conversation flow as a concept flow along the commonsense relations. The concept is guided by a graph attention mechanism that models the possibility of the conversation evolving towards different concepts. The conversation response is then decoded using the encodings of both utterance texts and concept flows, integrating the learned conversation structure in the concept space. Our experiments on Reddit conversations demonstrate the advantage of ConceptFlow over previous commonsense aware dialog models and fine-tuned GPT-2 models, while using much fewer parameters but with explicit modeling of conversation structures.

1. INTRODUCTION:

The rapid advancements of language modeling and natural language generation (NLG) techniques have enabled fully data-driven conversation models, which take user inputs (utterances) and directly generate natural language responses (Shang et al., 2015; Vinyals & Le, 2015; Li et al., 2016) . On the other hand, the current generation models may still degenerate dull and repetitive contents (Holtzman et al., 2019; Welleck et al., 2019) , which, in conversation assistants, lead to irrelevant, off-topic, and non-useful responses that would damage user experiences (Tang et al., 2019; Zhang et al., 2018; Gao et al., 2019) .

A promising way to address this degeneration challenge is to model conversations with the help of knowledge, for example, open-domain knowledge graph (Ghazvininejad et al., 2018) , commonsense knowledge base (Zhou et al., 2018a) , or background documents (Zhou et al., 2018b) . Recent research leverages these prior knowledge by grounding the conversation utterances to the external knowledge and integrating them as additional semantic representations; then response can be generated by conditioning on both the text inputs and the grounded semantics (Ghazvininejad et al., 2018; Zhou et al., 2018a; b) .

Integrating external knowledge as a semantic representation of the utterance and an additional input to the conversation model effectively improves the quality of generated responses (Ghazvininejad et al., 2018; Logan et al., 2019; Zhou et al., 2018a) . On the other hand, human conversations do not stay still on the same set of grounded semantics; instead, our dialog dynamically flows in the semantic space: we shift our discussions from one concept to another, chat about a group of related entities, and may switch dialog topics entirely (Fang et al., 2018) . Limiting the usages of knowledge only to the grounded ones, effective as they are, does not leverage semantics' full potential in modeling human conversations. This work presents ConceptFlow, (Conversation generation with Concept Flow), which leverages the commonsense knowledge graph to model the conversation flow in the latent concept space. Given a conversation utterance, ConceptFlow starts from the grounded knowledge, which in our case are the commonsense concepts appearing in the utterance, and extends to multi-hop concepts along the commonsense relations. Then the conversation flow is modeled in the extended concept graph using a new fine-grained graph attention mechanism, which learns to encode the concepts using central or outer graph. Mimicking the development conversation topic flow, the graph attentions guild the concept flow by attending on different directions in the concept flow.

The encoded latent concept flow is integrated to the response generation with standard conditional language models: during decoding, each token, word or concept, is sampled from ConceptFlow's context vector, which combines the encodings of the utterance texts and the latent concept flow. This enables ConceptFlow to explicitly model the conversation structure when generating responses.

Our experiments on a Reddit conversation dataset (Zhou et al., 2018a ) and a commonsense knowledge graph, ConceptNet (Speer & Havasi, 2012) , demonstrate the advantage of ConceptFlow. In both automatic and human evaluation, ConceptFlow performs significantly better than various seq2seq based generation models (Sutskever et al., 2014) , as well as previous methods that also leverage commonsense knowledge graph but as static memories (Zhou et al., 2018a; Ghazvininejad et al., 2018; Zhu et al., 2017) . Notably, ConceptFlow also outperforms two fine-tuned GPT-2 systems (Radford et al., 2019) , despite using much fewer parameters-Effective modeling of conversation structure can reduce the need of large parameter space.

We also provide extensive analyses and case studies to investigate the advantage of modeling conversation flow in the latent concept graph. Our analyses show that many of Reddit discussions are naturally aligned with the paths in the commonsense knowledge graph; expanding the latent concept graph multiple hops away from the initial grounded concepts significantly improves the coverage on the ground truth response. Our ablation study further confirms the effectiveness of our graph attention mechanism in selecting useful latent concepts and concepts appearing in golden responses, which help generate more relevant, informative, and less repetitive responses.

2. METHODOLOGY:

This section presents our Conversation generation with latent Concept Flow (ConceptFlow). As shown in Figure 1 , the ConceptFlow models the conversation flow along commonsense relations between concepts to generate meaningful responses.

2.1. PRELIMINARY ON GROUNDED CONVERSATION MODELS:

Given a user utterance X = {x 1 , ..., x m } with m words, conversation generation models often use an encoder-decoder architecture to generate a response Y = {y 1 , ..., y n }.

Typically, the encoder represents the user utterance X as a representation set H = { h 1 , ..., h m }. This is often done with a Gated Recurrent Unit (GRU):

EQUATION

where the x i is the embedding of word x i .

The decoder generates t-th response word according to the previous t -1 generated words y <t = {y 1 , ..., y t-1 } and the user utterance X:

EQUATION

)

The t-th token y t is generated according to the t-step decoder context representation s t :

EQUATION

where c t-1 is the context embedding of t -1-th time, y t-1 is the t -1-th generated word embedding and s t-1 is the decoder output representation of t -1-th time.

2.2. CONVERSATION GENERATION WITH LATENT CONCEPT FLOW:

This part introduces the flow concept candidate construction, the latent concept flow encoding, and the conditional conversation decoder to generate response.

2.2.1. CONSTRUCTING FLOW CONCEPT CANDIDATES:

ConceptFlow constructs a latent concept graph G for knowledge grounded conversation generation.

The latent concept graph G starts from the grounded concepts (zero-hop concepts G 0 ), which appear in the conversation utterance and grounded by entity linking. Besides the grounded concepts, ConceptFlow grows zero-hop concepts G 0 with one-hop concepts G 1 and two-hop concepts G 2 . G 0 and G 1 form the central concept graph G central , which is closely related to the current conversation topic. G 1 and G 2 construct an outer concept graph G outer , which models outer conversation flow.

Outer Subflow ! Central Concept " # " $ POST chat

/ " 0 ! Attention Latent Flow 1 " 2 # " $ # " $ # " $ # " $ # " $ 1 " 2 1 " 2 1 " 2 # " $

Latent concept flow consists of related concepts that can help understand the conversation. Next, we model conversation flow from zero-hop concepts, to one-hop concepts and then to two-hop concepts.

2.2.2. ENCODING LATENT CONCEPT FLOW:

This part describes the latent concept flow encoding of central flow concepts and outer flow concepts.

Central Flow Encoding. The central flow concept models the concept flow from zero-hop concepts to one-hop concepts using the interactions between zero-hops and one-hops. A multi-layer Graph Neural Network (GNN) (Sun et al., 2018) is used to encode concept e ∈ G central in central concept graph. The l-th layer representation e l i of concept e i is calculated by a single-layer feed-forward network (FFN) over three states:

EQUATION

where • is the concatenate operator. e l-1 j represents the concept e j 's representation of (l -1)-th layer. p l-1 represents user utterance representation of (l -1)-th layer.

The l-th layer user utterance representation is updated with the grounded concepts G 0 :

p l-1 = FFN( ei∈G 0 e l-1 i ).

(5)

The

f ej ->ei r ( e l-1 j

) aggregates the concept semantics of the relation r specific neighbor concept e j . It uses attention weight α ej r to control concept flow from e i :

EQUATION

where • is the concatenate operator and r is the relation embedding of r. The attention weight α ej r

is computed over all concept e i 's neighbor concepts according to the relation weight score and the Page Rank score (Sun et al., 2018) :

EQUATION

where PageRank(e l-1 j

) is the page rank score to control propagation of embeddings along paths starting from e i (Sun et al., 2018) and p l-1 is the (l -1)-th layer user utterance representation.

The 0-th layer concept representation e 0 for concept e is initialized with the pre-trained concept embedding e and the 0-th layer user utterance representation p 0 is initialized with the m-th hidden state h m from the user utterance representation set H. The result GNN encodings establishes the central concept flow between zero-hop and one-hop concepts using attentions.

Outer Flow Encoding. The outer flow models the concept flow from one-hop concepts to two-hop concepts. Given a concept e i from the one-hop concepts G 1 , all neighbor concepts e k ∈ G 2 are weighted to form the subflow g ei 's representation g ei :

EQUATION

where • is the concatenate operator. The e i and e k are embeddings for e i and e k , respectively. The attention score β e k is calculated to weight and aggregate concept triple (e i , r, e k ) to get g ei :

EQUATION

)

where r is the relation between the concept e i and its neighbor concept e k . The w r , w h and w t are learnable parameters. The outer concept flow models more diverse developments of the conversation and guides the flow with the subgraph encoding to more possible directions.

2.2.3. DECODING FROM LATENT CONCEPT FLOW:

This part presents how to generate the response Y using the latent concept flow.

Context Representation with ConceptFlow. The t-th time output representation s t of decoder is calculated by updating the t -1-th step output representation s t-1 with context representations:

EQUATION

where y t-1 is the t-1-th step generated token y t-1 's embedding. The context representation c t-1 is the concatenation of the text-based representation c text t-1 and the concept-based context representation

EQUATION

c text t-1 reads the user utterance representations H with the attention score a j t-1 (Bahdanau et al., 2014) :

EQUATION

)

where the attention a j t-1 weights over user utterance representations:

EQUATION

The concept-based representation

c concept t-1

is a combination of central concept flow encodings and outer flow encodings:

EQUATION

where the attention b ei t-1 weights over central concept representations:

EQUATION

The attention z g t-1 weights over outer flow representations:

EQUATION

)

Generating Words and Concepts. The conversation generator utilizes the t-th time output representation s t to decode t-th words in response from the word vocabulary and the concept vocabulary:

EQUATION

)

where w is the word embedding for word w, e i is the central concept representation for concept e i and e k is the two-hop concept e k 's embedding.

The generation probability of word w is calculated over word vocabulary. The generation probability of concept is separated into two parts: central concept e i 's probability over G 0,1 and outer concept over G 2 . The σ * is a gate used to control the token generation from these three probability distributions:

EQUATION

to choose words (σ * = 0), central concepts (σ * = 1) and outer concepts (σ * = 2) when generating the response.

Then we minimize the cross entropy loss L and optimize all parameters end-to-end:

EQUATION

where the y * t is the ground truth tokens for words or concepts. Metrics. A wide range of evaluation metrics are included from three evaluating aspects: relevance, diversity and novelty. PPL (Serban et al., 2016) , Bleu (Papineni et al., 2002) , Nist (Doddington, 2002) , ROUGE (Lin, 2004) and Meteor (Lavie & Agarwal, 2007) are used for relevance and repetitiveness; Dist-1, Dist-2 and Ent-4 are used for diversity, which is same with the previous work (Li et al., 2015; Zhang et al., 2018) . Zhou et al. (2018a) 's concept PPL favors concept grounded models and is reported in Appendix A.1. The Precision, Recall and F1 Score to generate golden concepts (those appear in the ground truth response) are used to evaluate the quality of learned latent concept flow.

3. EXPERIMENT SETTINGS:

Baselines. Six baselines are compared in our experiments. Seq2Seq (Sutskever et al., 2014) is the basic encoder-decoder for the language generation task. MemNet (Ghazvininejad et al., 2018) and CopyNet (Zhu et al., 2017) utilize extra knowledge in two different ways: maintain a memory to store and read concepts; copy concepts for the response generation. Both MemNet and CopyNet provide solutions to store and incorporate knowledge for conversation generation. The Commonsense Knowledge Aware Conversation Generation Model (CCM) (Zhou et al., 2018a ) leverages a graph attention mechanism to model local graphs, which further considers the graph structure for the improvement. The three models above use grounded graph as still knowledge and do not explicitly model conversation flow.

GPT-2 (Radford et al., 2019) , the pre-trained model that achieves the state-of-the-art in lots of language generation tasks, is also compared in experiment. We fine-tune the 124M GPT-2 in two ways: concatenate all conversations together and train like a language model (GPT-2 (lang)); extend the GPT-2 model with encode-decoder architecture and supervised with response data (GPT-2 (conv)).

Implement Details. The zero-hop concepts are initialized by matching the keywords in post to concepts in ConceptNet, the same with CCM (Zhou et al., 2018a) . Then zero-hop concepts are extended to their neighbors to form central concept graph. The outer concept flow usually contains lots of noise because of the large number of two-hop concepts. To select more related concepts for the conversation generation to reduce the computation cost, ConceptFlow first randomly selects 10% training data to train an initial version. Then we use the initial version's learned graph attention to select the top-100 two-hop concepts on all the rest data, and then conduct standard train, develop, and test process with the pruned graph. More details of this concept selection step can be found in Appendix C. TransE (Bordes et al., 2013) embedding and Glove (Pennington et al., 2014) embedding are used to initialize the representation of concepts and words, respectively. Adam optimizer with learning rate of 0.0001 is used to train the model. 

4. EVALUATION:

This section presents the quality of generated responses from the ConceptFlow, the ablation study for the roles of different modules, and case studies to evaluate the ConceptFlow.

4.1. CONVERSATION GENERATION QUALITY ESTIMATION:

Automatic Evaluation. The relevance, diversity, and novelty of generated responses with different evaluation metrics are presented in Table 1 and Table 2 .

In Table 1 , all the evaluation metrics compare the relevance between the generated response and the golden response. Our model outperforms all previous models by large margins. Responses generated by our model are on-topic and cover more necessary information. In Table 2 , Dist-1, Dist-2, and Ent-4 measure the word diversity of generated response, whereas the rest of metrics measure the repetitiveness comparing to the user utterance to avoid dull copying the input. Our model also presents a convincing balance to generate novel and diverse responses. GPT-2 (lang) performs more diversely, but ConceptFlow performs more novelty and more on-topic than both GPT-2 versions, perhaps due to its different decoding strategy.

Human Evaluation. The human evaluation mainly focuses on two testing scenarios: appropriateness and informativeness, which are important for conversation systems. Appropriateness indicates if the response is on-topic for the given utterance. Informativeness indicates the ability to provide new information instead of copying from the utterance (Zhou et al., 2018a) . All responses of sampled 100 case are selected from four best methods: CCM, GPT-2 (conv), ConceptFlow and Golden Response. The responses are scored from 1 to 4 by five judges.

The model performance is listed in Table 3 . The human evaluation is divided into two parts: Average Score and Best@1 ratio, where Best@1 ratio indicates the fraction of judges consider the corresponding response as the best. ConceptFlow outperforms all baseline models on all scenarios. This convincing result demonstrates the advantage of explicitly modeling conversation flow with semantics: ConceptFlow outperforms GPT-2 with one-third parameters. More details of human evaluation are presented in Appendix D. 

4.2. ABLATION STUDY:

This part studies the effectiveness of the learned latent ConceptFlow. Figure 2 shows golden concept coverage, effectiveness for golden concept selection and perplexity of response generation of four different strategies to select latent concepts. Base only considers central concept graph. Random, Gold, and Full add two-hop concepts in three different ways: Random selects concepts randomly, Gold selects all golden concepts with random negatives, and Full is our method that selects by learned graph attentions.

As shown in Figure 2 (a), Random has almost the same coverage with Base, while ConceptFlow (Full) performs better than Random by a large scale. This confirms the concept selection in Con-ceptFlow effectively selects more meaningful outer concepts for conversation generation. Then the effectiveness of two-hop concept selection strategies is presented in Figure 2(b) . Full outperforms all models with Precision, Recall and F1. The ConceptFlow filters unrelated concepts and chooses underlying concepts to enhance the central graph understanding.

The high-quality latent concept flow leads to ConceptFlow's advanced performances in Figure 2(c ). Interestingly, ConceptFlow even outperforms Gold in Perplexity, even Gold includes all two-hop concepts from the golden response. This shows that the "negatives" selected by ConceptFlow, even not directly appear in the target response, are also only topic and related, thus provide more meaningful information than Gold's random negatives. More results are presented in Appendix A.2. 

5. RELATED WORK:

Natural language generation (NLG) has achieved promising results with the sequence-to-sequence model (Sutskever et al., 2014) and helped build end-to-end conversation systems (Shang et al., 2015; Vinyals & Le, 2015; Li et al., 2016; Wu et al., 2019) . Recently, pre-trained language models, such as ELMO (Devlin et al., 2019) , BERT (Peters et al., 2018) and GPT-2 (Radford et al., 2016) , further improve the NLG performance with large-scale unlabeled data. Nevertheless, the degenerating irrelevant, off-topic, and non-useful response is still one of the main challenges in conversational generation (Tang et al., 2019; Zhang et al., 2018; Gao et al., 2019) .

Some work focuses on conversation generation with unstructured texts (Ghazvininejad et al., 2018; Vougiouklis et al., 2016; Xu et al., 2016; Long et al., 2017) , while others extract knowledge with Convolutional Neural Network (CNN) (Long et al., 2017) or store knowledge with memory network (Ghazvininejad et al., 2018) to generate better conversation response.

The structured knowledge graphs include rich semantics about concepts and relations. Lots of previous studies focus on domain-targeted dialog system based on domain-specific knowledge base (Xu et al., 2017; Zhu et al., 2017; Gu et al., 2016) . To generate the response with a large-scale commonsense knowledge base, Zhou et al. (2018a) and Liu et al. (2018) utilize graph attention and knowledge diffusion to select knowledge semantics for better user post understanding and response generation.

Different from previous research, ConceptFlow models the conversation flow explicitly with the commonsense graph and presents a novel attention mechanism using Graph Neural Network to guide the conversation flow in the latent concept spaces.

6. CONCLUSION:

In this paper, we present ConceptFlow, which models the conversation flow explicitly as transitions in the latent concept space in order to generate more meaningful responses. Our experiments on the Reddit conversation dataset illustrate the advantages of ConceptFlow over previous conversational systems that also use prior knowledge, as well as our fine-tuned GPT-2 systems, though the latter uses much more parameters. Our studies confirm the source of this advantage mainly derive from the high quality and high coverage latent concept flow, which is effectively captured by ConceptFlow's graph attentions. Our human evaluation demonstrates that ConceptFlow generates more appropriate and informative responses by explicit modeling of the latent conversation structure. 

B CASES:



Post:

i think it 's to get their parents to care for them . predators do n't give a crap . they target young animals because they are the weakest . CCM i think they are the weakest . they do n't give a shit about their children . GPT-2 (conv) i 'm sure they do n't care about their kids .

ConceptFlow:

i have a friend who does n't care about the parents at all . it 's a great way to get a job at once a year .

give him time ? you know he hit . 162 last year right ? red sox could n't wait to get rid of him , i do n't think he will be coming out of it CCM i think he 'll be fine . i think he 'll be fine . i think he 'll be a good starter for the next few seasons . GPT-2 (conv) i think he 'll be fine . he 's a great player , but he 's not a great player . ConceptFlow he 's not going to give up the money , but the media will just ignore him . Post there 's an offline mode for steam , and it works fine for me .

CCM i 'm not sure if it works for offline mode , but i 'm not sure if it works for steam or steam . i 'm not sure if it works for online or not . GPT-2 (conv) i 'm not sure if it 's offline or not , but i 've never had a problem with it . ConceptFlow you 're right . it 's not offline . you 're going to have to wait for the next update .

Post at this point it may be more beneficial for him to keep playing as much as he can before the next season . trick his body into thinking he is still in the season .

CCM:

i think he 's just going to be a great player . GPT-2 (conv) i 'm sure he 'll be fine . he 's a great player , but he 's not a top 10 player .

he 's been playing the same game for the last two years . he 's been playing the same way . 

the best part of the team is the way they are , and we do n't want to play the best team in the country .

:



Figures and Tables:

Figure 1: The Architecture of ConceptFlow. The latent concept flow is annotated with red arrows.

(a) Golden Concept Coverage. (b) Golden Concept Selection. (c) Response Generation.

Figure 2: Comparison of flow concept selection methods. Base only considers the central concepts. Random randomly selects two-hop concepts. Gold incorporates golden concepts in the response with random negatives. Full chooses two-hop concepts with ConceptFlow's graph attention.

Figure 3 presents a case of ConceptFlow to demonstrate model effectiveness. The attention score b ei and z g on central concepts and two-hop concepts are illustrated. The championship of zero-hop, fan of one-hop and team of two-hop receive more attention than others and are used by ConceptFlow to generate the response. On the other hand, some concepts, such as win and pretty, are filtered by the gate σ. More examples are listed in Appendix B.

Figure 3: Case Study (Best viewed in color). Left: Attention flow in commonsense concept graph, where zero-hop concepts, one-hop concepts and two-hop concepts are highlighted. Right: Attention scores over all concepts. Darker green indicates higher attention scores.

Dataset. All experiments use the Commonsense Conversation Dataset(Zhou et al., 2018a), which collects single-round dialogs from the Reddit site. This dataset contains 3,384,185 training pairs, 10,000 validation pairs and 20,000 test pairs. ConceptNet is used for our commonsense graph. It contains 120,850 triples, 21,471 concepts and 44 relations. For each example in the Commonsense Conversation Dataset, the average number of central concepts and two-hop concepts are 98.6 and 782.2, respectively.

Table content: <table/>

Relevance Between Generated and Golden Responses. The PPL results * of GPT-2 is not directly comparable because of the different vocabulary. More results can be found in Appendix A.1.

Table content: <table><tr><td>Model</td><td colspan="5">Bleu-4 Nist-4 Rouge-1 Rouge-2 Rouge-L Meteor</td><td>PPL</td></tr><tr><td>Seq2Seq</td><td>0.0098 1.1069</td><td>0.1441</td><td>0.0189</td><td>0.1146</td><td colspan="2">0.0611 48.79</td></tr><tr><td>MemNet</td><td>0.0112 1.1977</td><td>0.1523</td><td>0.0215</td><td>0.1213</td><td colspan="2">0.0632 47.38</td></tr><tr><td>CopyNet</td><td>0.0106 1.0788</td><td>0.1472</td><td>0.0211</td><td>0.1153</td><td colspan="2">0.0610 43.28</td></tr><tr><td>CCM</td><td>0.0084 0.9095</td><td>0.1538</td><td>0.0211</td><td>0.1245</td><td colspan="2">0.0630 42.91</td></tr><tr><td colspan="2">GPT-2 (lang) 0.0162 1.0844</td><td>0.1321</td><td>0.0117</td><td>0.1046</td><td colspan="2">0.0637 29.08  *</td></tr><tr><td colspan="2">GPT-2 (conv) 0.0124 1.1763</td><td>0.1514</td><td>0.0222</td><td>0.1212</td><td colspan="2">0.0629 24.55  *</td></tr><tr><td colspan="2">ConceptFlow 0.0246 1.8329</td><td>0.2280</td><td>0.0469</td><td>0.1888</td><td colspan="2">0.0942 29.90</td></tr></table>

Diversity and Novelty of Generated Response. Diversity is calculated within generated responses; Novelty is compared to the input post. More results are in Appendix A.1.

Table content: <table><tr><td/><td>Diversity(↑)</td><td colspan="3">Novelty w.r.t. Input(↓)</td></tr><tr><td>Model</td><td colspan="5">Dist-1 Dist-2 Ent-4 Bleu-4 Nist-4 Rouge-2 Rouge-L Meteor</td></tr><tr><td>Seq2Seq</td><td colspan="2">0.0123 0.0525 7.665 0.0129 1.3339</td><td>0.0262</td><td>0.1328</td><td>0.0702</td></tr><tr><td>MemNet</td><td colspan="2">0.0211 0.0931 8.418 0.0408 2.0348</td><td>0.0621</td><td>0.1785</td><td>0.0914</td></tr><tr><td>CopyNet</td><td colspan="2">0.0223 0.0988 8.422 0.0341 1.8088</td><td>0.0548</td><td>0.1653</td><td>0.0873</td></tr><tr><td>CCM</td><td colspan="2">0.0146 0.0643 7.847 0.0218 1.3127</td><td>0.0424</td><td>0.1581</td><td>0.0813</td></tr><tr><td colspan="3">GPT-2 (lang) 0.0325 0.2461 11.65 0.0292 1.7461</td><td>0.0359</td><td>0.1436</td><td>0.0877</td></tr><tr><td colspan="3">GPT-2 (conv) 0.0266 0.1218 8.546 0.0789 2.5493</td><td>0.0938</td><td>0.2093</td><td>0.1080</td></tr><tr><td colspan="3">ConceptFlow 0.0223 0.1228 10.27 0.0126 1.4749</td><td>0.0258</td><td>0.1386</td><td>0.0761</td></tr></table>

Human Evaluation on Appropriate (Appro.) and Informativeness (Infor.). The Average Score calculates the average of human judgment score. Best@1 Ratio indicates the fraction of judges consider the case as the best. The parameter number is also presented.

Table content: <table><tr><td>Model</td><td>Parameters</td><td>Average Score Appro. Infor. Appro. Infor. Best@1 Ratio</td></tr><tr><td>CCM</td><td>35.6M</td><td>1.802 1.802 17.0% 15.6%</td></tr><tr><td>GPT-2 (conv)</td><td>124M</td><td>2.100 1.992 26.2% 23.6%</td></tr><tr><td>ConceptFlow</td><td>35.3M</td><td>2.690 2.192 30.4% 25.6%</td></tr><tr><td>Golden Response</td><td>-</td><td>2.902 3.110 67.4% 81.8%</td></tr></table>

Some Cases Generated by Different Models. Tokens from seed concepts, one-hop concepts and two-hop concepts are highlighted. (conv) i 'm pretty sure it was a room with a door .ConceptFlow good luck , i 'm sure there are some good things in the house .

Table content: <table><tr><td>Post</td><td>actually i stayed at the building right next to the lighthouse . i believe it was a boiler room . another group was already sleeping at the lighthouse .</td></tr><tr><td>CCM</td><td>i 'm not sure if you 're joking , but i 'm not sure if you 're talking about the lighthouse . i 'm not sure if it was a blanket or a blanket .</td></tr><tr><td>GPT-2</td><td/></tr></table>

Post no i 've never felt like i had friends whoops CCM i 'm sorry , i feel you . GPT-2 (conv) i 've never felt like i had friends whoops . i 've never felt like i 've had friends whoops . ConceptFlow you 're gonna have to call your friend ! Post i 've known how to change the music , it does n't feel natural . thanks for the advice though . CCM i 'm not sure if you 're joking , but i 'm not sure if you 're being sarcastic or not . i 'm not sure if you 're being sarcastic or not , but i 'm not sure if you 're being sarcastic or not . i 'm pretty sure it 's just a video . GPT-2 (conv) i 've been playing with a lot of different music and it 's always been the same . ConceptFlow i did n't even notice the natural sound . i 'll check it out ! Post are milan and inter no longer top clubs given that they currently do n't have the finances to compete with the best ? CCM milan is n't a top club , but they are n't top clubs . they are n't the only ones who are n't in the world . GPT-2 (conv) i think they are . they are a club that has been in the top 4 for a long time .

Table content: <table/>

Peer review (iclr2020):
This paper proposes to make use of commonsense knowledge base in conversation generation. The general idea of introducing such KB is not wrong. The main approach of the proposed method is in Secs 2.2.2 and 2.2.3. Unfortuantely, the paper reads like code comments, rather than a scientific paper. Most of the model components are not well motivated. For example, the attention score of concepts is computed by softmax(.) * PageRank. The authors fail to justify why this formula is approximate and better than alternatives. It doesn*t look good to me. Suppose the prompt is about a rare concepts, whose neighbors are also rare. Then the PageRank scores are low for all neighbors, resulting in very low attention weights. A better approach seems to be post-normalization. The ablation test compares the model with random and golden concepts. However, it is unclear how each proposed model component (e.g., Central flow encoding and outer flow encoding) plays a role in the entire model. Also, it*s unclear how the proposed model is compared with an obvious and seemingly better treatment, for example, softmax(.) * PageRank vs. softmax(. + log PageRank). Besides, the paper does not tackle the key issue of multi-hop reasoning. One of the key problems is that the number of neighbors grows exponentially wrt the number of hops. Therefore, I would expect some routing techniques (for example, trained by RL) that maintain a list of tractable paths, but with more steps of hops. The paper instead proposes some esoteric attention mechanisms on one-hop/two-hop neighbors. In short, the proposed model is neither well motivated nor well tested. The technique is also pretty superficial. Minor: There*re some formatting issues. For example e_j -> e_i is better formatted as e_j ightarrow e_i.