Knowledge-Consistent Dialogue Generation with Knowledge Graphs
Authors: Minki Kang
Jin Myung Kwak
Jinheon Baek
Sung Ju Hwang
Venue: NeurIPS 2022 Submitted (2022)
Keywords: knowledge-grounded dialogue generation
knowledge graph

Abstract: Pre-trained generative language models have achieved impressive performances on dialogue generation tasks. However, when generating responses for a conversation that requires complicated factual knowledge, they are far from perfect, due to the lack of mechanisms to retrieve, encode, and reflect the knowledge in the generated responses. Unlike the methods working with unstructured text that are inefficient in retrieving and encoding the knowledge, some of the knowledge-grounded dialogue generation methods tackle this problem by leveraging the structured knowledge from the Knowledge Graphs (KGs). However, existing methods do not guarantee that the language model utilizes a relevant piece of knowledge for the given dialogue, and that the model generates dialogues which are consistent with the knowledge, from the KG. To overcome this limitation, we propose SUbgraph Retrieval-augmented GEneration (SURGE), a framework for generating knowledge-consistent, context-relevant dialogues with a KG. Specifically, our method first retrieves the relevant subgraph from the given KG, and then enforces consistency across the facts by perturbing their word embeddings conditioned on the retrieved subgraph. Then, it learns the latent representation space using graph-text multi-modal contrastive learning which ensures that the generated texts have high similarity to the retrieved subgraphs. We validate the performance of our SURGE framework on the OpendialKG dataset and show that our method does generate high-quality dialogues that faithfully reflect the knowledge from the KG. 

While retrieving the relevant documents from large-scale text corpus with information retrieval techniques significantly boosts the performance of dialogue agents [18, 24), the computational burden of searching for the relevant documents and embedding them on the fly could be high, which may compromise the responsiveness of the conversation agent. Thus, we instead consider the approach that utilizes the pre-compiled Knowledge Graph (KG) [2, 43] consisting of symbolic facts, which represent the entities as nodes and their relations as edges, in the form of a triplet, e.g., (Pride & Prejudice, written by: Jane Austen). Such KG-augmented dialogue generation models are highly efficient compared to retrieving from and augmenting with unstructured texts. This is because we Knowledge-Consistent Dialogue Generation with Knowledge Graphs
Anonymous Author(s)
Affiliation Address email
Abstract
Pre-trained generative language models have achieved impressive performances on dialogue generation tasks. However, when generating responses for conversation that requires complicated factual knowledge, they are far from perfect, due to the lack of mechanisms retrieve, encode, and reflect the knowledge in the generated responses. Unlike the methods working with unstructured text that are ineffi- cient in retrieving and encoding the knowledge, some of the knowledge-groundee dialogue generation methods tackle this problem by leveraging the structured knowledge from the Knowledge Graphs (KGs). However, existing methods do not guarantee that the language model utilizes a relevant piece of knowledge for the given dialogue, and that the model generates dialogues which are consis- tent with the knowledge, from the KG. To overcome this limitation, we propose SUbgraph netrieval-augmennt ing knowledge Knowledge-consistenn, consistent,
3
5
10
11
12
GEneration (SURGE), framework for generat- context-relevant dialogues with KG. Specifically, our method first retrieves the relevant subgraph from the given KG, and then enforces consistency across the facts by perturbing their word embeddings conditioned on the retrieved subgraph. Then, it learns the latent representation space using graph-text multi-modal contrastive learning which ensures that the generated texts have high similarity to the retrieved subgraphs. We validate the performance of our SURGE framework on the OpendialKG dataset and show that our method does generate high-quality dialogues that faithfully reflect the knowledge from the KG.
13
14
15
16
18
20
Introduction
21
Dialogue systems aim at conversing with humans by generating human-like responses, considering the context and history of the dialogue. Recently, thanks to the development of pre-trained language models (PLMs) for text generation [32, 34], neural dialogue agents are able generate fluent re- sponses. However, despite their satisfactory fluency, they often generate factually incorrect responses 26 due to lack of explicit knowledge. The problem can become worse, when the conversation requires accurate knowledge about certain subjects. Thus, to overcome such limitations, some of the recent methods access the external knowledge sources, for example, Wikipedia [7] Web [21], and then retrieve the documents containing the relevant knowledge for ongoing conversations.
29
25
27
28
29
31
39
34
35
36
37
Submitted to 36th Conference on Neural Information Processing Systems (NeurIPS 2022). Do not distribute.

---

Knowledge Graph
writer
User
Issue I. Inrelevant knowlodge
Issue 2. Inconsistent Generation
Could you recommend any book by Jane Austen?
1ap KG
87.4%
12.6%
Factually wrong with external entiyy
Jane Austen
Lady
Susan
Assist.ant
Arelewasr knowledge
Gold bnowe
Mary M.ri
???
Distracted by Irrelevant facts
Aus tatant
Assistant
Pride Prejudice
Sure, she wrote
Sure, she wrote The Secret Life of Man Who Was Born.
Steventon
Dialogue History
Steventon.
Figure I: Motivation. Existing wennwlldggrooudeee dialogue generation models with KG utilize the multi-hop subgraph for entities in the dialogue context (Jane Austen). However, they suffer from the following two problems: (1) irrelevant knowledge where only 12.6% of facts from 1-hop KG are useful to generate the target responses given a dialogue context, and (2) inconsistent generation including the factually wrong statement.
38 can directly retrieve entities from the context without searching for all candidate documents from
39 large text corpus (e.g. Wikipedia), and the retrieved facts succinctly encode the required knowledge in the most compact and effective form. Figure 1 shows example which illustrates how the KG represents facts as relations among entities, that help generate a knowledge roddeegg response.
41
42 Few recent works [41, 9, 50] use the KG to provide facts associated with the entities in the dialogue context to the conversation agents. However, they utilize all the triplets associated to the given entity, whose facts are mostly irrelevant to the dialogue context (e.g., Jane Austen was born in Steventon in Figure 1), which could mislead the model into generating factually incorrect responses. We found that about 87% of facts from 1-hop KG are irrelevant to the context in the OpendialKG dataset [29]. Moreover, encoding all the facts including the unnecessary ones is computationally inefficient [9].
43
44
45
46
47
48
Even after correctly retrieving the relevant facts from the KG, it is not straightforward to combine 49 the representations from two heterogeneous modalities: the dialogue context is represented as a text, meanwhile, the knowledge is represented as graph. Moreover, since the PLMs already have tons of pre-trained parameters trained on the unstructured texts, properly conditioning the structured graph to the PLM is highly important. If not done so, PLMs may generate inconsistent responses with regard
51
53 to the knowledge from the retrieved subgraph, whose phenomenon is known as hallucination [37, 8]
54 where PLMs generate responses with their own memorized yet incorrect knowledge.
55 In this work, we tackle such challenging and fundamental issues of mnowledge-consiseen dialogue
56 generation with KG In particular, at the first step, we propose context-relevant subgraph retrieval that retrieves only the relevant triplets from a large KG to prevent the model from generating context-
57
58 irrelevant responses. Notably, our subgraph retrieval method is end-to-end trainable jointly with
59 the generation objective by marginalizing the likelihood of the generated sentences over the latent variable of the retrieved subgraph [24]. Then, to encode the retrieved subgraph along with the input text sequence, we propose a graph encoding that is permutation and relation inversion invariant yet efficient. Furthermore, to ensure that the model does make use of the encoded knowledge when generating responses, we propose a multi-modal contrastive learning objective between the two different graph-text modalities to enforce the consistency across the retrieved facts and the generated texts. We refer to our framework as SUbgraph aetrieval-augmentt GEneration (SURGE).
60
61
63
65
66 We validate our SURGE framework on the OpendialKG [29] dataset against relevant baselines, with the T5-s [34] model as the base PLM. When evaluating the generated responses from the dialogue agents, the conventional metrics (e.g. BLEU [31], Rouge [27]) can not measure how faithfully the generated responses reflect the world knowledge. Thus, we introduce an additional performance metric, referred to as Knowledge-verffying Question Answering (KQA), which evaluates whether generated responses contain the correct knowledge with an extractive question answering model. The experimental results show that SURGE generates responses that not only agree with the gold knowledge but are also consistent with the retrieved knowledge from the KG.
67
68
69
71
73
74
Our main contributions can be summarized as follows:
75
We propose a context-relevant subgraph retrieval method for knowledge graph-augment dialogue generation, to extract only the relevant piece of the knowledge for the given context from the entire knowledge graph, for generating more appropriate responses to the ongoing conversation.
76
77
We propose invariant yet efficient graph encoder and multi-modal graph-text contrastive learning objective to ensure that the generated responses faithfully reflect the retrieved knowledge.
79
80
We validate SURGE against relevant baselines, demonstrating its efficacy in generating responses
81 that are more informative by retrieving and reflecting the relevant knowledge from the KG.
In this work, we denote the knowledge facts (i.e., set of triplets) in the knowledge graph.

---

82 2 Related Work
Pre-trained Language Model Large Pre-trained Language Models (PLMs) [32, 23, 34] that use the encoder-decoder architecture based on Transformers [42] have achieved great successes on language generation tasks. As they can accurately contextualize the given context and then generate human-like sentences, they are often used as the base architecture for the neural dialogue systems [49, 15]. Moreover, when the PLMs become larger, dialogue generation models have shown to generate high-quality responses [1], suggesting that pre-trained parameters do contain certain knowledge. However, despite the fluency of such PLM-based dialogue agents, they often generate factually incorrect responses that are unfaithful to the dialogue context but look plausible - widely known as a hallucination problem [28]. Thus, generating responses requiring specific and valid factual knowledge is still challenging. To tackle this, recent works propose to retrieve knowledge from external sources, and then use augment the neural dialogue agents [37, 40], discussed below.
84
85
86
87
88
89
90
91
92
93
94 Bhowledge-Groundee Dialogue The sources of external knowledge can be categorized into two types: documents from large unstructured corpora such Wikipedia [7] Web [30], and symbolic facts from Knowledge Graphs (KGs) [2. 43]. Firstly, Dinan et al. [7] propose a retrieval-based dialogue generation model, which links the pre-compiled documents retrieved from Wikipedia articles with the given dialogue context using the information retrieval [3]. Further, several works [19, 25, 40] propose to learn the document retrievers in end-to- and-to-end fashion, to generate the stowledge-groundee responses for the given dialogue. However, K KG-augmented dialogue generation models, which use structured KGs, are more efficient than the previous methods utilizing unstructured texts thanks the efficacy of KG for encoding knowledge, and consequently thus more preferable when responsiveness is important [26]. Regarding the dialogue generation with the KG, Moon et al. [29] introduce knowledge- googgggg-godedd dialogue dataset where each dialogue comes with the large-scale KG. Before the era of pre-trained language models, several works [41, 45, 48. 5, 50] have suggested sequence- to-sequence models that generate dialogue by conditioning the output word distribution with the entities from the KG. Further, Galetzka al. [9] propose an efficient way to encode all of the facts in the h-hop neighbors of the entities that appear in the dialogue history in the given KG, in order to reduce the number of input tokens used in the pre- pre-trained language model [32]. However, all of these methods simply match and retrieve all the facts for entities including irrelevant ones that appear in the dialogue history, which may mislead the agent into generating out-of-context responses. Our work differs from these existing works, since we aim at retrieving only the context-relevant subgraph among the I-hop facts with a novel subgraph retriever, which is end-to-end trainable along with the dialogue generation model.
96
97
98
99
100
101
102
103
104
105
108
107
108
109
110
111
112
113
3 Method
115
116
We first discuss the basic ingredients: graph neural networks and transformers. We then formalize the dialogue generation problem and describe the key components for our SUbgraph Retrieval-augne -ugmeettd
117
118
GEneration (SURGE) framework: context-relevant subgraph retrieval, invariant graph encoding, and graph-text contrastive learning. Figure 2 illustrates the overview of our framework.
120
3.1
Preliminaries
As we two different modalities, namely text and graph, we first define them, and then describe the neural networks to encode them. In particular, a text is defined as a sequence of tokens æ = (.., w, Vas, V. where v; is token and V is pre-defined vocabulary formed with specific tokenization algorithms [39]. On the other hand, a knowledge graph (KG) is a type of multi-relational graphs G {(e). r, } E & x R E, where е₁, and e are head and tail entities along with their relation r, and E and R are sets of entities and relations, respectively, i.c., en, e₁ E E and r E R.
122
123
124
125
128
127 To casily access different modalities in the same framework, we define the mapping function that maps entities and relations in the KG to the tokens in the text as follows: qe È + V* and q, R - V In other words, any entity e E E and relation r R can be mapped to sequence of l tokens
128
Vt: ge = x, and qr (x) = x,-- Such functions enable us to associate the KG symbol with the text.
129
130
131
Transformer A Transformer [42] is a neural architecture that embeds sequence of tokens while taking their relationships into account. It a basic building block of recent PLMs [6. 32]. Formally, assume that we have a sequence of tokens x = [21, ..., IN), Var E V. then goal of generative transformers is to generate a sequence of tokens ycr = [yi,..., Ut-1] 1), Vy: E V. with encoder Enc, decoder Dec and tokens' embedding function f. Thus, hidden state at time t for generating
132
134
3

---

Context-Relevant Subgraph Retriever Py (2)x)
Gruph-Teat Contrastive Learning
Knowledge Graph (f)
Austen;
Iron man 3
Lady
Iron Man
XB peeree
Pride Prejudioe;
Stan Lee
vrittentr
Subgruph (2)
Negative Subgraph (2')
ptece NCrrsh
E
Decodar Hadlen Siawe A
User
d(z)
Topa
Graph Encoding
Generate
Could you
Sivibuity
* (x,
(u(y, 2)
recommend any book
Sure, she wrote Pride
Jane Austen?
Fadedkinng
Tetioval Tica
(2(2)
Encoder
Decoder
& Prejudice and Lady
Dialogue
Jane Austen Lady Suran Fride & Prejucice
Susan.
Ilistory (x)
Could you recommmend bny bock by Jane.
Generator (PLM)
Figure 2: Framework Overview. Our framework, SURGE. consists of three parts. First, context-relevant
subgraph retriever P.o (2(w) retrieves the subgraph Z relevant to given dialogue history æ from knowledge
graph G (e.g.. 1-hop KG from entity Jane Austen; a). Specifically, we measure the similarity of a context and triplet embedding to compose the retrieval distribution Pe (2 x)) ($ 3.3). Then, we encode the retrieved subgraph 2 into the imput of the generator, using the graph encoding function (a 2) ($ 3.4), Finally, contrastive learning to enforce the model to generate a consistent response with the retrieved subgraph ($ 3.5).
Y1 is h4 = (ec(Enc( X), Y=,), where X = f(x) = [(( (xx)) and Yer f(yer) = (y1),, ), ..., J 1)]. We note that both Enc and Dec functions are permutation sensitive with positional encodings as in generic Transformer architecture [42, 46].
136
137
138
139
Graph Neural Network A Graph Neural Network (GNN) represents a node with its neighboring
140
nodes over the graph structure [12]. which is formalized as follows:
(1) where e, and е₁, are embeddings of entities (nodes) e, and eh, respectively, 9) fer (en, r, ei) 9} is set of neighboring entities of e,. AGG is function that aggregates embeddings of e, 's neighboring entities, and UPD is function that updates a representation of e, with the aggregated messages from AGG, at each iteration k.
e, (k+1)
CNN(*)
(c(*) = (K) AGG (fe).) Vea e((er))))),
141
143
145
3.2
Problem Statement
Here we formalize the problem of context-relevant subgraph retrieval for grounded dialogue generation. Given dialogue history x = [x1,...,,,,, ]. a model with generative PLMs first encodes the input tokens, and then models a probabilistic distribution p(y x) to generate output response = 1/1. ....... vrl. This problem requires a piece of specific knowledge for a conversation.
147
149
150
To that end, given dialogue history æ, we aim at retrieving subgraph 2 C G consisting of a set of triplets= = E where = = (eh. r, e;), which encodes relevant knowledge for ongoing conversation. Thus, the distribution of the context-relevant facts 2 is p( 2 (z)) and our final likelihood of generating responses then becomes p(ylar, 2). Then, to jointly optimize the objective of graph retrieval with response generation, we treat Z as latent variable and then marginalize the likelihood of the generative model over all possible latent variables for retrieved subgraphs Z, formalized as follows:
151
152
153
155
p(yll=) = Σ (6(2))) Po(y 2)= - II po (v a v1:111 ZCG
ZCG
where p (2 x)) is output distribution of the context-relevant subgraph retriever, and (o Z) is the target distribution of a snowledge-augmentee generator, parameterized as á and 6, respectively.
157
158
3.3
Context-Relevant Subgraph Retriever
159
We now provide a concrete description of our context-relevant subgraph retriever formalized in Eq. 2. We assume that a retrieval probability of each triplet in Z {21,... Zn } is independent. Then, for simplicity, we decompose the probability of retrieving a set of triplets p( p()zz) into the product of individual triplet retrieval probabilities, as follows: p( p(Z(x) p(21 (e) (x)p(((2))))) (2 (x). ... p(_,, (x).
161
162
163
From the aforementioned Eq. about pf p(Z)z), we can now focus on retrieving the only one triplet. Therefore, we define the retrieval of one triplet with an inner product of dense vectors between the
184
165
dialogue history x and the candidate triplet z, similarly to a dense retrieval model [11], J, as follows:
(2(2) exp(d(z) Tx()).
(3)

---

166
where d is a triplet embedding function and is a dialogue context embedding function. We can use a PLM for implementing q. but we need another effective method for that can reflect the property of the graph. Therefore, we propose the GNN-based tripler embedding method for realizing d.
167
168
169
Let consider a set of triplets associated the entities that appear in the given dialogue context {(e, r, et) or (e,.I,e) (e(e) S @)) as the retrieval candidates. To effectively represent the triplets consisting of entities and their relations as items, we use GNNs described in Section 3.1 for the triplet embedding function d. In our triplet retrieval, representing both nodes and edges, which are equally essential components for the multi-relational graph, is worthwhile represent an entire triplet. To
170
171
172
173
174
do so, we adopt the existing edge message passing framework [17] that transforms edges of the original graph to nodes of the dual hypergraph [38] (i.e., transforming 9 to G*), which allows us to
175
178
use existing node-level GNNs for representing edges of the original graph (See Section D.1 of the Supplementary File for more details). Formally, our triplet embedding function denoted as follows:
177
d(z) = -ML(((e>
|| |r || e,)), ea (eN(e 9), r = GNN(r: = CNN(ee; 9),
(4)
178
where: 2 = (en, r; et), and is the concatenation operator.
179
3.4 Invariant Graph Encoding
180
In this subsection, we then now specify the remaining operations for po æ. 2), which generates y conditioned on the two different modalities, namely text x and graph Σ. Before doing so, we first define the notion of graph encoding, whose goal is to leverage the retrieved subgraph information along with the dialogue history for response generation, which is formalized in Definition 3.1.
182
183
184
Definition 3.1. (Graph Encoding) Let sb(xr 2) be a graph encoding function. Then, given [_1,...,,N] and a subgraph 2, it first yields a new sequence x' = [K',..,, I'm Ii...., IN] where (i!....,''n, comes from qe(e) = I's and q-(r)) xt (e, +) E Z. Then, it embeds sequence X' = [f(x')...,, f(
185
sequence of tokens
186
((N)) f(111, Ln, I1,,.., INI), where f is the token embedding function, Consequently, X' 1/ x'= 2).
187
188
For instance, given sequence from the retriever, 2b 2) = f((a, d, b, b, e, a, a, d, c, .i.... xN\) with a = (a(a), = (b().. q=(c), d = (d), g- (e). which we term as the naive encoding. Due to its simplicity, it is widely used for a text-conditioned generation [24]. However, for graph encoding, it violates two important invariance properties: permutation invariance [47] and relation-inversion invariance, which
[x1.................. ÆN] and a subgraph 2 == {(a, d. b), (b. e. a. d.)
190
191
192
198
194
are formalized in Definition 3.2, 3.3.
195
Definition 3.2. (Permutation Invariance) For any set permutation n. (b, 2) = (b( n. 2), iee, an order of elements in a subgraph does not affect representation.
196
197
Definition 3.3. (Relation Inversion Invariance) Let a relation - d be an inverse relation to d, if d, b) = (b, -d, a) Va, b E E. Then, 1b(x, 2 U ((a, d, b)) <b(2, 2 U {(b, -d, a)}) for any subgraph 2.
198
199
Invariant Graph Encoding To meet both properties, we consider two additional operations set of triplets up to the navve encoding. We first define a SORT operator that returns the same output regardless of the order of input set elements, as follows:
201
202
SORT(= : Z) = SORT( Z), VT, T' E S,.,
(5)
where Sn is set of all possible permutations for elements. Moreover, we define INV operator that adds the inverse triplet of each triplet in the subgraph Z, as follows:
204
INV 2 2 u {(er, I, ex) (en, r, ex) E 2).
205
With above operations, we now define a more solid graph encoding function: (b SORT(INV(())). which satisfies both permutation and relation inversion invariance.
206
207
Invariant and Efficient Graph Encoding However, above encoding is not efficient since it requires the ((n) space complexity for encoding a graph with n triplets. To be more efficient, we newly define d that only encodes the unique nodes (entities) along the sequence, formalized as follows:
208
209
(xc, SORT = s((a, b, c, 21,.,,... xxN)),
210
where ENT(2) 2) returns the set of unique nodes in 2 and SORT is used to preserve the permutation invariance. This encoding is thus invariant but efficient since it only costs Okk), for k-entity
5

---

212 sequence where & < n. However, as it does not consider the relational information in Z. we further 213 perturb the entities' token embeddings with respect to their representations in Z. Specifically, for each entity E 22) we apply affine transformations from learnable Multi-Laayrr Perceptrons (MLP) on the token embedding of as follows:
215
B(f 2) = (1+ f(a) + 5,
7 = MLP1 (m) 8 (T) 7 UPD(f(o), AGGR( vb E M(a 2)))),
where G: R + Rd perturbs the embedding according to Z, AGGR is the relation- aware aggregation function for triplet (b, a) E 2 with (a)(a) and b = g..(b). In sum, we denote relation-aware invariant and efficient encoder 1p" formally represented as follows:
216
218
s (a,
[z. 2) = (( SORT (ERT(EN(()))) INV(2)),
where 3 can be applied to the sequence of representations, B Raad - Raxxd We conclude that our graph encoding satisfies both properties. For proofs, please see Section C of the Supplementary File.
219
220
221
3.5
Consistent Generation with Graph-Text Contrastive Learning
222
Although the previous schemes allow retrieving and encoding subgraphs that are relevant to the input dialogue history, the consistent generation with the given subgraph is further required, when generating responses with the factual knowledge. In other words, the model should be able to generate different sequences given different graphs with the same dialogue history.
223
224
225
226
However, we only access the single ground-truth response regardless of the retrieved knowledge, while the generative model is trained with teacher forcing. Thus, this setting can give rise to the problem of exposure bias [35]: the model is never exposed to other generated tokens during training. To overcome such limitations,
228
introduce novel graph-text contrastive learning method motivated by multi-modal contrastive learning [33]. Formally, for a single pair of a graph and text, the contrastive learning objective is defined follows:
230
exp( exp(siim(p) (<))), (()))/T) + 2 log exp(sim ((h)))
log Leont 2
(8)
En exp sim ((()),
Σ. ecp(sim/ (((2))
where 1 Em
m i=1 2 is the mean of graph representations from [z 2/m Z1 xx]
232
233 Enc(ab"(( (xx 2)). + Σt h4 is the mean of decoder representations, sim is the cosine similarity,
234
and & are learnable linear projection layers, and is learnable temperature parameter. Furthermore, En and Σ,, indicate the summation over negative samples, which are other texts or graphs within same mini-batch as in previous contrastive literature [4, 10, 13, 20, 22, 33J. With Eq. 8, the model can embed the correlated pairs closer together order to generate a consistent response to a given graph, i.e., given different graph, the model would generate different tokens for the same text.
235
237
239
3.6
Training
We train the entire model by maximizing the log-likelihood l(gl(r)) defined in Eq. 2 with respect to parameters of the retriever & and the generator However, computing the marginal probability over all possible subgraphs Σzcg Po Pa(Z)z)pp()) Z) is impractical. Therefore, as in existing works [11, 24], we approximate this by instead summing over h sampled subgraphs. Moreover, for each subgraph, we samples n triplets without replacement from the categorical distribution, parameterized by di 21, An Cal((g), (=(xx)) which results in Z = { }.
241
244
246
Our whole end-to-end training objective for retrieval augmentted generation is then defined follows:
247
Cret log Σ (2( Z.), Zz)x)
248
where we simplify the sampling over n triplets as the sampling over the subgraph distribution Ps (Z(z). We assume that we can access the gold subgraph for some data in training. Thus, we further add the supervised retrieval loss to introduce a semi-supervised retriever learning as follows:
249
250
Laup log Ps 2 (z))
(10)
251
where 2 is the available ground-truth subgraph. Combining all objectives in Eq. 8, 9, and 10, our final training objective is then defined as follows: L = Lret + Loup Lcont-
252

---

User
veriee
QA Context: Could you recommend a book written by Jane Austen?
Could you recommend a book written
Jun: Auric
Lady Sueen
by
Jane Austen
(b) Corresponding Fact
<Generated Responses
Cold
Response
Sure,
Lady Susan
Question: June Austen
?)
Accuptable Juspenses
She wrote
Emaa
Cancidate Searching
Sonse
Answer Candidates:
Lady Susan
about
Sense & Sensibility
Jaze Auaion
Sczubilty
Sense Sensibility
Emma
(a) Dialogue History
Jane AnarOn
(c) Candidates KG
(d) KQA example
Figure 3: KQA. (Left) An example where multiple responses are acceptable but the gold response cannot reflect all of them. (Middle) We first find the fact from the KG that reflects the relation between entities within the user input and gold response (b), and then search candidate facts from the KG (c). (Right) Corresponding KQA example. If a generated response contains the one of answer candidates, the KQA can predict it (success).
4
A Novel Metric: Knowledge QA
254
Existing automatic evaluation metrics, namely BLEU and ROUGE [31, 27], are limited in that they only consider the lexical overlaps of words without measuring the factual correctness of the generated responses. As shown in Figure 3 (a), there could be multiple correct responses, but existing metrics score them lower due to the lexical mismatch. To solve this issue, we propose Knowledge-verify Question Answering (KQA) which measures whether generated responses contain factually correct knowledge given the dialogue history. Compared to the existing metrics using question generation methods [14, 44], we automatically derive QA pairs for evaluation from the dialogue and the large- scale KG [2]. In particular, at first, the ground truth is entity mentioned in the gold response, and question is composed of an entity in a dialogue and the existing relation between two entities. Once
256
258
260
262
263
the question is determined, more answer candidates can be found through the searching in KG, which allows more acceptable responses that contain the correct knowledge as in Figure 3 (d).
264
5
Experiment
265
286
5.1
Experimental Setup
257 We conduct experiments on the OpendialKG dataset [29], which is a dialogue corpus associated with a large-scale Knowledge Graph (KG), namely Freebase [2] with 100k entities and IM facts. This dataset has 15K dialogues with 91K utterances. We note that 49% of the responses come with the gold knowledge, whereas others are not. Since the dataset does not provide a predefined data split, we randomly split it into train (70%), validation (15%), and test sets (15%). We use T5-small [34] for all experiments. We also conduct experiments on another dataset and language model and present the results in Section E of the Supplementary File. For details, see Section D of the Supplementary File.
268
289
270
271
272
273
274
5.2
Baselines and Our Models
275
We compare different variants of our SURGE framework against various KG-augmented dialogue generation models. No Knowledge. This model is only provided with the dialog history. thus no external knowledge is used. All Knowledge. This model provided with entire facts within I-hop subgraph of entities associated with the dialog history. Gold Knowledge. This model is provided with the exact gold knowledge, even in the test time if the gold knowledge exists. Space Efficient Encoding. This model takes all facts from the 1-hop subgraph of the entities as input. We use two different encoding methods introduced in [9], namely Space Efficient (series) and Space Efficient (parallel). EARL. This is an RNN-based model, where the entities are conditioned in response generation [50]. Random/Sparse Retrieval These models are provided with selected facts from a 1 hhhop subgraph, via the random sampling or the sparse retrieval BM25 [36]. Text-based Retrieval. This model is a variant of our framework where T5 encoder [34] used for d in Eq. 4 instead of GNNs similar to [16]. SURGE (unsupervised). Ours with retrieved context-relevant facts from
278
277
278
279
280
281
283
285
287
1- hhhop subgraph, where the retricval is trained without any supervision. SURGE semmisuperrvied).
Ours but the retriever is trained with supervision if the data has a gold fact. SURGE (contrastive).
289
Our full model jointly trains the retriever in a semi- eemi-supervised manner with the contrastive learning term. By default, all our models are trained with an invariant and efficient graph encoding.
5.3
Evaluation Metrics
252 We evaluate the generated responses using BLEU [31], ROUGE [27] and unigram overlap (F1) with the gold response. Along with these conventional text evaluation metrics, we also evaluate the results
7

---

Table I: Experimental results OpendialKG dataset.
indicates the model under the oracle setting using the
gold facts even the test time.
KQA
BLEU
ROUGE
Unigram
Method
EM
F1
B-1
B-2
B-3
B-4
R-1
R-2
R-L
Fi
No Knowledge
7.62
13.2
15.79
9.19
5.61
3.43
19.67
7.13
19.02
22.21
All Knowledge
30.06
34.95
15.95
9.98
6.72
4.65
20.96
8.50
20.21
24.34
Baselines
Space Efficient (series)
26.88
31.15
16.15
10.03
6.66
4.50
21.15
8.56
20.44
24.55
Space Efficient (parallel) EARI
28.90
33.19
16.33
10.22
6,81
4.64
21.42
8.85
20.68
24.87
24.52
27.09
11.49
6.34
4,06
2.75
15.36
4.37
14.61
16.88
Retrieval
Random Retrieval
21,05
26.09
15.70
9.52
6.12
3.99
20.21
7.88
19.55
23.28
variants
Sparse Retrieval (BM25)
19.32
24.55
15.63
9.44
6.05
3.96
20.05
7.67
19.37
23.10
Text-based Retrieval
31,00
35:95
16.87
10.64
7.23
5.07
20.63
8.53
19.89
24.16
SURGE (unsuperviiedd
37.35
42.24
18.10
11.65
7.99
5.59
22.14
9.50
21.23
25.91
Ours
SURGE (sent-supervsse)
39.57
44.13
18.21
11.74
8.08
5.68
22.11
9.41
21.22
25.91
SURGE (contrastive)
39.52
43.96
17.72
11.53
7.96
5.61
22.19
9.77
21.34
25.94
Oracle
Gold Knowledge
49.76
53.41
18.47
12.79
9.32
6.92
24.93
11.97
24.03
28.82
Gold Response
83.88
22
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
Unigram-
FM
4.
Retrieval Results
Table 2: Results knowledge-consistent response generation, where we compare three variants of our SURGE - unsuper-
35-
50-
60
30
26
25
10
vised, semi-supervised and contrastive, on
20 Kaow
Know X
10
Know
Know
Mixa
Hiigg
Fl and KFI as metrics.
Figure 4: Results of whether gold knowl- edge exists (Know O) or not (Know X) for the dialogue history. We note that T5 + Gold Knowledge exactly uses the gold knowledge for generating responses.
Figure 5: Knowl-
Method
edge retrieval results on the OpendialKG dataset, with MRR and H.ts@3.
SURGE (unsupervised)
27.78
24.00
SURGE (semi-supervised)
28.30
26.38
SURGE (contrustive)
28.
27.58
294
using our new metric, KQA (Section 4), which measures whether the generated responses contain proper knowledge. Lastly, we compute the Knowledge F1 (KFI) similarly as in Shuster et al. [40], which is a measure of unigram overlap between the retrieved knowledge and generated response.
296
297
5.4
Experimental Results and Analysis
296 In Table 1, we report the knowledge-grounde response generation performances of baselines and our SURGE. As shown in Table 1, our models significantly outperform all the baseline models, excluding the upper bound measure, Gold Knowledge, in all evaluation metrics. The high BLEU, ROUGE, and Fl refer that ours sufficiently learns the syntactic and semantic structure of the responses. Our models also achieve high Fl and EM scores in KQA. The high KQA scores indicate that the generated responses are formed with the correct facts, which are relevant the dialog context. Even the baseline models such as All Knowledge, Space Efficcent Encoding [9], and EARI [50], which are provided with all of 1-hop facts, underperform than ours. The result demonstrates that selecting relevant knowledge is critical in knowledge-augmentt response generation.
300
302
303
304
305
306
Figure 4 examines the generation performance further by categorizing the dialogues into two groups: those with gold knowledge and those without, When there is no gold knowledge, the model using Gold Knowledge suffers significant drop in all metrics and performs similarly to No Knowledge. On the contrary, ours significantly improves the unigram Fl and KQA scores even with the retrieved knowledge without using the exact gold knowledge.
308
310
312
Knowledge Retrieval Figure 5 shows the performances of knowledge retrieval methods, where we only measure the retrieval performance on dialogues that contain the gold knowledge. We use Mean Reciprocal Rank (MRR) and Hits @k as metrics. Note that our SURGE is differentiable retriever, which jointly learns to retrieve the context-relevant knowledge and then generate the corresponding responses, whereas Random and BM25 [36] retrieve the knowledge without learning. Therefore, our models outperform other retrieval approaches by a large margin (See Section G with Figure 4 of the Supplementary File for examples of baselines and ours). In addition, when our retriever is trained in a semi- suni-vupee manner, we observe the substantial performance gains from unsupervised learning, as the model can learn to retrieve the ground truth knowledge during training.
313
314
315
316
317
318
319
320
321
Generation We conduct an ablation study on our models to validate the knowledge consistency performance of the response generation by computing the Knowledge F1 (KFI) score [40]. To focus solely on the case where a given knowledge is consistently reflected the generated responses, use the gold knowledge rather than the retrieved one. We randomly perturb
mhowledge-Consistee
323
8

---

Context
Gold responee
Baseline response
SURGE response
(a) loved Moby Dick. Can
writlen by Herman
Moby Dick sailor. Do you like her work?
Moby Dick was written by Herman Melville. He also wrote The Whale.
you recommend something similar?
Melville in 1851, 1. It's
sometimes called The Whale.
(b) Do you know anything the actor Adam Brown?
Yes, he was in the mavie The Hobbit: An Unexpected Journy.
Adam Brown starred King Kong. Have you seen it?
Adam Brown starred The Hobbit:
The Desulation of Smaug and The Hobbit: The Battle of the Five Annies.
(4) Retrieved Subgraph from SURGE
(b) Retrieved Subgraph from SURGE
(Moby Dick; or, The Whole, written by, Herman Melville) (Moby Dick, written, Norman Corwin)
(The Hobbit: The Battle of the Five Arnies, starred _actors Adam Adam Brown)
(The Hobbit: An Unexpected Journey, sturred, actors, Adam Brown)
(Moby Dick, written_by by. Ray Bradbury)
(The Hobbit: The Desolation of Smaug, starred_ actors, Adam Brown)
Figure 6: Examples of the baseline (Space Efficient, paralicl) responses and SURGE responses. On both examples, the baseline generates statements which are factually wrong while SURGE successfully retrieves appropriate facts and generate the good response.
Table 3: Performance comparisons of Table 4: Human evaluation
variants of graph encodings, descrihed on Consistency, Informativeness, and Fluency.
in Section 3.4.
KUA
Knowledge Length
Consis.
Fluency
Method
EM
F1
Naive Invariant
18
42.18
62
All Knowledge
2.52
1.99
2.62
Figure 7: Visualization of the cm-
54
43.35
28.28
117
Space Efficiont
2,47
1.75
2.46
Fdficiint (enrity only)
38.80
43.06
30
bedding space from graph(star)- text(circle) contrastive learning.
Invariant Eificient
39.57
44.13
SURGE (ours)
2.71
2.39
2.92
325
the gold knowledge to ensure that responses are generated from the given knowledge rather than the trained knowledge. Table 2 shows that our model with contrastive learning term outperforms all others in the KF1, implying that the generated responses accurately reflect the encoded knowledge.
326
327
328
Sensitive A Analssis on Graph Encoding We further conduct an analysis on graph encoding variants introduced in Section 3.4. The knowledge length in Table 3 indicates the average token length used for graph encoding. Our Invariant & Efficient aO* performs the best against other variants, while using the lesser space at the graph encoding phase. Notably, simple Invariant achieves a comparable performance against Invariant & Efficient, but yields a longer sequence.
329
330
331
332
333
Retrieval and Generation Examples Figure 6 shows the examples of generated responses along with the retrieved knowledge. In particular, we compare our SURGE against Space Efficient (parallel) baseline. In example (a), the baseline response contains an incorrect fact distracted by the contextually irrelevant entity 'sailor'. Contrarily, SURGE successfully retrieves relevant facts from the KG then generates the factually correct response. This tendency is similar in example (b), where the baseline incorrectly generates the response with a wrong fact containing 'King Kong', meanwhile our SURGE retrieves context-relevant facts and generates informative response.
334
335
396
337
338
Human Evaluation We sample 30 responses of SURGE, All Knowledge, and Space Efficient on the OpendialKG test dataset [29], then conduct a human study of them. We recruit 46 annotators, and ask them to evaluate the quality of the generated responses by the 3 models given in random order, with 3 criteria- - consistency, informativeness, and fluency- - using a 3 point Likert-like scale. As shown in Table 4, ours obtains significantly (p-value 0.05) higher scores than others in all criteria, which is another evidence that our framework generates consistent, informative, and fluent responses.
341
343
345
346
347
Embedding Space Visualization We further visualize the multi-modal graph-text latent space in Figure 7. The visualization shows that, for the same dialogue with different subgraphs, our SURGE with graph-text contrastive learning (right) generates distinct response embeddings pertraining to different subgraphs, unlike the one without graph-text contrastive learning which shows less variety over responses for the same dialogue (left). We include zoomed Figure 7 in the Supplementary File.
348
350
Conclusion
352
353
We proposed a novel end-to-end framework for knowledge graph-augmented dialogue generation which retrieves context-relevant subgraph, encodes a subgraph with the text, and generates knowledge- consistent responses, called as SUbgraph Retrieval-augmentee GEneration (SURGE). Our results demonstrate the effectiveness of our framework in both quantitative and qualitative experiments in knowledge retrieval and response generation tasks. The analysis shows the contribution of each proposed component: retrieval, encoding, and graph-text representation learning. Our work suggests new direction to generate informative responses for knowledge graph-based dialogue task by empirically showing the importance of retrieving the more relevant subgraph knowledge rather than using all the relevant knowledge graphs when generating knowledge- weooudgg-ggounddd responses.
354
355
356
357
358
360

---

References
362
363
[1] Daniel Adiwardana, Minh- Shnh-TThngg Luong, David R. So, Jamie Hall, Noah Fiedel, Romal Thoppi- lan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, and Quoe V. Le. Towards human-like open-domain chatbot. CoRR, abs/2001 109977, 2020.
364
365
366
[2] Kurt D. Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the ACM SIGMOD International Conference Management of Data, SIGMOD 2008, Vancouver,
367
369
BC, Canada, June 10-12, 2008, pages 1247-1250, 2008.
370
[3] Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. Reading wikipedia to answer open-domain questions. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume I: Long Papers, pages 1870-1879, Association for Computational Linguistics. 2017.
371
372
373
374
[4] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597-1607. PMLR, 2020.
375
376
377 [5] Fuwei Cui, Hui Di, Hongjee Ren, Kazushige Ouchi, Ze Liu, and Jinan Xu. Syntactically diverse adversarial network for snowledge-grounded conversation generation. In Xuanjing Huang. Lucia Specia, and Scott Wen-tau Yih, editors, Findings of the Association for Computatioonal Linguistics: EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 16-20 November, 2021, pages 4620 4630. Association for Computational Linguistics, 2021. f wff.iiiinsss
378
Moens,
380
382
URL. atttps://cle /doii org/ 10.
[6]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:
384
386
Human Language Technologies. NAACL-HLA 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), pages 4171-4188 2019. URL https://doot org/10. 10. 18653/v1/n19- 14433.
388
389
[7]
Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. Wizard of wikipedia: Knowledge-powere conversational agents. In 7th International Con- ference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.
380
382
393
[8]
Nouha Dziri, Andrea Madotto, Osmar Zaïane, and Avishek Joey Bose. Neural path hunter: Reducing hallucination in dialogue systems via path grounding. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event/ / Punta Cana, Dominican Republic, 7-11 November, 2021, pages 2197-2214 Association for Computational Linguistics, 2021.
394
385
398
397
398
399 [9]
Fabian Galetzka, Jewgeni Rose, David Schlangen, and Jens Lehmann. Space efficient con- text encoding for non-task non-task-oriented dialogue generation with graph attention transformer. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACI/IJCNLP 2021, (Volume I: Long Pupers), Virtual Event, August 1-6, 2021, pages 7028-7041. Association for Computational Linguisties, 2021.
401
402
403
404
405
[10] Jean- BanttBaseeennn Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning. Advances in Neural Information Processing Systems, 01221271-21288 2020.
407
[1I] Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming- Wei Chang. Retrieval 409
410
augmented language model pre-training. In Proceedings of the 37th International Conference on Machine Learning, ICMI. 2020, 13-18 July 2020, Viruual Event, volume 119 of Proceedings PMLR, 2020.
411
of Machine Learning Research, pages
10

---

413
[12]
William L. Hamilton. Graph representation learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 14(3):1-159, 2020.
414
415
[13] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9729-9738, 2020.
416
417
[14]
Or Honovich, Leshem Choshen, Roee Aharoni, Ella Neeman, Idan Szpektor, and Omri Abend. Sq 2S: Evaluating factual consistency in knowledge- knowledge-grounded dialogues via question generation and question answering. In
419
Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pages 7856-7870. Association for Computational Linguistics, 2021.
421
423
URL https://doll org/ 10. 18653/v1/20211 emnlp-man haappmmin..... 619.
425
[15] Ehsan Hosseini- Asl,, Bryan McCann, Chien-Sheng Wu, Semih Yavuz, and Richard Socher. A simple language model for task-oriented dialogue. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020.
427
429
[16] Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux, and Jason Weston. Poly-encoders: Architectures and pre-training strategies for fast and accurate multi-sentence scoring. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia. April 26-30, 2020. OpenReview.net, 2020. URL SkxgnnNFvH.
430
431
wttp://wwwe///foormmne id=
435
434
[17] Jaehyeong Jo, Jinheon Back, Seul Lee, Dongki Kim, Minki Kang, and Sung Ju Hwang. Edge representation learning with hypergraphs. CoRR, 1ttp/2106.11844 2021.
435
436
[18] Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick S. H. Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 6769-6781. Association for Computational Linguistics, 2020.
438
439
440
[19] Byeongchang Kim, Jaewoo Ahn, and Gunhee Kim. Sequential latent knowledge selection for knowledge knowledge-grounded grounded dialogue. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020.
442
444
[20]
Minseon Kim, Jihoon Tack, and Sung Hwang. Adversarial self supervved contrastive learning, Advances in Neural Information Processing Systems, 33:2983-299 2020.
446
[21] Mojtaba Komeili, Kurt Shuster, and Jason Weston. Internet-augmented dialogue generation. CoRR, abs/2107.07566 2021.
447
[22] Scanie Lee, Dong Bok Lee, and Sung Ju Hwang. Contrastive learning with adversarial perturba- tions for conditional text generation. In International Conference on Learning Representations, 2021. URL https: / /owp///eee/eeeeeee//eeeeee new
449
450
hrC33P3..
451
[23]
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. BART: denoising sequence- sgguencc-to-seeuencce pre- training for natural language generation, translation, and comprehension. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault, editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages Association for Computational Linguistics, 2020.
453
455
457
[24] Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmmetted generation for knowledge-intensive NLP tasks. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020.
458
459
461
11

---

463
[25] Linxiao Li, Can Xu, Wei Wu, Yufan Zhao, Xueliang Zhao, and Chongyang Tao. pero-resouree wwwwlldgg-grouddee dialogue generation. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020.
484
465
466
[26]
Yu Li, Baolin Peng, Yelong Shen, Yi Mao, Lars Liden, Zhou Yu, and Jianfeng Gao. Knowledge- grounded dialogue generation with a unified knowledge representation. In Marine Carpuat, de Marneffe, and Iván Vladimir Meza Ruíz, editors, Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022, Seattle, WA, United States, July 10-15, 2022, pages 206-218 Association for Computational Linguistics, 2022. URL
468
470
472
https: acl anthology.y...... org/ 2tgg/022 2022. naacl 15.
474
[27] Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summariza- tion Branches Oul pages 74-81, Barcelona, Spain, July 2004. Association for Computational Linguistics. URL https: / http//aaaaaaa /aaclaaaaaotooyy....... org/W04-1013.
475
476
477
[28]
Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan T. McDonald. On faithfulness and factuality in abstractive summarization. In Dan Jurafsky, Joyce Chai, Natalic Schluter, and Joel R. Tetreault, editors. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACI. 2020, Online, July 5-10, 2020, pages 1906-1919. Association for Computational Linguistics, 2020.
478
479
480
481
[29] Seungwhan Moon, Pararth Shah, Anuj Kumar, and Rajen Subba. Opendialkg: Explainable 482
483
conversational reasoning with attention- walks over knowledge graphs. In Anna Korhonen, David R. Traum, and Lluís Marquez, editors, Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2. 2019, Volume I: Long Papers, pages 845 854. Association for Computational Linguistics, 2019.
485
486
487
[30] Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christo- pher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger. Kevin Button, Matthew Knight, Benjamin Chess, and John Schul- man. Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint, ar 1XXi:22122009332 2021.
489
480
481
492
[31] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6-12, 2002, Philadelphia. PA, USA. pages 311-318. ACL, 2002.
493
494
495
496
[32] Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
487
498
[33] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 14-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pages 8748 8763. PMLR, 2021.
500
502
504
[34] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 00:110011140::6 2020.
505
506
507
[35] Mare Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojeiech Zaremba. Sequence level training with recurrent neural networks. In Yoshua Bengio and Yann LeCun, editors, 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings, 2016.
508
509
510
511
[36] Stephen Robertson and Hugo Zaragoza. The probabilistic relevance framework: BM25 and beyond. Found. Trends Inf. Retr., 3(4):333-389, 2009.
512
12

---

518
[37] Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Eric Michael Smith, Y-Lan Boureau, and Jason Weston. Recipes for building an open- domain chatbot. In Paola Merlo, Jörg Tiedemann, and Reut Tsarfaty, editors, Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguisties: Main Volume, EACL. 2021, Online, April 19 23, 2021, pages 300-325. Association for Computational Linguistics, 2021.
514
515
516
517
518
519
[38] Edward Scheinerman and Daniel Ullman. Fractional graph theory: a rational approach to the theory of graphs. Courier Coporation, 2011.
520
521
[39]
Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers. The Association for Computer Linguistics, 2016.
522
523
524
525
[40]
Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. Retrieval aug- mentation reduces hallucination in conversation. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, Findings of the Association for Computational Linguistics: EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 16-20 November, 2021, pages 3784-3803 Association for Computational Linguistics, 2021.
526
527
528
529
[41] Yi-Lin Tuan, Yun-Nung Chen, and Hung-yi Lee. Dykgehat: Benchmarking dialogue generation grounding on dynamic knowledge graphs. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLPPIJCNNLP 2019, Hong Kong. China, November 3-7, 2019, pages 1855-1865, Association for Computational Linguistics, 2019.
531
532
539
534
[42] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin, Attention is all you need. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 5998-6008, 2017.
536
538
539
540
541
[43] Denny Vrandecic and Markus Krötzsch. Wikidata: a free collaborative knowledgebase. Com- mun. ACM, 57(11):78 (5, 2014.
[44] Alex Wang, Kyunghyun Cho, and Mike Lewis. Asking and answering questions to evaluate the factual consistency of summaries. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault, editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 5008-5020, Association for Computa- 450.
5M
546
tional Linguistics, 2020. URL https: //w.i org 10. 18653/ /01553/v1/20 v1/2020..002
[45] Sixing Wu, Ying Li, Dawei Zhang, Yang Zhou, and Zhonghai Wu. Diverse and informative dialogue generation with context-specific commonsense knowledge awareness. Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault, editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 5811-5820. Association for Computational Linguistics, 2020. URL https: //doi. org/ 10. N0.18653/v1/22 /0/0////1/2000 acl main. 515.
549
551
553
554
[46]
Chulhee Yun, Srinadh Bhojanapalli, Ankit Singh Rawat, Sashank J. Reddi, and Sanjiv Kumar. Are transformers universal approximators of sequence-to-sequees functions? In 8th Inter- national Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020.
596
558
[47] Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabás Póczos, Ruslan Salakhutdinov, and Alexander J. Smola. Deep sets. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA. USA, pages 3391-3401, 2017.
$59
561
13

---

583
[48] Houyu Zhang, Zhenghao Liu, Chenyan Xiong, and Zhiyuan Liu. Grounded conversation generation as guided traverses in commonsense knowledge graphs. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. 'ettreault, editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACI. 2020, Online, July 5-10, 2020, pages 2031-2043. Association for Computational Linguistics, 2020. URL http::t/s/seee //ooi. org/10. 10. 18653/v1/2020. mocl-main.111 184.
584
565
566
567
568
[49] Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jian- feng Gao, Jingjing Liu, and Bill Dolan. Dialogpt: Large-scale generative pre-training for conversational response generation. In ACL, system demonstration, 2020.
570
572
[50] Hao Zhou, Minlie Huang, Yong Liu, Wei Chen, and Xiaoyan Zhu. EARL: informative knowledge-groundee conversation generation with entity-agnoste representation learning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pages 2383-2395. Association for Computational Linguistics, 2021.
574
576
Checklist
577
578
1. For all authors...
579
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
581
(b) Did you describe the limitations of your work? [Yes] See Section of the Supplemen- tary File
582
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See Section A of the Supplementary File
584
585
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
5e6
587
2. If you are including theoretical results...
586
(a) Did you state the full set of assumptions all theoretical results? [N/A]
589
(b) Did you include complete proofs of all theoretical results? [Yes] See Section C. of the Supplementary File
590
591
3. If you ran experiments..
592
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [Yes] We provide it as the supplemental material.
593
594
595
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Section D of the Supplementary File
596
597
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [N/A]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Section D.3 of the Supplementary File
600
601
602
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
603
(a) If your work uses existing assets, did you cite the creators? [Yes]
604
(b) Did you mention the license of the assets? [N/A]
605
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
606
607
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
609
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
610
5. If you used crowdsourcing conducted research with human subjects...
14

---

612
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [Yes] See Section of the Supplementary File
613
614
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
617
15


Reviews:
[{"review_id": "q0WJr8-8IQ_", "reviewer": "Reviewer_JGjW", "summary": "This paper proposed a KG-based dialogue generation approach. \n\nThe proposed approach consists of a triplet selection, a subgraph construction and retrieval, a graph encoding and a basic encoder-decoder framework. \n\nContrastive learning is used for ensuring graph-text consistency in generation. \n\nA KQA metric is proposed for the evaluation of generated dialogue in fact-level.\n\nExperiments in a benchmark dataset verified the effectiveness of the proposed approach. ", "questions": "Please see the weaknesses. ", "limitations": "The lack of comparisons with SOTA methods limited the novelty of the proposed approach. \n\nThe only one experimental dataset limited the extension of the proposed approach and the common conclusions presented in the paper. ", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. A novel subgraph construction and retrieval scheme. \n\n2. The proposed invariant graph encoding considered the order and inverse relation of a triplet. \n\n3. The use of a loose evaluation metric for dialogue generation.\n\nWeaknesses:\n\n1. The lack of SOTA methods for empirical comparisons on triplets selection. \n\n2. The lack of verification of different graph-based neural networks for encoding. \n\n3. I am afraid p(Z|x) defined in this paper is not a probability. \n\n4. The lack of empirical comparisons with other triplets ranking methods or using other methods for the performance verification of dialogue generation. \n\n5. A minor point that only one dataset is used in this paper. I concerned the extension of the proposed approach.\n\n6. Another minor point is that there are a lot of grammar errors, a careful proofread is needed. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "UZeM8lu1A6V", "reviewer": "Reviewer_Pv3X", "summary": "- The authors introduce a method, SURGE (subgraph retrieval augmented generation) for generating consistent dialogue responses, augmented with knowledge from knowledge graphs. SURGE consists of several elements. The first is a context-relevant knowledge retriever, which simply retrieves knowledge from a graph given a dialogue context. The second is an invariant graph encoding; given a set of entity-relation tuples from a subgraph, the authors propose a method for jointly encoding the context and knowledge such that the encoding is invariant to both the order in which facts are presented, and the order in which entities are presented (i.e., relation-invariant). Finally, to encourage consistent generation, the authors propose a contrastive learning objective, which encourages model graph encodings (context, knowledge) to be similar to their decoded output representations, and dissimilar to decoded output representations given other sets of knowledge; as the authors say, “given a different graph, the model would generate different tokens fro the same graph” (L235).\n- The authors compare their method against others on the OpenDialKG dataset, measuring standard generation metrics as well as a new proposed Knowledge QA metric, in which the authors examine the performance of a QA model when given a question and model response as input. The authors find that their method outperforms baselines handily, in all metrics considered. Human evaluations and several ablation studies support the authors’ claims and methods.\n", "questions": "- Several times, the authors mention that KG models are more efficient or preferred more than those with unstructured knowledge (e.g., L36-37, L100-103); could you please explain why that is the case?\n- The authors mention that the Enc and Dec functions from a transformer are “permutation sensitive” (L135); is it not the case that Enc could in theory be permutation insensitive, due to bidirectional attention?\n- Do you have some clarity as to which specific situations the contrastive learning is better than using just semi-supervised, given that, in Table 1, both KQA and BLEU performance is better for the latter?\n", "limitations": "The authors adequately address several limitations within the supplementary material.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Originality\n- **Strengths**: The exact methods themselves may not be new (retrieval-augmented semi-supervised learning, contrastive learning, etc.), however the authors indeed provide a novel combination of techniques. The invariant graph encoding is novel as well, and it is clear how the work differs from previous contributions.\n\n### Quality \n- **Strengths**: The authors’ claims are generally well supported and technically sound; the authors present a complete work with several contributions that improve significantly compared to baseline performance. Human evaluations emphasize the gap between the authors’ proposed work and previous methods. The choices of using retrieval-training and invariance encodings are well supported with ablation studies.\n- **Weaknesses**: While the three proposed main models all outperform baselines (SURGE with unsupervised, semi-supervised, and contrastive learning losses), it is not clear which aspect of the model truly shines as the best contribution. The gap between SURGE and using sparse retrieval seems to imply that graph retrieval is driving nearly all of the performance gains; this intuitively makes sense, given that OpenDialKG was built using the Freebase KG. Beyond that, however, different methods yield different results; i.e., in some cases, semi-supervised outperforms contrastive, and vice-versa. The performance of naive encoding of (context, subgraph) is also not too far behind the invariance proposals. \n\n### Clarity\n- **Strengths**: The paper is well written and clearly outlines the proposed methods and the rationale for using them. Results are generally well presented.\n- **Weaknesses**: Figures and tables are very small and compact. While this is not as big of an issue when presenting numbers of graphs, this sees to really hurt other visualizations; Figure 2 (the framework overview) is a bit tough to follow, as are the results in Figure 7.\n\n### Significance\n- **Strengths**: This paper provides several methods that other researchers could use in their work, e.g., the invariance encodings (which is helpful in situations with lots of knowledge conditioning), or the various ways in which to train the retriever (e.g. via contrastive learning with the decoder representations). The paper also highlights how a good pipeline can improve performance significantly on a dialogue task requiring knowledge from a knowledge graph.\n- **Weaknesses**: The contributions may be limited to instances where a knowledge graph is the most appropriate knowledge source; though the authors claim that using such a knowledge source is more efficient than using unstructured documents, it still limits the types and amount of knowledge at one’s disposal, and it is not immediately clear how to apply this method to situations in which unstructured knowledge is preferred.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "O_a4WiIfOC", "reviewer": "Reviewer_75s5", "summary": "The work aims at avoiding generating factually incorrect responses by introducing knowledge from knowledge graph (KG) into a conversation. It retrieval only context-relevant sub-graph from the global KG to avoid unnecessary encoding overhead and propose an invariant graph encoder to encode the graph.", "questions": "None", "limitations": "limitations are sufficiently considered in Supplementary materials.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n\nThe proposed new graph encoding achieves permutation invariance and relation inversion invariance, which I like.\n\nWeaknesses:\n\n1. Inadequate baseline. Some classic and recent methods are not compared or discussed. For example, [1][2][3]\n\n2. Missing ablation study. The effectiveness of the newly-proposed graph encoder is unclear as there is no ablation experiment.\n\n[1]Wu, Sixing, et al. \"Diverse and informative dialogue generation with context-specific commonsense knowledge awareness.\" Proceedings of the 58th annual meeting of the association for computational linguistics. 2020.\n\n[2]Zhang, Houyu, et al. \"Grounded conversation generation as guided traverses in commonsense knowledge graphs.\" arXiv preprint arXiv:1911.02707 (2019).\n\n[3]Cui, Fuwei, et al. \"Syntactically Diverse Adversarial Network for Knowledge-Grounded Conversation Generation.\" Findings of the Association for Computational Linguistics: EMNLP 2021. 2021.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Reject