Knowledge-Consistent Dialogue Generation with Language Models and Knowledge Graphs
Authors: Minki Kang
Jin Myung Kwak
Jinheon Baek
Sung Ju Hwang
Venue: Submitted to ICLR 2023 (2023)
Keywords: knowledge-grounded dialogue generation
knowledge graph

Abstract: Pre-trained language models have achieved impressive performances on dialogue generation tasks. However, when generating responses for a conversation that requires factual knowledge, they are far from perfect, due to the absence of mechanisms to retrieve, encode, and reflect the knowledge in the generated responses. Some knowledge-grounded dialogue generation methods tackle this problem by leveraging the structured knowledge from Knowledge Graphs (KGs). However, existing methods do not guarantee that the model utilizes a relevant piece of knowledge from the KG before generating knowledge-consistent dialogues. To overcome this limitation, we propose SUbgraph Retrieval-augmented GEneration (SURGE), a framework for generating context-relevant and knowledge-consistent dialogues with a KG. Specifically, our method first retrieves the relevant subgraph from the KG, and then enforces consistency across facts by perturbing their word embeddings conditioned on the retrieved subgraph. Then, it learns a latent representation space using contrastive learning which ensures that the generated texts have high similarity to the retrieved subgraphs. We validate the performance of our SURGE framework on the OpendialKG and KOMODIS datasets and show that our method generates high-quality dialogues that faithfully reflect the knowledge from the KG. 

Under review as a conference paper at ICLR 2023
KNOWLEDGE- DIALOGUE GENERATION WITH LANGUAGE MODELS AND KNOWLEDGE GRAPHS
Anonymous authors Paper under double-blind review
ABSTRACT
Pre-trained language models have achieved impressive performances on dialogue generation tasks. However, when generating responses for conversation that requires factual knowledge, they are far from perfect, due to the absence of mech- anisms to retrieve, encode, and reflect the knowledge in the generated responses. Some www.wllge-gooundeee dialogue generation methods tackle this problem by leveraging the structured knowledge from Knowledge Graphs (KGs). However, existing methods do not guarantee that the model utilizes a relevant piece of knowl- edge from the KG before generating mnowledge-consistern dialogues. To overcome this limitation, we propose SUbgraph Retrieval-augmeented GEneration (SURGE), framework for generating context-relevant and inowledge-consistent dialogues with a KG. Specifically, our method first retrieves the relevant subgraph from the KG, and then enforces consistency across facts by perturbing their word embed- dings conditioned on the retrieved subgraph. Then, it learns latent representation space using contrastive learning which ensures that the generated texts have high similarity to the retrieved subgraphs. We validate the performance of our SURGE framework on the OpendialKG and KOMODIS datasets and show that our method generates high-qualit dialogues that faithfully reflect the knowledge from the KG.
INTRODUCTION
Dialogue systems aim at conversing with humans by generating human-like responses, given the dialogue context. While pre-trained language models (PLMs) (Radford et al., 2019; Raffel al., 2020) are capable of generating fluent responses, they often generate factually incorrect responses due to lack of explicit knowledge (Shuster al., 2021). To overcome such limitations, recent methods access the external knowledge sources, such as Wikipedia (Dinan et al., 2019) or Web (Komeili al., 2021), and then retrieve the relevant knowledge for ongoing conversations, In addition such document-based retrieval approaches, there also exists variety of works (Tuan et al., 2019; Wu et al., 2020; Zhang et al., 2020a; Cui et al., 2021; Zhou et al., 2021; Galetzka et al., 2021; Li et al., 2022), which focus the use of the Knowledge Graphs (KGs) (Bollacker et al., 2008; Vrandecic & Krötzsch, 2014) - a different form of the knowledge source which succinctly encodes the knowledge in the most compact and effective form in dialogue generation. Specifically, KGs consist of symbolic facts which represent entities as nodes and their relations as edges, in the triplet, e.g., (Pride & Prejudice, written by, Jane Austen) (See Figure 1), which can help generate a mhowledge-roounded response.
Most of the dialogue generation models with KGs (Galetzka et al., 2021; Li et al., 2022) utilize all the triplets associated with the entity in the dialogue context. However, not all of the facts relevant to the ongoing conversation (e.g., Jane Austen was born in Steventon in Figure 1), which could mislead the models from generating factually incorrect responses. We found that about 87% of facts from 1 1-hop KG are irrelevant to the context in the OpendialKG dataset (Moon et al., 2019). Moreover, encoding all the facts including the unnecessary ones is computationally inefficient (Galetzka et al., 2021; Rony et al., 2022). On the other hand, even after correctly retrieving the relevant facts, it is not straightforward to combine two heterogeneous modalities: the dialogue context is represented as a text, meanwhile, the knowledge is represented as graph. In other words, since PLMs already have tons of nre-traieed parameters trained on the unstructured texts, properly conditioning the structured graph to PLMs is highly important. Otherwise, PLMs may generate inconsistent responses disregarding the knowledge from the retrieved subgraph, which a phenomenon known as hallucination (Rohrbach al., 2018), where they generate responses with their memorized yet incorrect knowledge.

---

Under review as a conference paper at ICLR 2023
Knowledge Gruph
writer
User
Issue I. Imelevant knowledge
Issuc 2. Incousisteat Generation
wruther by
Could you recommend any book by Jane Austen?
Lap
12.6%
Factually wrong with extemnal entity
Jane Austen
lady Susan
Assistant
Amvleaar koowledge
Gold Inoraiidge
???
Distracted by Irrelevant facts
asaistant she wrote Steventon.
Assistant
Pride & Prejudice
Sure, she wrote The Secret Life of Man Who Was Born.
Steventon
Dialogue History
Figure I: Motivation. Existing mnowledge-grounddg dialogue generation models with KG utilize the multi-hop subgraph for entities in the dialogue context (Jane Austen). However, they suffer from the following two problems: (1) irrelevant knowledge where only 12.6% of facts from I-hop KG are useful to generate the target responses given dialogue context, and (2) inconsistent generation including the factually wrong statement.
In this work, we tackle such challenging and fundamental issues of knowledge-consistent dialogue generation with the KG. We propose an end-to-end dialogue generation framework that considers all aspects from knowledge retrieval, encoding, and reflection along the generation process. As a first step, propose context-relevan subgraph retriever that retrieves only the relevant triplets from the KG to prevent the model from generating context-irrelevant responses. Notably, our subgraph retrieval method embeds the KG considering the relational structure with the Graph Neural Network (GNN) (Kipf& & Welling, 2017) instead of using PLMs as in previous work (Li et al., 2022). Furthermore, it end-to-end trainable jointly with the generation objective by marginalizing the likelihood of the generated sentences over the latent retrieved subgraph (Guu et al., 2020; Lewis et al., 2020b). Then, encode the retrieved subgraph along with the input text sequence, we propose graph encoding that is permutation and relation inversion invariant yet efficient. Specifically, we devise the graph encoding method that reflects the graph structure onto the representation space of PLMs, instead of prepending them in front of the text sequence to avoid the computational burden. Furthermore, to ensure that the model does make use of the encoded knowledge when generating responses, we propose multi-modal contrastive learning objective between two different graph-text modalities to enforce the consistency across the retrieved facts and the generated texts. We call our framework SUbgraph netrieval-augmentt GEneration (SURGE).
We validate our framework on the OpendialKG (Moon et al., 2019) and KOMODIS (Galetzka et al., 2020) datasets against relevant basclines. Note that, when evaluating the generated responses from dialogue models, conventional metrics (e.g.. BLEU (Papineni et al., 2002), Rouge (Lin, 2004)) can not measure how faithfully the generated responses reflect the related knowledge in KGs. Thus, in evaluation, we further introduce an additional performance metric, referred to as Knowledge-verifying Question Answering (KQA), which evaluates whether the generated responses contain the correct knowledge with an additional extractive question answering scheme. The experimental results show that SURGE generates responses that not only agree with the gold knowledge but are also consistent with the retrieved knowledge from KGs.
Our main contributions can be summarized as follows:
We propose GNN-based context-relevan subgraph retrieval method for KG-augmented dialogue generation, to extract only the relevant piece of the knowledge for the dialogue context from the entire knowledge graph, for generating more appropriate responses the ongoing conversation.
We propose an invariant yet efficient graph encoder and graph-text contrastive learning objective ensure that the generated responses faithfully reflect the retrieved knowledge.
We validate SURGE against relevant baselines, demonstrating its efficacy in generating responses that are more informative by retrieving and reflecting the relevant knowledge from the KG.
RELATED WORK
Language Models
Pre-trained Language Models (PLMs) (Radford et al., 2019; Lewis et al., 2020a; Raffel et al., 2020) that use a Transformers-baate (Vaswani et al., 2017) encoder-decoder architecture have achieved great successes on language generation tasks. As they can accurately contextualize the given context and then generate human-like sentences, they are often used the base architecture for neural dialogue systems (Zhang et al., 2020b; Hosseini-Ast et al., 2020). Moreover, when PLMs become larger, dialogue models have shown to generate high-quality responses (Adiwardana et al., 2020), suggesting that pre-trained parameters do contain certain knowledge (Petroni et al., 2019). Despite the fluency of such MlM--ased dialogue agents, they often generate factually incorrect
2

---

Under review as a conference paper at ICLR 2023
responses that unfaithful to the context but look plausible - widely known as hallucination (Maynez et al., 2020), Thus, generating responses requiring specific and valid factual knowledge is still challenging. To tackle this, recent studies propose to retrieve knowledge from external sources, and then use it to augment dialogue models (Roller et al., 2021; Shuster et al., 2021).
Dialogue Generation with KGs The sources of external knowledge can be categorized into two types: documents from unstructured corpora (e.g., Wikipedia (Dinan et al., 2019), Web (Nakano et al., 2021)), and symbolic facts from Knowledge Graphs (KGs) (e.g.. Freebase (Bollacker et al., 2008), Wikidata (Vrandecic & Krötzsch, 2014)). Regarding dialogue generation tasks with KGs that we target, Moon et al. (2019) introduce a knowledge-grouneg
dialogue dataset where each dialogue comes with facts from the large-scale KG. Several works (Tuan et al., 2019; Wu et al., 2020; Zhang et al., 2020a; Cui et al., 2021; Zhou et al., 2021) suggest sequence-to-sen se-uuuenceee models trained from scratch, which focus on generating dialogue by conditioning the output word distribution with the entities from the KG. Further, Galetzka et al. (2021) propose an efficient way to encode all of the facts in the k-hop neighbors of the entities that appear in the dialogue history in the given KG, in order reduce the number of input tokens used in PLMs. On the other hand, Rony et al. (2022) propose to mask out weights for irrelevant facts in PLMs. However, all of these methods simply match and retrieve all facts for entities that appear in the dialogue context, which either may mislead the agent to generate out-of-context responses from irrelevant facts or can increase the computational overheads for prepending tokens for all facts in PLMs. Our work differs from those existing works, since we aim at retrieving only a context-relevant subgraph among all associated facts with a novel GNN-based subgraph retriever, which is end-to-end trainable along with a dialogue generation model.
3
METHOD
In this section, we first discuss the basic ingredients: Transformer and Graph Neural Network. We then formalize the dialogue generation problem and describe key components for our SUbgraph Retrieval- augmented GEneration (SURGE) framework: context-relevant subgraph retrieval, invariant graph encoding, and graph-text contrastive learning, Figure 2 illustrates the overview of our framework.
3.1 PRELIMINARIES
As we use two different modalities, namely text and graph, we first define them, and then describe the neural networks to encode them. In particular, a text is defined as sequence of tokens [.i,,..,<<] [.]. Vz, E V, where x; is a token and V is pre-defined vocabulary formed with specific tokenization algorithms (Sennrich et al., 2016). On the other hand, knowledge graph (KG) is type of nulti-relational graphs G = {(e,, r, } E E x R x E, where е and e, are head and tail entities (nodes) along with their relation (edge) r; and & and R are sets of entities and relations, respectively,
To easily access different modalities in the same framework, we define the tokenization (mapping) function that maps entities and relations the tokens used in Pre-trained Language Models (PLMs), represented as follows: q E EUR + V where is an arbitrary length varying across different entities and relations. In other words, any entity e E E and relation E R consisting of / tokens can be tokenized to a sequence of / tokens æ E V4: q(e) = Dr and q(r) = F,. For instance, an entity New York (i.c., e), is tokenized into two tokens 'New' and 'York', i.c., I. ['New', 'York'].
Transformer A Transformer (Vaswani et al., 2017) is a neural architecture that embeds a sequence of tokens while taking their relationships into account, which is the most basic building block of recent PLMS (Devlin et al., 2019; Radford et al., 2019). Formally, given a sequence of input tokens x [21,--., Ve, V, goal of generative transformers is to generate a sequence of tokens -1--11 = [/,...., 1t- Vy E V, with encoder Enc, decoder Dec, and tokens' embedding function f. Thus, a hidden state time for generating yt is h, Dec(Enc(X),, Yi:t)),, where f 1)]. We note that both Enc and Dec functions are permutation sensitive with positional embedding (Vaswani et al., 2017).
X = fxx = [f(z),, ),.,, f( )} and Y = -( = [f(]),
Graph Neural Network A Graph Neural Network (GNN) represents a node with its neighboring nodes over the graph structure (Hamilton, 2020), which is formalized as follows:
GNN(e; 9) UPD(e,, AGG(
{ eex Vea E (N(ee;))))
where g) = fex (e,, x, ex) e 9} is a set of neighboring entities of e,; e, and ej embeddings of entities (nodes) e, and e,; AGG is a function that aggregates embeddings of neighboring entities; and UPD is a function that updates e, with the aggregated messages from AGG.
3

---

Under review as a conference paper at ICLR 2023
Context-Relevant Subgraph Retriever Pa (2)x)
Graph-Text Contrastive Learning
Knowledge Graph (fF)
Jane Austen;
Iron man
Lady Susan;
Iron Man
wraflay tp
Pride Prejudice;
Stan Lee
Xlane decttres
GNN
Subgraph (2)
Negative Subgraph (2')
place sf Aurba
f
Docaddr Hidden Suawe
Usex
(xr)
d(x; f
Topio
* (x.
Graph Encoding
Generate
Pa 2)
Could you
recommend any book
Sure, wrote & Prejudice and Lady
Pa(4
Austen?
Eabolinn
Encoder
Decoder
Dialogne
Jane Austen Pride Prejucice
Susan.
History (x)
book
Generator (PLM)
Figure 2: Framework Overview. Our framework, SURGE, consists three parts. First, a
subgraph retriever Pwd (l)²) retrieves the subgraph Z relevant to the given dialogue history æ from a knowledge graph G (e.g.. 1-hop KG from entity Jame Austen; a). Specifically, we measure the similarity of context and triplet embedding compose the retrieval distribution (=(=()) ($ 3.3). Then, we encode the retrieved subgraph Z into the input of the generator, using the graph encoding function (b( 2) ($ 3.4). Finally, we contrastive learning to enforce the model generate consistent response with the retrieved subgraph ($ 3.5).
3.2
PROBLEM STATEMENT
Here we formalize the problem of context-relevant subgraph retrieval for knowledge grounded dialogue generation. Given dialogue history x [.1..... NN), model with generative PLMs first encodes the input tokens, and then models conditional distribution p(y az) to generate an output response y y/...... yrl. This problem requires a piece of specific knowledge for conversation.
To that end, given dialogue history æ, we aim at retrieving subgraph Z S G consisting of a set of triplets: = E Z where 2 = (ex, r, er), which encodes relevant knowledge for ongoing conversation. Thus, the distribution of the context-relevant facts 2 is p( (z)). and our final likelihood of generating responses then becomes p æ 2). Then, to jointly optimize the objective of graph retrieval with response generation, we treat Z as latent variable and then marginalize the likelihood of the generative model over all possible latent variables for retrieved subgraphs Z, formalized as follows:
T
p(y(0)) = Σ ZCg
= Σ Pa ( ()) 3C9
II
pa (v 2, yo:-11,,
where yo the start token for the generation, P& (Z(z) is an output distribution of the context-relevant subgraph retriever, and po (y| 2) is the target distribution of the knowledge-augnee augmented generator, parameterized as Φ and 6, respectively, which we specify in next few subsections.
3.3 GNN BASED CONTEXT- RELEVANT SUBGRAPH RETRIEVER
We now provide concrete description of our context-relevant subgraph retriever, i.c., p( ()) formalized in Equation 2. Given the dialogue history æ, we assume that a retrieval of each triplet in 2 = {=1,.... ................................ =n is independent. Then, for simplicity, we decompose the retrieval of set of triplets p( 2() a)) into the product of individual triplet retrieval, represented as follows: p(2 = p(2 (t)ppp((2)))))) xx) p((, (z), for n retrieved triples.
From the above decomposition, it is sufficient to focus on a single triplet retrieval. We define the score for the single triplet triplet with inner product between the embedding of dialogue history and the embedding of candidate triplet = (Guu et al., 2020), as follows:
(2(x)) x:) x exp(d(2) qxx)).
(3)
where a triplet embedding function and q a dialogue context embedding function. For any off-thh-shelf PLMs, but for d, we need another effective approach that captures the property of the graph. Therefore. we utilize the Graph Neural Networks (GNNs) for the triplet embedding function d to consider the relational structure between entities in the KG.
implementing g.
More specifically, we consider a set of triplets associated to the entities that appear in the given dialogue context: {(e, r, ex) or (e),z,e) q(e) C x> e. as the retrieval candidates. Then, to effectively represent triplets consisting of entities and their relations as items, we use GNNs described in section 3. In our triplet retriever, utilizing both nodes and edges, which equally essential
4

---

Under review as a conference paper at ICLR 2023
components for the multi-relationa graph, is worthwhile to represent an entire triplet. To do so, adopt the existing edge message passing framework (Jo et al., 2021) that transforms edges of the original graph to nodes of the dual hypergraph (Scheinerman & Ullman, 2011) (i.e., transforming G to 9*), which allows us to use existing node-level GNNs for representing edges of the original graph (See Appendix D.1 for more details). Formally, our triplet embedding function is denoted follows:
d(z; = MLP (fex r e:)), en GNN(e);; G), r GNN(r"; g"), GNN(e 9), (4)
and isstart (nc(X))/(end start + 1), if q() S otherwise
where: : = (ex, r, ex), ((e) (......, my}. 0 is zero vector, and || is the concatenation operator. If the entity e exists in a dialogue history æ. its node embedding (i.c., e²) becomes the mean of corresponding token representations on the PLM encoder Enc. Otherwise, a zero vector assigned to the initial node embedding. For relation embedding r we use the trainable relation embedding matrix. For more justifications on this function, please refer to Appendix D. Also, we experimentally verify that the use of GNN the triplet embedding yields the better retrieval performance compared to previous PLM-based methods (Humeau et al., 2020; Li et al., 2022), in Table and Figure 4.
3.4 INVARIANT GRAPH ENCODING
We then now specify the remaining operation of graph encoding, which determines how to condition the structural graph Z along with the sequential text sequence æ to generate y, with regard to the Pre-trained Language Models (PLMs). Let (zx Z) be graph encoding function, and there are mapping functions qe and gr, which map the entity and relation in a graph Z into the natural languages, respectively. Then, the simplest way to encode the graph along the text is to concatenate the mapped tokens of entities and relations in front of the given text input æ, as in previous works for triplet-conditioned text generation with PLMs (Li al., 2022; Ma al., 2022). For instance, given a)), this naive graph encoding
text x = [ _N] and a graph 2 = {(a, d. b), (b, a,), (a, function defined as follows: (ax Z) f((a, d, b, b, a, d, c, T1, ..... æN]) where a q(a). d = (c),, and so on. Also, f is token embedding function of the PLM. However, this naïve encoding function violates two important invariance properties for encoding a multi-rclational graph into the text sequence: permutation invariance (Zaheer et al., 2017) and -nveersion invariance, where the order of elements matters, which are formalized in Definition 3.1 and 3.2 as follows:
Definition 3.1. (Permutation Invariance) For any permutation π E S,, (xx Z) (b( ग 2). i.e., an order of elements in a subgraph does not affect a representational output.
Definition 3.2. (Relation Inversion Invariance) Let -d be an inverse relation to d, if (a. d, b) (b, -d. a) Ya, b E &. Then, sb(x, (x, Z U {(a, d, b)}) ab(x,, Z U {(b, -d, a)}) for any graph Z.
Invariant Graph Encoding To satisfy both properties, we consider two additional operations on a set of triplets up to the naive encoding. We first define SORT operator that returns the same output regardless of the order of input set elements, as follows:
SORT(T: (Tt 2) = SORT(T' 2), Vn, n' E S..
(5)
where S, is set of all possible permutations for n elements. Moreover, we define INV operator that adds the inverse triplet of each triplet in the graph Z, as follows:
INV = Z J {(e,, -I, ex) (es, I, ex) E Z}.
(6)
Based on the above SORT and INV operations, we can now define a more solid graph encoding function: (xx SORT(INV(Z)),, which satisfies both permutation and relation inversion invariance.
Invariant and Efficient Graph Encoding However, above encoding not efficient since it requires (n)) space complexity for encoding graph with n triplets. Thus, to make it efficient, we newly define v that only encodes the unique nodes (entities) along the sequence, formalized as follows:
(xx (ERT(EN(()))) = f(a b, ,.,,,,.,,NN,)
where ENT (Z) returns the set of unique entities in Z. This encoding meets both invariance properties but also efficient since it only costs Okk) for the entity where < n. However, as it does not consider the relational information in 2, we further perturb the token embeddings for entities in
5

---

Under review as a conference paper at ICLR 2023
PLMs with respect their graph representations in 2. Specifically, for each entity ENT(2), apply a learnable affine transformation on the token embedding of a as follows:
B(f = (1+ f(a) + 6,
7 (7))
8 MLP2(1) " GNN() 2),
where MLP is Multi-Layer Perceptron, 3 R² R perturbs the embedding according to Z, R-GNN is the relation-aware GNN (Schlichtkrull et al., 2018; Vashishth et al., 2020), f(a) is a token embedding of each entity which is used node embedding for R-GNN. In sum, denote relation-aware and invariant yet efficient encoding 25", defined as follows:
/ (x. 2) = 3 (ax (ORT(ENT(2)), INV(Z)).
where 3 can be applied to any sequence of representational inputs for texts and graphs. We conclude that our graph encoding satisfies both properties, and for proofs, please Appendix C. For better understanding, we include comprehensive illustration of Equation 7 in Appendix Figure
3.5 CONSISTENT GENERATION WITH GRAPH-TEx TEXT CONTRASTIVE LEARNING
Although the previous schemes allow retrieving and encoding subgraphs that are relevant to the input dialogue history, the consistent generation with the given subgraph is further required, when generating responses with the factual knowledge. In other words, the model should be able to generate different sequences when providing different subgraphs, for the same dialogue history.
However, only access the single ground-truv response regardless of the retrieved knowledge, while the generative model is trained with a teacher forcing. Thus, this setting can give rise to the problem of exposure bias (Ranzato al., 2016): the model is never exposed to other generated tokens during training. To overcome such limitations, we introduce a novel graph-text contrastive learning method motivated by multi-modal contrastive learning (Radford et al., 2021). Formally, for single pair of graph and text, the contrastive learning objective is defined as follows:
1
log exp(sim(((x).<(()))) + En (xp(ssm((())),(()))))
1
Leant
2 log
(8)
2
where = 1 Ei= 2, is the average encoder representations of the appended knowledge from Enc(bb "x, 2)) = [ ....... 2m, 21...., zv], = + - h, is the mean of decoder representa- tions, sim is the cosine similarity, and & are learnable linear projection layers, and is a learnable temperature parameter. Furthermore, Σn and Σg' indicate the summation over negative samples, which are other texts graphs within same mini-batch as in previous literature contrastive learning (Chen et al., 2020; Lee et al., 2021; Radford et al., 2021). With Equation 8, the model can embed the correlated pairs closer together in order generate consistent response given graph, ie., given a different graph, the model would generate different tokens for the same text.
3.6 TRAINING
We train the entire model, named as SUbgraph Retrieval-augmenter GEneration (SURGE), by maximizing the log-likelihood p(y ar) defined in Equation with respect to parameters of both the retriever ó and the generator 6.
Since computing the marginal probability over entire subgraphs is infcasible, we approximate by summing over A sampled subgraphs (Guu et al., 2020; Lewis et al., 2020b). Our end-to-ent training objective for retrieval-augment generation then defined as follows:
Lret log Σ Po(()xr)poyllo, 2),
2C1
where = samp e (p denotes sampling subgraphs over the subgraph distribution and each subgraph sampling is decomposed into sampling n triplets from ps(2:læ)vV E n] as in sub- section 3.3. We further assume that the gold subgraph is partially available in training. Thus, we can utilize the supervised retrieval loss to introduce semi-supervised retriever learning as follows:
Csup log P 2" [z))
(10)
where 2 is the available ground-trut truth subgraph. By combining all objectives in Equation 8, 9, and 10, our final training objective is then defined follows: L Lret Loup + Loont-

---

Under review as a conference paper at ICLR 2023
User
veite
QA Context: Could you recommmend
Could you recommend a book written
Ione Audens
Eady Seon
book written by Jane Austen?
by
Austen
(b) Corresponding
<Generated Response>
Reagonse
Sure. Lady Susan
Question: Jane Austen
vri ree ?)
Earr
acoegtable tesgonses
She wrote Emma
Candidate Searching
votte
Answer Candidates: Lady Susan
How about
Sense & Sensibility
vodte
Jass Juckan
Secsliilty
Sense Sensibility
Emima
(a) Dialogue History
Janc Atsion
(c) Candidates KG
(d) KQA example
Figure 3: KQA. (Left) An example where multiple responses acceptable but the gold response cannot reflect all of them. (Middle) We first find the fact from the KG that reflects the relation between entities within the user input and gold response (b), and then search candidate facts from the KG (c). (Right) Corresponding KQA example. If a generated response contains the one of answer candidates, the KQA can predict it (success).
4 A NOVEL METRIC: KNOWLEDGE-VERIFEE
QA
Existing automatic evaluation metrics, namely BLEU and ROUGE (Papineni et al., 2002; Lin, 2004), are limited in that they only consider the lexical overlaps of words without measuring the factual correctness of the generated responses. As shown in Figure 3 (a), there could be multiple correct responses, but existing metrics score them lower due to the lexical mismatch. To solve this issue, propose Knowledge-veriie responses contain factually correct knowledge given the dialogue history. We formulate extractive QA task (Rajpurkar et al., 2016) by automatically derive QA pairs from the dialogue and the large-scale KG in each dataset (See Figure 3). Then, we fine-tune BERT (Devlin et al., 2019) on synthetic KQA pairs to build QA model. To evaluate generated responses from dialogue generation model, we concatenate the dialogue history and the generated response then forward it into the trained QA model. If the QA model yields the correct answer, we regard this case as the generated response contains accurate knowledge. For more details, see Appendix D.
Question Answering (KQA) which measures whether generated
EXPERIMENT
5.1 EXPERIMENTAL SETUP
We conduct experiments on the OpendialKG dataset (Moon et al., 2019), which contains 15K dialogues with 91K utterances associated with a large-scale Knowledge Graph (KG), namely Free- base (Bollacker et al., 2008) with 100k entities and IM facts. Among them, 49% of the utterances come with the gold knowledge, whereas others are not. We randomly split the dataset into train (70%), validation (15%), and test sets (15%). We also examine KOMODIS dataset (Galetzka et al., 2020), which contains 7.5K dialogues associated with much smaller KG with 88k facts. As retrieval candidates, we use I-hop KG for OpendialKG and 2-hop KG for KOMODIS. Except Figure 2, most of the experiments are on OpendialKG dataset. We use T5-small (Raffel et al., 2020) for all experiments for the fair comparison. For more details, see Appendix D.
5.2 BASELINES AND OUR MODELS
We compare different variants of our SURGE framework against various KG-augmented dialogue generation models. No Knowledge. This model is only provided with the dialog history, thus no external knowledge is used. All Knowledge. This model is provided with entire facts within k-hop subgraph of entities associated with the dialog history. Gold Knowledge. This model is provided with the exact gold knowledge, even in the test time if the gold knowledge exists. Space Efficient Encoding. This model takes all facts from the k-hop subgraph of the entities as input. We two different encoding methods introduced in (Galetzka al., 2021), namely Space Efficient (series) and Space Efficient (parallel). EARL. The latest RNN-based model, where the entities are conditioned in response generation (Zhou et al., 2021). DifrKG. Dialogue generative model with differentiable path traversal (Tuan et al., 2022). Random/Sparse Retrieval. These models are provided with selected facts from a 1-hop subgraph, via the random sampling or the sparse retrieval - BM25 (Robertson & Zaragoza, 2009). Dense Retrieval This model is a variant of our framework where T5 encoder (Raffel et al., 2020) is used for d in Eq. 4 instead of GNNs similar to Bi-encoder and Poly-encoder (Humeau et al., 2020). SURGE (unsupervised). Ours with retrieved context-relevant facts from k-hop subgraph, where the retrieval is trained without any supervision. SURGE (semi-supervised), Ours but the retriever is trained with supervision if it exists. SURGE (contrastive). Ours with both semi-supervised retriever learning and contrastive learning term.
7

---

Under review as a conference paper at ICLR 2023
Table 1: Experimental results on OpendialKG dataset with T5 1m-sllll model. indicates the model under the incomparable oracle setting, which uses the gold facts even in the test time.
KQA
BLEU
ROUGE
Unigram F11
Method
EM
F1
B-1
B-2
B-3
B-4
R-1
R-2
R-L
Knowledge
12.25
20.69
15.79
9.19
5.61
3.43
19.67
7.13
19.02
22.21
All Knowledge
43.58
50.60
15.95
9.98
6.72
4.65
20.96
8.50
20.21
24.34
Bazelines
Space Emfieient (series)
36.60
42.64
16.15
10.03
6.66
4.50
21.15
8.56
20.44
24.55
Space Ellicient (paraflel) EARL
38.54
44.34
16.33
10.22
6.81
4.64
21.42
8.85
20.68
24.87
32.47
35.88
11.49
6.34
4,06
2.75
15.36
4.37
14.61
16.88
DinKG
12:25
20.99
15.68
9.13
5.60
3.46
19.50
7.07
18.84
22.26
Random Retrieval
31.72
38.95
15.70
9.52
6.12
3.99
20.21
7.88
19.55
23.28
Retrieval variants
Sparse Retrieval (BM25)
29.50
36.96
15.63
9.44
6.05
3,96
20.05
7.67
19.37
23,10
Dense Retrieval (Bi-encoder)
46.17
52.52
16.67
10.44
7.05
4.91
20.41
8.38
19.66
23.85
Dense Retrieval enoldeencoder)
46.05
52.57
17.56
11.01
7.45
5.18
20.66
8,46
19.87
24.24
SURGE (unsupervixed)
48.49
55.77
17.77
11.30
7.69
5,36
21.64
9.14
20.75
25.24
Ours
SURGE (semi-superised)
51.00
57.63
11:21
7.61
5.28
SURGE (contrastive)
50.45
57.70
17.29
11.04
7.54
5,28
21.35
8.98
20.48
25.10
Oracle
Gold Knowledge!
63.32
67.90
18.47
12.79
9.32
6.92
24.93
11.97
24.03
28.82
Gold Response
93.30
95.21
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
Reteieval Revults
Table 2:
Experimental results on KO-
Table Knowledge-consiitent response generation results under the condition where we use the modified gold subgraph instead of retrieved one, to solely evaluate the efficacy of contrastive learning.
35
MODIS dataset with T5-small model. For full experimental results, see Appendix E.
15-
KQA
BLEU
Miks
EM
F1
B-1
B-2
F1
Figure 4: Knowledge
Random
12.41 14.17
7.74
4.02 16.29
Method
F1
KFI
retrieval results on the
SE (xerias)
12.41 14.70
8.34
17.37
SURGE (unsupervised)
27.78
24.09
OpendialKG dataset, with metries of MRR and @
SE (parallel)
16,46
18.70
9.33
5.66
17.72
SURGE (semi-supervised)
28.30
26.38
SURGE (Ours) 17.30 19.50 11,51
7.18 19.51
SURGE (contrastive)
28.17
5.3
EVALUATION METRICS
We evaluate the generated responses using BLEU (Papineni et al., 2002), ROUGE (Lin, 2004) and FI score with the gold response. Along with these conventional text evaluation metrics, we also evaluate the results with our new metric, KQA (S 4), which measures whether the generated responses contain proper knowledge. Lastly, we compute the Knowledge F1 (KFI) (Shuster et al., 2021) to measure the unigram overlap between the retrieved knowledge and generated response.
5.4 EXPERIMENTAL RESULTS AND ANALYSIS
response generation performances of baselines and our SURGE on OpendialKG dataset. As shown in Table 1, our models significantly outperform all the baseline models, excluding oracles, in all evaluation metrics. The high BLEU, ROUGE, and F1 refer that ours sufficiently learns the syntactic and semantic structure of the responses. Our models also achieve high Fl and EM scores in KQA. The high KQA scores indicate that the generated responses are formed with the correct facts, which are relevant to the dialog context. Even the baseline models such as All Knowledge, Space Efficient Encoding (Galetzka et al., 2021), EARL (Zhou et al., 2021), and DiffKG (Tuan et al., 2022), which are provided with all of k-hop facts, underperform than ours. The result demonstrates that selecting relevant knowledge critical in generation. In Figure 2, we additionally report the experimental results on KOMODIS dataset to show applicability of our method to other dataset. Our SURGE (contrastive) also outperforms other baselines in KOMODIS dataset. For results with all metrics, please see Table 8 in Appendix E.
In Table 1, report the knowledge-grounde
-aggeeeeeeeeeeeeeddd response
Knowledge Retrieval Figure 4 shows performances of retrievers, for which we measure the performance on 45% of test dialogues containing the gold knowledge, with Mean Reciprocal Rank (MRR) and Hits@k as metrics. Our models outperform all basclines by large margins since ours has learnable retriever unlike others. Further, contrastive Icarning version of ours, including semi- supervised retriever training, outperforms an unsupervised version. See Appendix G for examples.
Soattedge-Consisteen Generation We conduct an ablation study on our models to validate the knowledge consistency performance of the response generation by computing the Knowledge F1 (KF1) score (Shuster al., 2021). To focus solely on the case where given knowledge is consistently reflected in the generated responses, we use the gold knowledge rather than the retrieved one. We randomly modify the tail entity of each gold knowledge ensure that responses generated
8

---

Under review as a conference paper at ICLR 2023
Context
Gold response
Baseline response
SURGE respouse
(a) loved Moby Dick. Can you recommend something
was writlen by Herman Melville in 1851. It's sometimes called The Whale.
Moby Dick sailor, Do you like her work?
Moby Dick was writlen by Herman Melville. He also wrote The Whale.
similar?
(b) Do you know anything the actor Adam Brown?
Yes. be was in the movic The Hobbit: An Unexpocted Journey.
Adam Brown starred King Kong. Have you
Adam Brown starred The Hobbit:
The Desolation of Smaug and The Hobbit: The Battle of the Five Arnies.
seen it?
(a) Retrioved Subgraph from SURGE
(b) Retrieved Suhgraph from SURGE
(Moby Dick; or, The Whale, writen_ by. Herman Melville) (Moby Dick, written, by. Norman Corwin)
(The Hobbit: The Battle of the Five Arnies, sturred acttors Adam Brown)
(The Hobbit: An Unexpected Journey, starred actors, Adam Brown)
(Moby Dick, written_by. by. Ray Bradbury)
(The Hobbit: The Desolation of Smaug, starred actors, Adam Brown)
Figure 5:
Examples of responses from the baseline (Space Efficient, parallel) and responses from SURGE.
Table 4: Performance comparisons of Table 5: Human evaluation variants graph encodings, described on Consistency, Informativeness, and Fluency. (p < 0.05)
in Section 3.4.
Method
EM
KQA
F1
Knowwledge Length
Method
Cousis.
Hlunnyy
Nalve
69.27
55.2 25.23
68.21
All Knowledge
2,52
1.99
2.62
Visualization of the em- bedding space from graph(star)- text(circle) contrastive learning.
Figure 6:
Invariant
$1.55
56.
121.82
Spuce Efficient
2,47
1.75
2.45
Edficient (ensity only)
99
49.00
15:42
lavariant Efficieat
51.00
57.63
15.42
SURGE
2.39
2.92
from the given knowledge rather than the trained knowledge. Figure 3 shows that model with contrastive learning term outperforms all others in the KFI, implying that the generated responses accurately reflect the encoded knowledge.
Sensitive Analysis on Graph Encoding We further conduct an analysis on graph encoding variants introduced in subsection 3.4. The knowledge length in Figure 4 indicates the average token length used for graph encoding. Our Invariant & Efficcient s/" performs the best against other variants, while using the lesser space at the graph encoding phase. Notably, simple Invariant achieves a comparable performance against Invariant & Efficient, but yields a longer sequence.
Retrieval and Generation Examples Figure 5 shows the examples of generated responses along with the retrieved knowledge. We compare SURGE against Space Efficient (parallel) baseline. In example (a), the baseline response contains an incorrect fact distracted by the contextually irrelevant entity 'sailor'. Contrarily, SURGE successfully retrieves relevant facts from the KG then generates the factually correct response. In example (b), similarly, the baseline generates the response with a wrong fact, meanwhile SURGE retrieves context-relevant facts and generates a informative response.
Human Evaluation We sample 30 responses of SURGE, All Knowledge, and Space Efficient on the OpendialKG test dataset, then conduct a human study of them. We recruit 46 annotators, and ask them to evaluate the quality of the generated responses by the 3 models given in a random order, with 3 criteria- consistency, informativeness, and fluency using a 3 point Likert- like scale. As shown in Figure 5, ours obtains significantly (p-value < 0.05) higher scores than others in all criteria, which is another evidence that our framework generates consistent, informative, and fluent responses. We also note that the informativeness score and KQA F1 score have a 0.42 Pearson correlation coefficient. This allows us to confirm that our KQA metric positively correlates with the human evaluation results.
Embedding Space Visualization We further visualize the multi-moda graph-text latent space in Figure 6. The visualization shows that, for the same dialogue with different subgraphs, our SURGE with graph-text contrastive learning (right) generates distinct response embeddings pertraining to different subgraphs, unlike the one without graph-text contrastive learning which shows less variety over responses for the same dialogue (left). We include zoomed Figure in the Appendix.
CONCLUSION
We proposed a novel end-to-end framework for knowledge-consistent dialogue generation which retrieves context-relevant subgraph, encodes a subgraph with the text, and generates knowledge- consistent responses, called as SUbgraph Retrieval- aaugeetedd GEneration (SURGE). Our results demonstrate the effectiveness of our framework in both quantitative and qualitative experiments in knowledge retrieval and response generation tasks. The analysis shows the contribution of each proposed component: retrieval, encoding, and graph-text representation learning. Our work suggests new direction to generate informative responses for knowledge graph- bash- dialogue task by empirically showing the importance of retrieving the more relevant subgraph knowledge rather than using all the relevant knowledge graphs when generating knowledge-grounded responses.

---

Under review as a conference paper at ICLR 2023
REPRODUCIBILITY STATEMENT
We attach the source code of our SURGE framework in the supplementary file to facilitate the reproducibility of our work. For experimental setups, we provide every details in Appendix D.
REFERENCES
Daniel Adiwardana, Minh- Luong, David R. So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, and Quoe V. Le. Towards human-like open-domain chatbot. CoRR, abs/2001 0abb/2001.09977,,, 2020.
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
Kurt D. Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the ACM SIGMOD International Conference on Management of Data, SIGMOD 2008, Vancouver, BC, Canada, June 10-12, 2008, pp. 1247-1250, 2008.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597-1607. PMLR, 2020.
Fuwei Cui, Hui Di, Hongjie Ren, Kazushige Ouchi, Ze Liu, and Jinan Xu. Syntactically diverse adversarial network for mnowledgegrooundd
grounded conversation generation. In Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Findings of the Association for Computational Linguistics: EMNLP 2021, Virtual Event / Punla Cana, Dominican Republic, 16-20 November, 2021, pp. 4620-4630. Association for Computational Linguistics, 2021. URL https: http///...o// org/
Moens,
/ / findings -enninggs-mmllpppp 394.
Jacob Devlin, Ming-Woo Chang, Kenton Lee, and Kristina Toutanova. BERT: pre- tre-training of deep bidirectional transformers for language understanding. In Proceedings the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HII 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume I (Long and Short Papers), pp. 4171-411888 2019. URL https: / /doi...../ org/ 10. 1138633/// /ev//l/lnl/// 1-11-1142
Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. Wizard of wikipedia: Knowledge-poweree conversational agents. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.
Fabian Galetzka, Chukwuemeka Uchenna Eneh, and David Schlangen. A corpus of controlled opinionated and knowledgeable movie discussions for training neural conversation models. In Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Asunción Moreno, Jan Odijk, and Stelios Piperidis (eds.). Proceedings of The 12th Language Resources and Evaluation Conference, LREC 2020, Marseille, France, May 11-16, 2020, pp. 565-573, European Language Resources Association, 2020. URL https: ht//a/caanntoooooooooooooo org/ 2020. 100.0r1ee...1.. 71/.
Fabian Galetzka, Jewgeni Rose, David Schlangen, and Jens Lehmann. Space efficient context encoding for non-task-oriented dialogue generation with graph attention transformer. In Proceed- ings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume Long Papers), Virtual Event, August 1-6, 2021, pp. 7028-7041. Association for Computational Linguisties, 2021.
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming- Wei Chang. Retrieval augmented language model pre-training. In Proceedings of the 37th International Conference Machine
10

---

Association Under review as a conference paper at ICLR 2023
Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 3929 -938. PMLR, 2020.
William L. Hamilton. Graph representation learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 14(3):1-15) 2020.
Ehsan Hosseini-Asi. Bryan McCann, Chien-Shena Wu, Semih Yavuz, and Richard Socher. A simple language model for task-oriented dialogue. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020.
Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux, and Jason Weston. Poly-encoders: Architec- tures and pre-training strategies for fast and accurate multi-sentence scoring. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URL htttps:///pppet /wtp//ppeeeeveeeeeeee ew. net/ forum? d=kkggnNF
Jachyeong Jo, Jinheon Back, Seul Lee, Dongki Kim, Minki Kang, and Sung Ju Hwang. Edge representation learning with hypergraphs. CoRR, abs/2106. 15845, 2021.
Thomas N. Kipf and Max Welling, Semi-supervised classification with graph convolutional networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017.
Mojtaba Komeili, Kurt Shuster, and Jason Weston. Internet-augmented dialogue generation. CoRR, Abs/2107.07566 2021.
Seanie Lee, Dong Bok Lee, and Sung Hwang. Contrastive learning with adversarial perturbations for conditional text generation. In International Conference on Learning Representations, 2021.
URL https:tt//// / /oppnreviwwweeeeeeoeoee net / forum? wwwpmmidddaaaaaaaaee
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettemmyer. BART: denoising sequence-to--eqqeenee pre-training for natural language generation, translation, and comprehension. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pp. for Computational Linguistics, 2020a.
Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Kütttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensivee NLP tasks. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020b.
Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. A diversity-promoting promoting objective function for neural conversation models. In Kevin Knight, Ani Nenkova, and Owen Rambow (eds.), NAACL. HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, June 12-17, 2016, pp. 110-119. The Association for Computational Linguistics, 2016. URL https: / doi www/oog//... 653/653/612 v1 1014
Yu Li, Baolin Peng, Yelong Shen, Yi Mao, Lars Liden, Zhou Yu, and Jianfeng Gao. Knowledge- grounded dialogue generation with unified knowledge representation. In Marine Carpuat, Marie-Catherine de Marneffe, and Iván Vladimir Meza Ruíz (eds.), Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022, Seattle, WA, United States, July 10-15, 2022, pp. 206 218. Association for Computational Linguistics, 2022. URL https::/aclanthhlleeee org/ 20g/20222 2022.00222222 naacl mminn.
Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pp. 74-81, Barcelona, Spain, July 2004. Association for Computational Linguistics. URL httpsttt///// / anthol ogy. 2
11

---

Under review as a conference paper at ICLR 2023
Iyy Loshchilov and Frank Hutter. Decoupled weight decay regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019, 2019. URL https: http:::///peeeeveewe / htw./eeww.eeeeeeeeeeeeeeeeeeeeee
Kaixin Ma, Hao Cheng. Xiaodong Liu, Eric Nyberg, and Jianfeng Gao. Open domain question answering with A unified knowledge interface. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (V (Volume I: Long Papers), ACI. 2022, Dublin, Ireland, May 22-27, 2022, pp. 1605-1620. Association for Computational Linguistics, 2022. URL https::///// http://00..r111111555/// 2022. -onccll-oong- 113.
Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan T. McDonald. On faithfulness and factuality in abstractive summarization. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguisties, ACL 2020, Online, July 5-10, 2020. pp. 1906-1919, Association for Computational Linguisties, 2020.
Seungwhan Moon, Pararth Shah, Anuj Kumar, and Rajen Subba. Opendialkg: Explainable con- versational reasoning with attention- walks over knowledge graphs. In Anna Korhonen, David R. Traum, and Lluís Mårquez (eds.), Proceedings of the 57th Conference of the Association for Computational Linguistics, ACI. 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pp. 845-854. Association for Computational Linguistics, 2019.
Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Floundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. Webgpt: Browser-assisted question-ansee answering with human feedback. arXiv preprint, arXiv:211..09332. 2021.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 62, 2002, Philadelphia, PA, USA, pp. 311-318. ACL, 2002.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary Vetoto Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems 32, pp. 8024-8035, 2019.
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick S. H. Lewis. Anton Bakhtin, Yuxiang Wu, and Alexander H. Miller. Language models as knowledge bases? In Kentaro Inui, Jing Jiang, Vincent Ng. and Xiaojun Wan (eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNIP 2019, Hong Kong. China, November 3-7, 2019, pp. 2463 -273.. Association for Computational Linguistics, 2019. URL https: //t::///ooi.. org 11t.o////....1111111111 D19 219-1250..
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9 2019.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18- 24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pp. 8748-878 8763. PMLR, 2021.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21:1 -1:1446711 2020.
12

---

Under review as a conference paper at ICLR 2023
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100, 000+ questions for machine comprehension of text. In Jian Su, Xavier Carreras, and Kevin Duh (eds.), Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pp. 2383-2392. The Association for Computational Linguisties, 2016. URL https: //ooi .rgg 0 183368653/// /d1 6264.
Marc' Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. Sequence level training with recurrent neural networks. In Yoshua Bengio and Yann LeCun (eds.), 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings, 2016.
Stephen Robertson and Hugo Zaragoza. The probabilistic relevance framework: BM25 and beyond. Found. Trends Inf. Retr. 3(4):333-389 2009.
Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Object hallucination in image captioning. In Ellen Riloff, David Chiang, Julia Hockenmaier, and unhi Tsujü (eds.), Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 November 4, 2018, pp. 4035 4045. Association for Computational Linguistics, 2018. URL https: http://oo...oggg//////
org 10. 1Bt..////1/////11//// 1437.
Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Eric Michael Smith, Y-Lan Boureau, and Jason Weston. Recipes for building an open- open-domain chatbot. In Paola Merlo, Jörg Tiedemann, and Reut Tsarfaty (eds.), Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, EACI. 2021, Online, April 19 23, 2021, pp. 300-325. Association for Computational Linguisties, 2021.
Md. Rashad Al Hasan Rony, Ricardo Usbeck, and Jens Lehmann. Dialokg: Knowledge-structure aware task-oriented dialogue generation. In Marine Carpuat, Marie-Cathering de Marneffe, and Iván Vladimir Meza Ruíz (eds.), Findings of the Association for Computational Linguis-
fics: NAACL 2022, Seattle, WA, United States, July 10-15, 2022, pp. 2557-2571. Associa- tion for Computational Linguistics, 2022. URL https: / org 10. 1Btg//111.1663//12222020 findings naacl. 195.
Edward Scheinerman and Daniel Ullman. Fractional graph theory: a rational approach to the theory of graphs. Courier Coporation, 2011.
Michael Sejr Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. In Aldo Gangemi, Roberto Navigli, Maria-Esther Vidal, Pascal Hitzler, Raphaël Troncy, Laura Hollink, Anna Tordai, and Mehwish Alam (eds.), The Semantic Web 15th International Conference, ESWC 2018, Heraklion, Crete, Greece, June 3-7, 2018, Proceedings, volume 10843 of Lecture Notes in Computer Science, pp. 593-607. Springer, 2018 URL https: / http:///...o///////// wtt..oogg/...000 978-3-31 310---19-9330 17-4 38..
Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computa- tional Linguistics, ACL. 2016, August 7-12, 2016, Berlin, Germany, Volume I: Long Papers. The Association for Computer Linguistics, 2016.
Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. Retrieval augmentation reduces hallucination in conversation. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Findings of the Association for Computational Linguistics: EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 16-20 November, 2021, pp. 3784-3803. Association for Computational Linguistics, 2021.
Yi-Lin Tuan, Yun-Nung Chen, and Hung-yi Lee. Dykgchat: Benchmarking dialogue generation grounding on dynamic knowledge graphs. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. EMNLP-IJCNLP 2019, Hong Kong. China, November 3-7, 2019, pp. 1855-1865. Association for Computational Linguistics, 2019.
13

---

Under review as a conference paper at ICLR 2023
Yi-Lin Tuan, Sajjad Beygi, Maryam Fazel-Zarandi, Qiaozi Gao, Alessandra Cervone, and William Yang Wang. Towards large-scale interpretable knowledge graph reasoning for dia- logue systems. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Find- ings of the Association for Computational Linguistics: ACI. 2022, Dublin, Ireland, May 22- 27, 2022, pp. 383 383-395. Association for Computational Linguistics, 2022. URL https: 101110001166///2222222 findings-acl...
I Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Ma- chine Learning Research, 9(86):2579-2605, 2008. URL http:t /mmr..mrrrrr.o http://lllrrr//////////////oo/lt wttpppperr////oeeo/ootttt..sssst html.
Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, and Partha P. Talukdar. Composition- multi-relational graph convolutional networks. In 8th International Conference on Learning Representations. ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 30: Annual Conference Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. 1998-6008, 2017.
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018.
Denny Vrandecic and Markus Krötzsch, Wikidata: a free collaborative knowledgebase. Commun. ACM, 57(10):778 2014.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of-the-arr natural language processing. In EMNLP 2020- Demos, Online, November 16-20, 2020, pp. 38-45, 2020. URL https::///// / /ttp:/oi.oooogooo org. 18 1861118633/111 00055//vv/222// 1ttp://11..1653//1222222220
emnlp demos... 6.
Sixing Wu, Ying Li, Dawei Zhang, Yang Zhou, and Zhonghai Wu. Diverse and informative dialogue generation with context-specific commonsense knowledge awareness. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (eds.). Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. ACL 2020, Online, July 5-10, 2020, pp. 5811 5820. 10. wttpo///.....//////../ vl/
Association for Computational Linguistics, 2020. URL httph: /ooi 2020. mann..main....mm 515.
Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabás Póczos, Ruslan Salakhutdinov, and Alexander J. Smola. Deep sets. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA. USA, pp. 3391-3401, 2017.
Houyu Zhang, Zhenghao Liu, Chenyan Xiong, and Zhiyuan Liu. Grounded conversation generation as guided traverses in commonsense knowledge graphs. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (cds.), Proceedings of the 58th Anmual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pp. 20433. Associa- tion for Computational Linguistics, 2020a. URL http:: // doii org 10. 863 10.53311/22200 acl-maing inn.iii...n 84.
Yizhe Zhang. Sigi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. Dialogpt: Large-scale generative pre-training for conversational response generation. In ACL, system demonstration, 2020b.
Hao Zhou, Minlic Huang, Yong Liu, Wei Chen, and Xiaoyan Zhu. EARL: informative knowledge- grounded conversation generation with entity-agnostic representation learning. In Proceedings of
14

---

Under review as a conference paper at ICLR 2023
the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pp. 2383-2395 Association for Computational Linguistics, 2021.
15

---

Under review as a conference paper at ICLR 2023
A DISCUSSION ON LIMITATION AND POTENTIAL IMPACT
Limitation As briefly discussed in Appendix G, our work limited in multiple dimensions primarily in terms of dataset, retrieval, and generation. First, the benchmark dataset is limited. Despite the fact that there are several public Knowledge Graph (KG) available (Vrandecic & Krötzsch, 2014; Bollacker et al., 2008), only one dataset (Moon al., 2019) provides both the diverse of dialogue and the corresponding large-scale KG. This circumstance may limit the rigorous evaluation of our framework's 's adaptability various settings. Future work may study applying our approach for wider range of dialogue datasets based on Wikipedia (Dinan et al., 2019) by leveraging existing public large-scale KG such as Wikidata (Vrandecic & Krötzsch, 2014). Second, the search space for retrieving context-relevant subgraphs can be expanded. Our SURGE framework now runs I-hop KG that is rooted to entities the given dialogue history. Finding the entity within the text, on the other hand, necessitates precise named entity extraction and entity linking. Therefore, future work may investigate extending our approach to a framework that can retrieve the context-relevant subgraph among entire KG instead of 1-hop KG. Third, there is still room for improvement in generation quality since we generate enowledge-enhanced responses with a small-scale Pre-trained Language Model (PLM) for efficiency. Such PLMs occasionally fail to generate natural sentences with high quality (Raffel et al., 2020). Future work could aim to improve generation quality using small-scale PLM.
Broader Impact Our proposed knowledge-grounded dialogue generation model is essential for designing user-friendly real-world AI systems. Among various types dialogue generation models, knowledge grouudeggroddedddee dialogue models are trained to interact with users and convey factual information to users in natural languages. Their conversational features can be adapted to any user interfaces that connect the bilateral interaction between human and computer. We believe that the conversational interfaces can enhance the users' experiences and reduce the users' efforts learning how to use the systems. However, knowledge-groundes dialogue models can become vulnerable generating offensive, harmful contents or responses with misinformation depending on the users or data. When deploying the models the real world, in addition to generating realistic responses, they also need to be robust to adversarial feedback from malicious users and biases inherited in pre-training corpus, or else they could malfunction. Along with the quantitative and qualitative evaluations on generated responses, it is worthwhile to examine robustness of the dialogue models.
training
16

---

Under review as a conference paper at ICLR 2023
B NOTATIONS
We organize the notations we used for formally describing our method in Table 6.
Table A list of notations that we used for defining our method.
V
pre-defined vocabulary of tokens for pre-trained language models (text)
E
pre-defined vocabulary of entities (symbol)
R
pre-defined vocabulary of relations (symbol)
knowledge graph symbols written in typewrite font input sequence (vector) input tokens (scalar)
x
NN
//
VT
output sequence and tokens multi-re celti-rrllaionall graph, such as knowledge graph retrieved subgraph: Z g
G
Z
triplet (edge): 2
tokenization (mapping) function of KG symbol to sequence of tokens representation function for retrieval triplet representation function for retrieval Transformer Encoder
g(-)
d(-)
Enc
Dec
Transformer Decoder
token (word) embedding function
generator parameter
retriever parameter
set encoding function
perturbation function B
set permutation
n
the number of triplets in a retrieved subgraph Z the number of samples in marginalization term encoder hidden state (single token)
k
z
Z
encoder hidden states (sequence of tokens)
h
decoder hidden state (single token)
H
decoder hidden states (sequence of tokens)
X
input embeddings after token embedding function (sequence)
Y
output embeddings after token embedding function (sequence)
C PROOFS
In this section, we first show that naïve encoding function &b in Section 3.4 is neither permutation invariant nor relation inversion invariant, formalized in Proposition C.I. After that, we prove that our invariant and efficient encoding function 2/" with graph-conditioned token embedding perturbation is both permutation invariant and relation inversion invariant, formalized in Proposition C.2.
Proposition C.1. nalve encoding function up is neither permutation invariant relation inversion invariant.
Proof. We prove this by contradiction.
Suppose [#1,..., #n] and 2 {(a, d, b), (b, a), (a, c)). Moreover, let 2' (a, c)} be one of permutations of Z with the permutation order π = (2, 1,).
{(b, ((b,e,a), (a, (a,
From the definition of naïve encoding. (< Z) [a, d, b, b, e, a, a, d, æI.... xn and easy to notice that (0 2) +
(a zz) = [b, e, a, a, d, b, a, d, c, c,.,.... v,). Therefore, (xx 2'), thus the naïve encoding not permutation invariant.
We then show naive encoding is not relation inversion invariant. Suppose 2 d,b,, (b, (c., a)). where (a, c) E Z is changed to its inverse relation (c, (c, -d, a). Then, (0x 2") [a, d, b, b, e, a, c, -d, a, I1, [,,] that is different against 1/ Z): (xx, 2) * (x,, z"). Therefore, the naïve encoding function not relation inversion invariant.
17

---

Under review as a conference paper at ICLR 2023
In conclusion, from the above two counterexamples, prove that naive encoding function 2/ is neither permutation invariant nor relation inversion invariant.
We now provide proof of the permutation invariance and the relation inversion invariance of our invariant and effective graph encoding *²,, described in Section 3.4. Before starting the proof, we first revisit the permutation invariant property of graph neural networks that sum, mean and max operators are permutation invariant for the input set of AGGR. Thus, if we use sum, mean, max for AGGR, then the token embedding perturbation function /3 naturally satisfies the permutation invariance property. In other words, ((X, 2) 3(X, 2), where X = =b(x, SORT(ENT(2))) for any permutation x.
Proposition C.2. Invariant and efficient encoding sp" is both permulation invariant and relation inversion invariant.
Proof. Suppose x [zi,...., and Z = {(, d) (b, a), d.), We first consider the permutation invariance for any permuted set 2' = ग 2. While and 2' can have different orders of elements thus the outputs of ENT(2) and ENT(2") could be different, we always obtain the same output with the usage of the SORT operator for encoding. In other words, SORT(ENT(Z)) = (BRT(EN(ZZ') holds due to the definition of the SORT operation in Eq. 5 of the main paper. Therefore, (xx SORT (ORT((ETTT))))
(xx SORT(ENT( holds.
Further, since the token embedding perturbation function B(,, Z) along with sum, max, or mean in AGGR is also permutation invariant with regards to any permutation 2, conclude invariant and efficient encoding x/" is permutation invariant.
We finally prove the relation inversion invariance property of き³ Suppose Z" (3 U E) where / E 2 is any triplet a set and is inverse of Then, ENT(2) = ENT(2") that is trivial as (ZT returns the set of only unique nodes in Z. Therefore, (bx (ORTT(())( (ax, SORT(ENT [)))) correspondingly holds.
The remaining step to conclude the proof to show the following equality: (,, INV(2)) = B(, INV(2")), to conclude that ( 2) B(b((,, SORT(ENT(Z")), INV(Z")). We note that
(t (x, 2") from
B(b((, (ORT(E) INV(2)) INV(2) I2"( as INV makes any graph as bidirectional one by the definition in Eq. of the main paper. Therefore, ((, INV(2)) B(, INV(2")) holds, and the relation inversion invariance property of up" holds.
D EXPERIMENTAL SETUP
In this section, we introduce the detailed experimental setups for our models and baselines. Specif- ically, describe the details implementation, dataset, training and model in the following subsections of D.I, D.2, D.3 and D.4, one by one.
D.1 IMPLEMENTATION DETAILS
We use the T5-small (Raffel et al., 2020) as the base Pre-trained Language Model (PLM) for all experiments. For the pre-trained checkpoint, we use the version that the authors released. For all implementations, we use Pytorch (Paszke et al., 2019). To easily implement the language model, we use the huggingface transformers library (Wolf et al., 2020).
Retriever Details In this paragraph, we describe the implementation details of our context-relevant subgraph retriever, including the triplet embedding and dialogue context embedding for the retriever.
For the dialogue history embedding function we use the existing pre-trained language model (PLM). Specifically, the parameters of it not to trained. We then instead add a Multi-Layer Perceptron (MLP) on top of it, to give point-wiss wise attention (Bahdanau et al., 2015) each token, whereby all tokens
the encoder part of the T5-small model (Raffel et al., 2020) and freeze
18

---

Under review as a conference paper at ICLR 2023
Romance
Lady Susan
writer
novel
genTa
by
Pride & Prejudice
wrieten_br by
Jane Austen
book
Place_ oo birth
weittwo_ by
(a) Node-level
Sense and Sensibility
writeen by
Steventon
Message Passing
Emma
aggregator
Hypergraph Duality-
gUIZE
xricten,
Place birth
(b) Edge-level Message Passing
Nritton_ by
by
Figure 7: GNN-based Triplet Representation Retrieval. To represent each triplet with regards to its graph structure, we use the message passing on both nodes and edges. (a) Node-level Message Passing. To represent the entity Sense and Sensibility, the message from its neighbors - the entity Jane Austen - is aggregated. (b) Edge-level Message Passing. To represent the relation wr tten by, the messages from relations associated to green hyperedge aggregated, We do not draw self-loops and inverse edges for simplicity.
equally considered in the sentence encoding, Formally,
eXp(MLP(2))
g(x))
Zi, Z |z zn] = Enc(X), O2
Vi
2==1 eXp(MMP(())
where is scalar, and MLP is a Multi- Perceptron consisting of two linear layers and ReLU nonlinearity.
For obtaining triplet representations, we need to embed the entity (node) and relation (edge) into the latent space. Similar to the token embedding matrix used in PLMs, we can introduce the entity and relation embedding matrices. However, since the number of entities used in Freebase of OpendialKG (Moon et al., 2019) is too large compared to the number of tokens in T5 (100,814 14 vs 32,000) (Raffel et al., 2020), it is inefficient to introduce the trainable entity embedding matrix for the retriever. Furthermore, the use of standalone entity embedding matrix might be sub-optimal in terms of generalization since there dataset.
evidence that all entities large-scale KG emerge in training
Thus, instead reuse the contextualized representation from the PLM encoder, embed each node if the corresponding entity exists in the dialogue context. Formally, suppose that there is a triplet ((e,, I, e.)) in the I-hop subgraph G, which satisfies the following condition: <(ex) C or q(e)) C x. If so, we can know the position of the mapped entity within the dialogue history: [Estarts ..., Fend]
q(en) from ((e)) S x. Therefore, the node embedding for the entity en is snnd obtained by Ent Emb(ee)) = ln(ex)) Eisstart Enc( X); iff q(en) S æ. If the entity mention does not exist in the dialogue history. we use the zero vector as the node embedding, For edge embedding, we use the trainable relation embedding matrix R. E TR. [| 128 to represent the edge, since the number of relations is relatively small (1,357).
With our node and edge representations, we now focus on representing the triplet in Eq. 4 of the main paper for its retrieval. In particular, we the Graph Neural Networks (GNNs) for encoding triplets, where we obtain the node representations from the Graph Convolutional Network (GCN) (Kipf & Welling, 2017) that a widely used architecture for representing the nodes with respect to their graph structures. However, for representing the edges, we use the Edge Hypergraph Graph Neural Network (EHGNN) used in et al. (2021), due to its simplicity but effectiveness for edge representations. We summarize our triplet representation in Figure 7.
Graph Encoding Details In this paragraph, we describe the implementation details of the token embedding perturbation function & used in our Invariant and Efficient graph encoding introduced
19

---

Under review as a conference paper at ICLR 2023
Transformer Encoder
CompGCN
(CEmmaa))
Token Embedding Layer f())
write
ICLady Susan")
Nzito
[CJane Austen')
Retrieved Subgraph
[Jane Austen] [Lady Susan] [Emma] Could you recommend any
Figure 8: Comprehensive diagram for Invariant and Efficient graph encoding. Our proposed graph encoding first concats the sorted list of entities in front of the dialogue history. Then, we form the learnable affine transformation 7, 6 for each entity using relation-aware GNN such CompGCN (Vashishth et al., 2020).
in Section 3.4. To be aware of the relation of the graph over GNNs, we use the simplified version of CompGCN (Vashishth et al., 2020). For architectural details, instead of using the different linear layers to distinguish the inverse relation from its opposite relation, we use the same linear layer. Also, we use subtraction the specific composition operator for reflecting relations in CompGCN.
Then, we form the learnable affine transformation based on the aggregated representation from GNN layers, to perturb the token embeddings with respect to their graph information as in Equation 7 of the main paper. particular,
Vo E Waa; 2)1)): 77 R-GNN(f(a)) (a)): (CCC)
7 MLP1( 6 -LLP)((n)
(1+ f(a) + 5,
where MIP, and MLP are learnable MLPs consisting of two linear layers with ReLU nonlinearity. In Figure illustrate comprehensive diagram of Equation which enables Invariant and Efficient graph encoding to understand the structure of the retrieved subgraph Z.
Contrastive Learning Details For contrastive learning, we initialize 7 in Equation as 0.01.
KQA Details In this paragraph, describe the implementation details for Knowledge- verifying Question Answering (KQA) introduced in section 4. For building the QA dataset, we first gather the dialogue sessions where the gold response contains the entity from the whole OpendialKG dataset. Then, we extract the triplet from the given whole KG where the head entity is placed within the dialogue history and the tail entity is placed within the gold response. We build a QA training dataset based on the extracted triplets and corresponding dialogue session. To diversify the training data, we replace the tail entity of each triplet with plausible candidate entities within KG and change the entity in the response following the changed entity on the triplet. As result, we obtain the QA dataset size 200k. We train the BERT-base (Devlin et al., 2019) with the constructed QA dataset. We hold out 10% of data for validation and obtain the fine-tuned BERT model with 88.89 Fl score on the hold-out validation set. When we apply the fine-tuned QA model on the evaluation of the generated responses, rebuild the QA evaluation set with the generated response instead of gold response as illustrated in Figure 3 of the main paper.
D.2 DATASET DETAILS
We mainly conduct experiments on OpendialKG (Moon et al., 2019), which provides the parallel dialogue corpus corresponding to the existing large-scale Knowledge Graph (KG) named Free- base (Bollacker et al., 2008). The provided large-scale KG consists of total 1,190,658 fact triplets over 100.813 entities and 1,358 relations. This dataset is collected from 15K numan-to-humae role-playing dialogues, having multi-turns, from which we pre-process that each assistance response the label and its corresponding dialogue history is the input. Although some of the data contain
20

---

Under review as a conference paper at ICLR 2023
the gold knowledge that is useful for generating the response the ongoing conversation, found that 51% of data has no gold knowledge. To overcome this limitation, we additionally find entities from the dialogue history using the Named Entity Recognition module in spaCy', and then include the extracted entities' corresponding triplets in the KG to the dataset. For entity linking, we use the exact match. Since the dataset does not provide the pre-defined data split, randomly split sessions into train (70%), validation (15%), and test sets (15%). We also conduct experiments on KOMOIDS (Galetzka et al., 2020) dataset and follows the same preprocessing as OpendialKG dataset.
D.3 TRAINING DETAILS
All experiments constrained to be done with single 48GB Quadro 8000 GPU. SURGE training needs 12 GPU hours. For all experiments, we select the best checkpoint on the validation set. We fine-tune the SURGE for 30 epochs on the training set, where we set the learning rate le-4, weight decay as 0.01, learning rate decay warmup rate as 0.06, maximum sequence length for dialogue history as 256, maximum sequence length for knowledge as 128, and batch size as 24. For retrieval, we use the subgraph size n as 3, and sample size k for marginalization as 4. We use the Adam (Loshchilov Hutter, 2019) optimizer for training. For fair evaluation, we apply the same training setting to all baselines if applicable.
D.4 MODEL DETAILS
In this subsection, we describe the details of baselines and models used in our experiments, as follows:
1. No Knowledge: This model is provided with only the dialog history. No knowledge is used to generate responses.
2. Gold Knowledge:
This model is provided with the dialogue history along with its exact gold knowledge for the gold response. Thus, since this model uses such gold knowledge, we expect the results of it as the upper bound of the task.
Space Efficient (series): This model is provided with all the knowledge which are related the entities that appeared in the dialogue history (Galetzka et al., 2021), by matching the entities in the dialogue history and the entities the KG. In particular, this model encodes the entities and their relations explicitly in the words in the encoder part.
Space Efficient (parallel): This model is mostly the same the above model space Efficient (series)- except the knowledge encoding part. Specifically, it encodes the entities in the words like the above, whereas, encoding the relation between entities in the segmentation block of the entities Galetzka et al. (2021).
5. EARL: This model uses the RNN-based encoder-decoder architecture with the entity-agnosiee representation learning (Zhou et al., 2021), with all the provided knowledge associated with the entities in the dialogue history. Specifically, this model first calculates the probability of words obtained by encoding the entities in the KG, and then uses such probabilities to generate a word in the decoding phase.
DiffKG: This model (Tuan et al., 2022) uses a differentiable path reasoning, which is jointly trainable along with the dialogue generation. After the path reasoning, the entities in the reasoning path are naively appended front of the dialogue history, then concatenated input is forwarded to the pre-trained language model.
This model is provided with entire facts from k-hop subgraphs of entities that appeared in the dialogue history. However, instead of encoding all the knowledge in one-hop subgraph as in Space Efficient, this model randomly samples them, which are then used for generating responses.
Random Retrieval:
8. Sparse Retrieval (BM25): This model is also provided with entire facts from k-hop subgraphs of entities. To sample relevant facts to the dialogue history among the entire facts, this model uses BM25 (Robertson & Zaragoza, 2009) that is sparse retrieval model. To be specific, let assume we have dialogue history and its corresponding facts from k-hop subgraphs of matched entities. Then, to run BM25, we first concatenate components of each fact consisting of two entities and
hhtps///spcy.io//
21

---

Under review as a conference paper at ICLR 2023
35
Method
MRR
@its@1
His@3
Hits@5
His 10
Hits@:
knowlogg Length
61
Random Retrieval
8.67
3.11
7.89
10.86
17.94
66.22
60
Sparse Retrieval (BM25)
9.89
16.67
61.21
25
59
Dense Retrieval (Bi-encuder)
Dense Retrieval (Puly-cncoder)
21.47
20.89
11.99
12.13
23.79
24.12
30.31
31,51
39.26
41.43
67.26
68.03
50 St AURGE. (unsupevisee))
21.24
10:46
34.06
31.97
44.17
74.74
57 SURGE (semi-supervise))
22.53
13:40
24.79
31.84
42.37
69.33
15
SURGE (contrastive)
25.98
16.67
28.50
35.72
46.11
74.31
Num Facts in Retrieved Subgraph
Figure 9: (Left:) Performances of our SURGE by varying the number of facts for retrieving the subgraph (i.c., varying the number of triplets in the subgraph) from threc, five, to ten, with the length of sequence for knowledge (knowledge length) and FI scores of KQA as evaluation metrics, (Right:) We additionally report the knowledge retrieval performances, with MRR and Hits @ K as evaluation metrics.
Table 7: Experimental results OpendialKG dataset with BART-base.
KQA
BLEU
ROUGE
Unigram F1
Method
EM
B-1
B-2
B-3
B-4
R-1
R-2
R-L
No Knowledge (BART-buse)
31.17
37.54
17.38
10.79
7.16
4.81
20.64
8.22
19.92
24.36
Space Efficient (BART buse, Series)
48.49
53.83
18.56
11.85
8.01
5,56
22.36
9.43
21.48
26.38
Space Efficient (BART-base, Porallel)
55.06
12.19
8.35
22.63
9.79
21.76
26.79
SURGE (BART-base, seml-sapervised. 10)
50.84
57.35
17.80
11.12
7.48
5.18
18.64
7.27
17.77
22.07
SURGE (75-small semi-superrised. 3)
51.32
17.63
11,28
7.41
21.74
9.18
20.85
25.57
SURGE (75-small, semn-suupervised 10)
54.50
61.65
17.70
11.37
7.81
5.50
21.55
9.09
20.65
25.44
one relation, and tokenize the dialogue history and the facts for obtaining corpus and queries, respectively, for BM25. After that, BM25 calculates the lexical overlapping score between the dialogue context (corpus) and the one-hop fact (query), from which we use the relevant facts having top-k scores by BM25.
9. Dense Retrieval (Bi-encoder, Poly-encoder): This model uses trained language model for the triplet embedding of the retriever instead of using GNN. Specifically, we consider each triplet as single sentence (e.g, (Jane Austen, write, Susan) - "Jane Austen write Susan"" and embed them with the pre-trained language model. For scoring, we use both bi-encoder and poly-encoder architectures (Humeau et al., 2020).
10. SURGE (unsupervised): Our basic subgraph strrevallaugmmened generation framework that is provided with entire facts from k-hop subgraphs of entities. In particular, this model trains the structure-aware subgraph retriever without any guidance of the gold knowledge (i.e.. ground truth knowledge for the dialogue history is not given). In other words, for the given dialogue context, this model implicitly learns to retrieve the context-relevant knowledge, and then generates the response with the retrieved knowledge.
11. SURGE (semi-supervisel): Our subgraph eetrieval-augmentee generation framework with semi- supervised learning of graph retrieval, with provided entire facts from k-hop subgraphs of entities. Unlike the unsupervised version of SURGE, this model trains the retriever select the gold knowledge if the dialogue context has such knowledge during training.
12. SURGE (contrastive): Our full subgraph trastive learning of graph-text modalities as well as the supesvupe learning of graph retrieval, with provided entire facts from k-hop subgraphs of entities. Unlike aforemennioned frameworks of ours, this additionally enforces the model to faithfully reflect the retrieved knowledge in the input, to the generated response with contrastive learning.
generation framework with the con-
E ADDITIONAL EXPERIMENTS
E.
VARYING THE NUMBER OF FACTS IN SUBGRAPHS
We experiment our SURGE framework with varying the number of facts in retrieval, which are then used in our graph encoding function to condition the encoded graph information for response generation. Specifically, in Figure 9, we report the length of sequence for knowledge (knowledge length) and Fl scores measured by KQA for SURGE framework, with different numbers
22

---

Under review as a conference paper at ICLR 2023
Table 8: Experimental results on KOMODIS dataset with T5-small.
KQA
BLEU
ROUGE
Unigram
Method
EM
F1
B-1
B-2
B-3
B-4
R-1
R-2
R-L
F1
Random
12.41
14.17
7.74
4.02
2.46
1.68
21.79
4.00
21.44
16.29
Space Efficient (Series)
12.41
14.70
8.34
5.13
3.77
3.05
22.36
4.85
22.06
17.37
Space Efficient (Parallel)
16.46
18.70
9.33
5.66
4.06
3.20
22.80
4.12
22.47
17.72
SURGE (unsupervised)
16.18
18.51
11.46
7.10
5.15
4.07
23,49
5.77
23.09
18.70
SURGE (semi-supervised)
16.62
19.48
11.28
6.98
5.05
3.98
23,58
5.79
23.21
18.68
SURGE (contrastive)
17.30
19.50
11.51
7.18
5.20
4.10
24.13
6.17
23.74
19,51
of facts within retrieved subgraph: the semi-supprvised model without the contrastive loss. We expect that the performance of our SURGE will increase as increase the number of facts within the retrieved subgraph, since the model can leverage more numbers of knowledge for response generation. As shown in Figure 9, we observe the significant performance improvements facts, while the performance difference between the three and five is marginal. We suggest that this result should be interpreted with the retrieval results on the right side of Figure 9, where about 40% of retrieved subgraphs including the ten different facts contain at least one necessary knowledge, thus the generation performance boosted according to the improvement in retrieval.
[3.5, 10). Note that, in this experiment, we only use
using ten facts against using three and five
E.2 DISCUSSIONS ON USING LARGER PLMS
Notably, we observe that the use of larger Pre-trained Language Models (PLMs) three times more number parameters compared T5-small that we use does not result in better performance for the knowledge-grounded grounded dialogue task. Specifically, in Table 7, we report the experimental results of selected baselines and our SURGE semi-supervised model with BART-base (Lewis al., 2020a) as the base PLM. We want clarify that the BART-base model has 220M parameters, which is about three times larger than the number of parameters of the T5-small model (60M).
We first observe that BART-base shows decent performance without any knowledge (No Knowledge) compared to the no-knowledge case of T5-sm,,,, verifying that the larger PLM generally contains more factual knowledge within its pre-trained parameters. Moreover, BART-base obtains higher scores in the simple word overlap metrics such as BLEU (Papineni et al., 2002) and ROUGE (Lin, 2004), whose results further confirm that larger PLM generate more natural syntactically better sentences than the smaller one, thanks to its parameter size.
On the other hand, we find that BART-base is less suffered from the irrelevant knowledge issue (i.e., conditioning irrelevant knowledge for the given context when generating responses) than T5-small, therefore, the performance of Spuce Efficient Encoding KQA is quite high. However, the use of BART-base does not result in significant improvement on the KQA metric for our SURGE framework. Moreover, ours with T5-small shows better performance than ours with BART-base in terms of KQA scores, when the number of facts within the retrieved subgraph is 10: n 10. This result suggests that the quality of the generated response- having relevant knowledge the given context might depend on the performance of the subgraph retriever whose goal is to retrieve the context-relevant knowledge, rather than the inherent performance of PLMs.
E.3 FULL EXPERIMENTAL RESULT ON KOMODIS
In the main paper, we mostly focus on OpendialKG dataset (Moon et al., 2019), since it is the largest and most realistic public datasets that provides both dialogues across diverse domains and corresponding large-scale Knowledge Graph (KG) (Bollacker et al., 2008). To verify the effectiveness of our SURGE framework, the existence of the large-scale KG and the importance of relevant fact searching is important since focus the real-world scenario where the response generation requires the relevant fact acquirement from the large-scale KG.
raise the question regarding the versatility of our method other datasets. To alleviate the issue, we conduct additional experiments on another dataset named KOMODIS (Galetzka al., 2020), which is also KG-based dialogue dataset. Compared to OpendialKG, KOMODIS does
However,
23

---

Under review as a conference paper at ICLR 2023
Table 9: (Left:) Performance evaluation with the diversity metric named Distinet. (Right) Ablation study results on GNN variants in our modules.
Methad
Diad Dia
KQA
BLEU
ROUGE
Unigram
No Kevnledge
6:c6
1571
Kosstelge
Method
EM
F1
B-1
B-2
B-3
B-4
R-1
R-2
R-L
F1
SEE (Series)
8.49
21.77
Fq 4.
16 56.10
17.42 10.96
7.39
5.17
21.10
20.25
24.79
SET (Paralle)
8.78
70
EARL
Bq CompGCN - GCN
48.61
55.53
17.48
10.97
7.34
5.05
21.23
8.73
20.37
24.77
Spani Retricval (B3325)
7.65
10.6
SURGE (semissupervised) 51.00
57.63
17.70
11.21
7.61 5.28
21.43
8.85
20.57
25.07
SUIGE berzi saperciad)
18.18
27.85
not provide the corresponding large-scale KG and most of responses do not require the knowledge. Therefore, we only measure the automatic evaluation to evaluate the performance of each method on KOMODIS dataset. In Table we present the experimental results the KOMODIS dataset. Results obviously show that our SURGE framework shows superior performance against baselines on the additional dataset. Therefore, we can conclude that our method can generalize other datasets beyond the opendialKG dataset.
E.4 DIVERSITY EVALUATION
In the main paper, evaluate model generation performance primarily on its quality. We measure the distinct metric (Li et al., 2016), which is one of the most popular metrics for evaluating the diversity of the generative model, to evaluate the performance of each model in more diverse aspects. In Figure 9 left, we report the performance of baselines and our models in distinct metric. Our SURGE framework generates more diverse responses than all other baselines, according to the results.
E.5 ABLATIONS STUDIES ON GNN DESIGN CHOICES
We use two different types of Graph Neural Networks (GNN) in our SURGE framework. One is the Graph Convolutional Network (GCN) (Kipf & Welling, 2017), which is used to embed each node entity the entire 1-hop subgraph in the triplet embedding function d of the main paper Equation 4. Another is Composition- Based Multi-Relation (CompGCN) (Vashishth et al., 2020), which is used embed each entity by considering the relations between entities in the token embedding perturbation function 3 of the main paper Equation 7. In this subsection,
Graph Convolutional Networks
conduct ablation studies both GNN design choices. First of all, we replace the GCN in Equation 4 with Graph Attention Network (GAT) (Velickovic et al., 2018) to validate the effect of the GNN design choices on the node embedding in the triplet embedding function. Then, we run experiments by changing CompGCN in Equation 7 to GCN to see how important the relationships are in the graph encoding. We present the results on Figure right. Results indicate that the use of GAT in Equation does not have any impact the performance a lot. However, the use of relation-aware GNN is highly important in effective and efficient graph encoding, since removing the relation awareness of GNN reduces the performance of our model a lot.
F HUMAN EVALUATION
In this section, we describe the details of human evaluation used in section 5 of the main paper. We request the annotators evaluate the responses generated from two baselines (i.e., ALL Knowledge and Space Efficient) and our SURGE framework in response to the given dialogue context, according to three criteria consistency, informativeness, and fluency. Figure 10 the instructions provided to each annotator. Specifically, regarding the consistency metric, we ask annotators to check whether the generated response makes sense in the context of the conversation. For informativeness. we ask annotators to check whether the response contains correct and enough information, whereby experiment participants are recommended to use the internet search, to check whether the response contains correct facts. In addition to this, also provide the tallouue-related facts from Freebase as a reference for fact checking for annotators. For fluency, we ask annotators to check whether the response is grammatically correct and naturally sound.
24

---

Under review as a conference paper at ICLR 2023
Dialogue Evaluation A (1 out of 15)
Given d'alouu resporis on the right
agenti),
Please rele each acale from for each criterion joonsistency. intormativeeesa, fluency).
Required
Please keep mind these criteria scoring piic the quality cuch reaper so the noutrul sparingly
-ondetencyy examplc)
David Guerts? Good Reeponae lwould les recommend titmnum
Contect:
thdd Reeponeer liee Devid Guerta
Info mat venees: Doss the response contaln corne and enough information? We recommenc you use the inteme search is hethee the respones containe correct facte. examplee)
Contect Eo you know anything about the actor Adom Brown?
Good Respo see Adom Brown star red in the movie The Hobeih An Unexpecceed. Loumey tid Beepeneet tho intormantnt ont: don't know
Respp ferong
Brown starred sing Nong.
example)
Good Respo
Bed Reaperoa: ca thane millarrder dhaidder midfielder micfielcar midhialde::
Human Evaluation Instructions. To measure the qualitative performances of the generated responses, annotators are provided with the following instruction on three criteria consistency. informativeness. and fluency.
Figure 10:
G RETRIEVAL AND GENERATION EXAMPLES
In this section, we provide the examples for knowledge retrieval and response generation, for the given dialogue history.
Embeeding Space Visualization In Figure 11, we present a larger version of Figure in the main paper. Specifically, embed the hidden representations before the projection layer for each graph (star) and the embedding of the generated text (circle) through the dimensionality reduction using t-SNE (van der Maaten & Hinton, 2008). As mentioned in the main paper, the visualization highlights that our SURGE framework with graph-text contrastive learning generates more distinct responses to different subgraphs, unlike the without graph-text contrastive learning which shows less variety over responses even with different graphs.
Retrieval Examples We provide the retrieval examples of various models, such as random retrieval, sparse retrieval and our SURGE models. In particular, in the first (top) example of Figure 12, we are given a dialogue context in regard to books for Richard Maxwell, and baselines including random and BM25 retrievers select the facts associated the entity Richard Maxwell, which are but irrelevant to the ongoing conversion, for example, (Richard maxwell, is-a Theatre director), Also, as shown the second (bottom) example of Figure 12, observe that the simple term-based matching model (i.e., BM25) cannot contextualize the current and previous dialogues, but retrieves the facts associated to frequent words, for example, song, which are less meaningful for the user's question. In contrast baselines, as our SURGE framework trains retriever in an end-to-end fashion, it first contextualizes the given dialogue context, and then accurately retrieves relevant knowledge.
Generation Examples We provide the generation examples from our model. To be specific, we provide the dialogue context along with its corresponding retrieved subgraph and generated response obtained from our SURGE framework. In Figure 13 and Figure 14, provide the correct examples: our model retrieves a context-relevan subgraph, but also generates a factual response from retrieved knowledge. On the other hand, in Figure 15, provide the failure cases. In particular, shown in the first row Figure 15, the fact in the knowledge graph could be ambiguous or inaccurate, as defines the release year of the book Wicked as both 2008 and 2014. Moreover, further provide
25

---

Under review as a conference paper at ICLR 2023
Contrastive X
Contrastive O
Large version of Figure in the main paper. Stars indicate the embedding of graph and circles indicate the embedding decoder hidden states (text), respectively.
Figure 11:
the failure example on retrieval in the second row of Figure 15, where the user asks about the Bourne Legacy, while the dialogue agents retrieve the irrelevant knowledge to the question. Finally, show the common problem in PLMs in the last row Figure 15, where the generative model repeats the meaningless words at the end, while the retriever correctly selects the relevant knowledge.
26

---

Under review as a conference paper at ICLR 2023
Dialogue Context
Could you recommend any books written by Richard Maxwell?
Gold Knowledge
Richard maxwell, wwritten by, a tale of two cities
Random Knowledge
Richard maxwell, sibling, jan maxwell
Screnwriter, ais-a, Richard maxwell
Theatre director, a, Richard maxwell
BM25 Knowledge
Richard maxwell, is-a, Theatre director
Screenwriter ais-a, Richard maxwell
Richard maxwell, organization founded, new york city players
Our Knowledge
Richard maxwell, -written by, a tale of two cities
Richard maxwell, sibling, Jan maxwell
Dialogue Context
I like A dam Levine.
B: OMG me too! love that song Moves Like
Like Jagger: A: Yes, Love that too. is really fun. Can you tell me more.
B: Did you know it's considered power pop song?
A: No, did'n. Do you know Love the way you Lie?
Gold Knowledge
Song, -kind of composition, Love the way you lie Love the way you lie, composer, Eminem
Random Knowledge ledge
Blue monday, kind of composition, Song The look of love, kind of composition, Song Bad romance, kind of composition, Song
BM25 Knowledge
Song, ~kind of composition, This charming man Behati prinsloo, ~spouse (or domestic partner), dam levine Song, kknd of composition, safe & sound
Our Knowledge
g minor, -key, Love the way you lie Eminem, -composer, Love the way you lie Skylar grey, -composer, Love the way you lie
dialogue history with its corresponding gold knowledge as well as retrieved knowledge from random retrieval and sparse retrieval baselines and from our SURGE framework. The retrieved fact is represented as the format of (head, relation, tail), where --symbol in the front of relation (i.e., ~relation) the retrieved knowledge denotes the inverse relation.
Figure 12:
Examples
27

---

Under review as a conference paper at ICLR 2023
Dialogue Context
A: Do you like Rob Marshall? My friends have recommend his films.
Retrieved Knowledge
Nine, has has_ s__geee,, Drama
Pirates of Caribbean: On Stranger Tides, directed by, Rob Marshall
Memoirs Geisha, directed by. Rob Marshall
Generated Response
B: Yes, he wrote Pirates of the Caribbean: On Stranger Tides, Memoirs of a Geisha Nine and many others. Have you either those?
Dialogue Context
Are there any works by Chris Cleave, which you can recommend me?
B: Certainly, wrote The Other Hand, Incendiary, and Old. Have you read them?
No, haven't read The Other Hand of the ones you mentioned. What genre is it?
Retrieved Knowledge
The Other Hand, release_yeet yeare 2008 2008, rrelease rrelease_year year, Other Hand
The Other Hand, has genre, Literary fiction
Generated Response
B: It's Literary fiction, and was released in 2008.
Dialogue Context
A: Who wrote Of Mice and Men?
It was written by John Steinbeck
A: Didn't he also write The Red Pony?
Yes He also wrote The Red Pony
A: Can you suggest other titles by him to read?
Retrieved Knowledge
Cannery Row, written by, John Steinbeck
The Grapes of Wrath, written by, John Steinbeck
Tortilla Flat, written by, John Steinbeck
Generated Response
B: Yes also wrote Tortilla Flat
Figure 13: Examples the dialogue history with its corresponding retrieved knowledge and generated response
from our SURGE framework. The fact is represented as the format of the front of relation (i.e., rerlltioon in the retrieved knowledge denotes the inverse relation. In this example, we only provide the correct cases of both retrieval and generation.
(head, relation, tail), where ~symbol in
28

---

Under review as a conference paper at ICLR 2023
Dialogue Context
Could you recommend some movies by director Simon Wood?
Retrieved Knowledge
The One That Got Away, written by, Simon Wood
Simon Wood, -written by, The One That Got Away
Author, -is-a, Simon Wood
Generated Response
Simon Wood directed The One That Got Away. Have you seen that?
Dialogue Context
I like David McCullough. Could you recommend any books of him?
B: Sure, He wrote The Greater Joumey: Americans In Paris. Also, he wrote some documentary and Indie films.
Thank you for the information. When was The Greater Journy: Americans released?
Retrieved Knowledge
The Greater Journey: Americans In Paris, release_ yearee_erra__e 2011
2011, rreleaee year, The Greater Journey: Americans In Paris Paris, written by, Edward Rutherfurd
Generated Response
B: It was released in 2011. It's great book.
Dialogue Context
A: I like Tony Tanner, what do you think?
Retrieved Knowledge
Sense Sensibility, written by, Tony Tanner
Tony Tanner, -written by, Mansfield Park
Mansfield Park, written by, Tony Tanner
Generated Response
B: think he is a great actor. He starred in Sense and Sensibility and Mansfield Park.
Figure 14: Examples the dialogue history with its corresponding retrieved knowledge and generated response from our SURGE framework. The fact is represented as the format of the front of relation (i.e., rerlltioon in the retrieved knowledge denotes the inverse relation. In this example, we only provide the correct cases of both retrieval and generation.
(head, relation, tail), where ~symbol in
29

---

Under review as a conference paper at ICLR 2023
Dialogue Context
A: Do you know what books Jennifer L. Armentrout has written?
B: She wrote Wicked, White House Kiss and Wait For You. Are you interested in her other
books as well?
A: No thanks. What genre is Wicked?
Wicked is Play that the Pulitzer Prize for Drama. Would you like information
about it?
Sure, sounds interesting
Retrieved Knowledge
Wicked, releasc_ year, 2008
2008, latlcrrec_cecaaa Wicked
Wicked, release_ years 2014
Generated Response
B: It is play written by Jennifer L. Armentrout and released in 2008.
Dialogue Context
A: I like the movie Armageddon.
Did you know Armageddon is written by Tony Gilroy?
A: Oh, didn't also write The Bourne Legacy?
Retrieved Knowledge
Michael Clayton, starred actors, Tony Gilroy
Michael Clayton, written by, Tony Gilroy
Proof of Life, written by, Tony Gilroy
Generated Response
Yes, he did. He also wrote Michael Clayton.
Dialogue Context
like Deadpool or any movies like Deadpool.
Retrieved Knowledge
Deadpool, starred actors, Ryan Reynolds
Deadpool, starred actors, Stefan Kapicic
Deadpool, starred actors, Ed Skrein
Generated Response
Deadpool is great movie. Stefan Kapicic starred in it. Stefan Kapicic also starred in The Last Man and The Last Man.
Figure 15: Examples of the dialogue history with its corresponding retrieved knowledge and generated response from SURGE framework. The fact represented the format of (head, relation, tail), where wsymbol in the front of relation (i.e., erelation) the retrieved knowledge denotes inverse relation. In this example, we only provide the failure cases due the problem on data (first row), retrieval (second row), and generation (third row).
30


Reviews:
[{"review_id": "wcAgQsOilM8", "reviewer": "Reviewer_6h9N", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes subgraph retrieval-augmented generation (SURGE), a framework for knowledge-grounded dialogue. It can be divided into 3 major components:\n(i) a subgraph-retriever, which retrieves only  context-relevant triples from a KG,\n(ii) a permutation and inversion invariant graph encoding scheme, and\n(iii) a contrastive learning objective to enforce consistency between the retrieved triples and generated text\n\nThe authors also propose a new metric called knowledge-verified question-answering (KQA) to evaluate the generated responses. They evaluate the method on OpendialKG and KOMODIS. Both see fair improvements over the baselines on KQA and minimal improvements on lexical overlap metrics like bleu and rouge. \n\nThe paper is well-written and understandable. However,\n(i) some space could be saved in section 3.4 and used for ablations and analysis.\n(ii) I would have liked to see more information about sensitivity analysis on graph encodings and a discussion on why the authors think retrieving proper subgraphs is easier/more important than say, teaching the decoder to look for/ignore relevant/irrelevant knowledge conditioned on the history. Some toy experiments would have cemented the premise of the paper.\n(iii) contrastive training does not give any significant benefit. Why is that? One would intuitively think otherwise. Some analysis there would be appreciated.\n\nThere are minor writing errors. E.g.: in section 5.2, the last line should be SURGE (contrastive)", "summary_of_the_review": "-", "strength_and_weaknesses": "-", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "T21bZj-oCE5", "reviewer": "Reviewer_ypx1", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper provides an approach to perform knowledge grounded dialogue generation. Their GNN based approach retrieves a relevant subgraph from the KG to efficiently generate an appropriate response. They also propose a permutation and relation inversion invarient yet efficient way to encode the graph information along with the text using a contrastive learning objective that ensures that the generated responses reflect the retrieved knowledge. Finally, they propose a novel metric called Knowlege-verifying Question Answering to evaluate whether the generated responses contain the correct knowledge.", "summary_of_the_review": "This paper provides a novel approach to filter knowledge from KGs efficiently and make dialogue responses follow the retrieved knowledge.  This is a well written paper with adequately supported claims using extensive experiments and analysis. The metric proposed in this work can be used by the community for evaluating knowledge grounded generation better. The strengths far outweigh the weaknesses, which are mostly to do with clarifications. ", "strength_and_weaknesses": "Strengths:\n* This paper is really well written with extensive experiments and analysis to support the claims in the paper.\n* Their approach and proposed metric can be a valuable contribution to the community in the space of knowledge grounded generation especially since it is more efficient than previous approaches using KGs.\n*  They talked about their design choices and assumptions in detail.\n\nWeakness:\n* Some of the parts in this paper could be better explained, especially the new proposed metric in section 4. It was unclear what this metric is exactly. \n* This paper lacks comparison with approaches that use commonsense KGs in a similar way.\n* Using subgraphs is not entirely novel. (Wenhao Yu, Chenguang Zhu, Zaitang Li, Zhiting Hu, Qingyun Wang, Heng Ji, and Meng Jiang. 2022. A Survey of Knowledge-Enhanced Text Generation. ACM Comput. Surv. Just Accepted (March 2022). https://doi.org/10.1145/3512467 talk about it in much more detail), though the way the authors find a subgraph might be novel.\n* The mapping function is unclear. Why did you split New York into New and York? Why not just use the base model's tokenizer? If you did use the tokenizer, then instead of calling this a \"mapping function\" you can simply refer it as \"the entity name was directly passed to the PLM to get its encoding\". It is also unclear how you did entity linking to get the KG entities from the context. Do you do exact match? \n* What is a) in Figure 2 caption?\n* From reading section 3.3 It is not clear if you're using all triplets present in the KG at each step to filter and obtain a subgraph. After reading the appendix (limitations) as well as line 2 in last para of 3.3, it's clear that you're using only the k hop neighborhood of the entities present in the context. Again, how do you do the linking? Also, wont you miss out on potential entities that are not exactly mentioned in the context? Did you consider also using relations? (eg: (Barack Obama, president of, Usa) is present in your KG but the context is \"Who is the top-person of the american armed forces\" or something, then you'll miss out on Barack Obama).\n  * In the limitations, you do discuss limitations on entity linking but have not provided how you are doing the same (I might have missed it).\n* Please describe the relation embedding matrix in the main paper section 3.3.\n* Human evaluation using just 30 samples might not be sufficient for statistical significance. If possible, please try with 100 samples. In any case, please report statistical significance.\n* The metric is only slightly correlated with human judgement and more evaluation of the metric is needed before it can be widely used. \n\n\nQuestions:\n* How did you arrive at the 87% figure for facts that are irrelevant to the context in OpendialKG dataset (2nd para 4th line in Intro)?\n* Why did you choose zero vector for initialization of node embeddings?\n* What is EM that is reported in Table 1? ( I think you can fix this by describing the proposed metric a bit more).\n* What is F1 vs KF1 in Table 3?\n\n\nGrammar and nitpicks:\n* The related works section lacks some discussion on works which use GNNs in different ways in dialogue systems. Some quick google search examples at the end.\n* Figure 2 caption last line - \"finally we use ~a~ contrastive learning to enforce\".\n\nSuggestions:\n* Update the hyperlinks to Figures, and tables to include link in the \"Figure\" or \"Table\" keyword too instead of just linking the number.\n* Please add more details in section 4. \n\nSome other works that use GNNs in dialogue (from a quick google search) (I'm not asking you to cite them just that it would be nice to talk about other dialogue works that use GNNs) :\n1) - Chen, L., Lv, B., Wang, C., Zhu, S., Tan, B., & Yu, K. (2020). Schema-Guided Multi-Domain Dialogue State Tracking with Graph Attention Neural Networks. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05), 7521-7528.\n2) Mauajama Firdaus, Nidhi Thakur, and Asif Ekbal. 2020. MultiDM-GCN: Aspect-guided Response Generation in Multi-domain Multi-modal Dialogue System using Graph Convolutional Network. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2318–2328, Online. Association for Computational Linguistics.\n3) W. Nie, R. Chang, M. Ren, Y. Su and A. Liu, \"I-GCN: Incremental Graph Convolution Network for Conversation Emotion Detection,\" in IEEE Transactions on Multimedia, doi: 10.1109/TMM.2021.3118881.\n4) X. Zhao, L. Chen and H. Chen, \"A Weighted Heterogeneous Graph-Based Dialog System,\" in IEEE Transactions on Neural Networks and Learning Systems, doi: 10.1109/TNNLS.2021.3124640.\n5) L. Qin, W. Che, M. Ni, Y. Li and T. Liu, \"Knowing Where to Leverage: Context-Aware Graph Convolutional Network With an Adaptive Fusion Layer for Contextual Spoken Language Understanding,\" in IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 1280-1289, 2021, doi: 10.1109/TASLP.2021.3053400.\n6) Rishabh Joshi, Vidhisha Balachandran, Shikhar Vashishth, Alan Black, and Yulia Tsvetkov. 2021. Dialograph: Incorporating interpretable strategygraph networks into negotiation dialogues. In International Conference on Learning Representations.\n7) [AttnIO: Knowledge Graph Exploration with In-and-Out Attention Flow for Knowledge-Grounded Dialogue](https://aclanthology.org/2020.emnlp-main.280) (Jung et al., EMNLP 2020)\n8) Zhou, Li and Kevin Small. “Multi-domain Dialogue State Tracking as Dynamic Knowledge Graph Enhanced Question Answering.” ArXiv abs/1911.06192 (2019): n. pag.\n9) C. -M. Wong et al., \"Improving Conversational Recommender System by Pretraining Billion-scale Knowledge Graph,\" 2021 IEEE 37th International Conference on Data Engineering (ICDE), 2021, pp. 2607-2612, doi: 10.1109/ICDE51399.2021.00291.\n10) Liu, Z., Niu, Z., Wu, H., & Wang, H. (2019). Knowledge Aware Conversation Generation with Explainable Reasoning over Augmented Graphs. EMNLP.\n11) Jing Li, Qingbao Huang, Yi Cai, Yongkang Liu, Mingyi Fu, Qing Li, Topic-level knowledge sub-graphs for multi-turn dialogue generation, Knowledge-Based Systems, Volume 234, 2021,\n12) Chen, Yu, Lingfei Wu, and Mohammed J. Zaki. \"Toward Subgraph Guided Knowledge Graph Question Generation with Graph Neural Networks.\" arXiv preprint arXiv:2004.06015 (2020).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "N_5tlJt__7", "reviewer": "Reviewer_YCpx", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes an approach to knowledge-grounded dialogue. The key problems the paper intends to tackle is the selection and encoding of relevant knowledge according to the dialogue context. To this end, subgraphs are selected using the context. Subgraph encoding is asked to meet some conditions: permutation invariance and relation inversion invariance. Two tricks are used to satisfy these conditions: by sorting the triples, and by adding the inverse relation.\nThe proposed method is compared to a set of baselines, in which either random knowledge selection, or a selection using T5 is used. The experiments show that the proposed method outperforms the baselines.\nIn addition, a knowledge verifying metric - KF1 - is used to measure if the generated response is consistent with the known knowledge.\nThe overall ideas of locating relevant subgraphs and doing subgraph encoding are interesting.", "summary_of_the_review": "The idea is very well motivated. The paper proposes some way to encode the retrieved subgraphs so that the pieces of knowledge can be better encoded.\nHowever, there are questions about the way the subgraphs are encoded. It may lead to undesirable cases.", "strength_and_weaknesses": "The idea of selecting relevant subgraphs from a knowledge graph is well motivated. This is a known problem. There are approaches proposed to do it. This paper also relies on an existing tool to identify the entities in the knowledge graph and to select subgraphs.\nThe two conditions for subgraph encoding are important in some cases. This paper proposes interesting ideas to satisfy them.\nHowever, the tricks used to satisfy these conditions may raise some questions:\n1. The permutation invariance condition is satisfied by sorting the triples. This is a kind of normalization, that transforms a subgraph to a normalized form. Even though the output will be permutation invariant, it does not require that the encoding process after the transformation be permutation invariant. The latter is what one intend to obtain by imposing the condition. So, it is not sure that making this transformation would make the encoding better.\n2. The encoding process then considers a set of triples as a sequence of tokens. that is submitted to a PLM for encoding. In this step, one may lose the specific triple structure for relations. Interpreting a set of triples as a sentence makes the encoding easier, but may alter the meaning.\n3. In the approach to increase efficiency, it seems that the entities from different triples are mixed up, then sorted. The purpose is to encode an entity only once. However, this process would alter the meaning of the initial subgraph. For example, the initial set of triples [(a,r1,b), (c,r2,d)] would become the same as [(a,r2,c),(d,r1,b)] after this operation. I understand that this operation would transform both into a set of entities like [a,b,c,d,r1,r2] (this part is not very clear). However, the two different sets of triples may mean very different things and we may not want to unify them.\n4. The relation inversion invariance is imposed to all the repations. If the relation is kept the same after inversion, this does not make sense. It seems that the inversed relation is different:  ~relation. In this case, I do not see what new information this inversion could add. For example, saying that (a, born-in, b) would be the same as saying (b,  ~born-in, a). What's the benefit of adding the latter?\n\nIn the experiments, it seems that the way subgraphs are retrieved is quite simple. There could be more sophisticated methods in the literature. For example [1] proposes a knowledge selection network to make the selection. Given this fact, it is unsure that the method is compared to the state-of-the-art knowledge selection approaches.\n\n[1] Yutao Zhu, Jian-Yun Nie, Kun Zhou, Pan Du, Hao Jiang, and Zhicheng Dou, Proactive Retrieval-based Chatbots based on Relevant Knowledge and Goals, SIGIR 2021.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "KyRNk2RO0d4", "reviewer": "Reviewer_p6ze", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes SURGE, a GNN-based context-relevant subgraph retrieval method for KG-augmented dialogue generation. The proposed method considers the permutation invariance and relation inversion invariation, and uses graph-text contrastive learning to generate knowledge-consistent response.", "summary_of_the_review": "With my listed weakness, I would suggest considering more about the line of knowledge graph attention and reasoning works into motivation, related work, and experiments (some works are listed above), or adjusting the scope of the current claim.", "strength_and_weaknesses": "### Strengths\n* The paper is overall written clearly.\n* The proposed method is a thoughtful combination of important details such as the invariant graph encoding and graph-text contrastive learning, which convincingly can help the training.\n* The proposed evaluation metric KQA could be useful.\n\n### Weaknesses\n* The paper misses references to prior works on dialogue generation priorly or jointly extracts subgraphs, which to my understanding are closely related to this work, such as:\n  * Jung, Jaehun, Bokyung Son, and Sungwon Lyu. \"Attnio: Knowledge graph exploration with in-and-out attention flow for knowledge-grounded dialogue.\" In EMNLP 2020. \n  * Zhou, Hao, Tom Young, Minlie Huang, Haizhou Zhao, Jingfang Xu, and Xiaoyan Zhu. \"Commonsense knowledge aware conversation generation with graph attention.\" In IJCAI 2018.\n  * Tuan, Yi-Lin, Sajjad Beygi, Maryam Fazel-Zarandi, Qiaozi Gao, Alessandra Cervone, and William Yang Wang. \"Towards Large-Scale Interpretable Knowledge Graph Reasoning for Dialogue Systems.\" In Findings of ACL 2022.\n  * Yang, Shiquan, Rui Zhang, and Sarah Erfani. \"GraphDialog: Integrating Graph Knowledge into End-to-End Task-Oriented Dialogue Systems.\" In EMNLP 2020.\n* Since the proposed method is a combination of subgraph retrieval and dialogue generation, I would like to see comparisons to prior works that also do dialogue generation attending to parts of a graph, e.g., the methods in above listed references.\n* About the main motivation across the paper, I have a few concerns:\n  * The title is too general according to the scope of this paper. If the main novelty of the proposed method is the subgraph-retrieval augmented method for dialogue generation, I would suggest the title to be more specific, e.g., mentioning “subgraph-retrieval”.\n  * About the second paragraph in the introduction, it may not always be true in real applications that only a few hops in a KG can compose a good response. I would suggest revising the paragraph to be more general instead of motivating by one specific dataset that was collected for research purpose and only contained few cases in the real world.\n  * The paper claims to jointly retrieve context-relevant subgraph, however, in the experiments, it seems that the “subgraph-retrieval” is based on a manually selected subgraph candidates beforehand i.e., only considering 1-hop in OpenDialKG and 2-hop in KOMODIS. Am I misunderstanding this?\n  * Following question c, if the proposed method is based on pre-selected candidates, many prior works already consider such cases (as listed in question (a)). How general is this method when applying to cases with a variant number of hops? How would you analyze its scalability and complexity?\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "KX14KB8klXt", "reviewer": "Reviewer_KmrR", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "What is the goal of the paper?\n\n* Knowledge-consistent dialogue generation with the KG\n\nWhat has been done before?\n\n* Many existing methods do not guarantee that the model utilizes a relevant piece of knowledge from the KG before generating knowledge-consistent dialogues. The ones which do simply match and retrieve all facts for entities that appear in the dialogue context, which either may mislead the agent to generate out-of-context responses from irrelevant facts or can increase the computational overheads for prepending tokens for all facts in PLMs. This work differs from those existing works, since it aims at retrieving only a context-relevant subgraph among all associated facts with a novel GNN-based subgraph retriever, which is end-to-end trainable along with a dialogue generation model.\n\nWhat are the contributions of the paper?\n\n* Propose SUbgraph Retrieval-augmented GEneration (SURGE), a end-to-end dialogue generation framework for generating context-relevant and knowledge-consistent dialogues with a KG that considers all aspects from knowledge retrieval, encoding, and reflection along the generation process.\n\n* Introduced an additional performance metric, referred to as Knowledge-verifying Question Answering (KQA), which evaluates whether the generated responses contain the correct knowledge with an additional extractive question answering scheme.\n\n* Proposed a GNN-based context-relevant subgraph retrieval method for KG-augmented dialogue generation, to extract only the relevant piece of the knowledge for the dialogue context from the entire knowledge graph. \n\n* Proposed an invariant yet efficient graph encoder and a graph-text contrastive learning objective to ensure that the generated responses faithfully reflect the retrieved knowledge. \n\n* Evaluated SURGE against relevant baselines on the OpendialKG and KOMODIS datasets , demonstrating its efficacy in generating responses that are more informative by retrieving and reflecting the relevant knowledge from the KG.\n\n\n\n\n", "summary_of_the_review": "Paper has many strengths with scope of improvements in writing. \n\n\nStrengths\n\n* Empirically showed the importance of retrieving the more relevant subgraph knowledge rather than using all the relevant knowledge graphs when generating knowledge-grounded response\n\n* Proposed multiple components for retrieval, encoding, and graph-text representation learning\n\n* Better results than baselines on OpendialKG and KOMODIS datasets\n\n* Source code is available for reproducibility\n\n\nWeaknesses\n\n* KQA correlation with human ratings is small\n\n* Is human evaluation performed on only 30 examples?\n", "strength_and_weaknesses": "\nStrengths\n\n* Empirically showed the importance of retrieving the more relevant subgraph knowledge rather than using all the relevant knowledge graphs when generating knowledge-grounded response\n\n* Proposed multiple components for retrieval, encoding, and graph-text representation learning\n\n* Better results than baselines on OpendialKG and KOMODIS datasets\n\n* Source code is available for reproducibility\n\n\nWeaknesses\n\n* KQA correlation with human ratings is small\n\n* Is human evaluation performed on only 30 examples?\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "-URrrzN6ha-", "reviewer": "Reviewer_CSao", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Existing knowledge-grounded dialogue generation models with KG suffer from irrelevant knowledge and inconsistent generation. To attack the above two problems, this paper proposes a GNN-based context-relevant model to retrieve relevant subgraphs from KG, an invariant yet efficient graph encoder, and a graph-text contrastive learning objective which overcome the exposure bias in teacher forcing to guarantee knowledge-consistent generation. Extensive experiments have been conducted on OpendialKG and KOMODIS and the results show the superiority of SURGE over baselines.", "summary_of_the_review": "see above.", "strength_and_weaknesses": "**Strengths**\n\nThe paper is generally well-written. \n\nThe study of generating context-relevant and knowledge-consistent dialogues with a KG is important and interesting. I appreciate the authors' investigation of this important problem. \n\nThe three components (the GNN-based context-relevant subgraph retrieval method, the graph encoder, and a graph-text contrastive learning objective) seem to be new. Experimental results show the empirical advantage of the new methods.\n\n**Weaknesses**\n\nThe overall framework is not new, very similar to (Lewis et al., 2020b). In fact, the authors should make this point more explicit.\n\nAlthough the three components are somewhat new, neither is of high originality, and neither is throughly investigated (see more comments below). This paper seems to belong to a kind of assembly innovation: putting several pieces together, while each piece brings a small adjust. I suggest the authors to focus on major novelty and fully validate the novelty.\n\nMore ablation studies are needed to show some significant benefit of each component, by fixing other components while modifying one component. Now the significance of each component is not clear. It is not good to simply say that \"We empirically verify that the use of GNN as the triplet embedding yields the better retrieval performance compared to previous PLM-based methods.\"\n\nIn Table 1 and Table 3, the advantages of SURGE (contrastive) are not significant, compared with SURGE (unsupervised) and SURGE (semi-supervised), especially considering SURGE (contrastive) also consists of semi-supervised learning.\n\nIn Table 1, SURGE (unsupervised) outperforms the other two SURGE performance on BLEU, ROUGE and Unigram, is there any explanation for that?\n\nUsing only unique entities does not guarantee permutation invariant.\n\nSome problems with writing:\n\nSection 3.4 (INVARIANT GRAPH ENCODING) is hard to be understood, even after reading the Appendix.\n\nThe two examples in Figure 1 illustrate the two problems - irrelevant knowledge and inconsistent generation. Somewhat confusing. \"Irrelevant facts\" and \"factually wrong\" actually may express almost the same meanings.\n\nmaaping -> mapping\n\nduplicate definition of SURGE (semi-supervised) on the bottom of page 7\n\nSome places should be re-written to be clearer in both grammar and meaning:\n\nwe utilize the Graph Neural Networks (GNNs) in the triplet embedding function \n-> we utilize the Graph Neural Networks (GNNs) for the triplet embedding function ?\n\nThe 2nd paragraph, Section 3.1: Any entity and relation are mapped to a sequence of l tokens. Is the length l fixed for different entities and relations?\n\n--- **To reply to Follow-up Response from the authors (Provided here, since the comment button is disabled 2022/12/12)** ---\n\nUsing $\\eta$ alone should be an important baseline to be compared.\n\nIf you think that Eq.(7) is the first in developing GNN-based entity embedding in the literature (not necessarily for dialog generation), then you can clearly say so in the paper. Suppose this is yes, then the contribution should be stated around this, rather than putting more emphasis on \"Invariant and Efficient Graph Encoding\". To my understanding, \"Invariant and Efficient\" are actually down to only encoding the unique entities, which essentially is entity embedding. Clarity issues remain.\n\nI can see the ablation result in Table 3. My concern is that the overall performance of SURGE (contrastive) is limited. In real applications, the model needs to work with retrieved knowledge.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject