Sequential Latent Knowledge Selection for Knowledge-Grounded Dialogue
Authors: Byeongchang Kim
Jaewoo Ahn
Gunhee Kim
Venue:  (2020)
Keywords: dialogue
knowledge
language
conversation

Abstract: Knowledge-grounded dialogue is a task of generating an informative response based on both discourse context and external knowledge. As we focus on better modeling the knowledge selection in the multi-turn knowledge-grounded dialogue, we propose a sequential latent variable model as the first approach to this matter. The model named sequential knowledge transformer (SKT) can keep track of the prior and posterior distribution over knowledge; as a result, it can not only reduce the ambiguity caused from the diversity in knowledge selection of conversation but also better leverage the response information for proper choice of knowledge. Our experimental results show that the proposed model improves the knowledge selection accuracy and subsequently the performance of utterance generation. We achieve the new state-of-the-art performance on Wizard of Wikipedia (Dinan et al., 2019) as one of the most large-scale and challenging benchmarks. We further validate the effectiveness of our model over existing conversation methods in another knowledge-based dialogue Holl-E dataset (Moghe et al., 2018).

Published as a conference paper at ICLR 2020
SEQUENTIAL LATENT KNOWLEDGE SELECTION FOR KNOWLEDGE-GROUNDED DIALOGUE
Byeongchang Kim
Jaewoo Ahn
Gunhee Kim
Department of Computer Science and Engineering
Scoul National University, Seoul, Korea
fbyecngchang.ur k jae snu.ac. kr gunhee@snu. ac.kr
http://vision snu ac. www.wss.......ccc...cooo http:///c...ooaa..ee.c////ooo///
ABSTRACT
Knowledge-grounded dialogue is task of generating informative response based on both discourse context and external knowledge. we focus on bet- ter modeling the knowledge selection in the multi-turn alogue, we propose a sequential latent variable model as the first approach to this matter. The model named sequential knowledge transformer (SKT) can keep track of the prior and posterior distribution over knowledge; as a result, it can not only reduce the ambiguity caused from the diversity in knowledge selec- tion of conversation but also better leverage the response information for proper choice of knowledge. Our experimental results show that the proposed model improves the knowledge selection accuracy and subsequently the performance of utterance generation. We achieve the new state-of-ther thrtthe-rtt performance on Wizard of Wikipedia (Dinan et al. (019)) as one of the most large-scale and challenging benchmarks. We further validate the effectiveness of our model over existing con- versation methods in another knowledge-basee dialogue Holl-E dataset (Moghe) et al. 2018).
di-
INTRODUCTION
Knowledge-grouut dialogue task of generating an informative response based on both dis- course context and selected external knowledge (Ghazvininejad et al. (2018). For example. it is more descriptive and engaging to respond "I've always been more of a fan of the American football team from Pittsburgh, the Steelers!" than "Nice, I like football too." (Dinan & Weston 2019). As it has been one of the key milestone tasks in conversational research (Zhang et al., 2018)) a majority of previous works have studied how to effectively combine given knowledge and dialogue context to generate an utterance (Zhang et al. 2018: Li et al. 20096) Parthasarathi Pineau 2018 Madotto al. 2018: Gopalakrishhan gellggggggg dialogue by decomposing into two sub-problems: first selecting knowledge from large pool of candidates and generating response based on the selected knowledge and context.
al.
2019) Recently, Dinan et al. (2019) proposed to tackle the
In this work, investigate the issue of knowledge selection in the multi-turn knowledge- -goounddgg dialogue, since practically the selection of pertinent topics is critical to better engage humans in conversation, and technically the utterance generation becomes easier with more powerful and consistent knowledge selector in the system. Especially, we focus on developing a sequential latent variable model for knowledge selection, which has not been discussed in previous research. We believe it brings several advantages for more engaging and accurate knowledge-baad based chit-chat. First, it can correctly deal with the diversity in knowledge selection of conversation. Since one can choose any knowledge to carry on the conversation, there can be one-to-many relations between dialogue context and knowledge selection. Such multimodality by nature makes the training a dialogue system much more difficult in data-driven way. However, if we can sequentially model the history of knowledge selection in previous turns, we can reduce the scope of probable knowledge candidates current turn. Second, the sequential latent model can better leverage the response information, which makes knowledge selection even more accurate. is naturally easy to select the knowledge in the pool once the response is known, because the response is generated based on the

---

Published as a conference paper at ICLR 2020
Table 1: Accuracy of knowledge selection with and without knowing the response. We test with GRU (Cho al., 2014), Transformer [aswann] et al. 2017) and BERT (Devlin et al. 2019 the sentence encoder. For human evaluation, we randomly sample 20 dialogues and ask human annotators to select the most likely knowledge sentence from the pool.
Well, help make sure people not drown or get injured while near the water!
Appcostor
(1) lifoguard
who supervises the sufety hileguards are stroeg swimmess and trained in la some areas, lifeguards are of the emergency...
2) Despite the considerable amount accivity (t- 1) The season officially sturted on Vay the
(4) These dates conventionally delimix the period
Task : Knowledge Selection
Methods GRU
w/ response
W/ response
I've heard that in some places, lifeguards also
20.0
66.0
help with other sorts of emergencies!
Transformer
22.5
70.4
Task Response Generation
BERT
23.4
78.2
Transformer GT history
25.4
70.4
Figure example of wizard's tasks in knowledge knowledge-grounded grounded conversation of Wizard of Wikipedia (Dnann et al. 2019).
BERT GT history
27.3
79.2
Random
2.7
Human
17.1
83.7
selected knowledge. Our sequential model can keep track of prior and posterior distribution over knowledge, which are sequentially updated considering the responses in previous turns, and thus we can better predict the knowledge by sampling from the posterior. Third, the latent model works when the knowledge selection labels for previous dialogue not available, which is common in practice. For example, multiple people have discussion about given documents, knowledge selection of previous turns is done by others. The latent model infer which knowledge others are likely to select and use.
Finally, the contributions of this work follows.
1. We propose novel model named sequential knowledge transformer (SKT). To the best of our knowledge, our model is the first attempt to leverage a sequential latent variable model for knowledge selection, which subsequently improves knowledge-grounde chit-chat.
2. Our experimental results show that the proposed model improves not only the knowledge selection accuracy but also the performance of utterance generation. As result, we achieve the new state-of-the-ar performance on Wizard of Wikipedia (Dinan et al. 2019) and a knowledge-annodee version of Holl-E (Moghe et al. 2018) dataset.
PROBLEM STATEMENT AND MOTIVATION
As a main testhed of our research, we choose the Wizard of Wikipedia (WoW) benchmark (Di- inan et al.) 2019), since it is one of the most large-scale and challenging datasets for open-domain multi-turn knowledge-ben basedd dialogue. Moreover, the dataset can evaluate the algorithm's 's ability for solving the two subproblems of knowledge selection and response generation. That is, it provides ground-truth labels of knowledge selection and clear grounding between the pairs of selected knowl- edge and response. In our experiments, we also evaluate Holl-E (Moghe et al. 2018) another dataset for knowledge -goouddggg dialogue, after collecting clearer labels of knowledge sentences.
The Flow of Conversation. The WoW (Dinan al.) 2019) deals with chit-chat dialogue task where two speakers discuss depth about a given topic. One speaker (coined as Wizard) is to be both engaging and knowledgeable over Wikipedia to supplement its knowledge. The other speaker (Apprentice) is curious and eager learn about the topic. With example Figure [ the conversation flow takes place as follows.
the topic with access an information retrieval (IR) system
1. One topic chosen among 1,431 topies and shared between the two speakers.
2. Given an apprentice's ' utterance and a wizard's previous utterance, the IR system retrieves relevant knowledge, which includes the first paragraph of top 7 articles each for wizard and apprentice and the first 10 sentences of the original Wikipedia page of the topic (e.g. the lifeguard wikipage). The knowledge pool contains 67.57 sentences average. Then.
2

---

Published as a conference paper at ICLR 2020
the wizard must choose single relevant sentence from them (knowledge selection) and construct an utterance (response generation).
3. The conversation repeats until minimum number of turns (5 each) reaches.
The Motivation of Sequential Latent Models. The goal of the task is to model the wizard that solves the two subproblems of knowledge selection and response generation (Dinan et al. 2019). In the knowledge selection step, a single relevant knowledge sentence is chosen from pool of candi- dates, and in the response generation step, a final utterance is generated with the chosen knowledge and dialogue context. This pipeline is originally proposed to tackle open-domain TexIQA (Che eet al. 2017); for example, Min et al.] (2018) show its effectiveness for single-document TexIQA, to which the key is locate the sentences that contain the information about the answer a question.
For knowledge- wwwleegg-grounded dialogue, however, there can be one-to-many relations between the dia- logue context and the knowledge to be selected unlike Tex Except direct question about con- text, one can choose any diverse knowledge to carry on the conversation. Therefore, the knowledge selection in dialogue diverse (i.e. multimodal) by nature, which should be correctly considered in the model. It is our main motivation propose sequential latent variable model for knowledge selection, which has not been studied yet. The latent variable not only models such diversity of knowledge but also sequentially track the topic flow of knowledge in the multi-tury dialogue.
Another practical advantage of the sequential latent model lies in that it is easy to find which knowl- edge is chosen once the response is known, since the response is written based on the selected knowledge. Table D clearly validates this relation between knowledge and response. In the WoW dataset, knowing response boosts the accuracy of knowledge sentence selection for both human and different models. These results hint that knowledge selection may need to be jointly modeled with response generation in sequence of multi-turn chit-chats, which can be done by the sequential latent models.
APPROACH
We propose a novel model for knowledge-groundee conversation named sequential knowledge transformer (SKT), whose graphical model illustrated in Figure 2 It is a sequential latent model that sequentially conditions on previously selected knowledge to generate a response.
We will use S t < T iterate over dialogue turns, 1 S. m S M and S iterate over words in the utterance of apprentice and wizard, and < 1 S L to denote knowledge sentences in the pool. Thus, T is the dialogue length, M and N are the length of each utterance of apprentice and wizard, and L is the size of the knowledge pool.
N to respectively
The input to our model at turn previous turns of conversation, which consists of utterances from apprentice x ..., x", utterances from wizard where k' (k'd) = k "..,....''. wizard's response y*. Below, we discuss sentence embedding, knowledge selection and utterance decoding in our approach. Note that our technical novelty lies in the knowledge selection model, while exploiting existing techniques for text encoding and utterance decoding.
and the knowledge pool kl k!..., k', The output the model is selected knowledge k' and the
Sentence Encoding. We represent an apprentice utterance x' to an embedding h' using BERT (Devlin et al. 2019) and average pooling over time steps (Cer et al. 2018):
H = BERT base (Iaf; xhrl) E I x768 h = avgpool H
R 768
(1)
Likewise, the utterance of Wizard y²11 embedded as " and knowledge sentences are as (h(') h ...., b.... Each apprentice-wine wizard utterance pair by [b',: ht] dialog turn is jointly represented through a GRU (Cho et al. 2014 layer: d' = dRUUoo (d' hh,,)) 768
Sequential Knowledge Selection. Compared to previous works, we make two significant mod- ifications. First, we regard the knowledge selection as a sequential decision process instead a single-step decision process. Second, due to the diversity of knowledge selection in dialogue, model it as latent variables. As result, we can carry out the joint inference of multi-turns of knowl- edge selection and response generation rather than separate inference turn by turn.
3

---

Published as a conference paper at ICLR 2020
Dialogue Context
x',y
x²
x²
y²
x3
y
Train/eeh
an .y yoges Prior
Test only
Train only
Knowledge Pool
Sumpling
k2
Sampling
k3
(p kel)
Posterior
A graphical representation of the proposed sequential knowledge transformer (SKT) model. At the third turn, the goal is to generate wizard's response (y²) given dialogue context (xS3 3.). Our model sequentially infer which knowledge is likely be used (kS*), from which the utterance yd is generated.
Figure 2:
There have been much research on sequential latent variable models (Chung et al., 2015 Fraccaro et 2016 Goyal et al. 2017 Aneja al. 22019; Shankar & Sarawagi 2019). For example. Shankar & Sarawagi (2019) propose posterior attention model that represents the attention of seq2seq
models as sequential latent variables. Inspired by them, we factorize the response generation with latent knowledge selection and derive the variational lower bound as follows:
log l(yl())x) = log II
Po po y k "TO (k'
(2)
k'
Σ Eng (kt-1) [ (Ke) (ke) flog (x y k*] - DxL || TO(K)))
(3)
where (k') is shorthand for go (o ySt k ket) and (k') for Fe (k pa'kl [xSe yet, brevity. Note that po(y"l-) is decoder network, To(kk) is categorical conditional distribution of knowledge given dialogue context and previously selected knowledge, and qo (k') is an inference network to approximate posterior distribution pa (k') (k"IxS* y k
ket for
The conditional probability of generating wizard's response y' given dialogue context S and st, can be re-written from Eq. as follows:
ply'Ix ~
II (k') 1=1 Ke'
[ kt
Poly' _st,k")To(kk))
(4)
The detailed derivation can be found in Appendix. Eq.44) means that we first infer from the knowl- edge posterior which knowledge would be used up to previous turn t -- 1, estimate the knowledge for current turn from prior knowledge distribution and generate an utterance from the inferred knowledge. Figure 2 shows an example of this generation process at / = 3. We parameterize the decoder network po. the prior distribution of knowledge no. and the approximate posterior da with deep neural networks as will be discussed.
From the posterior distribution 9o (k'-1) we draw a sample k and then update Ty and go with the sentence embedding of sampled knowledge (h (h(=):)) and the embeddings of previous and current current knowledge pool (h'²) to
utterances (d' d'yy: h²)). We use attention mechanism compute knowledge distribution given the dialogue context. This process is modeled as
To (k' |x . st k softmax (porio [h(] ... h (1)) E R
(5)
St y k = softmax ( [b4] ..... h 1T E R g (k') (k' [k'Ix St
(6)
where
Aprior W prior (Id h GRU 2
,h 1))),,
Gost W pooet ((d GRUmee - .n: [1))
(8)
de is the hidden state of Git and we initialize do de 0 768
and W priors W post
(768-)) are the parameters. We here use the GRU (i et al.
2017 Aneja et al.
2019 to
sequentially condition previously selected knowledge my and 46-
4

---

Published as a conference paper at ICLR 2020
Finally, sample knowledge kj over attention distribution in Eq. 6 and pass it to the decoder. At test time, we select the knowledge with the highest probability over distribution in Eq. (5)
Decoding with Copy Mechanism. We generate the wizard's response turn t, given rent context xf and selected knowledge sentence k-- We feed their concatenated embedding H zk. = H H to the decoder po- To maximize the effect of selected knowledge for response generation, we choose the Copy mechanism (Xia et al.] 2017 Li et al.] 2019b) with Transformer decoder (Vaswani al. 2017). We obtain the output word probability (Zhao et al. 2019a):
h Decoder(HHa,,
K', V T..K',
hi, W
H zk, W H Hk. W.
Pl, (w) = softmax W out by), copy (w) (o
(10)
(w.) = (1 (1-a,,n)) ag,, * pt, (w) + Ot, * Pt.n
copy
copy (w),
(11)
copy Σ Pt.n (w) V and is sigmoid. Finally, we select the word with the highest probability vn+1 arg wax at,a (w) where V is the dictionary. Unless the word y EOS token, repeat generating the next word by feeding vh+1 to the decoder.
where copy (WT
3.1 TRAINING
Obviously, there is large gap in knowledge selection accuracy between training with or without true labels (e.g. 23.2 of E2E Transformer MemNet with labels vs 4.8 of PostKS without labels in Table As one way to take advantage of true labels for training of latent models, prior research has employed auxiliary losses over latent variables (Wen et al., 2017 Zhao et al. (017). Similarly, we use the knowledge loss from Dinan et al. 2019) (i.e. the cross-entropy loss between predicted and true knowledge sentences) as an auxiliary loss for the latent variable. Thus, the training objective is
combination of the variational lower-bound from Eq. (3) and the auxiliary knowledge loss as
1 1
C=
(kt-1 [B (k') [log pe S y K!)]
-DKLlookk) To(k')) log (k())
(12)
Knowledge lass
where kg is sampled knowledge from go (k' ySt, ket), kf true knowledge, and À is hyperparameter. Note that knowledge is sequentially sampled from attention distribution in Eq. (6.. We train our model by mini-batch gradient descent. We approximate the expectation by drawing one sample from the posterior with Gumbel-Softmaa function (Jang et al., 2017: Maddison al. 2017b). Further details of optimization can be found in Appendix.
4 EXPERIMENTS
We evaluate our model mainly on the Wizard of Wikipedia (Dinan et al. 2019) and additionally Holl-E (Moghe et al., 2018) as another knowledge-groune chit-chat dataset. We quantitatively and qualitatively compare our approach with other state-of-the-al models.
4.1 DATASETS
Wizard of Wikipedia. It contains 18,430 dialogues for training, 1,948 dialogues for validation and 1,933 dialogues for test. The test set is split into two subsets, Text Seen and Test Unseen. Test Seen contains 965 dialogues on the topics overlapped with the training set, while Test Unseen contains 968 dialogues on the topics never seen before in training and validation set.
Holl-E. It contains 7,228 dialogues for training, 930 dialogues for validation and 913 dialogues for test. A single document is given per dialogue; the documents include about 58 and 63 sentences on average for training/validation and test set, respectively. The dataset provides spans in the document additional information to provide which parts of the document is used generate response. However, the span labels are rather inconsistent; for example, they are often shorter than a single sentence contain multiple consecutive sentences. Thus, we collect new set of ground-truth (GT)
5

---

Published as a conference paper at ICLR 2020
Table 2: Quantitative results on the Wizard of Wikipedia dataset (Dinan et al. 2019). The method with []] does not use the knowledge loss. The scores of E2E Transformer MemNet and Transformer (no knowledge)* are from the original paper. The variant (BERT vocab)² re-runned using the authors' code, since the vocabulary is different from original paper due to the use of BERT.
Test Scen
Test Unseen
Method
PPL
R-1
R-2
Acc
PPL
R-1
R-2
Acc
Random knowledge selection
8.4
1.4
2.7
8.0
1.2
2.3
Repeat utterance
14.5
14.1
2.9
Transformer (no knowledge) (Dinan et al.
2019
41.8
17.8
87.0
14.0
E2E Transformer MemNet' (Dinan al.
2019
63.5
16.9
22.5
97.3
14.4
12.2
E2E Transformer MemNet (BERT vocab)
53.2
17.7
4.8
23.2
137.8
13.6
1.9
10.5
PostKS' (Lian 2019
79.1
13.0
1.0
4.8
193.8
13.1
1.0
4.2
E2E BERT
53.5
16.8
4.5
23.7
105.7
13.5
2.2
13.6
PostKS + Knowledge Loss
54,5
18.1
5.3
23.4
144.8
13.5
2.0
9.4
E2E BERT + PostKS
54.6
17.8
5.3
25.5
113.2
13.4
2.3
14.1
E2E BERT + PostKS + Copy
52.2
19.0
6.5
25.5
83.4
15.6
3,9
14.4
Ours
52.0
19.3
6.8
26.8
81.4
16.1
4.2
18.3
Table 3: Quantitative results on the Holl-E dataset multiple references test set.
MMoghe et al.
2018
with single reference and
Single Reference
Multiple References
Method
PPL
R-1
Acc
PPL
R-1
R-2
Acc
Random knowledge selection
7.4
1.8
1.9
10.3
3.6
3.5
Repeat last utterance
11.4
1.5
13.6
2.0
E2E Transformer MemNet Dinan et al. 2019
140.6
20.1
10.3
22.7
83.6
24.3
12.8
32.3
PostKS (Lian et al. 2019
196.6
15.2
6.0
1.5
114.1
19.2
7.9
3.2
E2E BERT
112.6
25.9
18.3
28.2
66.9
31.1
22.7
37.5
PostKS Knowledge Loss
135.1
19.9
10.7
22.5
81.9
23.8
12.9
32.2
F2F BERT PostKS
119.9
27.8
20.1
27.6
66.7
33.7
25.8
37.3
E2F BERT PostKS Copy
47.4
29.2
22.3
27.8
27.9
35.9
29.0
37.8
Ours
48.9
29.8
23.1
29.2
28.5
36.5
29.7
39.2
knowledge per document that it is similar to that of WoW where all of the GT knowledge are in the form of sentences. Basically, we select the sentence that includes the span as the GT knowledge sentence. If the span is given sentences containing the span as GT. If no span is given, we use the no passages used tag as GT, which amounts 5% of all GT labels. indicates that the gold utterance generated with no knowledge grounding and the model should predict the label of no passages used for this sample to be correct. We make our new set of GT annotations available in the project page.
select the minimum number of consecutive multiple sentences,
4.2 EXPERIMENTAL SETTING
Evaluation Metries. We follow the evaluation protocol of WoW (Dinan al. 2019). We measure unigram Fl (R-1), bigram Fl (R-2) and perplexity (PPL) for response generation, and the accuracy for knowledge selection. For n- r-gram metrics, we remove all the punctuations and (a, an, the) before computing the score. We remind that lower perplexity and higher n-gram (R-1, R-2) scores indicate better performance.
The test set for Holl-E is split into two subsets, single reference and multiple references. The dataset basically provides single response per context (denoted as single reference). However, for some conversations, more responses (e.g. 2-13) are collected from multiple annotators per context (mul- tiple references). For evaluation of multiple references, we take the best score over multiple GTs by following Moghe et al. (2018). For knowledge accuracy, regard the model's prediction is correct it matches at least one of the correct answers.
We closely compare with two state-of-the-ar knowledge knowledge-grounded dialogue models. The first one is E2E Transformer MemNet (Dinan et al.) 2019), which uses a Transformer memory network for knowledge selection and Transformer decoder for utterance prediction. The second
Baselines.

---

Published as a conference paper at ICLR 2020
one is PostKS (Lian et al. 2019), which uses the posterior knowledge distribution pseudo-label for knowledge selection. For fair comparison, we replace all GRU layers in PostKS with Transform- ers. We also compare with four variants of these models as ablation study: (i) E2E BERT, where we replace the Transformer memory network with pre-trained BERT, (ii) PostKS Knowledge loss, where we additionally use the knowledge loss, (iii) E2E BERT PostKS, which combines all the components of baselines, and (iv) E2E BERT + PostKS + Copy, where we additionally use the copy mechanism with the Transformer decoder.
We use official BERT tokenizer to tokenize the words and use pre-defined BERT vocabulary (V 30522) to convert token to index" All the baselines use the exactly same inputs with our model except PostKS, which does not make use of knowledge labels as proposed in the original paper.
4.3
QUANTITATIVE RESULTS
Table compares the performance of different methods on the Wizard of Wikipedia dataset. Our model outperforms the state-of-the-art sooleege-grounded dialogue models in all metrics for knowledge selection (accuracy) and utterance generation (unigram FI, bigram F1). The PostKS that is trained with no knowledge label shows low accuracy on knowledge selection, which is slightly better than random guess. However, it attains better performance than E2E Transformer MemNet with the knowledge loss in the WoW Test Seen, which shows that leveraging prior and posterior knowledge distribution is effective for knowledge-grounded dialogue, although using sequential la- tent variable improves further. BERT improves knowledge selection accuracy, but not much in TextQA because of diversity in knowledge selection of conversation. The E2E BERT + PostKS + Copy performs the best among baselines, but not good ours, which validates that sequential latent modeling is critical for improving the accuracy of knowledge selection and subsequently ut- terance generation. Additionally, the performance gaps between ours and baselines are larger in Test Unseen. It can be understood that the sequential latent variable can generalize hetter. Adding the copy mechanism to the baseline substantially improves the accuracy of utterance generation, but barely improves the knowledge selection, which also justifies the effectiveness of the sequential latent variable. Transformer (no knowledge) shows the lowest perplexity in the WoW Test Seen, and it is mainly due to that it may generate only general and simple utterances since no knowledge is grounded. This behavior can be advantageous for the perplexity, while the other knowledge-based models take risk of predicting wrong knowledge, which is unfavorable for perplexity.
compares the performance of our model on Holl-E dataset. Similarly, our model outperforms all the baselines in all metrics. One notable trend is that BERT considerably reduces the perplexity in all models, which may be due to that the dataset size of Holl-E is much smaller than WoW and BERT prevents overfitting (Hao et al. 2019).
4.4 QUALITATIVE RESULTS
Single-Ture Turn Human Evaluation. We perform user study to complement the limitation of auto- matic language metrics. We evaluate several aspects of utterance generation using the similar setting in Guu et al. (2018). We randomly sample 100 test examples, and each sample is evaluated by three unique human annotators on Amazon Mechanical Turk (AMT). At test, we show dialogue context and generated utterance by our method or baselines. We ask turkers to rate the quality of each ut- terance in two aspects, which are referred to Li et al. (2019a): (i) Engagingness: how much do you like the response? and (ii) Knowledgeabiliy: how much is the response informative? Each item is scored from to 4 to avoid catch-all category in the answer (Dalal et al.) 2014), where means not at all, 2 is a little, 3 is somewhat, and 4 is a lot. To mitigate annotator bias and inter-annotator vari- ability, we adjust human scoring with Bayesian calibration (Kulikov al. 2019). Note that human evaluation on knowledge selection is not possible, since any knowledge could be fine for given context, which is key motivation for our sequential latent model diversity of knowledge selection.
Table 4 summarizes the results of the single-turn human evaluation, which validates that annotators prefer our results to those of baselines. Again, the performance gaps between ours and baselines are larger in Test Unseen, thank to better generality of our sequential latent model.

---

Published as a conference paper at ICLR 2020
Table 4: Single-turn human evaluation results on the Wizard of Wikipedia. We report the mean
ratings and their standard errors of different methods for engagingness and knowledgeability scores. TMN stands for E2E Transformer MemNet (Dinan et 2019).
Test Seen
Test Unseen
Raw
Calibrated
Raw
Calibrated
Method
Engage
Knowledge
Engage
Knowledge
Engage
Knowledge
Engage
Knowledge
PostKS
1.65 (0.05) 1.72 (0.06)
1.51 (0.02) 1.72 (0.01)
1.66 (0.06) 1.74 (0.06)
1.38 (0.02) 1.60 (0.02)
TMN
2.57 (0.05)
2.47 (0.06)
2,41 (0.02)
2.49 (0.01)
2.39 (0.06)
2.21 (0.06)
2.12 (0.02) 2.05 (0.02)
Ours
2.59 (0.05)
2.53 (0.06)
2.45 (0.02)
2.55 (0.01)
2.52 (0.06)
2.35 (0.06)
2.26 (0.02)
2.21 (0.02)
Human
3.14 (0.05)
3.09 (0.05)
3.00 (0.02)
3.12 (0.01)
3.11 (0.05) 2.99 (0.05)
2.83 (0.01) 2.85 (0.02)
Table 5: Multi-turn human evaluation results the Wizard of Wikipedia. We report the averages and standard deviations (in parentheses).
Method
Test Seen
Test Unseen
E2E Transformer MemNet
Dinan et al. 2019)
2.36 (1.38)
2.10 (0.96)
Ours
2.39 (0.99)
2.38 (1.01)
Human DDnaa et al.
2019)
4.13 (1.08)
4.34 (0.98)
Multi-turn Human Evaluation. We add another human evaluation results in a multi-turn setting using the evaluation toolkit from Wizard of Wikipedia (Dinan al. 2019). Humans are paired with one of the models and chat about a specific topic (given a choice of 2-3 topics) for 3-5 dialogue turns. After conversation, they score their dialogue partners on a scale of 1-5, with the rating indicating how much they liked the conversation. We collect the votes for 110 randomly sampled conversations from different turkers.
Table 5 compares the results of different methods for the multi-turn evaluation. Human annotators prefer our results to those baselines with larger gap in Test Unseen.
Dialogue Examples. Figure 3 shows selected examples of utterance prediction. In each set, we show dialogue context, human response, and utterances generated by our method and baselines. Thanks to the use of latent variables, our model can better capture the changes in dialogue topics and thus generate more appropriate responses.
5 RELATED WORK
Knowledge-basee based conversations have been studied much including collecting new datasets (Qin
et al. 2019 Zhang et al. 2018 Ghazvininejad et al. 2018 Zhou et al. 2018 Dinan et al.) 2019 Moghe et al. 2018)
developing new models [Lian al. 2019 Li al. 2019b Yavuz et al.
2019). Most works on the models have less investigated the knowledge selection issue but instead focused on how to effectively combine given knowledge and dialogue context to improve response informativeness. For example. Ghazvininejad al. 2018 aid Seq2Seq model with external knowledge memory network, and ct al. (2019b) propose an Incremental Transformer to encode multi-turn utterances along with knowledge in related documents. Recently, Dinan al. (2019) propose both dataset of Wizard of Wikipedia and a model to leverage the two-step procedure of selecting knowledge from the pool and generating response based on chosen knowledge and given context.
2019 Zhao et al. 2019b Dinan et al. 2019 Liu et al.
One of the most related models to ours may be Lian et al. (219)) who also focus on the knowledge selection issue in the two-stage www.ledg-groundeer dialogue. However, our work is novel that model it as sequential decision process with latent variables and introduce the knowledge loss. Thanks to these updates, our model achieves significantly better performance as shown in the experiments.
Sequential Latent Variable Models. There have been many studies about sequential latent vari- able models. Chung et al. (2015) propose one of the earliest latent models for sequential data, named VRNN. Later, this architecture extended to SRNN (Fraccaro et al., (016)) and Z-Forcing (Goyal al. 2017). There have been some notable applications of sequential latent models, including doc-
8

---

Published as a conference paper at ICLR 2020
Seen Test (Topic: Italian Cuisine)
Unseen Test (Topic: Hunting)
A: love chicken parmigianaa us well, but I think my ultimate
W: That true but we always have to watch out for excessive
favorite is beef lasagna... Extra choese please!
hunting. It has caused some species to be endangered.
Chicken with sauce and mozzarella....
still my heart!
Yes agree. don't believe in the useless hunting that poachers
A: Truthfully, anything with cheese is the best
do. Its eruel.
(Ours) i love pizza too it 's tradiiional italian dish
(Ours) agree, poaching has been defined as the illegal hunting or capturing of wild animals.
consisting of ycasted flatbread typically topped with tomato
sauce and cheese
(TMN) i thinks so , 'n not sure if you 're talking about that poodles are the second most intelligent breed behind the poodle-
(TMN) cheese!
poaching, but
(E2E KL)) i and condiments.
mine tupped with vegetables, meats
(E2E BERT i agree i think
great way catch fish (Human) agreed i remember reading one time that unless you
(Human) especially cheddar cheese! it the second most popular cheese in the use !
plan to kill the animals its not considered hunting.
Figure 3: Examples of generated responses by our model and baselines on Wizard of Wikipedia. TMN stands for E2E Transformer MemNet, and A and W for apprentice and wizard. Examples with selected knowledge sentences can be found at Appendix
ument summarization (Li et al. 2017). image captioning (Aneja al.) 2019) and text generation (Saa et al. 2019). Another related class of sequential latent models may be latent atlention mod- ely (Deng al. [0188] Wang al.) 2018 Yang et al., 2017), which exploit the latent variables to model the attention mapping between input and output sequences. Although our method is partly influenced by such recent models, is novel to propose sequential latent model for the knowledge- grounded chit-chat problem.
CONCLUSION
This work investigated the issue of knowledge selection in multi-turn knowledge- grouuddd dia- logue, and proposed sequential latent variable model, for the first time, named sequential knowl- edge transformer (SKT). Our method achieved the new offte-of-thearr performance on the Wiz- ard of Wikipedia benchmark (Dinan et al.) 2019) and a dataset (Moghe al.) 2018). There are several promising future directions beyond this work. First, we can explore other inference models such as sequential Monte Carlo methods using filtering vari- ational objectives (Maddison et al. (017a). Second, we can study the interpretability of knowledge selection such as measuring the uncertainty of attention (Heo et al. 2018).
version of Holl-E
ACKNOWLEDGMENTS
We thank Hyunwoo Kim, Chris Dongjoo Kim, Soochan Lee, Junsoo Ha and the anonymous review- ers for their helpful comments. This work was supported by SK T-Brain corporation and Institute of Information & communications Technology Planning Evaluation (IITP) grant funded by the Korea government (MSIT) (No.2019-0-01082, SW StarLab). Gunhee Kim is the corresponding author.
REFERENCES
Jyoti Aneja, Harsh Agrawai, Dhruv Batra, and Alexander Schwing. Sequential Latent Spaces for Modeling the Intention During Diverse Image Captioning. In ICCV, 2019.
Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. Enriching Word Vectors with Subword Information. In TACL, 2016.
Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St John, Noah Con- stant, Mario Suujardd-Cespedeer Steve Yuan, Chris Tar, et al. Universal Sentence Encoder. arXiv:1803.11175. 2018.
Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. Reading Wikipedia to Answer Open- Domain Questions. In ACI., 2017.

---

Published as a conference paper at ICLR 2020
Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning Phrase Representations Using RNN Encoder- Decoder for Statistical Machine Translation. In EMNLP, 2014.
Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron C Courville, and Yoshua Ben- gio. A Recurrent Latent Variable Model for Sequential Data. In NIPS, 2015.
Dev K Dalal, Nathan T Carter, and Christopher Lake. Middle Response Scale Options are Inap- propriate for Ideal Point Scales. J. Bus. Psychol., 29(3):463 478, 2014.
Yuntian Deng, Yoon Kim, Justin Chiu, Demi Guo, and Alexander Rush. Latent Alignment and Variational Attention. In NIPS, 2018.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre- Training of Deep Bidirectional Transformers for Language Understanding. In NAACL-HILT 2019.
Emily Dinan and Jason Weston. Advances in Conversational AI. https://ai: facebook. 2019.
Com/ com/blooe
Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. Wizard of Wikipedia: Knowledge-Powered Conversational Agents. In ICLR, 2019.
Sergey Edunov, Myle Ott, Michael Auli, David Grangier, and Marc' Aurelio Ranzato. Classical Structured Prediction Losses for Sequence to Sequence Learning. In NAACL-IILT, 2017.
Marco Fraccaro, Søren Kaae Sønderby, UIrich Paquet, and Ole Winther. Sequential Neural Models with Stochastic Layers. In NIPS, 2016.
Marjan Ghazvininejad, Chris Brockett, Ming- Weig Chang, Bill Dolan, Jianfeng Gao, Wen-tau Yih, and Michel Galley. A Knowledge-Grounded Neural Conversation Model. In AAAI, 2018.
Xavier Glorot and Yoshua Bengio. Understanding the Difficulty of Training Deep Feedforward Neural Networks. In AISTATS, 2010.
Karthik Gopalakrishnan, Behnam Hedayatnia, Qinlang Chen, Anna Gottardi, Sanjeev Kwatra, Anu Venkatesh, Raefer Gabriel, Dilek Hakkani-Tihe and Amazon Alexa Al. Topical-Chat: Towards Knowledge-Groundt Grounded Open-Domain Conversations. In Interspeech, 2019.
Anirudh Goyal Alias Parth Goyal, Alessandro Sordoni, Marc-Alexandre Côté, Nan Rosemary Ke, and Yoshua Bengio. Z-Forcing: Training Stochastic Recurrent Networks. In NIPS, 2017.
Kelvin Guu, Tatsunori B Hashimoto, Yonatan Oren, and Percy Liang. Generating Sentences by Editing Prototypes. TACL, 6:437-450, 2018.
Yanu Hao, Dong, Furu Wei, and Ke Xu. Visualizing and Understanding the Effectiveness of BERT. In EMNLP, 2019.
Jay Heo, Hae Beom Lee, Saehoon Kim, Juho Lee, Kwang Joon Kim, Eunho Yang, and Sung Ju Hwang. Uncertainty-Aware Attention for Reliable Interpretation and Prediction. In NIPS, 2018.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical Reparameterization with Gumbel-Softmax. In ICLR, 2017.
Diederik Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. In ICLR. 2015.
Ilya Kulikov, Alexander Miller, Kyunghyun Cho, and Jason Weston. Importance of a Search Strategy in Neural Dialogue Modelling. In INLG, 2019.
Margaret Li, Jason Weston, and Stephen Roller. ACUTE-EVAL: Improved Dialogue Evaluation with Optimized Questions and Multi-tum Comparisons. arXiv: 190.000077 2019a.
Piji Li, Wai Lam, Lidong Bing, and Zihao Wang. Deep Recurrent Generative Decoder for Abstrac- tive Text Summarization. In EMNLP, 2017.
10

---

Published as a conference paper at ICLR 2020
Zekang Li, Cheng Niu, Fandong Meng, Yang Feng, Qian Li, and Jie Zhou. Incremental Transformer with Deliberation Decoder for Document Grounded Conversations. In ACL, 2019b.
Rongzhong Lian, Min Xie, Fan Wang, Jinhua Peng, and Hua Wu. Learning Select Knowledge for Response Generation in Dialog Systems. In IJCAI, 2019.
Zhibin Liu, Zheng-Yo Yu Niu, Hua Wu, and Haifeng Wang. Knowledge aware conversation generation with reasoning on augmented graph. In EMNLP, 2019.
Chris Maddison, John Lawson, George Tucker, Nicolas Heess, Mohammad Norouzi, Andriy Mnih, Arnaud Doucet, and Yee Teh. Filtering Variational Objectives. NIPS, 2017a.
Chris J Maddison, Andriy Mnih, and Yee Whye Teh. The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables. 2017b.
Andrea Madotto, Chien-Sheng Wu, and Pascale Fung. Mem2Seq: Effectively Incorporating Knowl- edge Bases into End-to-End Task Misk-Oriented Dialog Systems. In EMNLP. 2018.
Sewon Min, Victor Zhong, Richard Socher, and Caiming Xiong. Efficient and Robust Question Answering from Minimal Context over Documents. In ACL, 2018.
Nikita Moghe, Siddhartha Arora, Suman Banerjee, and Mitesh M Khapra. Towards Exploiting Background Knowledge for Building Conversation Systems. In EMNLP, 2018.
Prasanna Parthasarathi and Joelle Pineau, Extending Neural Generative Conversational Model using External Knowledge Sources. In EMNLP, 2018.
Gabriel Pereyra, George Tucker, Jan Chorowski, Eukasz Kaiser, and Geoffrey Hinton. Regularizing Neural Networks by Penalizing Confident Output Distributions. In ICLR, 2017.
Lianhui Qin, Michel Galley, Chris Brockett, Xiaodong Liu, Xiang Gao, Bill Dolan, Yejin Choi, and Jianfeng Gao. Conversing by Reading: Contentful Neural Conversation with On-Demand Machine Reading. In ACL, 2019.
Shiv Shankar and Sunita Sarawagi. Posterior Attention Models for Sequence to Sequence Learning. In ICLR, 2019.
Zhihong Shao, Minlic Huang, Jiangtao Wen, Wenfei Xu, and Xiaoyan Zhu. Long and Diverse Text Generation with Planning-based Hierarchical Variational Model. In EMNLP. 2019.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention All You Need. NIPS, 2017.
Weiyue Wang, Derui Zhu, Tamer Alkhouli, Zixuan Gan, and Hermann Ney. Neural Hidden Markov Model for Machine Translation. In ACI., 2018.
Tsing-Hsien Wen, Yishu Miao, Phil Blunsom, and Steve Young. Latent intention dialogue models. In ICML, 2017.
Yingce Xia, Fei Tian, Lijun Wu, Jianxin Lin, Tao Qin, Nenghai Yu, and Tie- Yan Liu. Deliberation Networks: Sequence Generation Beyond Pass Decoding. In NIPS, 2017.
Zichao Yang, Zhiting Hu, Yuntian Deng, Chris Dyer, and Alex Smola. Neural Machine Translation with Recurrent Attention Modeling. EACL, 2017.
Semih Yavuz, Abhinav Rastogi, Guan-Lin Chao, and Hakkani-Tur Dilek. DeepCopy: Grounded Response Generation with Hicrarchical Pointer Networks. In SIGDIAL, 2019.
Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. Per- sonalizing Dialogue Agents: I Have Dog, Do You Have Pets Too? In ACI,, 2018
Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. Learning Discourse-Ievel Diversity for Neural Dialog Models using Conditional Variational Autoencoders. In ACI,, 2017.
11

---

Published as a conference paper at ICLR 2020
Wei Zhao, Liang Wang, Kewei Shen, Ruoyu Jia, and Jingming Liu. Improving Grammatical Error Correction via Pre- Training a Copy-Augmented Architecture with Unlabeled Data. In NAACL- HLT, 2019a.
Xueliang Zhao, Chongyang Tao, Wei Wu, Can Xu, Dongyan Zhao, and Rui Yan. A Document- Grounded Matching Network for Response Selection in Retrieval-based Chatbots. In IJCAI, 2019b.
Kangyan Zhou, Shrimai Prabhumoye, and Alan W Black. A Dataset for Document Grounded Con- versations, In EMNLP, 2018.
A DERIVATION OF CONDITIONAL PROBABILITY
In Section
re- re-write the conditional probability of wizard's response y" given dialogue context
x St and from Eq. (2) to Eq. 4.. We can simply derive as follows:
p(y|x)
(13)
I kt
Po P S
y k Tokk))
(by Eq. (2)
(14)
po (x x po((') Σ po Sx y (s mokk))
(by Bayes' rule)
(15)
Po ( y (oo(k)) Σ Po po(y"Ix5' y 1* t (k*))
(16)
k'
No (k')
Σ po (x y kk mo())) (x and ytt are given)
(17)
R p(y' St y ).
(18)
where (kk)) is an approximated posterior distribution and (kk)) is true posterior distribution.
B TRAINING DETAILS
All the parameters except pretrained parts are initialized with Xavier method (Glorot & Bengio 2010). We use Adam optimizer (Kingma & Ba 2015) with 31 0.9, 32 0.999, le 07. For the models without BERT. we set the learning rate to 0.001 and initialize the embedding matrix with fastText (Bojanowski et al. 2016) trained on the Common Crawl corpus. For the models with BERT, we set the learning rate to 0.00002 and initialize encoder weights with BERT-Ba Uncased pretrained weights. We apply label smoothing (Pereyra al.) [017]] [Edunov al., 2017 Vaswani et al. 2017) for both knowledge selection and response generation, and set 0.1 and 0.05 for each. We set the temperature of Gumbel-Softtax to = 0.1 and the hyperparameter for the knowledge loss to À = 1.0. For efficiency, we batch the dialogues rather than individual turns. We train our model up to 5 epochs on two NVIDIA TITAN Xp GPU.
C KNOWLEEDGE SELECTION ACCURACY OVER TURNS
Table 6 compares the knowledge selection accuracy of different methods for each turn the Wiz- ard of Wikipedia. Thanks to the sequential latent variable, our model consistently outperforms other methods for all turns in knowledge selection accuracy. Notably, in all models, the accuracy signif- icantly drops after the first turn, which is often easily predictable as a topic definition sentence. shows the diversity nature in knowledge selection, discussed in Section 2
D QUANTITATIVE RESULTS ON
SETTING
Table shows the results of our model with partial knowledge labels on the Wizard of Wikipedia. We attain better performance with more labeled knowledge data for training as expected. Furthermore,
12

---

Published as a conference paper at ICLR 2020
Table 6: Knowledge selection accuracy for each turn on the Wizard of Wikipedia (Dinan et al,
2019 The method with [*] uses no knowledge loss. TMN stands for E2E Transformer MemNet.
Test Seen
Test Unseen
Method
1st
2nd
3rd
4th
5th
1st
2nd
3rd
4th
5th
PostKS" (Lian et al.
2019)
3.6
3.6
4.1
7.0
9.5
3.4
3.0
4.7
4.1
9,9
PostKS + Knowledge Loss
55.4
19.3
10.7
7.0
26.0
3.8
4.0
TMN (Dinan et al. 2019)
55.8
19.5
10.4
7.6
6.2
25.9
7.0
4.1
4.2
6.1
E2F BERT + PostK.
56.5
20.6
13.7
10.4
9.2
36.0
8.1
6.1
6.8
5.7
Ours
59.1
20.6
15.8
12.8
9.1
52.9
8.8
8.4
6.4
10.7
Table 7: Performance of model with partial knowledge labels Wizard of Wikipedia Dinan et al. 2019).
Test Seen
Test Unseen
Method
PPL
R-I
R-2
Acc
PPL
R-1
R-2
Acc
E2E Transformer MemNet' Dinan 2019)
63.5
16.9
22.5
97.3
14.4
12.2
E2E Transformer MemNet (BERT vocab)
53.2
17.7
4.8
23.2
137.8
13.6
1.9
10.5
Ours
$2.0
19.3
6.8
26.8
81.4
16.1
4.2
18.3
1/2 knowledge labeled
49.0
19.2
6.6
25.1
77.8
16.1
4.1
16.7
1/4 knowledge labeled
45.7
18.7
6.1
22.4
78.0
15.8
3.6
13.8
1/8 knowledge labeled
45.3
18.6
6.0
21.0
79.9
15.7
3.6
12.3
knowledge loss
54.7
17.1
4.6
0.3
88.2
15.5
3.4
0.1
our model achieves competitive performance with less label. For instance, our model using only 1/4 labeled training data comparable E2E Transformer MemNet and even better in Test Unseen. As result, our sequential latent knowledge selection model can be utilized in semi-supervised method without severe drop in the performance.
E EXAMPLES WITH SELECTED KNOWLEDGE
Figure and show selected examples of knowledge selection and response generation. In each set, given dialogue context, we compare selected knowledge and generated utterances by our method and baselines with human ground truths.
13

---

Published as a conference paper at ICLR 2020
Seen Test (Topic: Italian Cuisine)
A: I love chicken parmigiana as well, but think my ultimate favorite is beef lasagna...
Extra cheese please!
W: Chicken with sauce and mozzarella.... Be still my heart!
A: Truthfully, anything with cheese is the best
Task 1: Knowledge Selection
Task 2: Response Generation
Pizza is a traditional Italian dish consisting of yeasted flathread typically topped with tomato sauce and cheese and baked in an oven.
love pizza too it is traditional italian dish consisting of yeasted flatbread typically topped with tomato sauce and cheese
Ours
(from "Pizza" Wikipedia page)
TMN
no passages used
love cheese
can also be topped with additional vegetables, meats, and condiments, and can
like mine topped with vegetables, meats and condiments.
E2E
BERT
be made without cheese.
*KL
(from "Pizza" Wikipedia page)
is the sccond-mos most popular cheese in the US (behind mozzarella), with average annual consumption of 10 lb (4.5 kg) per
espccially choddar cheese it the second most popular cheese in the use
Human
capita.
(from "Cheddar cheese" Wikipedia page)
Figure 4: Examples of selected knowledge and generated responses by our model and baselines on the WoW Seen Test set.
Unseen Test (Topie: Hunting)
A: Yes, that is the best way do it. Apparently in some areas the government will actually pay money to hunt for
W: That but always have watch out for excessive hunting. has cause some species be endangered.
A: agrec. don't believe in the useless hunting that poachers do. It's cruel.
Task 1: Knowledge Selection
Task 2: Response Generation
Poaching has traditionally been defined the illegal hunting or capturing of wild animals, usually associated with land use rights.
agrec. poaching has been defined as the illegal hunting capturing of wild animals.
Ours
(from "Poaching" Wikipedia page)
i thinks so m not sure if you re talking about poaching, know that poodles are the second most intelligent breed behind the poodle.
TMN
no passages uuee
E2E
Hunting can also means pest control. i agres. think great way to catch fish. (from "Poaching" Wikipedia page)
BERT
*KL
It is also not considered hunting to pursue animals without intent to kill them, as in wildlife photography, birdwatching, or scientific research (from "Hunting" Wikipedia page)
agreed, i remember reading time that unless you plan to kill the animals its not considered hunting.
Human
Figure 5: Examples of selected knowledge and generated responses by our model and baselines on the WoW Unseen Test set.
14


Reviews:
[{"review_id": "ryxezvS0tB", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Post author response edit: The authors did a good job of addressing many of the concerns of reviewers. I believe with these new results (esp to reviewer 4), they will have a stronger version for the camera ready. I'm bumping up my recommendation for this reason.\n\n\n\nThe authors propose a novel architecture for selecting knowledge in knowledge-grounded multi-turn dialogue. Their knowledge selection module uses a sequential latent variable scheme, and is claimed to be able to both handle diversity of knowledge selection in conversation as well as leverage the information from the response. The proposed model yields state of the art on two relevant benchmark datasets in terms of perplexity and F1, and scores higher in human evaluations as well.\n\nThe paper is relatively well-written, and the authors offer extensive insight into their approach, providing relevant equations and diagrams where necessary. The approach is well-motivated, and the experiments indicate that the model indeed helps on all evaluation fronts. A variety of baselines are considered and are shown to be inferior, in nearly every metric. I did not spend a lot of effort to try to understand their factorization, but the intuition makes sense, and their use of gumbel softmax provides a clear avenue to fix some of the hard-backprop issues apparent in the original Dinan et al. paper. I also appreciate the addition of the knowledge loss to the PostKS baseline: it’s a good effort to make the baseline as good as possible.\n\nA few things bother me with the paper. The primary one is it concerns me a bit that the BERT pretraining does not improve significantly over the E2E transformer memnet (with just bert vocabulary). Unless I’m missing something, that model contained NO pretraining, so I would expect massive improvements. A sanity check there would be checking ppl with gold knowledge: if that doesn’t significantly improve, then I suspect the authors have something really weird about the pretraining or fine tuning. However, It also appears to me that replacing the GRU with a transformer in PostKS might be unfair: Transformers are way more data hungry than RNNs, and so both variants should be tried (though I would be okay with the loser being relegated to a footnote or appendix).\n\nThe human evaluations are not as convincing as the authors propose them to be, especially the difference in the “Test Seen” case. It is unclear to me why the authors believe that their “model’s merit would be more salient” in a multi-turn setting, and I think such an experiment would be good to show - or, at the very least, an indication that such an experiment was tried but results were not considered due to reasons X,Y, Z, etc. Overall, the rough improvement that is being provided in the first-stage of the two stage setting seems rather minor (23% -> 26% accuracy; 2.21 -> 2.35 human eval), and that the task remains extremely difficult\n\nQuestions\n* I know the Dinan et al. models, at human evaluation time, hardcoded to not pick the same knowledge twice. Do you have a similar restriction? If not, maybe you can at least say that you manage to get rid of the need for that!\n* As mentioned earlier, I would be curious to see multi-turn human evaluations. I understand this is expensive and a large ask.\n* How are the examples in figure 3 chosen? Are they generally indicative of what is seen throughout the human evaluation?\n* It would be useful to see a qualitative example of the model’s knowledge selection process when comparing to other models, rather than just the utterance generation (which is not the novel contribution of the paper).\n\nNits\n* Small grammatical errors dealing with subject-verb agreement (plurals mostly).\n* Using “-” instead of n/a in tables would make it mildly easier to see digest and see where metrics don’t make sense.\n", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I did not assess the derivations or theory."}, "extra_text": {}}, {"review_id": "Bkxtl24c5r", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper looks at the problem of knowledge selection for open-domain dialogue. The motivation is that selecting relevant knowledge is critical for downstream response generation.\nThe paper highlights the one-to-many relations when selecting knowledge which makes the problem even more challenging. It tries to address this by taking into account the history of knowledge selected at previous turns.\nThe paper proposes a Sequential Latent Model which represents the knowledge history as some latent representation. From this methodology they select a piece of knowledge at the current turn and use it to decode an utterance. The model is trained in a joint fashion to learn which knowledge to select and on generating the response. As the two are strongly correlated. Additionally there is an auxiliary loss to help correctly identify if the knowledge was correctly selected. Additionally a copy mechanism is introduced to try to copy words from the knowledge during decoding.\nThe experiments are run on the Wizard of Wikipedia dataset where there are annotations for which knowledge sentence is selected and on Holl-E, where they transform the dataset to have a single sentence tied to a response.\nFor automatic metrics there is significant improvement over baselines for correctly selecting a piece of knowledge and generating a response. Additionally there is human evaluation that also shows significant improvement.  Their model also seems to generalize well to domains that were not seen during training time over baselines models.\n\nThe contribution of the paper is the novel approach to selecting knowledge for open-domain dialogue. This work is significant in that by improving knowledge selection we see a subsequent improvement in response generation quality which is the overall downstream task within this problem space.\nI believe this paper should be accepted because of the significant and novel approach of modeling previous knowledge sentences selected. The linking of this knowledge selection model to topic tracking as stated in the paper is of clear importance, as ensuring topical depth and topical transition are two key aspects for open-domain dialog.\n \nFeedback on the paper\nIn Figure 3, please provide the knowledge sentence that was selected.\nPlease provide the inter-annotator agreement for human evaluation.\nI think it would be interesting to see what is the copy mechanism actually adding in terms of integration of knowledge vs the WoW MemNet approach. Are those two truely comparable because one does not have copy?\nFor Related Work, also cite Topical-Chat: Towards Knowledge-Grounded Open-Domain Conversations\n\nSmall grammatical errors\n\"Recently, Dinan et al. (2019) propose to tackle\" -> \"Recently, Dinan et al. (2019) proposed to tackle\"\n\"which subsequently improves the knowledge-grounded chit-chat.\" -> \"which subsequently improves knowledge-grounded chit-chat.\"\n\n\nSome questions for the authors in terms of future direction\nHow is the performance of the model impacted with longer dialog context vs shorter?\n\nThe Holl-E dataset was transformed from spans of knowledge to a single knowledge sentence. It would be interesting to see what happens when the knowledge selected is over multiple sentences.\n\nThe knowledge pool currently consists of 67.57 sentences on average. How will this method scale as the amount of knowledge sentences grows?\n\n\n\n\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "B1ehFnYiYr", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a sequential latent variable model for knowledge selection in dialogue generation. More specifically, the authors extended the posterior attention model (Shankar and Sarawagi, 2019) to the latent knowledge selection problem. The proposed model achieved higher performances than previous state-of-the-art knowledge-grounded dialogue models on Wizard of Wikipedia and Holl-E datasets.\n\nThis work presents a reasonable ideas with new state-of-the-art results in both quantitative and qualitative evaluations.\nAnd overall the paper reads well.\n\nBut I think it could be further improved with the following points:\n- Could you describe the updates from the previous sequential latent variable models more clearly? It would help to further highlight the contribution of this work. Now it might not be very clear enough for those who are not familiar with the previous work.\n- In the introduction, the authors claim the following three advantages of the proposed method: reduced scope of knowledge candidates, better utilization of response information, and weakly-supervised inference with no labels.\nBut I'm not very convinced whether the experimental results indicate the aspects clearly enough. More detailed analysis should be added to support the contributions.\n- The current experiments mainly focus on end-to-end dialogue generation performances. But it would be also interesting to see more detailed aspects of knowledge-selection itself in both quantitative and qualitative manners. I guess this analysis can be done based on the sampled or selected knowledge from the attention distribution.\n- Could you possibly add some ablation studies to show the effectiveness of each component? Especially, I'm curious about the results of the proposed model without knowledge loss.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a sequential latent variable model for the knowledge selection task for knowledge grounded dialogues. Experimental results demonstrate improvements over the previous SOTA in the WoW, knowledge grounded dialogue dataset, through both automated and human evaluation. All reviewers scored the paper highly, but they also made several suggestions for improving the presentation. Authors responded positively to all these suggestions and provided updated results and other stats. The paper will be a good contribution to ICLR.
Decision: Accept (Spotlight)