Generating Dialogue Responses From A Semantic Latent Space
Authors: Wei-Jen Ko
Avik Ray
Yilin Shen
Hongxia Jin
Venue:  (2020)
Keywords: dialog
chatbot
open domain conversation
CCA

Abstract: Generic responses are a known issue for open-domain dialog generation. Most current approaches model this one-to-many task as a one-to-one task, hence being unable to integrate information from multiple semantically similar valid responses of a prompt. We propose a novel dialog generation model that learns a semantic latent space, on which representations of semantically related sentences are close to each other. This latent space is learned by maximizing correlation between the features extracted from prompt and responses. Learning the pair relationship between the prompts and responses as a regression task on the latent space, instead of classification on the vocabulary using MLE loss, enables our model to view semantically related responses collectively. An additional autoencoder is trained, for recovering the full sentence from the latent space. Experimental results show that our proposed model eliminates the generic response problem, while achieving comparable or better coherence compared to baselines.

Under review as a conference paper at ICLR 2020
GENERATING DIALOGUE RESPONSES FROM A SE- MANTIC LATENT SPACE
Anonymous authors Paper under double-blind review
ABSTRACT
Generic responses known issue for open-domain dialog generation. Most current approaches model this one-to-many task as one-to-one task, hence being unable integrate information from multiple semantically similar valid responses of prompt. We propose novel dialog generation model that learns semantic latent space, on which representations of semantically related sentences are close to each other. This latent space is learned by maximizing correlation between the features extracted from prompt and responses. Learning the pair relationship between the prompts and responses as a regression task on the latent space, instead classification on the vocabulary using MLE loss, enables our model to view semantically related responses collectively. An additional autoencoder is trained, for recovering the full sentence from the latent space. Experimental results show that our proposed model eliminates the generic response problem, while achieving comparable or better coherence compared to baselines.
INTRODUCTION
The eqquence-t--sqqeenee (Sutskever et al. 2014) framework has become a popular choice for de- signing open-domain neural response generation systems (Vinyals & Le 2015). Recently, trans- former (Vaswani et al., 2017) based models have received increasing attention (Wolf et al. 2018)_ Those models typically involves maximizing the probability of the ground truth response given the input prompt, trained using a cross entropy loss on the 1 1-of-n coding vocabulary.
However, it is known problem that models tend to generate bland and uninformative responses, such as "/ don't know" (Serban et al., 2016 Sordoni et al., 2015). despite the presence of more specific responses in the training data than generic responses. This problem is not caused by model optimization error or decoding search error (Holtzman et al. 2019); on the contrary this is a result of a fundamental deficiency of the end-to-end maximum likelihood objective, which models the one-to-mant task as a one-to-one task.
The specific issue is the following. A sequence-to soseuuueeeeeeeeeeeeeeeeeeeee decoder trained using maximum likeli- hood objective tends to prefer uninformative stop word with higher frequency in training data, over more informative words with lower frequency, because the model loss treat each informative word independently and ignoring their semantic relatedness. A similar effect happens on the utterance level when using beam search decoding. Beam search aims find the most probable utterance. Each sentence is treated independently, and the model is unable to use semantic similarity between different utterances (Qiu et al. 2019). When specific responses collectively have high probability, it is diluted by the large number of variations and possibilities of specific answers. On the other hand, generic responses have much fewer variations, thus they become the most probable response sequences. An alternative decoding method to beam search is sampling (Holtzman et al., 2019 Fan jet al. 2018), which does not suffer from this problem. However, sampling does not consider the subsequent words during decoding, and the randomness in word choice makes it prone to generating implausible responses and grammatical errors.
Aiming to take into account the semantic relatedness of those diverse specific responses, propose a novel model that learns a shared latent space with semantic information, on which semantically related sentences tend to be close to one another. The generation process could be separated into two steps. The first step predicts a sentence vector the response on the semantic latent space. Since predicting vector is regression problem in the latent space instead of classification in the

---

Under review as a conference paper at ICLR 2020
vocabulary in MLE loss, our model is able to learn that most of the probability mass of the response is around the cluster of possible specific responses. The second step constructs the full response sentence from the predicted vector.
Specifically, our semantic latent space learned by maximizing the canonical correlation (Hotelling 1936) between the features extracted from the prompt and the response. This objective preserves information most correlated with the other sentence in the pair. Since semantically similar responses likely to correspond latent space. Generic responses correlate with a different and much larger set of inputs than specific responses, so they will have very different representations in the latent space. This is illustrated in Figure Souuee[showe showing our model's 's representations of prompts and responses on t-SNE plot.
similar set of prompts, they will have similar representations in the
Prompt D3 (Light red)
Prompt D, (Light green)
instruments you play"
what you living
Response Dy (Dark red)
Response Dy (Dark green)
practice the piano every day
work as an clementary teacher
can play anything on my electric violin.
am an olympic gymnast
am learning the guitar
aside from nursing i work bar for school.
like the drums
i'm janitor but also play music night. you since fired lound job insurance.
used play clarinet.
play tromhone, alto sax, haritome, and trumpet... you
work part time as hartender but don't drink any alcohol mechanical engineering my
my parents taught flute
Prompt D2 (Light blue)
Prompt D, (Light yellow)
what is your Lavorite color
ya are you a female
Response Dy (Dark blue)
Response Dy (Dark yellow)
my lavorite color is green and whats yours yellow. superher
yes am woman.
not
i'm average
you
blue color makes happy
Jemale
mine is orange!
am just bay with heart outside my body 12 year old female
like rainhow colors, you
red. bluc, green, and yellow am thinking purple too strungely my favorite grey
Prompt D2 (Brown)
Response D, (Black)
what do you do for living"
I don't It know.
40
20
0
-20
-40
-20
40
Figure 1: t-SNE Visualization of the semantic latent space. The representations of the sentences in (a) are plotted in (b). Multiple semantic related responses are close to each other and close to the corresponding prompt, while generic responses are far away.
For the second decoding step, we train an additional autoencoder. The decoder part is used for constructing the response sentence from the shared latent space. The autoencoder representation is split into two parts. One part correlates with the prompt and the other is independent the prompt. This design models that some information of the response is related the prompt, while other
2

---

Under review as a conference paper at ICLR 2020
information might be completely irrelevant (e.g. follow up question). The encoder of the correlated part shared with the canonical correlated feature extractor for responses, while the uncorrelated part is learned independently of the prompt and regularized using normal distribution. The decoder is only trained on the autoencoder task. It simply recovers the sentence from its semantic latent representation, and it is not conditioned on the prompt. Since the semantics of the response and the decoding are learned separately, we could still perform beam search for the most probable sequence during inference but without the generic response problem.
We conduct experiments on two dialog datasets, and show that our model generates coherent and specific responses.
2 RELATED WORK
Several previous models also uses the idea of learning on sentence vector representations. Luo et al. 2018) used two autoencoders to learn the semantic representations of inputs, and learned utterance-level dependency between those representations. (Gao et al., 2019) fused the autoen- coder and seq2seq feature space, so that the distance and direction from a predicted response vector roughly matches the relevance and diversity. Those methods partially addresses this issue, but the problematic end-to-end MLE loss still plays an important role in both methods. In our work, completely remove the end-to-end loss term (our autoencoder loss does not backpropagate through the prompt encoder), encoder) the matching of the input and response is learned only shared semantic latent space. (Qiu et al., 2019) proposed a two-stage generation process, which predicts the average of the reference responses
intermediate task, but it requires training data with multiple sponses. (Kumar& & T'vetkov 2019) explored predicting continious word vectors on the token level in seq2seq models instead of using softmax classifcation.
To tackle the generic response problem, eL. 6016a) maximized mutual information in decoding or reranking. (Zhou et al. (0117) trained multiple response mechanisms model diversity. (?) split the generation into segments and allow attention to attend to both the prompt and the response to improve diversity. Several works use explicit specificity metries to manipulate the specificity of the responses, either using reinforcement learning(Li et al., 2016b), sample reweighting Bibauw f. or a gaussian mixture model (Zhang et al.) 2018a). Frequency based metrics such as IDF are commonly used. (Ko et al. 2019) proposed using specificity metric trained on discourse relation pair data.
&
Deep canonical correlation analysis (Andrew et al. 2013) has previously been used various tasks including feature learning (Wang et al. 2015), caption retrieval (Yan & Mikolajczyk 2015). multi- label classificaiton (Ych et 2017)) image cross-reconstrructin (Chanda et al. 2016), and multilin- gual word similarity (Rotman et al.
2018). (Mallinar & Rosset 2018) experimented on performing
deep canonical correlation analysis (DCCA) sequential data with recurrent network.
3
OUR METHOD
Given example dialogue (Prompt, Response) pairs (Dz, Dy) from open-domain dialogue datasets, our goal is to generate a coherent and non-generic response when given an unseen prompt. The structure of model is depicted in Figure It consists of three encoders; Prompt Encoder F., Correlated Response Encoder Fy, and Uncorrelated Response Encoder Fu. The final response is generated from a semantic latent vector via a Decoder Gy. During training all three encoders and the decoder are tuned. However, for inference/testing only the Prompt Encoder and the Decoder are utilized. We will now describe each of these components in more detail.
3.1 LEARNING THE CORRELATED SEMANTIC LATENT SPACE
Our primary aim is to learn a latent space where semantic related sentences (both prompts and responses) are close to each other. For this we employ canonical correlation analysis (CCA) (Hotelling 1936) between the prompt and response pairs, which maximizes the correlation of the extracted features with the other sentence in the pair. Since semantically similar responses are likely to correspond to similar set of prompts, semantic similar sentences will have similar represen-
3

---

Under review as a conference paper at ICLR 2020
D2'
Testing
0 am Fine
Training
Reconstruction loss
G
G
Maximizing Correlation
Y,
Regularization
Correlated Response
Uncorrelated
Prampt Encoder Fx
No
Encoder Fx
Response
H.How are you?
WhtWhat doing?
+ Watching
Figure 2: Architecture of our model.
tations in the CCA encoded space. Generic responses could be responses much larger set of prompts, so they will have very different representations in the latent space.
We use two recurrett neural networks Fa, Fy feature extractors to map prompts and responses into the shared featured space respectively. Using the definition of CCA, we maximize the total correlation of each dimension between X F.(D,), Y Fy (D,) as follows.
XX) - -Y-) Σ(X m
max Σ corr(X', vi)
max
m
(1)
E(E: m
X() Y.)2
1=1
m
subject to the condition
- - Xx) - X3) = Σ P:) YY)= -0
for each j pair. i, the indices of the feature dimension, and is the index of the example pair in the batch, and & the number of feature dimensions. The condition ensures that the different uncorrelated, to avoid redundant representations.
dimensions in the representation
To make the two feature spaces and Y shared, we add the following conditions to the mean and variance of both representations, inspired by (Yeh et al. 2017).
X y:
(3)
Σ (x =1
(4)
for all dimensions i.
When prompt X and response Y are perfectly correlated, and these two conditions perfectly hold, X will be equal to Y, so this makes X and Y interchangeable during inference. This is desirable because we don't have access Y during inference.
Using the two conditions, Equations] 1
becomes;
R Σ
max Σ corr(X', (X*, (X))
max
X Vi m m
min
Σ 1=1
M
X m
(5)
1=1 m
subject to,
Σ X
Σ
yi yi = m
(6)
372
7
4

---

Under review as a conference paper at ICLR 2020
We can formulate the total CCA loss from equation 3 to equation 6 as,
Le =
Σ
Σ
(x
yin) 2 +
X1
abs(
Σ
Xm)
+ absi
mm)
==1
m
m
m
i
+r2 abs(
(X )2 1) abs(
[(mm) 11
+
Σ
abs Σ
Xm m)
X
abs
Σ
Y Y
where 22, As are tunable hyper-parameters.
3.2 GENERATING THE RESPONSE FROM THE SEMANTIC LATENT SPACE
Since X and Y are interchangeable in the semantic space, we directly use the features extracted from the prompt X to approximate the response features Y. Now we want to generate the response sentence D from these response features. For this purpose, an additional autoencoder is trained on all the responses, simultaneously with the CCA. The autoencoder consists of encoder Fg. and decoder Gy. both of which are recurrent networks. The parameters of the encoder are shared with the semantic feature extractor of the responses. During inference, features extracted from the prompt X is directly fed into the decoder to generate the response sentence.
D' (F,(()
(8)
We also experimented adding domain discriminative adversarial training (Tzeng et al. 2017) between X and Y. but it did not improve the results. This shows that our conditions (1).)).(4)
already makes the distribution of the two encoders sufficiently similar.
Generating sentences from a continuous space is known to produce ungrammatical text (Bowman) ett al. 2016). To address this issue, randomly replace some of the autoencoder input word tokens by the unknown word token (unk). This serves three purposes. First, makes the decoder more robust, and able to generate grammatical responses when there is noise in the decoder input. This is important because the decoder input is from a different encoder during inference. Second, it prevents the autoencoder from overfitting too early before the CCA objective converges. Finally, the learned hidden representation would be more high-level, includinn semantic knowledge. This is desirable since the same representation is also used for learning the CCA for the semantic latent space.
Note that, this different from the word dropout used in (Bowman et al.] 2016). They only replace decoder inputs, while we replace both encoder and decoder inputs, since we want to prevent overfit- ting and make the extracted features include more high level semantic information of the sentence instead of focusing on individual words.
3.3 CORRELATED AND UNCORRELATED REPRESENTATIONS
When encoding the response with F,, the autoencoder and the CCA loss have conflicting objectives. The autoencoder task requires the representation to preserve all the information in the sentence for reconstruction. The CCA tasks aims to preserve only the information correlated with the prompt and discard all other irrelevant information. For example, a paraphrase pair are likely to be responses of the same set of prompts, they should have the same representation under CCA objective, but the autoencoder objective forces the representations to be different, to enable reconstruction of the exact sentences. A response could also include a topic change, which makes part of the response completely irrelevant with the prompt, and the irrelevant information shouldn't be in the CCA representation.
To model this issue, we separate the autoencoder representations into the correlated part Y, which correlates with the prompt, and the uncorrelated part Y.. The correlated part learns both the autoen- coder task and the CCA task. The uncorrelated part is only trained for autoencoder reconstruction.
During training. Gy learns to reconstruct from the concatenation of the correlated and uncorrelated representations. The reconstruction is trained using cross entropy loss.
D' (Y;
Fu(y)))
5

---

Under review as a conference paper at ICLR 2020
La CrossEntroppo D(y
(10)
During testing, Gy generates the response from the CCA semantic representation of the prompt and vector R representing the uncorrelated part of the response.
D'y = (P.) R))
(11)
By adding additional regularization to the uncorrelated representation Y, during training, we en- courage normal distribution with zero mean and unit variance. Hence during inference we can sample R. from this distribution or use a fixed prior to approximate Yu.
The formulation of the regularization is the same variational autoencoders (Kingma & Welling 2013). An encoder recurrent network Fu predicts a mean μ and variance o² for each dimension, and Y, is sampled from that multivariate normal distribution. The predicted μr and o² is regularized by the KL divergence with unit normal distribution.
L --EE + + (o -log((o
(12)
With this regularization, R can either be set to all zero mean, or be randomly drawn from a unit Gaussian distribution. We found that the generated sentence is insensitive this choice, so the two ways generate exactly the same sentence more often than not. Despite the insensitivity, we calculated the ratio between the KI. loss and the autoencoder reconstruction loss, and found that the ratio consistently increases during training, indicating that there is no posterior collapsing (Chen (ct al., 2017). We also found that adding the uncorrelated representation allows both the CCA loss and the autoencoder reconstruction loss to converge to significantly lower value. Inspection on generated sentences showed that there are obvious improvement on coherence, at the cost of slightly increased grammatical errors.
To avoid introducing excessive noise while using R as an approximation, we use low dimension for the uncorrelated part. A high dimension results in worse performance in our experiments.
During training, the gradient of all loss terms are weighted summed and all parameters are updated together. The total loss is:
C= AaLe + AgLL + d
(13)
where λ4, As, As are hyper-parameter
3.4 ATTENTION
One shortcoming of this model is that it does not have attention mechanism, it couldn't dynam- ically focus on different parts of the prompt during generation. We also experiment on a variant of model with Luong al. 2015)'s attenion. Similar previous works, the key and value is from the RNN hidden state of the prompt encoder Fz, and the query is the hidden state of the response de- coder Gy. To prevent nullifying the main purpose of model design: removing end-to-end MLE training, we create a bottleneck to limit the end-to-end information flow before concatenating the attention output vector with the hidden state. The bottleneck fully connected layer that reduces the attention output vector into low dimension.
4 EXPERIMENTS
4.1 DATASETS
We conduct experiments on two datasets: PersonaChat (Zhang et al. 2018b) and DailyDialog (Li et al. 2017). PersonaChat is chit-chat dataset collected by crowdsourcing. We don't use the personas in the dataset since they are not related to our work. We use the ConvA12 version, which contains 122,499 prompt-response pairs for training. We use 3,000 pairs for validation and 4,801 pairs for testing. DailyDialog is collection of conversations in daily life for English learners. We remove those prompt-response pairs in the validation and test set that also appears in the training set, which resulted in about 30% of pairs removed in the test set. The final dataset has 76,052 pairs for training, 5,334 pairs for validation, and 4,738 pairs for testing.

---

Under review as a conference paper at ICLR 2020
4.2 METHODOLOGY
We compare with the vanilla Seq2seq model with attention (Luong et al. 2015) trained using MLE
loss, decoded using beam search and nucleus sampling (Holtzman et al. (019). We also compare with previous work MMI-anti (Li et al. (2066)) and SpaceFusion (Gao et al. 2019). MMI-anti is based on mutual information, penalizes frequent responses with an anti-language model. Space- Fusion is re cently proposed method which learns a fused common space representation of the Seq2seq dialogue generation task and the autoencoder task. We use the authors' code.
Parameters: We use 1 layer GRU for all encoders and decoders. The correlated representation size is 512, the uncorrelated representation size 10. We implement Fy and Fu as different output dimensions of the same GRU. For compared methods we use hidden layer size 522. The word em- bedding dimension is 128. We use Adam (Kingma & Ba 2015) optimizer with learning rate 0.001, 31 0.9, B2 0.999. Batch size is 64. {11, λ3. X1, Xs, Ao) is set to (3.9.6.25.0005.22220011 they are tuned to make the conditions (2), (3), (4) enforced properly. The number of epochs trained is tuned on the validation set. Since there is no obvious automatic metric to evaluate the response quality, it is validated by human inspection on the generated sentences on the validation set for all models. Under this criterion, the MLE and MMI models are stopped considerably later than the minimum validation perplexity, since stopping earlier makes the responses more generic. For our model with attention, the attention bottleneck has dimension 10.
4.3 AUTOMATIC EVALUATION
Human evaluation is the only reliable way to evaluate the generated sentences, and the automatic metrics are only for reference. (Liu al. 2016) showed that automatic metries have low correlation with human judgements. However, since there are no good alternatives, those automatic metrics are still widely used. We use the following automatic evaluation metries: (1) BLEU-1 and BLEU-2 (Papineni et al.] 2002) (2) Embedding Average cosine similarity (Foltz et al. 1998) between gener- ated and gold response. The sentence-level embedding are computed by averaging the embeddings of each word in the sentence. This metric reflects the coherence of the response.
(3) dist-1 and
dist-2 (Li et al. 2016a), which evaluates the diversity of the generated responses. They respectively calculate the count of distinct unigrams and bigrams, divided by the total number of words in all
responses. Those metrics are also used in (Zhang et al. 2018a Xu et al. 2018 Gu et al. 2019 Qiu
et al. 2019).
Table shows the results. Our model outperform baselines on BLEU and embedding similarity. In terms of dist score, our model is not higher than MMI, since the responses model generate are much longer.
4.4 HUMAN EVALUATION
We use crowdsoureing for human evaluation. Following the standards of (Shao et al. 2017 Liu
eet al. 2018 Zhang et al. 22118a Qiu et al. 2019), for each model, we let 3 workers label 500 pairs of randomly sampled data. Our evaluation involved pool of around 400 distinct workers in total. We ask them to rate the specificity and coherence of each response, on Likert scale from 0 to 3. We report the average of all annotations for both metrics. Since a good response has to be both coherent and non-generic, we also report the useful information (UI) score, which is multiplying specificity multiplied by coherence for each sentence, and then take average over all sentences.
On PersonaChat dataset, SpaceFusion and sampling could generate very specific responses, but they do not improve coherence. Our model generates the most coherent responses, and the specificity score outperforms Seq2Seq and MMI. On DailyDailog dataset, sampling scores highest on speci- ficity, but the responses are not coherent. Our model with attention greatly improves specificity compared to other basclines, while maintaining similar level of coherence. Adding attention to our model improves both coherence and specificity on the DailyDialog dataset, but harms specificity on the PersonaChat dataset. Examples of generated responses are shown in the appendix. Our models performs best on UI score for both datasets. The improvements are significant.
We also evaluated the grammatical correctness of our model on PersonaChat. About 11% of sen- tences contain major grammatical errors that cause difficulty understanding of the sentence. 18% contain minor errors that don't 't affect the understanding of the sentence. 71 % of the sentences
7

---

Under review as a conference paper at ICLR 2020
grammatically correct. This shows that most of the responses of model are acceptable by human, and comprehensibility is not a major problem while we generate responses from a continuous space. Example model responses are shown in appendix A.
automatic metries
human evaluation
Dataset
Model
Blcu-1
Blcu-2
Sim
Dist-1
Dist-2
Coh.
Spec.
MLE+beam search
0.146
0.0640
0.854
0.0128
0.0397
1.09
0.98
MLs+linp
0.148
0.0605
0.851
0.0313
0.131
1.05
1.39
1.65
MMI
0.124
0.0551
0,838
0.0517
0.217
1.18
1.08
1.54
SpaceFusion
0.176
0.0715
0.855
0.0233
0.0766
1.05
1.66
1.84
PersonaChat
Ours
0.191
0.0746
0.871
0.0363
0.179
1.29
1.35
1.97
Ours4Attention
0.182
0.0712
0.868
0.0360
0.171
1.32
1.22
1.76
Ours w/o Uncorrelated
0.179
0.0729
0.866
0.0305
0.127
0.96
1,40
1,46
Ours w/o Denoising
0.167
0.0556
0.868
0.0332
0.252
MI.E+beam search
0.100
0.0394
0.763
0.0440
0.152
1.25
0.73
0.84
MI Elssampling
0.142
0.0413
0.790
0.0620
0.389
0.81
1.56
1.18
MMI
0.0939
0.0369
0.764
0.0697
0.270
1.29
0.89
1.09
DailyDialog
SpaceFusion
0.146
0.0595
0.792
0.0531
0,216
1.21
1.08
Ours
0.170
0.0575
0.807
0.0457
0.191
1.18
1.45
1.71
Attentin
0.171
0.0558
0.805
0.0530
0.213
1.24
1.51
1.74
Table 1: Result comparison all models PersonaChat and DailyDialog datasets using different automatic and human evaluation metrics. Our model generates specific and coherent responses.
4.5 ABLATION STUDY
We compare our full model with two variants and test the contribution of different parts in our model. We use the PersonaChat dataset for this experiment. The w/o Uncorrelated part model does not have the representation Y., the autoencoder reconstruction is solely based on Y. which also learns the CCA task. In the w/o Denoising model, we do not replace random words with (unk) in the autoencoder input. The results are in Table I
Without the uncorrelated part, all automatic metrics decrease. Human inspections sentences show that there is obvious decrease coherence, showing that allowing uncorrelated information pairs. Without denoising,
important for the learning the correlation between the prompt- the generated sentences contains many grammatical errors. All automatic metrics decreased except dist-2, which is caused by the ungrammaticality, Since the sentences are obviously unacceptable by humans, we did not perform human evaluation. This shows that denoising critical for our model to generate grammatical responses.
4.6 VISULIZING THE SEMANTIC SPACE
In order to verify that the shared latent space successfully encodes semantic information, we vi- sualize the representaions of some sentences Figure [ The dimension reduction is performed using t-SNE (van der Maaten 2009) trained on 1,000 prompt representations and 1,000 response representations in the test set.
The light red point is the latent representation of the sentence "what instruments do you play ?" encoded by the prompt encoder Fz. The seven dark red points are possible responses encoded by the response encoder Fy such as "i practice the piano every day ", "i am learning the guitar Similarly, the light and dark blue, green, and yellow points shows multiple possible responses to other three different questions.
We can see that those semantically related responses to the same question are clustered, showing that the latent space is indeed able to capture semantic information. The questions' representation is close to the cluster of their corresponding responses, demonstrating that our model has successfully learned from the collection of semantically similar possible responses.
We also visualize "i don't 't know " in black, and the most frequent generic response generated by the Seq2Seq model trained PersonaChat, "what do you do for living ?" in brown, using the
8

---

Under review as a conference paper at ICLR 2020
response encoder Fy. Those generic responses much farther away from the question than specific responses, thus they are unlikely to be generated.
CONCLUSION
In this paper, we proposed a novel open domain dialogue generation framework that first predicts a semantic sentence vector, and then reconstruct it into sentence. Our model jointly learns denois- ing autoencoder and a semantic latent space using CCA. The autoencoder representation consist of two parts, one correlates with the prompt, and one is regularized to follow unit normal distribution. This design enables learning from the semantic relatedness of different responses. Our model could generate coherent responses, and doesn't t suffer from the generic response problem unlike previous approaches. Evaluation on open domain dialog datasets concretely demonstrates the efficacy of our model.
REFERENCES
Galen Andrew, Raman Arora, Jeff Bilmes, and Karen Livescu. Deep canonical correlation analysis. In ICML, 2013.
Samuel R Bowman, Luke Vilnis, Oriol Vinyals, Andrew Dai, Rafal Jozefowicz, and Samy Bengio. Generating sentences from continuous space. CONLL, 2016.
Sarath Chanda, Mitesh M Khapra, Hugo Larochelle, and Balaraman Ravindran. Correlational neural networks. In Neural Computation, 2016.
Xi Chen, Diederik P. Kingma, Tim Salimans, Yan Duan, Prafulla Dhariwal, John Schulman, Ilya Sutskever, and Pieter Abbeel. Variational lossy autoencoder. In ICLR, 2017.
Angela Fan, Mike Lewis, and Yann Dauphin. Hierarchical neural story generation. ACL, 2018.
Peter W. Foltz, Walter Kintsch, and Thomas K. Landauer. The measurement of textual coherence with latent semantic analysis. In Discourse Processes, 1998.
Xiang Gao, Sungjin Lee, Yizhe Zhang, Chris Brockett, Michel Galley, Jianfeng Gao, and Bill Dolan. Jointly optimizing diversity and relevance neural response generation. NAACL, 2019.
Xiaodong Gu, Kyunghyun Cho, Jung- Woo Ha, and Sunghun Kim. Dialogwae: Multimodal response generation with conditional wasserstein auto-encoder. In ICLR, 2019.
Ari Holtzman, Jan Buys, Maxwell Forbes, and Yejin Choi. The curious case of neural text degener- ation. In arXiv cs. CL, 2019.
Harold Hotelling. Relations between two sets of variates. In Biometrika, 1936.
Diederik P. Kingma and Jimmy Lei Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
Diederik Kingma and Max Welling. Auto-encoding variational bayes. In arXiv Mt, 2013.
Wei-Jen Ko, Greg Durrett, and Junyi Jessy Li. Linguistically-informed specificity and semantic plausibility for dialogue generation. In NAACI,, 2019.
Sachin Kumar and Yulia Tsvetkov. Von mises-fisher loss for training sequence to sequence models with continuous outputs. In ICLR, 2019.
Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. A diversity-promotin objective function for neural conversation models. In NAACL, 2016a.
Jiwei Li, Will Monroe, Alan Ritter, Michel Galley, Jianfeng Gao, and Dan Jurafsky. Deep inforcement learning for dialogue generation. In EMNLP, 2016b.
Yanran Li, Hui Su. Xiaoyu Shen. Wenjie Li, Ziqiang Cao, and Shuzi Niu. Dailydialog: A manually labelled multi-turn dialogue dataset. In IJCNLP, 2017.

---

Under review as a conference paper at ICLR 2020
Pierre Lison and Serge Bibauw. Not all di-alogues are created equal: Instance weighting forneual conversational models. In SIGDIAL, 2017.
Chia-Wei Liu, Ryan Lowe, Iulian V. Serban, Michael Noseworthy, Laurent Charlin, and Joelle Pineau. How not to evaluate your dialogue system: An empirical study of unsupervised eval- uation metrics for dialogue response generation. In EMNLP, 2016.
Yahui Liu, Wei Bi, Jun Gao, Xiaojiang Liu, Jian Yao, and Shuming Shi. Towards less generic responses in neural conversation models: statistical re-weighting method. In EMNLP, 2018.
Liangchen Luo, Jingjing Xu, Junyang Lin, Qi Zeng, and Xu Sun. An auto-encoder matching model for learning utterance-level semantic dependency in dialogue generation. In EMNLP, 2018.
Thang Luong, Hieu Pham, and Christopher D. Manning. Effective approaches to attention-based neural machine translation. In EMNLP, 2015.
Neil Mallinar and Corbin Rosset. Deep canonically correlated Istms. In Undergraduate honors thesis, Johns Hopkins University, 2018.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. The measurement of textual co- herence with latent semantic analysis. In ACL, 2002.
Lisong Qiu, Juntao Li, Wei Bi, Dongyan Zhao, and Rui Yan. Are training samples correlated?
learning to generate dialogue responses with multiple references. In ACL, 2019.
Guy Rotman, Ivan Vulic, and Roi Reichart. Bridging languages through images with deep partial canonical correlation analysis. In ACL, 2018
Iulian V. Serban, Alessandro Sordoni, Yoshua Bengio, Aaron Courville, and Joelle Pineau. Building end-to-end dialogue systems using generative hierarchical neural network models. In AAAI, 2016.
Louis Shao, Stephan Gouws, Denny Britz, Anna Goldie, Brian Strope, and Ray kurzwei. Gener- ating high-quality and informative conversation responses with gepuence-to-sqqeeeee models. In EMNLP, 2017.
Alessandro Sordoni, Michel Galley, Michael Auli, Chris Brockett, Yangfeng Ji, Margaret Mitchell, Jian-Yun Nie, Jianfeng Gao, and Bill Dolan. A neural network approach to context-sensitive generation of conversational responses. In NAACL, 2015.
Ilya Sutskever, Oriol Vinyals, and Quoe V. Le. Sequence to sequence learning with neural networks. In NeurIPS, 2014.
Eric Tzeng, Judy Hoffmann, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In CVPR, 2017.
Laurens van der Maaten. Learning a parametric embedding by preserving local structure. In AIS- TATS, 2009.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurlPS, 2017.
Oriol Vinyals and Quoc V. Le. A neural conversational model. In ICML, 2015.
Weiran Wang, Raman Arora, Karen Livescu, and Jeff Bilmes. On deep multi-view representation learning. In ICML, 2015.
Thomas Wolf, Victor Sanh, Julien Chaumond, and Clement Delangue. Transfertransfo: A transfer learning approach for neural network based conversational agents. In NeurIPS CAI Workshop, 2018.
Xinnuo Xu, Ondej Duek, Ioannis Konstas, and Verena Rieser. Better conversations by modeling, filtering, and optimizing for coherence and diversity. In EMNLP, 2018.
Fei Yan and Krystian Mikolajezyk. Deep correlation for matching images and text. In CVPR, 2015.
10

---

Under review as a conference paper at ICLR 2020
Chih-Kuan Yeh, Wei-Chieh Wu, Wei-Jen Ko, and Yu Yu-Chiang Frank Wang. Learning deep latent spaces for multi-label classification. AAAI, 2017.
Ruqing Zhang, Jiafeng Guo, Yixing Fan, Yanyan Lan, Jun Xu, and Xueqi Cheng. Learning to control the specificity neural response generation. In ACL, 2018a.
Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. Per- sonalizing dialogue agents: have a dog, do you have pets too? In ACL, 2018b.
Ganbin Zhou, Ping Luo, Rongyu Cao, Fen Lin, Bo Chen, and Qing He. Mechanism-awwareeeurti machine for dialogue response generation. In AAAI, 2017.

---

Under review as a conference paper at ICLR 2020
A
EXAMPLE MODEL RESPONSES
PersonaChat
Prompt
wow love to see traditional dance
MLE++heam search
do you have favorite color color blue.
MMI
yes,
Spacefusion
am not sure am college student.
Ours
its fun i've always wanted to for dance class.
Prompt
do you to school at all
MLE+beam search
not have a job i want to be a doctor sometimes but i do like photography
MMI
Spacefusion
i do not i am college student.
Ours
don't i'm studying to go college.
Prompt
i live in nebraska i don't ton't think have ever been to ohio before.
MLE+beam search
oh wow what do you do for living ?
MMI
oh do you do volunteer work oh wow love to go to beach..
Spacefusion
Ours
wow i have been there.. do you have a favorite place ?
DailyDialog
Prompt
Can you operate computers skillfully? ?
MLE+beam search
Yes am
MMI
Yes have special job
Spacefusion
Yes can have any other Word 2003 and can live in other areas
OutstAntentoon
No. am not familiar with both Java and C Programming Languages..
Prompt
Exporters must ensure that their product satisfies customers "nceds. wants and likes
MLE+hhem search
Do you any other questions
MMI
We have to buy a new set
Spacefusion
I am calling to see you But I've got new customer here. But I don't want to be aware of their house
OurstAttention
That means that have commission about selling electronics. The prices
perfect for you to promote the commission and the prices are cheaper..
Prompt
closes 7:00 on Sundays
MLE+beam search
You're welcome can help you.
MMI
How do you get that
Spacefusion
Nice to sec you The alarm isn't a fool.
OurstAAtenntin
Hold on have some other classes available.
12


Reviews:
[{"review_id": "r1xLvGk6Yr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a method for generating dialogue responses that are not generic. The authors propose maximizing correlation between latent representations of responses and prompts, such that the response is encouraged to contain information relevant to the prompt. The method consists of two parts, the first part predicts a latent representation of the response while the second part decodes a response from the latent representation.\n\nThe idea is conceptually intuitive, however I have some questions regarding the details:\n- It's surprising that this works at all, since the inference representations seem to come from an entirely different distribution than in training (e.g. Figure 2)\n- Given that this works, I'd like to see an ablation in which Y is removed (e.g. we go from X and Yu to Gy), so as to evaluate the utility of the \"correlated response encoder\". This is effectively what's happening during inference anyway.\n- I am skeptical about the early stopping criteria, which involve human evaluation. I also don't find the justification convincing (\"there is no obvious automatic metric\") --- why don't we just use the automatic metrics listed in 4.3?\n- Can the authors please comment on the choice of evaluation? It seems like the evaluation metrics are different from what's typically used (e.g. see http://convai.io for PersonaChat). Why don't the authors use these metrics instead? This, combined with the early stopping criterion, make it difficult to evaluate the effectiveness of the proposed method in light of prior work.\n\nIn terms of the writing, I think the authors could do with more polish:\n- there are many generic claims which can be more detailed (e.g. abstract: enables our model to view semantically related responses collectively; achieving [...] better coherence)\n- some terms are not clearly defined (e.g. semantics, one-to-many task vs. one-to-one task, canonical correlated feature extractor)\n- the introduction is overly verbose, much of it should be in the methodology section. there should be an overview of experimental results and analysis in the intro.\n- In 3.3, it doesn't seem like you \"enforce\" a normal distribution. If I understand correctly, you use KL to encourage a normal distribution. ", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I did not assess the derivations or theory."}, "extra_text": {}}, {"review_id": "Bkl5T12pFB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "[Summary]\nThis paper proposes a dialogue generation model that learns a semantic latent space. In this paper, the authors tackle the generic response issue. The proposed model consists of three (prompt/correlated response/uncorrelated response) encoders and a decoder. The authors add an objective that maximizes the correlation between prompt and response features to the existing autoencoder structure, which prevents the generation of generic responses. \n\n[Comments]\nThe paper is well-organized and clearly written but has some weakness:\n- There seems to be a need for more detail on comparisons of papers that previously tackle the generic response problem.\n- In addition to the proposed main architecture, there are so many things combined (ex. denoising, attention). It seems like it's doing well with additional things rather than the model structure. In fact, the results of the model without denoising in the ablation study show poor performance.\n- The test pool for human-evaluation is too small (only 3 workers). It seems necessary to perform the human-evaluation on a larger number of participants. I think that it is a very important issue since the generic response addressed as the main problem in this paper can only be evaluated by human-evaluation.\n\n[Questions]\n- What are the criteria for specificity and coherence used in human-evaluation? If there are some criteria provided to the participants, it would be better to include a description together.\n- Can you provide an intuitive or theoretical explanation of denoising (to add a <unk>)?", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "B1loCsNZqS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "1. Summary: The authors proposed to alleviate the generic response problem in open-domain dialog generation by generating a response to a prompt from a semantic latent vector. This vector needs to be located close to the latent vector of the corresponding prompt. To this end, they employ canonical correlation analysis and auto-encoder to learn the mapping from a sequence of text to a semantic latent vector. To model the variations and topic shifts that may happen in responses, they also use a separate intermediate vector in the auto-encoder of generating responses.\n2. Overall assessment: While this paper is quite fun to read, it is not innovative enough and it lacks some critical experiments and error analysis to be accepted this time. I'll elaborate on these problems in my comments below. \n3. Strengths:\n3.1 This paper is well written. The motivation and the main idea are well explained and pretty easy to understand. Although there are some details missing, it doesn't prevent readers from understanding and enjoying this paper.\n3.2 The use of human evaluation is a great plus. Subtle characteristics of text, such as readability, coherence, and specificity cannot be well justified by automatic evaluation. Human evaluation is a must for these aspects. It's great to see the authors include this in this paper.\n4. Weakness and questions:\n4.1 It seems the model in this paper can be connected to adversarial learning from some angles. It would be great to see such analysis in this paper.\n4.2 The experiments in this paper do not look thorough enough. There are many more things should be included, such as error analysis, comparisons over different variations of the proposed model and so on. What if we remove $Y_u$ in both training and inference but keep the overall model the same? How important is $Y_u$ and what's its effect? Many such questions haven't been answered in this paper.\n4.3 Embedding average cosine similarity seems to be too simple to be used for evaluation as it omits the contextual information and semantic meaning of a sentence.\n4.4 It would be interesting to see the Dist-1 and Dist-2 scores of the gold standard responses. It's a good reference to let us know ho diversified the real responses are.\n4.5 I'd like to see some explanation of how the raters rate specificity and coherence. What are the standards they use? What are the instructions the author give to them?\n4.6 Are improvements significant in Table 1? It seems the proposed model is not performing very stably over different metrics. Some anlysis on this would be great.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a response generation approach that aims to tackle the generic response problem. The approach is learning a latent semantic space by maximizing the correlation between features extracted from prompts and responses. The reviewers were concerned about the lack of comparison with previous papers tackling the same problem, and did not change their decision (i.e., were not convinced) even after the rebuttal. Hence, I suggest a reject for this paper.
Decision: Reject