SlowGuess/ACL_2020_OCR huggingface dataset:
For all methods, we set word embedding dimension to 300 with the pre-trained GloVe (Pennington et al., 2014). Following Qin et al. (2019), our vocabulary contains top 30k frequent tokens. We use bi-LSTMs with the hidden dimensions of 512 and the dropout rate of 0.4 in our encoders. We optimize models by Adam with an initial learning rate of 0.0005 and the batch size of 32. All conversation contexts/responses/documents are truncated to have the maximum length of 30/30/500. For training, we set  $\lambda$  as 1 in the loss of student models after tuning. For inference, we apply a top- $k$  random sampling decoding (Edunov et al., 2018) with  $k=20$ . The validation set is for early stopping. Aforementioned implementation details can be found in our codes<sup>1</sup>.
1. Seq2Seq (Sutskever et al., 2014). The standard Seq2Seq model that leverages only the conversational context for response generation.
2. MemNet (Ghazvininejad et al., 2018). A knowledge-grounded conversation model that uses a memory network to store knowledge facts.
3. GLKS (Ren et al., 2020). It applies a global knowledge selector in encoding and a local selector on every decoding step.
4. Conversation with Machine Reading (CMR) (Qin et al., 2019). The state-of-the-art model on the CbR task, which is also our base model (Sec 3.2). Here, we use the full model of CMR (called  $\mathrm{CMR + w}$  in (Qin et al., 2019)), since the full model outperforms other CMR's variants on most metrics. We further apply the copy mechanism (See et al., 2017) to this base model ( $\mathbf{CMR} + \mathbf{Copy}$ ).
5. Four variants of our proposed models: RAM_T denotes our Response-Anticipated Memory-based model with RTI, and RAM_T+Copy denotes its
copy version. RAM_P and RAM_P+Copy denote our model with RPI and its copy variant.
Following all metrics in Qin et al. (2019), we evaluate all methods by both automatic and human evaluations. For automatic evaluations, we evaluate the responses in three aspects:
1. Appropriateness.
We use three metrics to evaluate the overall quality of a response: BLEU-4 (Papineni et al., 2002), Meteor (Banerjee and Lavie, 2005), and NIST (Doddington, 2002). NIST is a variant of BLEU that measures n-gram precision weighted by the informativeness of n-grams.
2. Grounding. We measure the relevance between documents and generated responses to reveal the effectiveness of responses exploiting the document information. We define #overlap as the number of non-stopword tokens in both the document  $D$  and the generated response  $\hat{R}$  but not in contexts  $X$ . We calculate the precision P and recall R as
$$
\# \text {o v e r l a p} = | (D \cap \hat {R}) \backslash X \backslash S |, \tag {13}
$$
$$
\mathrm {P} = \frac {\# \text {o v e r l a p}}{| \hat {R} \backslash S |}, \mathrm {R} = \frac {\# \text {o v e r l a p}}{| D \backslash S |}, \tag {14}
$$
where  $S$  denotes the stopwords list.  $F1$  is the harmonic mean of precision  $\mathbf{P}$  and recall  $\mathbf{R}$ .
We further propose to measure the effectiveness of exploiting the document information considering the ground-truth. In this way, we evaluate how many ground-truth information models can exploit from the document. We define  $\# \mathrm{overlap}_{GT}$  as the number of non-stopword tokens in the document  $D$ , the generated response  $\hat{R}$  and the ground-truth  $R$  but not in contexts  $X$ . The precision and recall
$^{1}$ https://github.com/tianzhiliang/RAM4CbR
Appropriateness	Grounding	Informativeness	Len
NIST	BLEU	Meteor	P	R	F1	PGT	RGT	F1GT	Ent4	Dist1	Dist2
RAM_T	2.510	1.43%	7.74%	4.46%	0.26%	0.49%	1.04%	0.08%	0.15%	9.900	0.053	0.290	15.1
RAM_P	2.353	1.40%	7.59%	3.89%	0.21%	0.41%	0.97%	0.07%	0.13%	9.891	0.049	0.279	14.9
RAM_T (Teacher)	2.539	1.43%	7.85%	4.47%	0.26%	0.49%	1.05%	0.08%	0.15%	9.904	0.053	0.290	15.1
RAM_P (Teacher)	2.551	1.47%	7.88%	4.56%	0.27%	0.50%	0.99%	0.08%	0.16%	9.900	0.053	0.287	15.1
RAM_T_Binary	2.560	1.63%	7.91%	3.75%	0.21%	0.40%	0.87%	0.07%	0.12%	9.890	0.052	0.283	15.1
RAM_P_Binary	2.403	1.51%	7.63%	3.55%	0.18%	0.38%	0.85%	0.07%	0.12%	9.887	0.046	0.274	14.6
Table 2: Performance comparison on our model variants. Line1&2: our models trained by the full teacher-student framework. Line3&4: our models trained with the teacher model only. Line5&6: our models with binary weight matrices. Bold values are the best results among the first four lines; underlines mark the best ones among the first two and last two lines.
H-Appr	H-Ground	H-Info
Human	2.986	2.521	3.007
Seq2Seq	1.902	1.564	2.040
MemNet	1.872	1.574	2.105
GLKS	2.073	1.593	2.071
CMR	2.188	1.678	2.219
CMR+Copy	2.063	1.773	2.075
RAM_T	2.259	1.714	2.312
RAM_P	2.213	1.682	2.231
RAM_T+Copy	2.109	1.861	2.240
RAM_P+Copy	2.114	1.775	2.115
are as following,
$$
\# \text {o v e r l a p} _ {G T} = | (D \cap \hat {R} \cap R) \backslash X \backslash S |, \tag {15}
$$
$$
\mathrm {P} _ {G T} = \frac {\# \text {o v e r l a p} _ {G T}}{| \hat {R} \backslash S |}, \mathrm {R} _ {G T} = \frac {\# \text {o v e r l a p} _ {G T}}{| D \backslash S |}, \tag {16}
$$
where  $F1_{GT}$  is the harmonic mean of precision  $P_{GT}$  and recall  $R_{GT}$ .
3. Informativeness. Ent-  $n$  (Mou et al., 2016) measures responses' informativeness with the entropy of the n-gram count distribution. Dist-  $n$  (Li et al., 2016a) evaluates the diversity of responses via the proportion of unique n-grams among all responses.
For human evaluations, we hire five annotators from a commercial annotation company to evaluate 200 randomly selected test samples, and results from different models are shuffled. The annotators evaluate on a 5-point scale in three aspects: overall quality (H-Appr), relevance with documents (H-Ground), and informativeness (H-Info).
In this part, we first show the performance of all methods in Sec 5.1. Then, we validate the effectiveness of response anticipation on CbR in Sec 5.2 by comparing the top similar tokens with the response using their representations in the memory. We also compare more variants of our model
Table 3: Human annotation results.
Top10 tokens	Top20 tokens
Emb-M	Emb-B	Emb-M	Emb-B
CMR	0.482	0.356	0.571	0.420
RAM_T_Soft	0.745	0.520	0.867	0.616
RAM_P_Soft	0.518	0.441	0.634	0.493
Table 4: Similarity between important document tokens picked by gold responses and the accumulated attention weights in the models.
in Sec 5.3, including the token importance versus pairwise importance, and each method with continuous weights versus their variants with binary weights. At last, we conduct a case study in Sec 5.4.
Results of all models on automatic and human evaluations are shown in Table 1 and Table 3. MemNet outperforms Seq2Seq on most metrics, which validates that it is important to utilize document information in CbR. However, MemNet only slightly improves on Grounding. Both GLKS and CMR outperform MemNet on most metrics, indicating that it matters how to construct the document memory used in conversation models for CbR. Compared with CMR, CMR+Copy is more competitive on Grounding but weaker on other metrics.
Our proposed models outperform other competing methods on all metrics, including automatic and human evaluations. For models without the copy mechanism, RAM_T performs the best. For models with copy, RAM_T+Copy and RAM_P+Copy excel CMR+Copy on most metrics. Overall, our proposed strategy works well on both the model with and without copy mechanism. We will compare RAM_T and RAM_P in details in Sec 5.3.
5.2 Effectiveness of Response Anticipation
In this section, we investigate whether anticipating response contributes to building a better document memory. We first calculate the semantic similarity between each document token and the response us
Case 1	Case 2
Document	fa premier league was the fourth season of the competition, since its formation in 1992. due to the decision to reduce the number of clubs in the premier league from 22 to 20, only two clubs were promoted instead of the usual three , middlesbrough and bolton wanderers.	darko milicic. darko milicic ( serbian cyrillic. serbian pronunciation. born june 20, 1985) is a serbian former professional basketball player . he is 2.13 m ( 7 ft 0 in ) , and played center .
Context	at least we qualified for a european competition we're capable of winning now	that darko milicic, who was drafted 2nd overall in the 2003.nba draft is currently an apple farmer in serbia.
Seq2Seq	i do n’t really need to take a time and a bit more and i think he ’s saying it was n’t in an accident .	he is so happy when i ’m not in 0ame universe as the first time.
MemNet	i am not saying i was a kid .	you know what ? is there anything in a book ?
GLKS	i have a pretty good chance of being the first person i know !	i think a lot of people are still able to get in a hour
CMR	well , at what point do you think about how they are getting play for ?	i remember my comment on my post. and i am not sure why but my point is that he has the best score that will always get a good
CMR+Copy	they are , but not the same as the first one .	he also played the same game, is there title to be a team
RAM_T	i think we have num teams playing the premier league team. in my opi-nion he was not a good player, but the united kingdom was in the europa	i love him the next time i play for num years, so that is probably the only option i understand.
RAM_T+Copy	they are the best player in the world.	he also played the second one, but that doesn’t mean it was num years ago.
Figure 3: Test samples with generated responses of all models. A colored word in the responses indicate that it has similar words with documents or contexts, which are marked in the same color.
Figure 4: The accumulated attention weights of documents tokens on RAM_T and CMR on Case 1 in Fig. 3. We only show top tokens in both methods here.
ing their Glove embeddings, and select top  $K$  document tokens. Next, we accumulate the attention weights of each token in all attention distributions in the self-attention weights  $\mathbf{A}$  in Eq. 1, i.e. summation over each column of  $\mathbf{A}$ . Then we select the top  $K$  tokens according to their accumulated attention weights. Here, we set  $K = 10, 20$ . We apply metrics in Liu et al. (2016) to calculate the similarity of two token sets extracted above, including maximal tokens-tokens embedding similarity (Emb-M) and bag-of-word embedding similarity (Emb-B). A higher similarity score indicates more response information anticipated by the model. Table 4 shows the results of our two models RAM_T and RAM_P as well as CMR (We use the original self-attention matrix  $\mathbf{A}$  for the above calculation for CMR). Results demonstrate that our model is able to output more response-anticipated self-attention distributions, which benefits generating a response close to the ground truth.
5.3 Analysis on Different Model Variants
Token importance vs Pairwise importance. We compare our model variants with different strategies to construct the response-aware/anticipated weight matrix, i.e. RAM_T (Eq. 5) and RAM_P (Eq. 9). We not only compare their overall performance by the teacher-student framework (Eq. 8 & 12) but also the teacher model only (Eq. 12).
The first four rows in Table 2 shows the results. We have an interesting finding that RAM_P underperforms RAM_T in the full teacher-student framework, but outperforms RAM_T on the mode with teacher model only on most metrics. This result is actually consistent with our discussion in Sec 3.5 that RAM_P has a higher capacity to carry more information in G, thus its teacher model yields better performance. However, for the student model, RAM_P is more difficult to converge to a good local optimum due to more parameters to be estimated, resulting in that its overall performance may not exceed that of RAM_T.
Continuous weight vs Binary weight. We also compare the model variants with continuous weight (Eq. 5) and binary weight (Eq. 6). The last two rows in Table 2 give the results of the variants of RAM_T and RAM_P with a binary G. We can see that both RAM_T and RAM_P with a binary weight matrix performs better on Appropriateness, which means a sparse G on the attention matrix can help select more concise information to construct the memory. Nevertheless, models with a continuous
weight matrix can generate more informative responses owing to their ability to access broader and more information from the document.
Table 3 shows two test samples with generated responses of all models. For Case 1, Seq2Seq and MemNet cannot generate responses relevant to either the document or context. CMR catches the topic "sports", while GLKS and  $\mathrm{CMR + Copy}$  use "first person" and "first one" to reflect "only two" mentioned in the document. The response of RAM_T contains information related to both document ("num teams" and "premier league") and context ("europa"). RAM_T+Copy is also highly relevant to the document and the context, and copies "player" from the document. For Case 2, the first four methods have little relation to the document or the context. CMR+Copy mentions "played". Our models mention "played" and "num years". By examining the cases, our method shows promising improvements over existing methods. However, generation on the CbR task is very challenging and there is still a huge space to improve.
We plot the accumulated attention weights of RAM_T and CMR as in Sec 5.2 of the document tokens on Case 1. Fig. 4 shows that RAM_T's attention highlights "num" and "premier", and thus it generates the above words in its response.
Focusing on the CbR task, we propose a novel response-anticipated document memory to exploit and memorize the document information that is important in response generation. We construct the response-anticipated memory by a teacher-student framework. The teacher accesses the response and learns a response-aware weight matrix; the student learns to estimate the weight matrix in the teacher model and construct the response-anticipated document memory. We verify our model on both automatic and human evaluations and experimental results show our model obtains the state-of-the-art performance on the CbR task.
Research on this paper was supported by Hong Kong Research Grants Council under grants 16202118 and 16212516 and Tencent AI Lab Rhino-Bird Focused Research Program (No. GF202035).
Shubham Agarwal, Ondrej Dusek, Ioannis Konstas, and Verena Rieser. 2018. A knowledge-grounded multimodal search-based conversational agent. In EMNLP, pages 59-66.
Satanjeev Banerjee and Alon Lavie. 2005. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Workshop on ACL, pages 65-72.
Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2018. Wizard of wikipedia: Knowledge-powered conversational agents. In ICLR.
George Doddington. 2002. Automatic evaluation of machine translation quality using n-gram cooccurrence statistics. In HLTCon, pages 138-145.
Sergey Edunov, Myle Ott, Michael Auli, and David Grangier. 2018. Understanding back-translation at scale. In EMNLP, pages 489-500.
Marjan Ghazvininejad, Chris Brockett, Ming-Wei Chang, Bill Dolan, Jianfeng Gao, Wen-tau Yih, and Michel Galley. 2018. A knowledge-grounded neural conversation model. In AAAI, pages 5110-5117.
Eric Jang, Shixiang Gu, and Ben Poole. 2016. Categorical reparameterization with gumbel-softmax. In ICLR.
Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016a. A diversity-promoting objective function for neural conversation models. In *NAACL*, pages 110-119.
Jiwei Li, Will Monroe, Alan Ritter, Dan Jurafsky, Michel Galley, and Jianfeng Gao. 2016b. Deep reinforcement learning for dialogue generation. In EMNLP, pages 1192-1202.
Chia-Wei Liu, Ryan Lowe, Iulian Vlad Serban, Mike Noseworthy, Laurent Charlin, and Joelle Pineau. 2016. How not to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. In EMNLP, pages 2122-2132.
Xiaodong Liu, Yelong Shen, Kevin Duh, and Jianfeng Gao. 2018. Stochastic answer networks for machine reading comprehension. In ACL, pages 1694-1704.
Chuan Meng, Pengjie Ren, Zhumin Chen, Christof Monz, Jun Ma, and Maarten de Rijke. 2020. Refnet: A reference-aware network for background based conversation. In AAAI.
Nikita Moghe, Siddhartha Arora, Suman Banerjee, and Mitesh M Khapra. 2018. Towards exploiting background knowledge for building conversation systems. In EMNLP, pages 2322-2332.
Lili Mou, Yiping Song, Rui Yan, Ge Li, Lu Zhang, and Zhi Jin. 2016. Sequence to backward and forward sequences: A content-introducing approach to generative short-text conversation. In COLING, pages 3349-3358.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In ACL, pages 311-318.
Prasanna Parthasarathi and Joelle Pineau. 2018. Extending neural generative conversational model using external knowledge sources. In EMNLP, pages 690-695.
Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In EMNLP, pages 1532-1543.
Lianhui Qin, Michel Galley, Chris Brockett, Xiaodong Liu, Xiang Gao, Bill Dolan, Yejin Choi, and Jianfeng Gao. 2019. Conversing by reading: Contentful neural conversation with on-demand machine reading. In ACL, pages 5427-5436.
Pengjie Ren, Zhumin Chen, Christof Monz, Jun Ma, and Maarten de Rijke. 2020. Thinking globally, acting locally: Distantly supervised global-to-local knowledge selection for background based conversation. In AAAI.
Abigail See, Peter J Liu, and Christopher D Manning. 2017. Get to the point: Summarization with pointer-generator networks. In ACL, pages 1073-1083.
Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2017. Bidirectional attention flow for machine comprehension. In ICLR.
Iulian V Serban, Alessandro Sordoni, Yoshua Bengio, Aaron Courville, and Joelle Pineau. 2016. Building end-to-end dialogue systems using generative hierarchical neural network models. In AAAI, pages 3776-3783.
Iulian Vlad Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron Courville, and Yoshua Bengio. 2017. A hierarchical latent variable encoder-decoder model for generating dialogues. In AAAI, pages 3295-3301.
Lifeng Shang, Zhengdong Lu, and Hang Li. 2015. Neural responding machine for short-text conversation. In ACL, pages 1577-1586.
Yiping Song, Cheng-Te Li, Jian-Yun Nie, Ming Zhang, Dongyan Zhao, and Rui Yan. 2018. An ensemble of retrieval-based and generation-based human-computer conversation systems. In *IJCAI*, pages 4382-4388.
Yiping Song, Zhiliang Tian, Dongyan Zhao, Ming Zhang, and Rui Yan. 2017. Diversifying neural conversation model with maximal marginal relevance. In IJCNLP, pages 169-174.
Sainbayar Sukhbaatar, Jason Weston, Rob Fergus, et al. 2015. End-to-end memory networks. In NIPS, pages 2440-2448.
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In NIPS, pages 3104-3112.
Zhiliang Tian, Wei Bi, Xiaopeng Li, and Nevin L Zhang. 2019. Learning to abstract for memory-augmented conversational response generation. In ACL, pages 3816-3825.
Zhiliang Tian, Rui Yan, Lili Mou, Yiping Song, Yansong Feng, and Dongyan Zhao. 2017. How to make context more useful? an empirical study on context-aware neural conversational models. In ACL, pages 231-236.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NLPS, pages 5998-6008.
Petar Velicković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2018. Graph attention networks. In ICLR.
Ashwin K Vijayakumar, Michael Cogswell, Ramprasath R Selvaraju, Qing Sun, Stefan Lee, David Crandall, and Dhruv Batra. 2018. Diverse beam search: Decoding diverse solutions from neural sequence models. In AAAI.
Wenquan Wu, Zhen Guo, Xiangyang Zhou, Hua Wu, Xiyuan Zhang, Rongzhong Lian, and Haifeng Wang. 2019a. Proactive human-machine conversation with explicit conversation goal. In ACL, pages 3794-3804.
Yu Wu, Furu Wei, Shaohan Huang, Yunli Wang, Zhoujun Li, and Ming Zhou. 2019b. Response generation by context-aware prototype editing. In AAAI, pages 7281-7288.
Chen Xing, Wei Wu, Yu Wu, Jie Liu, Yalou Huang, Ming Zhou, and Wei-Ying Ma. 2017. Topic aware neural response generation. In AAAI.
Liu Yang, Junjie Hu, Minghui Qiu, Chen Qu, Jianfeng Gao, W Bruce Croft, Xiaodong Liu, Yelong Shen, and Jingjing Liu. 2019. A hybrid retrieval-generation neural conversation model. In CIKM, pages 1341-1350.
Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. 2017. Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. In ACL, pages 654-664.
Hao Zhou, Tom Young, Minlie Huang, Haizhou Zhao, Jingfang Xu, and Xiaoyan Zhu. 2018. Commonsense knowledge aware conversation generation with graph attention. In *IJCAI*, pages 4623-4629.