SlowGuess/NAACL_2022_OCR huggingface dataset:
Training Data	Models	Wizard Seen	Wizard Unseen	CMU_DOG
PPL	F1	D-1	D-2	PPL	F1	D-1	D-2	PPL	F1	D-1	D-2
Reddit Corpus	BART	40.1	18.4	0.076	0.355	42.9	18.4	0.049	0.237	75.8	9.8	0.021	0.131
ZRKG	41.1	18.9	0.055	0.246	42.7	18.8	0.037	0.179	53.8	12.2	0.015	0.094
Our Model	35.9	19.3	0.082	0.383	38.4	19.2	0.060	0.292	60.4	12.2	0.028	0.186
Reddit Corpus + 10% annotated data	BART	32.7	18.9	0.073	0.357	35.0	18.8	0.049	0.235	49.5	10.1	0.019	0.110
ZRKG	29.1	19.1	0.072	0.309	31.6	18.9	0.048	0.209	38.0	13.7	0.010	0.062
Our Model	28.6	20.4	0.073	0.366	30.7	20.0	0.052	0.270	40.8	14.4	0.015	0.122
Table 2: Automatic evaluation results. Numbers in bold mean that the improvement to the best performing baseline is statistically significant (t-test with  $p$ -value  $< 0.05$ ).
Wikipedia (Wizard) (Dinan et al., 2019) and CMU Document Grounded Conversations (CMU_DOG) (Zhou et al., 2018c). We choose the Reddit Corpus published by (Li et al., 2020) as  $\mathcal{D}$  for pre-training. Since it is abundant in expression style as a corpus from online forum, the two latent variables could be well initialized. We use part of the training data of Wizard and CMU_DOG as  $D_{sty}$  respectively, for these two datasets are distinctive in expression style and differ from each other. The dialogues in CMU_DOG tend to be causal and short, with most utterances irrelevant to knowledge while the responses in Wizard are usually long and knowledgeable, as some phrases are directly extracted from wiki articles.
More details about the datasets are described in the appendix.
In this paper, we mainly consider two experimental setups, corresponding to the two aspects of knowledge expression. To explore how our model can be used to control the distribution of different kinds of segments (knowledge-related and knowledge-irrelevant), we first train the model on the Reddit Corpus and then fine-tune it on a small amount of examples in Wizard and CMU_DOG, respectively. To verify whether our model can generate the knowledge-related segments in the desired style, we still train the model on the Reddit Corpus, and use a style tag to control the generation process. In this experimental setup, we are primarily concerned with generating with two kinds of styles, positive and negative, where  $z_{t} = 2 \cdot \min(1, z_{t})$  tells the model to generate a response in positive sentiment and  $z_{t} = 3 \cdot \min(1, z_{t})$  is for response in negative sentiment.
Evaluation Metrics. Following Dinan et al. (2019), we choose PPL and unigram F1 as the
metrics to evaluate the appropriateness. We further use Distinct-1/2 (D-1/2), which are calculated as ratios of distinct unigrams and bigrams in responses respectively, to evaluate the distinctness. We also employ classification accuracy as the evaluation metrics for style control experiments. Due to space limitation, we provide automatic evaluation results on more metrics (i.e., BLEU-1, METEOR, and ROUGE-L) in the appendix.
To further verify whether our model could learn structure style and content style, we randomly sample 300 examples from Test Seen of Wizard, and the test set of CMU_DOG respectively, and recruit 6 well-educated native speakers to do qualitative analysis on the responses generated by our model and all baselines. The annotators need to judge the quality of the responses from four aspects (i.e., fluency, context coherence, knowledge relevance and style consistency), and assign a score from  $\{0,1,2\}$  (representing "bad", "fair" and "good" respectively) to each response for each aspect. The agreement among all annotators is measured via Fleiss' kappa (Fleiss, 1971). More details about the setup of human evaluation as well as the results on learning content style are provided in the appendix.
For the exploration of structure style, we select the following models as baselines: (1) BART (Lewis et al., 2020): a model that achieves state-of-the-art performance on various text generation tasks. Note that our model degrades into BART once we remove the module indicator Z and the boundary indicator M; (2) Zero-resource Knowledge-grounded Conversation (ZRKGC) (Li et al., 2020)<sup>6</sup>: a model that is based on UniLM (Dong et al., 2019) and optimized with Generalized EM method.
For the content style, we consider the following models as baselines: (1) Emotional Chatting
<sup>5</sup>We exploit Roberta trained on the SST-2 training set (Socher et al., 2013) as the evaluator.
$^{6}$ https://github.com/nlpxucan/ZRKGC
4We provide the evaluation results of training on the whole Wizard and CMU_DOG in the appendix.
Models	Wizard Seen	CMU_DOG
Fluency	Context Coherence	Knowledge Relevance	Style Consistency	Kappa	Fluency	Context Coherence	Knowledge Relevance	Style Consistency	Kappa
BART	1.68	1.56	1.52	1.34	0.64	1.62	1.57	1.55	1.31	0.63
ZRKGCT	1.62	1.59	1.55	1.36	0.65	1.61	1.53	1.65	1.56	0.66
Our Model	1.71	1.64	1.66	1.77	0.60	1.61	1.66	1.63	1.76	0.74
Table 3: Human evaluation results on learning structure style.
Machine (ECM) (Zhou et al., 2018a)<sup>7</sup>: a model which can generate appropriate responses not only content-relevant but also emotional consistent; (2) variant of DialogGPT (Zhang et al., 2019b): we add a sentiment indicating token at the first of the sequence and explore whether such simple heuristics works for controlling knowledge expression; (3) CTRL (Keskar et al., 2019)<sup>8</sup>: a large-scale model trained on conditional codes to govern the style and content of generation.
Our model and all baselines are trained on the identical Reddit Corpus to maintain fairness.
4.4 Results on Learning Structure Style
In this section, we demonstrate the effectiveness of our segmentation-based generation framework in both low-resource setting and zero-resource setting and empirically verify that our model can learn structure style with a few annotated examples.
In zero-resource setting, we trained our model on the Reddit Corpus published by Li et al. (2020) and tested on Wizard and CMU_DOG respectively. Automatic evaluation results are shown in Table 2. It could be observed that: (1) our model significantly outperforms ZRKGC and BART on most metrics and achieves the new state-of-the-art performance on Wizard. It is impressive that our model exceeds BART in CMU_DOG especially since the proposed model degrades into BART without two sequential latent variables Z and M. The result serves as strong evidence for the effect of two latent variables, which enable the model to learn complex expression style in Reddit Corpus to handle flexible expression in CMU_DOG. By contrast, BART is far from satisfying with only a regular decoder. (2) our model exceeds ZRKGC significantly in terms of Distinct metrics, for ZRKGC mainly focuses on leveraging external knowledge sources for response generation, but falls short on expression diversity. In the low-resource setting, after training our model on the Reddit Corpus, we then fine-tune it with only  $10\%$  training size of Wizard and CMU_DOG
respectively (i.e.,  $\mathcal{D}_{sty}$  in Sec 3.1) to adjust  $p(z_{t})$  and  $p(m_{t})$  to a new structure style. When provided with only  $10\%$  training data, our model gets obvious improvement ( $\sim 1\%$  increase in F1) in contrast with BART ( $\sim 0.5\%$  increase in F1) and ZRKGC ( $\sim 0.2\%$  increase in F1), proving that the proposed model can learn more sophisticated structure style through quick adjustment on a specific dataset with little cost.[9]
Human Evaluation. Table 3 shows human evaluation results on learning structure style. It could be observed that: (1) our model is significantly superior to others on style consistency, indicating that the model can learn a consistent expression style with very little data. (2) our model has better performance on context coherence and knowledge relevance, tallying with its impressive performance in the low-resource scenario.
Fine-tuning with Limited Annotated Data. We first train the model on the Reddit Corpus and then fine-tune it with the amount of annotated data (e.g., Wizard and CMU_DOG) gradually increasing from  $2\%$  to  $10\%$ . To have a more intuitive understanding of the effects of latent variables Z and M, we compare the proposed model with BART, which generates the response with a single decoder. The evaluation results are shown in Figure 2. It can be concluded from the result that: (1) our model can learn the expression style of a particular dataset more efficiently. As the training data increases, our model has a more significant improvement in terms of the F1 metric; (2) our model performs better in meager resources since there is a considerable gap between our model and BART when the training data is close to  $0\%$ ; (3) the expression style of CMU_DOG can be learned with less data because the model has a significant change in performance after using  $2\%$  CMU_DOG training data.
Refashioning of Knowledge-related Segments. To know how our model adjusts to different
After trained with  $10\%$  annotated data, the diversity of our model decreases for it fits the specific expression style of Wizard or CMU_DOG.
7https://github.com/thu-coai/ecm
8 https://github.com/salesforce/ctrl
(a) Wizard Seen
(b) Wizard Unseen
Figure 2: Performance of different models wrt. training data size.
(c) CMU_DoG
Models	Wizard Seen	Wizard Unseen	CMU_DoG
positive	negative	positive	negative	positive	negative
F1	Acc	F1	Acc	F1	Acc	F1	Acc	F1	Acc	F1	Acc
ECM	10.5	55.8	10.2	60.7	10.1	55.7	10.1	57.6	7.6	41.5	8.3	55.4
DialoGPT	12.1	54.1	12.1	46.9	12.0	56.0	12.0	45.0	9.2	44.9	9.2	55.1
CTRL	15.3	71.9	14.9	55.3	14.9	75.0	14.6	52.3	9.3	70.2	9.2	61.7
Our Model	19.7	70.3	19.2	70.7	19.4	73.1	19.2	69.9	12.7	74.8	12.2	68.0
Table 4: Evaluation results on sentiment control. Numbers in bold mean that the improvement to the best performing baseline is statistically significant (t-test with  $p$ -value  $< 0.05$ ).
Figure 3: The effect of fine-tuning on different data.
datasets, we compare the knowledge-related segments before and after trained with annotated data from two aspects: (1) the average proportion of knowledge-related segments  $(pklg)$  in a sentence; (2) the average proportion of words belonging to knowledge-related segments  $(lklg)$ . The motivation behind is that the frequency and length of these two kinds of segments generally indicates how well the latent variable is learned to capture the knowledge expression structure. We identify these two kinds of segmentation by comparing their lexical similarities with the knowledge. Figure 3 reports the results. The results indicate that our model could learn the underlying structure style of both datasets, with the great difference of  $pklg$  and  $lklg$  before and after fine-tuning as evidence. After fine-tuning with Wizard data,  $pklg$  drops to 0.26 while the  $lklg$  grows up a bit, indicating that the knowledge-related segments generated by our model are fewer and longer, which tallies with the fact that the responses in Wizard are probably directly copied from background knowledge. How-
ever, after CMU.DoG data is fed to the model, both  $pklg$  and  $lklg$  shrink drastically, which agrees with the fact that crowd-sourcing workers converse more liberally online and the responses are less relevant to the background knowledge.
4.5 Results on Learning Content Style
We further investigate whether the proposed model could express knowledge with the desired sentiment. Specifically, we introduce two sets of style adapters to endow knowledge expression in two different sentiments, namely positive and negative. So in this scenario, it is required that responses are not only coherent with context but also limited in positive or negative sentiment. To apply ECM on knowledge-grounded conversation, we label the sentiment category for each response with a classifier pre-trained on the SST (Socher et al., 2013) training set. For DialogGPT, we similarly annotate each response with a sentiment category and append the sentiment token before the context tokens. The evaluation results are shown in Table 4. We can conclude that: (1) The proposed model outperforms all baseline models in terms of all metrics, which indicates that our model can control the sentiment of knowledge expression and guarantee high quality of the generated responses; (2) Simply adding a sentiment indicating token at the beginning of the sequence can not effectively control the style of knowledge expression, as the performance of DialogGPT on sentiment control is poor; (3) Although ECM is designed for sentiment con
trol, it still fails to perform well in this task, proving that sentiment control in the knowledge-grounded conversation is rather difficult. Besides, ECM can only control the sentiment of the whole response but is helpless to manage every knowledge-related segment at a fine-grained level.
We explore knowledge expression in knowledge-grounded conversation and break down the expression style of a response into the structure of the response (structure style) and the style of the content in each part (content style). We propose a variational segmentation-based generation model to discover the underlying expression style in response. Specifically, we introduce two latent variables to model these two aspects of expression style respectively and induce an evidence lower bound of the likelihood. Evaluation results on two benchmarks of the task indicate that our model can learn the structure style with little cost and generate responses in desired content style without any human-annotated data.
It's crucial for an open-domain dialogue system to be able to automatically detect and discover the underlying structural pattern of a sentence. With the ability to handle a variety of expression styles, whether positive or negative, serious or casual, our work suggests that we are getting closer to the goal of creating an artificial intelligent dialogue system that can freely communicate with humans, which is beyond the wildest dreams of most AI and NLP researchers. However, a detailed survey should be undertaken in advance to consider the immediate audience's and developers' interests, as well as any potential stakeholder groups.
Furthermore, knowledge-grounded dialogue systems have the potential to fabricate facts and distribute rumors and false information, particularly when the source of external background knowledge is unreliable. If the knowledge candidate set is contaminated by fake news, the response generated by the dialogue system is likely to suffer from the "hallucination" issue. Controlling the source of knowledge sentences, such as paragraphs extracted from the wiki, authoritative news sites, or authoritative product documents, is a necessary and practical strategy.
Thanks for the anonymous reviewers for their constructive comments. This work was supported by the National Key Research and Development Program of China (No. 2020AAA0106600), National Natural Science Foundation of China (NSFC Grant No. 62122089 and No. 61876196), and Beijing Outstanding Young Scientist Program (No. BJWZYJH012019100020098). Rui Yan is also supported by Beijing Academy of Artificial Intelligence (BAAI).
Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2019. Wizard of wikipedia: Knowledge-powered conversational agents. In ICLR.
Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, and Hsiao-Wuen Hon. 2019. Unified language model pre-training for natural language understanding and generation. In Advances in Neural Information Processing Systems, pages 13042-13054.
Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological bulletin, 76(5):378.
Tingchen Fu, Xueliang Zhao, Chongyang Tao, Ji-Rong Wen, and Rui Yan. 2022. There are a thousand hamlets in a thousand people's eyes: Enhancing knowledge-grounded dialogue with personal memory. arXiv preprint arXiv:2204.02624.
Matthew D Hoffman, David M Blei, Chong Wang, and John Paisley. 2013. Stochastic variational inference. Journal of Machine Learning Research, 14(5).
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for nlp. In International Conference on Machine Learning, pages 2790-2799. PMLR.
Po-Sen Huang, Chong Wang, Sitao Huang, Dengyong Zhou, and Li Deng. 2017. Towards neural phrase-based machine translation. arXiv preprint arXiv:1706.05565.
Bernd Huber, Daniel McDuff, Chris Brockett, Michel Galley, and Bill Dolan. 2018. Emotional dialogue generation using image-grounded language models. In CHI, page 277. ACM.
Eric Jang, Shixiang Gu, and Ben Poole. 2016. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144.
Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher. 2019. Ctrl: A conditional transformer language model for controllable generation. arXiv preprint arXiv:1909.05858.
Byeongchang Kim, Jaewoo Ahn, and Gunhee Kim. 2020. Sequential latent knowledge selection for knowledge-grounded dialogue. arXiv preprint arXiv:2002.07510.
Taesup Kim, Sungjin Ahn, and Yoshua Bengio. 2019. Variational temporal abstraction. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, pages 11570-11579.
Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In ICLR.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871-7880.
Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2015. A diversity-promoting objective function for neural conversation models. *NAACL*, pages 110-119.
Jiwei Li, Michel Galley, Chris Brockett, Georgios Spithourakis, Jianfeng Gao, and Bill Dolan. 2016. A persona-based neural conversation model. In ACL, pages 994-1003.
Linxiao Li, Can Xu, Wei Wu, Yufan Zhao, Xueliang Zhao, and Chongyang Tao. 2020. Zero-resource knowledge-grounded dialogue generation. arXiv preprint arXiv:2008.12918.
Rongzhong Lian, Min Xie, Fan Wang, Jinhua Peng, and Hua Wu. 2019. Learning to select knowledge for response generation in dialog systems. arXiv preprint arXiv:1902.04911.
Christopher D Manning, Mihai Surdeanu, John Bauer, Jenny Rose Finkel, Steven Bethard, and David McClosky. 2014. The stanford corenlp natural language processing toolkit. In Proceedings of 52nd annual meeting of the association for computational linguistics: system demonstrations, pages 55-60.
Seungwhan Moon, Pararth Shah, Anuj Kumar, and Rajen Subba. 2019. Opendialkg: Explainable conversational reasoning with attention-based walks over knowledge graphs. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 845-854.
Nasrin Mostafazadeh, Chris Brockett, Bill Dolan, Michel Galley, Jianfeng Gao, Georgios Spithourakis, and Lucy Vanderwende. 2017. Image-grounded conversations: Multimodal context for natural question and response generation. In Proceedings of
the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 462-472.
Abigail See, Stephen Roller, Douwe Kiela, and Jason Weston. 2019. What makes a good conversation? how controllable attributes affect human judgments. arXiv preprint arXiv:1902.08654.
Iulian Vlad Serban, Alessandro Sordoni, Yoshua Bengio, Aaron C Courville, and Joelle Pineau. 2016. Building end-to-end dialogue systems using generative hierarchical neural network models. In AAAI, volume 16, pages 3776-3784.
Iulian Vlad Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron C Courville, and Yoshua Bengio. 2017. A hierarchical latent variable encoder-decoder model for generating dialogues. In AAAI, pages 3295-3301.
Lifeng Shang, Zhengdong Lu, and Hang Li. 2015. Neural responding machine for short-text conversation. In ACL, pages 1577-1586.
Xiaoyu Shen, Ernie Chang, Hui Su, Cheng Niu, and Dietrich Klakow. 2020. Neural data-to-text generation via jointly learning the segmentation and correspondence. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7155-7165.
Kurt Shuster, Samuel Humeau, Antoine Bordes, and Jason Weston. 2018. Engaging image chat: Modeling personality in grounded dialogue. arXiv preprint arXiv:1811.00945.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 1631-1642.
Chongyang Tao, Shen Gao, Mingyue Shang, Wei Wu, Dongyan Zhao, and Rui Yan. 2018. Get the point of my utterance! learning towards effective responses with multi-head attention mechanism. In *IJCAI*, pages 4418-4424.
Yi-Lin Tuan, Yun-Nung Chen, and Hung-yi Lee. 2019. Dykgchat: Benchmarking dialogue generation grounding on dynamic knowledge graphs. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1855-1865.
Oriol Vinyals and Quoc Le. 2015. A neural conversational model. arXiv preprint arXiv:1506.05869.
Chong Wang, Yining Wang, Po-Sen Huang, Abdelrahman Mohamed, Dengyong Zhou, and Li Deng. 2017. Sequence modeling via segmentations. In International Conference on Machine Learning, pages 3674-3683. PMLR.
Yansen Wang, Chenyi Liu, Minlie Huang, and Liqiang Nie. 2018. Learning to ask questions in open-domain conversational systems with typed decoders. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2193-2203.
Sam Wiseman, Stuart M Shieber, and Alexander M Rush. 2018. Learning neural templates for text generation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3174-3187.
Chen Xing, Wei Wu, Jie Liu, Yalou Huang, Ming Zhou, and Wei-Ying Ma. 2017. Topic aware neural response generation. In AAAI, pages 3351-3357.
Can Xu, Wei Wu, Chongyang Tao, Huang Hu, Matt Schuerman, and Ying Wang. 2019. Neural response generation with meta-words. arXiv preprint arXiv:1906.06050.
Hainan Zhang, Yanyan Lan, Liang Pang, Jiafeng Guo, and Xueqi Cheng. 2019a. Recosa: Detecting the relevant contexts with self-attention for multi-turn dialogue generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3721-3730.
Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? arXiv preprint arXiv:1801.07243.
Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2019b. Dialogpt: Large-scale generative pre-training for conversational response generation. arXiv preprint arXiv:1911.00536.
Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. 2017. Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. In ACL, pages 654-664.
Xueliang Zhao, Chongyang Tao, Wei Wu, Can Xu, Dongyan Zhao, and Rui Yan. 2019. A document-grounded matching network for response selection in retrieval-based chatbots. In IJCAI, pages 5443-5449.
Xueliang Zhao, Wei Wu, Chongyang Tao, Can Xu, Dongyan Zhao, and Rui Yan. 2020a. Low-resource knowledge-grounded dialogue generation. In International Conference on Learning Representations.
Xueliang Zhao, Wei Wu, Can Xu, Chongyang Tao, Dongyan Zhao, and Rui Yan. 2020b. Knowledge-grounded dialogue generation with pre-trained language models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3377-3390.
Hao Zhou, Minlie Huang, Tianyang Zhang, Xiaoyan Zhu, and Bing Liu. 2018a. Emotional chatting machine: Emotional conversation generation with internal and external memory. In Thirty-Second AAAI Conference on Artificial Intelligence.