SlowGuess/NAACL_2022_OCR huggingface dataset:
Jian Guan, Yansen Wang, and Minlie Huang. 2019b. Story ending generation with incremental encoding and commonsense knowledge. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):6473-6480.
Prakhar Gupta, Yulia Tsvetkov, and Jeffrey P. Bigham. 2021a. Synthesizing adversarial negative responses for robust response ranking and evaluation. In *Findings of the Association for Computational Linguistics: ACL/IJCNLP* 2021, Online Event, August 1-6, 2021, volume ACL/IJCNLP 2021 of *Findings of ACL*, pages 3867-3883. Association for Computational Linguistics.
Prakhar Gupta, Chien-Sheng Wu, Wenhao Liu, and Caiming Xiong. 2021b. Dialfact: A benchmark for fact-checking in dialogue. arXiv preprint arXiv:2110.08222.
Behnam Hedayatnia, Karthik Gopalakrishnan, Seokhwan Kim, Yang Liu, Mihail Eric, and Dilek Hakkani-Tur. 2020. Policy-driven neural response generation for knowledge-grounded dialog systems. In Proceedings of the 13th International Conference on Natural Language Generation, pages 412-421, Dublin, Ireland. Association for Computational Linguistics.
Peter Henderson, Koustuv Sinha, Nicolas Angelard-Gontier, Nan Rosemary Ke, Genevieve Fried, Ryan Lowe, and Joelle Pineau. 2018. Ethical challenges in data-driven dialogue systems. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, AIES '18, page 123-129, New York, NY, USA. Association for Computing Machinery.
Ioana Hulpus, Jonathan Kobbe, Heiner Stuckenschmidt, and Graeme Hirst. 2020. Knowledge graphs meet moral values. In Proceedings of the Ninth Joint Conference on Lexical and Computational Semantics, pages 71-80, Barcelona, Spain (Online). Association for Computational Linguistics.
Haozhe Ji, Pei Ke, Shaohan Huang, Furu Wei, Xiaoyan Zhu, and Minlie Huang. 2020. Language generation with multi-hop reasoning on commonsense knowledge graph. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 725-736. Association for Computational Linguistics.
Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong, and Richard Socher. 2019. CTRL: A conditional transformer language model for controllable generation. CoRR, abs/1909.05858.
Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017. DailyDialog: A manually labelled multi-turn dialogue dataset. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 986-995, Taipei, Taiwan. Asian Federation of Natural Language Processing.
Bill Yuchen Lin, Xinyue Chen, Jamin Chen, and Xiang Ren. 2019. KagNet: Knowledge-aware graph networks for commonsense reasoning. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2829-2839, Hong Kong, China. Association for Computational Linguistics.
Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74-81.
Yanxiang Ling, Fei Cai, Xuejun Hu, Jun Liu, Wanyu Chen, and Honghui Chen. 2021. Context-controlled topic-aware neural response generation for open-domain dialog systems. Inf. Process. Manag., 58(1):102392.
Chia-Wei Liu, Ryan Lowe, Iulian Serban, Mike Noseworthy, Laurent Charlin, and Joelle Pineau. 2016. How NOT to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2122-2132, Austin, Texas. Association for Computational Linguistics.
Ryan Lowe, Michael Noseworthy, Iulian Vlad Serban, Nicolas Angelard-Gontier, Yoshua Bengio, and Joelle Pineau. 2017. Towards an automatic turing test: Learning to evaluate dialogue responses. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1116-1126.
Yukun Ma, Khanh Linh Nguyen, Frank Z. Xing, and Erik Cambria. 2020. A survey on empathetic dialogue systems. Information Fusion, 64:50-70.
Bodhisattwa Prasad Majumder, Harsh Jhamtani, Taylor Berg-Kirkpatrick, and Julian J. McAuley. 2020. Like hiking? you probably enjoy nature: Person-grounded dialog with commonsense expansions. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 9194-9206. Association for Computational Linguistics.
Chaitanya Malaviya, Chandra Bhagavatula, Antoine Bosselut, and Yejin Choi. 2020. Commonsense knowledge base completion with structural and semantic context. Proceedings of the 34th AAAI Conference on Artificial Intelligence.
Ninareh Mehrabi, Pei Zhou, Fred Morstatter, Jay Pujara, Xiang Ren, and Aram Galstyan. 2021. Lawyers are dishonest? quantifying representational harms in commonsense knowledge resources. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5016-5033, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Tong Niu and Mohit Bansal. 2018. Polite dialogue generation without parallel data. Transactions of the Association for Computational Linguistics, 6:373-389.
Martha Palmer, Daniel Gildea, and Paul Kingsbury. 2005. The Proposition Bank: An annotated corpus of semantic roles. Computational Linguistics, 31(1):71-106.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311-318.
Jinghui Qin, Zheng Ye, Jianheng Tang, and Xiaodan Liang. 2020a. Dynamic knowledge routing network for target-guided open-domain conversation. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pages 8657-8664. AAAI Press.
Lianhui Qin, Vered Shwartz, Peter West, Chandra Bhagavatula, Jena D. Hwang, Ronan Le Bras, Antoine Bosselut, and Yejin Choi. 2020b. Back to the future: Unsupervised backprop-based decoding for counterfactual and abductive commonsense reasoning. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 794-805. Association for Computational Linguistics.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
Ananya B Sai, Akash Kumar Mohankumar, and Mitesh M Khapra. 2020. A survey of evaluation metrics used for nlg systems. arXiv preprint arXiv:2008.12009.
Karin Sevegnani, David M. Howcroft, Ioannis Konstas, and Verena Rieser. 2021. Otters: One-turn topic transitions for open-domain dialogue. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, pages 2492-2504. Association for Computational Linguistics.
Peng Shi and Jimmy Lin. 2019. Simple bert models for relation extraction and semantic role labeling. arXiv preprint arXiv:1904.05255.
Haoyu Song, W. Zhang, Yiming Cui, Dong Wang, and T. Liu. 2019. Exploiting persona information for diverse generation of conversational responses. In *IJCAI*.
Charles Spearman. 1961. The proof and measurement of association between two things. *Appleton-Century-Crofts*.
Robyn Speer, Joshua Chin, and Catherine Havasi. 2017a. Conceptnet 5.5: An open multilingual graph of general knowledge.
Robyn Speer, Joshua Chin, and Catherine Havasi. 2017b. Conceptnet 5.5: An open multilingual graph of general knowledge. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31.
Hao Sun, Guangxuan Xu, Jiawen Deng, Jiale Cheng, Chujie Zheng, Hao Zhou, Nanyun Peng, Xiaoyan Zhu, and Minlie Huang. 2021. On the safety of conversational models: Taxonomy, dataset, and benchmark. arXiv preprint arXiv:2110.08466.
Jianheng Tang, Tiancheng Zhao, Chenyan Xiong, Xiaodan Liang, Eric P. Xing, and Zhiting Hu. 2019. Target-guided open-domain conversation. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pages 5624-5634. Association for Computational Linguistics.
Chongyang Tao, Lili Mou, Dongyan Zhao, and Rui Yan. 2017. Rubber: An unsupervised method for automatic evaluation of open-domain dialog systems. arXiv preprint arXiv:1701.03079.
Peifeng Wang, Nanyun Peng, Filip Ilievski, Pedro Szekely, and Xiang Ren. 2020. Connecting the dots: A knowledgeable path generator for commonsense question answering. In *Findings of the Association for Computational Linguistics: EMNLP* 2020, pages 4129-4140, Online. Association for Computational Linguistics.
Wenquan Wu, Zhen Guo, Xiangyang Zhou, Hua Wu, Xiyuan Zhang, Rongzhong Lian, and Haifeng Wang. 2019. Proactive human-machine conversation with explicit conversation goal. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3794-3804, Florence, Italy. Association for Computational Linguistics.
Jing Xu, Da Ju, Margaret Li, Y-Lan Boureau, Jason Weston, and Emily Dinan. 2020. Recipes for safety in open-domain chatbots. arXiv preprint arXiv:2010.07079.
Kevin Yang and Dan Klein. 2021. FUDGE: controlled text generation with future discriminators. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, Online, June 6-11, 2021, pages 3511-3535. Association for Computational Linguistics.
Pengcheng Yang, Lei Li, Fuli Luo, Tianyu Liu, and Xu Sun. 2019. Enhancing topic-to-essay generation with external commonsense knowledge. In Proceedings of the 57th Conference of the Association
for Computational Linguistics, ACL 2019, Florence, Italy, July 28-August 2, 2019, Volume 1: Long Papers, pages 2002-2012. Association for Computational Linguistics.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with BERT. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net.
Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. 2017. Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, pages 654-664. Association for Computational Linguistics.
Peixiang Zhong, Yong Liu, Hao Wang, and Chunyan Miao. 2021. Keyword-guided neural conversational model. Proceedings of the AAAI Conference on Artificial Intelligence, 35(16):14568-14576.
Peixiang Zhong, Di Wang, and Chunyan Miao. 2019. An affect-rich neural conversational model with biased attention and weighted cross-entropy loss. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 7492-7500.
Hao Zhou, Tom Young, Minlie Huang, Haizhou Zhao, Jingfang Xu, and Xiaoyan Zhu. 2018. Commonsense knowledge aware conversation generation with graph attention. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, pages 4623-4629. International Joint Conferences on Artificial Intelligence Organization.
Pei Zhou, Karthik Gopalakrishnan, Behnam Hedayatnia, Seokhwan Kim, Jay Pujara, Xiang Ren, Yang Liu, and Dilek Hakkani-Tur. 2021a. Commonsense-focused dialogues for response generation: An empirical study. In Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 121-132, Singapore and Online. Association for Computational Linguistics.
Pei Zhou, Karthik Gopalakrishnan, Behnam Hedayatnia, Seokhwan Kim, Jay Pujara, Xiang Ren, Yang Liu, and Dilek Hakkani-Tur. 2021b. Think before you speak: Using self-talk to generate implicit commonsense knowledge for response generation. arXiv preprint arXiv:2110.08501.
Pei Zhou, Behnam Hedayatnia, Karthik Gopalakrishnan, Seokhwan Kim, Jay Pujara, Xiang Ren, Yang Liu, and Dilek Hakkani-Tur. 2021c. Think before you speak: Learning to generate implicit knowledge for response generation by self-talk. In Proceedings of the 3rd Workshop on Natural Language Processing for Conversational AI, pages 251-253, Online. Association for Computational Linguistics.
Caleb Ziems, Jane A Yu, Yi-Chia Wang, Alon Halevy, and Diyi Yang. 2022. The moral integrity corpus: A benchmark for ethical dialogue systems. arXiv preprint arXiv:2204.03021.
Asli Çelikyilmaz, Elizabeth Clark, and Jianfeng Gao. 2020. Evaluation of text generation: A survey. ArXiv, abs/2006.14799.
A Implementation Details for CODA
A.1 Training Details for CODA
Model training: We code our models using Pytorch and Huggingface library. We use validation loss to do model selection. The KPG-wc, KPGht and CRG models are all based on GPT-2 small architecture. We use batch size of 10 for GPT-2 models. We use Adam optimizer with initial learning rate of  $1e - 4$ . We use GeForce RTX 2080 GPUs for training models. All existing code used and datasets were CC-BY 4.0 or open sourced by original authors.
Decoding paths and responses: For decoding paths using the KPG models, we use temperature of 0.7 and nucleus sampling with top-p set to 0.9. We use the same decoding strategy and hyperparameters for decoding responses using CRG model. Concept Extraction: Entities are extracted from the context, target and response to generate and align paths using the KPG models. For a sentence s, we first extract the set of noun and verb phrases from the sentence using NLTK. We design simple grammar rules to convert some phrases to a more concise forms that are similar to the kinds of nodes present in ConceptNet,e.g., "watching the star" is converted to "watch stars". We use NLTK's POS tagging combined with the following grammar rules: (1) Nouns and Adjectives, terminated with Nouns  $< \mathrm{NN}.*| \mathrm{JJ}>$ * $< \mathrm{NN}.*>$  (2) Verb and verb phrases  $< \mathrm{RB}.?>$ * $< \mathrm{VB}.?>$ * $< \mathrm{JJ}>$ * $< \mathrm{VB}.?>$ + $< \mathrm{VB}>?$ . We normalize the verbs using NLTK. The final set of entities consist of the noun and verb phrases. We exclude phrases such as "today", "enough" which are sometimes incorrectly detected as entities.
Sub-selecting entity pairs during training of CRG model: For every context-target pair, we have n number of pair of head-tails entities. We score an entity pair by calculating the inverse document frequencies (computed using Gutenberg English corpus) of the entity tokens and summing up the maximum value found for a token in each entity in the pair. For training phase, we keep the topD
pairs of entities. The value of top D is selected based on validation performance and comes out typically between 1-3.
Knowledge graph details: The number of nodes in the ConceptNet resource we have used<sup>4</sup> is 382226. We perform random walks on the graph with paths of length from 1 to 6 and get a total of 3883671 number of paths.
Edges in the knowledge path: We discard some edge types which are regarded to be uninformative and offer little help for our task following Wang et al. (2020). They include RelatedTo, Synonym, Antonym, DerivedFrom, FormOf, Etymologically-DerivedFrom and EtymologicallyRelatedTo. Since the nodes in ConceptNet are directional, we also add inverse edges during path sampling. For example the path "ecosystem <- PartOf <- organism" can be sampled as "ecosystem_isPartOf organism" where the underscore indicates a reverse edge.
A.2 Clause Identification for Data Augmentation
For target creation, given a dialogue context  $c$  and its response  $r$ , we first break the response  $r$  into sentence clauses. For example, given a context "Is my booking complete?" and the response "your reservation is confirmed. now i need your phone number," we extract a clause  $t$  "i need your phone number" as the target candidate  $t$ . For clause extraction we use Allennlp's SRL parser which is trained using a BERT-based model (Shi and Lin, 2019) and is based on PropBank (Palmer et al., 2005). It identifies the arguments associated with the predicates or verbs of a sentence predicates (verbs or events) in a sentence and classifies them into roles such as agent, patient and instrument. For the example above, it identifies "need" as a predicate with agent "i" and instrument "your number".
A.3 Data Augmentation for CODA
We filter data from the dailydialog dataset based on a threshold set to 0.7 for data augmentation. This threshold was selected using empirical performance of thr CODA model. For CODA-ONLYDA model which does not use knowledge paths, the context, target and transition response is used directly in training the CRG decoder of CODA-ONLYDA model. But for CODA model which uses the knowledge paths, the dailydialog data is
$^{4}$ www.github.com/wangpf3/Commonsense-Path-Generator
$^{5}$ github.com/allenai/allennlp
Context: i enjoy staring up at the sky.
Target: i like to spend a lot of my free time with my pet.
Response 1: I like stargazing outside with my pet. (0.99)
Response 2: I like stargazing outside. (0.05)
Response 3: I like walking with my pet. (0.01)
Response 4: My pet is a big star. (0.02)
Context: i make blogs.
Target: i have a large family with babies.
Response 1: I want to blog about my children. (0.99)
Response 2: My family has a lot of babies. (0.05)
Response 3: My blogs are very famous. (0.01)
Table 6: Stress testing the Target-Coherence metric. We show sample responses and TC score for the responses in brackets.
converted to the same format as Otters data, that is, we first do entity detection on the target component of the responses as well as the dialogue context. Then we generate a set of paths for each pair of entities. The CODA model is first trained on paths from the filtered dailydialog data and then finetuned on the Otters dataset which follows the same knowledge path format. The maximum dialogue history length is set to 2 for dailydialog dataset.
A.4 Target Coherence Metric
In Table 6, we provide examples for stress testing the Target-Coherence metric. TC scores for the responses are shown in brackets. Simply repeating or addressing either the target or context gets a low TC score. For example the response "I like stargazing outside" is not a smooth transition and gets a low TC score, while "I like stargazing outside with my pet" is a smooth transition and gets a high TC score. In Figure 4 we present an overview of the mechanisms used for generating negative samples for training the Target-Coherence metric. For negative examples, 1) Given gold response r, and context c, we sample a random negative target t', which creates a response which does not transition towards the target t, 2) Given gold response r, and target t, we sample a random negative context c', which creates a response which is not coherent to the context c, 3) Given gold context c, and target t, we either sample a random negative response r' or generate a response r' conditioned on random c' or t', which creates a response which does not transition to target t or is coherent to context c.
B Training Details of Baselines
Training GPT-2 Fudge model Yang and Klein (2021) proposed a future discriminator based decoding technique. The Fudge discriminator uses a discriminator trained to distinguish good response
continuations from the poor ones and guides the GPT2 based decoder towards responses that are coherent to both the source and target sentences. The Fudge discriminator needs positive and negative sample data for training. We train the discriminator to distinguish a good response from a bad (not coherent to target or context). The input to train the discriminator (a LSTM model) is the concatenation of the context sentence, followed by the target sentence and finally the tokens of a response r with tokens k. The discriminator then learns to predict 1 if the next token in the response at position k belongs to the gold response or 0 if the token is a random one. We train the Fudge discriminator by preparing negative instances using the same techniques we use to train the Target-Coherence model - sampling random negative responses, responses coherent to the context but not to the target, and responses coherent to the target but not to the context.
Training CS-Pretrain model The model is based on the commonsense story generation model from Guan et al. (2020) We create training data for the CS-Pretrain model by using the same sampled paths we use for training the KPG-wc model. The paths are converted into textual format by converting edges into text sequences. The model is only pretrained with general commonsense paths and then fine-tuned on Otters dataset in a manner similar to the GPT-2 baselines (i.e. without paths). Our experiments show that pretraining with commonsense model does not help with target-guided task, probably since the task needs target conditional commonsense and general commonsense knowledge only confuses the model during decoding.
Training Concept-Predict leverages a concept prediction strategy from Qin et al. (2020a). The input to the model is the context and target and it predicts a single concept based on closeness to the target. The concept is then fed as an input to the CRG model along with the context and target sentences.
Training CODA-ONLYDA: CODA variant that uses Dailydialog augmentation and does not use commonsense paths from KPG models in the CRG model. Therefore the model consists of only a CRG model (no KPG models) which take the context and target sentences as inputs.
Training CODA-NoEDGE CODA variant that uses only entities and no edges from the path. For example the path "favorite city is the location which has bicycle shop is a dependency of ride bi
Target	Keywords
i need your address	send money; visit; mail; send gift; send coupon
you should spend time with your friends	don’t be alone; mental health; be happy;
you can try our restaurant	best ingredients ; cheapest food; free delivery
our new recipe is best selling	fat free; healthy; protein; tasty
i am the best financial advisor	get rich quickly; sound advice; money management
you should have a positive attitude	mental health; others will help; peace
we should always avoid fighting	peace; happiness; injury; understand other people
i want to come to united states	freedom ;democracy; money; job; american dream; education
everyone should get vaccinated	public health; reduce hospital burden; live longer; pandemic; be safe
we should donate to charity	help poor; make a difference; give assistance; feel good; social benefits
Table 7: The set of manually created targets and keyword set used for each target.
Figure 5: Amazon mechanical turk interface for human ratings collection
cycle” is converted to “favorite city bicycle shop ride bicycle”, which is fed as input to the CRG model.
Training CODA-NoALIGN: variant that relies on only KPG-ht for training and inference. Does not select paths based on alignment with responses. The paths used during training the CRG model come from KPG-ht instead of KPG-wc.
Training CODA-KBPATH: variant that samples paths directly from ConceptNet using the algorithm proposed in Lin et al. (2019). Given a pair of context and target concept, we use their algorithm to sample an actual path directly from ConceptNet. The model is pretrained on Dailydialog augmented data and fine-tuned on Otters with the sampled paths from ConceptNet. The model suffers from missing entities and missing links between entities in ConceptNet which is solved by CODA.
C Human-in-the-loop Experiment