SlowGuess/ACL_2020_OCR huggingface dataset:
CoQA Baseline: Conversational Question Answering (CoQA) (Reddy et al., 2019) is a large-scale ConvQA dataset aimed at creating models which can answer the questions posed in a conversational setting. Since we are generating conversational responses for QA systems, it is sensible to compare against such ConvQA systems. We pick one of the best performing BERT-based CoQA model from the SMRCToolkit (Wu et al., 2019) as a baseline. $^{10}$  We refer to this model as the CoQA baseline.
QuAC Baseline: Question Answering in Context
<sup>9</sup>http://forum.opennmt.net/t/english-chatbot-model-with-opennmt/184
10 one of the top performing model with available code.
is another ConvQA dataset. We use the modified version of BiDAF model presented in (Choi et al., 2018) as a second baseline. Instead of a SEQ2SEQ generation, it selects spans from passage which acts as responses. We use the version of this model implemented in AllenNLP (Gardner et al., 2017) and refer to this model as the QuAC baseline.
STs+BERT Baseline: We also compare our generation models with the technique that created the SS and  $\mathbf{SS}+$  training datasets (i.e. the responses generated by STs ranked with the BERT response classifier).
We validate all the SEQ2SEQ models on the human annotated SG data (Table 1).
3.2 Evaluation on the SQuAD 2.0 Dev Set
To have a fair and unbiased comparison, we create a new 500 question sample from the SQuAD 2.0 dev set (SQuAD-dev-test) which is unseen for all the models and baselines. This sample contains  $\sim 20\%$  of the questions that cannot be handled by the STs (parser errors). For such questions, we default to outputting the answer-phrase as the response for the STs+BERT baseline. For the CoQA baseline and the QuAC baseline, we run their models on passages (corresponding to the questions) from SQuAD-dev-test to get their responses.
To demonstrate that our models too can operate in a fully automated setting like the CoQA baseline and the QuAC baseline, we generate their responses using the answer spans selected by a BERT-based SQuAD model (instead of the gold answer span from the SQuAD-dev-test).
For automatic evaluation we compute validation perplexity of all SEQ2SEQ generation models on SG data  $(3^{rd}$  column in Table 3). However, validation perplexity is a weak evaluator of generation models. Also, due to the lack of human-generated references in SQuAD-dev-test, we cannot use other typical generation based automatic metrics. Therefore, we use Amazon Mechanical Turk to do human evaluation. Each response is judged by 5 annotators. We ask the annotators to identify if the response is conversational and answers the question correctly. While outputting answer-phrase to all questions is trivially correct, this style of response generation seems robotic and unnatural in a prolonged conversation. Therefore, we also ask the annotators to judge if the response is a complete-sentence (e.g. "it is in Indiana") and not a sentence-fragment (e.g. "Indiana"). For each question and response pair, we show the annotators five options
Model	Data	PPL	a	b	c	d	e
X	✓	X	✓	✓	correct answer
X	X	✓	✓	✓	complete-sentence
-	-	-	X	✓	grammaticality
CoQA B.	-	-	13.80	82.20	1.20	0.60	2.20
QuAC B.	-	-	5.20	3.80	46.40	2.80	41.80
STs+BERT B.	-	-	0.00	18.20	0.20	13.80	67.80
PGN	SS	6.60	1.00	7.00	9.00	16.20	66.80
PGN	SS+	3.83	1.00	3.00	8.40	17.60	70.00
PGN-Pre	SS	4.34	0.20	4.60	9.80	17.40	68.00
PGN-Pre	SS+	3.31	0.40	4.80	9.00	16.20	69.60
GPT-2	SS	4.69	1.00	5.00	13.20	18.60	62.20
GPT-2	SS+	2.70	0.80	4.20	8.20	16.80	70.00
GPT-2-Pre	SS	3.23	0.40	2.80	8.20	19.00	69.60
GPT-2-Pre	SS+	2.74	0.80	2.40	7.80	17.00	72.00
D-GPT	SS	2.20	0.40	2.40	8.60	13.00	75.60
D-GPT	SS+	2.06	0.40	2.60	7.80	13.20	76.00
D-GPT (o)	SS+	2.06	0.00	3.00	0.00	13.80	83.20
Table 3: Human evaluation results of all the models and baselines on sample of SQuAD-dev-test. In the first three rows B. stands for baseline. In the last row "(o)" stands for oracle. In Column 3 PPL stands for validation perplexity. All the values are percentage (out of 100) of responses from each model that belong to specific option(a to e) selected by annotators.
based on the three properties (correctness, grammaticality, and complete-sentence). These five options (a to e) are shown in the Table 3 header. The best response is a complete-sentence which is grammatical and answers the question correctly (i.e. option e). Other options give us more insights into different models' behavior. For each response, we assign the majority option selected by the annotators and aggregate their judgments into buckets. We present this evaluation in Table 3.
We compute the inter-annotator agreement by calculating Cohen's kappa (Cohen, 1960) between individual annotator's assignments and the aggregated majority options. The average Cohen's kappa (weighted by the number of annotations for every annotator) is 0.736 (i.e. substantial agreement).
The results reveal that CoQA baseline does the worst in terms of option e. The main reason for that is because most of the responses generated from this baseline are exact answer spans. Therefore, we observe that it does very well in option b (i.e. correct answer but not a complete-sentence). The QuAC baseline can correctly select span-based informative response  $\sim 42\%$  of the time. Other times, however, it often selects a span from the passage which is related to the topic but doesn't contain the correct answer i.e. (option c). Another problem with this baseline is that it is restricted by the input passage and many not always be able to find a valid span that answers the questions. Our STs+BERT baseline does better in terms of option e compared to the other baselines but it is limited by the STs
and the parser errors. As mentioned earlier,  $\sim 20\%$  of the time this baseline directly copies the answer-phrase in the response which explains the high percentage of option b.
Almost all models perform better when trained with SS+ data showing that the additional data from Natural Questions and HarvestingQA is helping. Except for the PGN model trained on SS data, all other variants perform better than STs+BERT baseline in terms of option e. The GPT-2 model trained on SS data from scratch does not perform very well because of the small size of training data. The pretraining with OpenSubtitles questions boosts its performance (option  $\in \%$  for GPT-2-Pre model variants  $>$  option  $\in \%$  for GPT-2 model variants). The best model however is D-GPT when finetuned with SS+ dataset. While retaining the correct answer, it makes less grammatical errors (lower  $\%$  in option c and d compared to other models). Furthermore with oracle answers it performs even better (last row in Table 3). This shows that D-GPT can generate better quality responses with accurate answers. We provide some sample responses from different models in Appendix A.
3.3 Evaluation on CoQA
In this section, we test our model's ability to generate conversational answers on the CoQA dev set, using CoQA baseline's predicted answers. The CoQA dataset consists of passages from seven different domains (out of which one is Wikipedia) and conversational questions and answers on those
Model	a	b	c	d	e
CoQA B.	12.0	78.0	5.0	2.0	3.0
D-GPT	2.0	5.0	16.0	20.0	57.0
D-GPT (o)	0.0	7.0	0.0	16.0	77.0
Table 4: Human evaluation results of D-GPT model (trained on SS+ dataset) vs CoQA model on sample of 100 question answers from filtered CoQA dev set. (o) stands for oracle answers. Options a to e are explained in Table 3 header.
passages. Due to the conversational nature of this dataset, some of the questions are one word ( $\sim 3.1\%$ ), like "what?", "why?" etc. Such questions are out-of-domain for our models as they require the entire context over multiple turns of the conversation to develop their response. Other out-of-domain questions include unanswerable ( $\sim 0.8\%$ ) and yes/no ( $\sim 18.4\%$ ) questions. We also don't consider questions with answers  $>5$  words ( $\sim 11.6\%$ ) as they are typically non-factoid questions. We take a random sample of 100 from the remaining questions. This sample contains questions from a diverse set of domains outside of the Wikipedia (on which our models are trained). This includes questions taken from the middle of a conversation (for example, "who did they meet ?") which are unfamiliar for our models. We perform a human evaluation similar to §3.2 on this sample. We compare CoQA against D-GPT trained on the SS+ dataset (with CoQA's predictions input as answer- phrases). The results are shown in Table 4.
This evaluation reveals that the D-GPT model is able to successfully convert the CoQA answer spans into conversational responses  $57\%$  of the time (option e). D-GPT gets the wrong answer  $18\%$  of the time (option a and c), because the input answer predicted by the CoQA baseline is also incorrect  $17\%$  of the time. However with oracle answers, it is able to generate correct responses  $77\%$  of the times (option e). The weighted average Cohen's kappa (Cohen, 1960) score for all annotators in this evaluation is 0.750 (substantial agreement). This result demonstrates ability of our model to generalize over different domains and generate good conversational responses for questions when provided with correct answer spans.
Question Generation (QG) is a well studied problem in the NLP community with many machine learning based solutions (Rus et al., 2010; Heilman
and Smith, 2010; Yao et al., 2012; Labutov et al., 2015; Serban et al., 2016; Reddy et al., 2017; Du et al., 2017; Du and Cardie, 2017, 2018). In comparison, our work explores the opposite direction, i.e. (generating conversational humanlike answers given a question). Fu and Feng (2018) also try to solve fluent answer response generation task but in a restricted setting of movie related questions with 115 question patterns. In contrast, our generation models can deal with human generated questions from any domain.
Learning to Rank formulations for answer selection in QA systems is common practice, most frequently relying on pointwise ranking models (Severyn and Moschitti, 2015; Garg et al., 2019). Our use of discriminative re-ranking (Collins and Koo, 2005) with softmax loss is closer to learning a pairwise ranking by maximizing the multiclass margin between correct and incorrect answers (Joachims, 2002; Burges et al., 2005; Koppel et al., 2019). This is an important distinction from TREC-style answer selection as our ST-generated candidate responses have lower semantic, syntactic, and lexical variance, making pointwise methods less effective.
Question Answering Using crowd-sourcing methods to create QA datasets (Rajpurkar et al., 2016; Bajaj et al., 2016; Rajpurkar et al., 2018), conversational datasets (Dinan et al., 2018), and ConvQA datasets (Choi et al., 2018; Reddy et al., 2019; Elgohary et al., 2018; Saha et al., 2018) has largely driven recent methodological advances. However, models trained on these ConvQA datasets typically select exact answer spans instead of generating them (Yatskar, 2019b). Instead of creating another crowd-sourced dataset for our task, we augment existing QA datasets to include such conversational answer responses using the STs + BERT trained with softmax loss.
In this work, we study the problem of generating fluent QA responses in the context of building fluent conversational agents. To this end, we propose an over-generate and rank data augmentation procedure based on Syntactic Transformations and a best response classifier. This method is used to modify the SQuAD 2.0 dataset such that it includes conversational answers, which is used to train SEQ2SEQ based generation models. Human evaluations on SQuAD-dev-test show that our models generate
significantly better conversational responses compared to the baseline CoQA and QuAC models. Furthermore, the D-GPT model with oracle answers is able to generate conversational responses on the CoQA dev set  $77\%$  of the time showcasing the model's scalability.
We would like to thank the reviewers for providing valuable feedback on an earlier draft of this paper. This material is based in part on research sponsored by the NSF (IIS-1845670), ODNI and IARPA via the BETTER program (2019-19051600004) DARPA via the ARO (W911NF-17-C-0095) in addition to an Amazon Research Award. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of NSF, ODNI, ARO, IARPA, DARPA or the U.S. Government.
Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, et al. 2016. Ms marco: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268.
Christopher Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton, and Gregory N Hullender. 2005. Learning to rank using gradient descent. In Proceedings of the 22nd International Conference on Machine learning (ICML-05), pages 89-96.
Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wentau Yih, Yejin Choi, Percy Liang, and Luke Zettlemoyer. 2018. QuAC: Question answering in context. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2174-2184, Brussels, Belgium. Association for Computational Linguistics.
Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1):37-46.
Michael Collins and Terry Koo. 2005. Discriminative reranking for natural language parsing. Computational Linguistics, 31(1):25-70.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers),
pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2018. Wizard of wikipedia: Knowledge-powered conversational agents. arXiv preprint arXiv:1811.01241.
Xinya Du and Claire Cardie. 2017. Identifying where to focus in reading comprehension for neural question generation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2067-2073, Copenhagen, Denmark. Association for Computational Linguistics.
Xinya Du and Claire Cardie. 2018. Harvesting paragraph-level question-answer pairs from Wikipedia. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1907-1917, Melbourne, Australia. Association for Computational Linguistics.
Xinya Du, Junru Shao, and Claire Cardie. 2017. Learning to ask: Neural question generation for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1342-1352, Vancouver, Canada. Association for Computational Linguistics.
Ahmed Elgohary, Chen Zhao, and Jordan Boyd-Graber. 2018. A dataset and baselines for sequential open-domain question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1077-1083, Brussels, Belgium. Association for Computational Linguistics.
Yao Fu and Yansong Feng. 2018. Natural answer generation with heterogeneous memory. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 185-195, New Orleans, Louisiana. Association for Computational Linguistics.
Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson F. Liu, Matthew Peters, Michael Schmitz, and Luke S. Zettlemoyer. 2017. Allennlp: A deep semantic natural language processing platform.
Siddhant Garg, Thuy Vu, and Alessandro Moschitti. 2019. Tanda: Transfer and adapt pre-trained transformer models for answer sentence selection. arXiv preprint arXiv:1911.04118.
Albert Gatt and Ehud Reiter. 2009. Simplenlg: A realisation engine for practical applications. In Proceedings of the 12th European Workshop on Natural Language Generation (ENLG 2009), pages 90-93.
Michael Heilman and Noah A. Smith. 2010. Good question! statistical ranking for question generation. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 609-617, Los Angeles, California. Association for Computational Linguistics.
Thorsten Joachims. 2002. Optimizing search engines using clickthrough data. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 133-142. ACM.
John Judge, Aoife Cahill, and Josef Van Genabith. 2006. Questionbank: Creating a corpus of parse-annotated questions. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, pages 497-504. Association for Computational Linguistics.
Dan Klein and Christopher D Manning. 2003. Fast exact inference with a factored model for natural language parsing. In Advances in neural information processing systems, pages 3-10.
Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander M. Rush. 2017. OpenNMT: Open-source toolkit for neural machine translation. In Proc. ACL.
Marius Koppel, Alexander Segner, Martin Wagener, Lukas Pensel, Andreas Karwath, and Stefan Kramer. 2019. Pairwise learning to rank by neural networks revisited: Reconstruction, theoretical analysis and practical performance. arXiv preprint arXiv:1909.02768.
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natural questions: A benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:453-466.
Igor Labutov, Sumit Basu, and Lucy Vanderwende. 2015. Deep questions without deep understanding. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 889-898, Beijing, China. Association for Computational Linguistics.
Roger Levy and Galen Andrew. 2006. Tregex and tsurgeon: tools for querying and manipulating tree data structures. In Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC'06), Genoa, Italy. European Language Resources Association (ELRA).
Ankur Parikh, Oscar Tackström, Dipanjan Das, and Jakob Uszkoreit. 2016. A decomposable attention model for natural language inference. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2249-2255, Austin, Texas. Association for Computational Linguistics.
Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532-1543, Doha, Qatar. Association for Computational Linguistics.
Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2227-2237, New Orleans, Louisiana. Association for Computational Linguistics.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog, 1(8).
Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don't know: Unanswerable questions for SQuAD. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 784-789, Melbourne, Australia. Association for Computational Linguistics.
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383-2392, Austin, Texas. Association for Computational Linguistics.
Sathish Reddy, Dinesh Raghu, Mitesh M. Khapra, and Sachindra Joshi. 2017. Generating natural language question-answer pairs from a knowledge graph using a RNN based question generation model. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 376-385, Valencia, Spain. Association for Computational Linguistics.
Siva Reddy, Danqi Chen, and Christopher D. Manning. 2019. CoQA: A conversational question answering challenge. Transactions of the Association for Computational Linguistics, 7:249-266.
Vasile Rus, Brendan Wyse, Paul Piwek, Mihai Lintean, Svetlana Stoyanchev, and Cristian Moldovan. 2010. The first question generation shared task evaluation challenge.
Amrita Saha, Vardaan Pahuja, Mitesh M Khapra, Karthik Sankaranarayanan, and Sarath Chandar. 2018. Complex sequential question answering: Towards learning to converse over linked question answer pairs with a knowledge graph. In Thirty-Second AAAI Conference on Artificial Intelligence.
Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1073-1083, Vancouver, Canada. Association for Computational Linguistics.
Iulian Vlad Serban, Alberto García-Durán, Caglar Gulcehre, Sungjin Ahn, Sarath Chandar, Aaron Courville, and Yoshua Bengio. 2016. Generating factoid questions with recurrent neural networks: The 30M factoid question-answer corpus. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 588-598, Berlin, Germany. Association for Computational Linguistics.
Aliaksei Severyn and Alessandro Moschitti. 2015. Learning to rank short text pairs with convolutional deep neural networks. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, pages 373-382. ACM.
Jörg Tiedemann. 2009. News from opus-a collection of multilingual parallel corpora with tools and interfaces. In *Recent advances in natural language processing*, volume 5, pages 237-248.
Stephen Wan, Mark Dras, Robert Dale, and Cecile Paris. 2006. Using dependency-based features to take the'para-farce'out of paraphrase. In Proceedings of the Australasian Language Technology Workshop 2006, pages 131-138.
Jindou Wu, Yunlun Yang, Chao Deng, Hongyi Tang, Bingning Wang, Haoze Sun, Ting Yao, and Qi Zhang. 2019. Sogou Machine Reading Comprehension Toolkit. arXiv e-prints, page arXiv:1903.11848.
Xuchen Yao, Gosse Bouma, and Yi Zhang. 2012. Semantics-based question generation and implementation. Dialogue & Discourse, 3(2):11-42.
Mark Yatskar. 2019a. A qualitative comparison of CoQA, SQuAD 2.0 and QuAC. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2318-2323, Minneapolis, Minnesota. Association for Computational Linguistics.
Mark Yatskar. 2019b. A qualitative comparison of coqa, squad 2.0 and quac. In Proceedings of the 2019 Conference of the North American Chapter of
the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers).
Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2019. Dialogpt: Large-scale generative pre-training for conversational response generation.
A Sample responses from different models