SlowGuess/ACL_2020_OCR huggingface dataset:
fine-tune with SPOLIN. In another configuration, we also fine-tune directly with SPOLIN on top of GPT-2. The original GPT-2 implementation prepends the personalities given in Persona-chat to the dialogue sequence input before tokenization. For fine-tuning to datasets apart from Persona-chat, we simply do not prepend any auxiliary information to the dialogue sequence input.
Automatic metrics that rely on n-gram overlap, such as BLEU, ROUGE, and METEOR, are often used for generative models when there is little variability in the target output (Papineni et al., 2002; Lin, 2004; Banerjee and Lavie, 2005). However, there can be a wide variety of responses that qualify as a good yes-and, a problem common to open-domain generation tasks. An adequate evaluation of our models requires assessing the main yes-and criteria: agreement with the context and the quality of the new relevant contribution, both of which are not feasible with the aforementioned metrics. Therefore, we ask human evaluators to compare the quality of the yes-ands generated by various models and the actual response to the prompt in SPOLIN that is used as the input.
We ask human evaluators to rank a set of four responses given a prompt, comparing the responses of a model trained only with SPOLIN, a model trained with an existing dialogue corpus, a model trained with both, and the actual response pair from
the development set, denoted as "Gold." These four responses are randomly ordered for each question to prevent evaluators from developing a bias for responses that frequently have a good or poor response in a set order, as shown in Figure 5. The evaluators are permitted to provide the same rank for different responses if they are equal in quality. This evaluation set contains 100 such prompts, and each is evaluated twice by different evaluators. The results of the average ranking and some of the examples generated by the models are shown in Table 4.
Results show that models trained only with SPOLIN or with SPOLIN and another dialogue dataset are preferred to the models trained only with another dialogue dataset, although in the case of DailyDialog, the average ranking improves only by at most 0.06 after fine-tuning with SPOLIN. However, even the responses generated by models trained with SPOLIN are not ranked as well as the actual responses in the development set, indicating our models are still inferior to professional human improviser quality.
5 Extracting from Other Corpora
The approach to classifier-based mining we describe in Section 2.2 can naturally be applied to other dialogue corpora. We thus next consider mining the gigantic (441M sentence) OpenSubtitles (Lison and Tiedemann, 2016) collection. As OpenSubtitles contains undesirable material, such as subtitles for media with minimal dialogue, we instead mine from the (3.3M sentence) SubTle corpus (Ameixa et al., 2013), a preprocessed subset of OpenSubtitles that heuristically combines subtitle sequences into dialogue form.
By iterating through half of this corpus, we collect more than 40,000 yes-ands from it alone, which, when added to SPOLIN, yields what we call SPOLIN-extended, which contains about 68,000 yes-ands, more than 2.5 times the size of the core SPOLIN. Heuristics for finding alternations mean that SubTle's utterances are shorter than those in Spontaneanation and Cornell, so once the proportion of utterances longer than the average length of in Spontaneanation and Cornell (18.5 words) is less than  $40\%$ , we stop further collection in the remainder of the dataset. SPOLIN-extended is available in the same public repository as SPOLIN. Details of the iterative process as applied to SubTle are in the appendix.
Dataset	Avg Rank ↓	Example Prompt	Example Responses
Persona-chat	3.67	I know alotta women and I'm sure she remembers me.	oh my goodness, i don't know her
SPOLIN	3.41	Yeah, she's a bit of a mystery.
Persona-chat+SPOLIN	3.17	So you remember her? I remember her in the shower.
Gold	1.92	She does. From when you were a boy.
Cornell	3.73	I made this beautiful salmon mousse that just looked like a puddle of spit.	What?
SPOLIN	3.39	And it's delicious!
Cornell+SPOLIN	3.34	That's the kind of thing you do when you're in love.
Gold	2.01	It was genius. It's making me hungry thinking about it.
DailyDialog	3.37	Excuse me. Is anybody in here? I'd like a bagel. Hello? Is anybody in here?	I'm sorry, sir. I'm not in the mood. I'm not in the mood.
SPOLIN	3.32	I'm in here. I'm just trying to make sure I can get a bagel.
DailyDialog+SPOLIN	3.31	Oh, yeah, the guy who left the bagel.
Gold	1.87	I can help you. The problem is that the bagels are burned.
Table 4: Average human ranking of responses to prompts from Spontaneanation generated by models trained with SPOLIN, an existing dialog corpus, or both, based on the yes-and criteria. Rank is scaled from 1 to 4, lower is better.
Dataset	Source	Size*
DailyDialog (Li et al., 2017b)	Crowdsourced	104K
Cornell Movie-Dialogs Corpus (Danescu-Niculescu-Mizil and Lee, 2011)	Movie scripts	304K
Persona-chat (Zhang et al., 2018)	Crowdsourced	162K
The Ubuntu Dialogue Corpus (Lowe et al., 2015)	Ubuntu chat logs	7M
Twitter Triple Conversations (Sordoni et al., 2015)	Social media	6K
OpenSubtitles (Lison and Tiedemann, 2016)	Subtitles	441M sentences
SubTle (Eng) (Ameixa et al., 2013)	Subtitles	3.3M pairs
London-Lund Corpus (Greenbaum and Svartvik, 1990)	Various sources	500K words
London-Lund Corpus 2 (Pöldvere et al., 2017)	Various sources	500K words
SPOLIN (yes-and only)	Improv, Movie scripts	26K pairs
SPOLIN-extended (yes-and only)	Improv, Movie scripts, subtitles	68K pairs
Table 5: A survey of publicly available English language text-based corpora frequently used for open-domain dialogue systems. The last two rows are our contribution. *Size is measured as the number of total utterances (dialogue turns) unless otherwise specified.
Many works have identified the same issues of repetitive or non-committal responses generated by neural conversational systems that are at least partially related to the lack of sufficiently high quality yes-ands we deal with in this work; approaches that mitigate these problems vary. The majority of recent works focus on diversifying the responses by modifying the training and decoding objectives (Li et al., 2016a,b, 2017a, 2016c; Xu et al., 2017; Shao et al., 2017). Other methods introduce latent variables to encourage diversity (Serban et al., 2017; Zhao et al., 2017). Some explore methods of re-weighing training instances that encourage diversity (Liu et al., 2018; Lison and Bibauw, 2017; Du and Black, 2019).
Our approach is complementary to all the model
based approaches described here, as it simply deals with the production of a particularly useful corpus, that can be used to fine-tune on top of these methods.
We provide a survey of publicly available text-based datasets frequently used for open-domain dialogue systems and discuss their limitations for our purpose of generating grounded responses (see Table 5 for an overview).
DailyDialog is a collection of multi-turn dialogue with manually annotated emotion and intent labels (Li et al., 2017b). Danescu-Niculescu-Mizil and Lee (2011) created the Cornell Movie-Dialogs Corpus, a compilation of dialogue sequences paired with meta data about the movie and characters. Persona-chat provides dialogue sequence coupled with corresponding personas (Zhang et al., 2018).
The Ubuntu Dialogue Corpus contains 1 million dialogue turns extracted from Ubuntu chat logs, which discuss Ubuntu-related technical support (Lowe et al., 2015). The Twitter Triple Corpus is a dataset of 4K dialogue triples extracted from Twitter (Sordoni et al., 2015). OpenSubtitles is a huge collection of subtitles that span various genres, but the absence of speaker turn annotations make it difficult to modify into dialogue format (Lison and Tiedemann, 2016). Ameixa et al. (2013) use heuristics to reformat OpenSubtitles into dialogues with some limited success. Clark and Schaefer (1989) illustrate grounding in conversations with examples from the London-Lund Corpus (Greenbaum and Svartvik, 1990), a corpus of full conversations annotated with prosodic and paralinguistic features. A second version of the corpus was compiled with the same annotations standards as the first using more recent spoken and text data (Pöldvere et al., 2017).
These corpora were not collected with the criteria for yes-ands in mind. Even for datasets with dialogue taking place in a similar domain as improv, they naturally contain only a small proportion of yes-ands. However, the relatively large sizes of these datasets still make them useful for dialogue systems. They can be used effectively for grounded conversations if the yes-ands or other desirable dialogue acts can be filtered out or given higher weights in training to enforce their characteristics in the responses generated.
Our data collection approach is similar to the method of Yarowsky (1995), which formalizes the bootstrapping mechanism of iteratively improving a classifier and label unlabeled data. The main difference from the Yarowsky algorithm and our approach is that, rather than using a fully automated process for increasing training data, we use a probability threshold to regulate recall, followed by human judgment to ensure high precision.
Apart from Clark and Schaefer (1989) there have been other taxonomies of grounding. For example, Traum (1999) considers six categories; among these are acknowledge and continue, which, taken together, map nicely to yes-and. Magerko et al. (2009) and Fuller and Magerko (2010) note the importance of establishing common ground in improv.
Inspired by yes-ands in improv, we carefully construct SPOLIN, a collection of dialogue pairs with responses that are not only coherent with dialogue context but also initiate the next relevant contribution. We extract high-quality yes-ands from Spontaneanation and build a classifier with them, which is then used to mine additional yes-ands from the Cornell Movie-Dialogs Corpus. We further use our mining technique to elicit a corpus of more than 68,000 yes-and turn pairs, easily the largest collection of this dialogue act known to exist. From human evaluations of dialogue models trained with various data configurations we find that SPOLIN is useful—when including it we are able to build models that can generate yes-ands more consistently than when we leave it out. Nevertheless, our models are still inferior at producing good yes-ands when compared to professional improvisers. We plan to continue our data-driven approach for grounded conversations by expanding our dataset through our iterative data collection process with other larger text-based open-domain dialogue corpora and extend our work to model and collect longer conversations exhibiting more complex improv-backed turns.
Many thanks to Nanyun Peng and Xinyu Wang for key contributions in a preliminary study, to Paul F. Tompkins, Colin Anderson, and Earwolf for allowing us to include yes-ands extracted from Spontaneanation in SPOLIN, to Paul Elsberg, Risa Harms, P.T. McNiff, and Peter Schell for initial inspiration, and to Jordan Boyd-Graber for feedback on the final draft. This material is based on research sponsored by the AFRL and DARPA under agreement number FA8650-18-C-7878. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the AFRL, DARPA, or the U.S. Government.
David Ameixa, Luís Coheur, and Rua Alves Redol. 2013. From subtitles to human interactions: Introducing the SubTle corpus. Technical report, INESC-ID.
Satanjeev Banerjee and Alon Lavie. 2005. METEOR:
An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pages 65-72, Ann Arbor, Michigan. Association for Computational Linguistics.
David Bohm and Lee Nichol. 2004. On Dialogue. Routledge classics. Routledge.
Allison Bruce, Jonathan Knight, Samuel Listopad, Brian Magerko, and Illah R. Nourbakhsh. 2000. Robot improv: using drama to create believable agents. Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).
Carlos Busso and Shrikanth S Narayanan. 2008. Scripted dialogs versus improvisation: Lessons learned about emotional elicitation techniques from the IEMOCAP database. In Ninth annual conference of the international speech communication association.
Herbert H Clark and Edward F Schaefer. 1989. Contributing to discourse. Cognitive science, 13(2):259-294.
Cristian Danescu-Niculescu-Mizil and Lillian Lee. 2011. Chameleons in imagined conversations: A new approach to understanding coordination of linguistic style in dialogs. In Proceedings of the 2nd Workshop on Cognitive Modeling and Computational Linguistics, pages 76-87, Portland, Oregon, USA. Association for Computational Linguistics.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Wenchao Du and Alan W Black. 2019. Boosting dialog response generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 38-43, Florence, Italy. Association for Computational Linguistics.
Daniel Fuller and Brian Magerko. 2010. Shared mental models in improvisational performance. In Proceedings of the Intelligent Narrative Technologies III Workshop, INT3 '10, New York, NY, USA. Association for Computing Machinery.
Sidney Greenbaum and Jan Svartvik. 1990. The London-Lund corpus of spoken English, volume 7. Lund University Press.
Charna Halpern, Del Close, and Kim Johnson. 1994. Truth in comedy: The manual of improvisation. Meriwether Publishing.
Ari Holtzman, Jan Buys, Maxwell Forbes, and Yejin Choi. 2020. The curious case of neural text degeneration. In Proceedings of the Eighth International Conference on Learning Representations.
Minlie Huang, Xiaoyan Zhu, and Jianfeng Gao. 2020. Challenges in building intelligent open-domain dialog systems. ACM Transactions on Information Systems, 38(3):1-32.
Keith Johnstone. 2017. Impro: Improvisation and the Theatre. Performance Books. Bloomsbury Publishing.
Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016a. A diversity-promoting objective function for neural conversation models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 110-119, San Diego, California. Association for Computational Linguistics.
Jiwei Li, Will Monroe, and Dan Jurafsky. 2016b. A simple, fast diverse decoding algorithm for neural generation. arXiv preprint arXiv:1611.08562.
Jiwei Li, Will Monroe, Alan Ritter, Dan Jurafsky, Michel Galley, and Jianfeng Gao. 2016c. Deep reinforcement learning for dialogue generation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1192-1202, Austin, Texas. Association for Computational Linguistics.
Jiwei Li, Will Monroe, Tianlin Shi, Sébastien Jean, Alan Ritter, and Dan Jurafsky. 2017a. Adversarial learning for neural dialogue generation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2157-2169, Copenhagen, Denmark. Association for Computational Linguistics.
Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017b. DailyDialog: A manually labelled multi-turn dialogue dataset. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 986-995, Taipei, Taiwan. Asian Federation of Natural Language Processing.
Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74-81, Barcelona, Spain. Association for Computational Linguistics.
Pierre Lison and Serge Bibauw. 2017. Not all dialogues are created equal: Instance weighting for neural conversational models. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, pages 384-394, Saarbrücken, Germany. Association for Computational Linguistics.
Pierre Lison and Jörg Tiedemann. 2016. OpenSubtitles2016: Extracting large parallel corpora from movie and TV subtitles. In LREC.
Yahui Liu, Wei Bi, Jun Gao, Xiaojiang Liu, Jian Yao, and Shuming Shi. 2018. Towards less generic responses in neural conversation models: A statistical re-weighting method. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2769-2774, Brussels, Belgium. Association for Computational Linguistics.
Ryan Lowe, Nissan Pow, Iulian Serban, and Joelle Pineau. 2015. The Ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems. In Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 285-294, Prague, Czech Republic. Association for Computational Linguistics.
Brian Magerko, Waleed Manzoul, Mark Riedl, Allan Baumer, Daniel Fuller, Kurt Luther, and Celia Pearce. 2009. An empirical study of cognition and theatrical improvisation. In Proceedings of the Seventh ACM Conference on Creativity and Cognition, page 117-126, New York, NY, USA. Association for Computing Machinery.
Lara J. Martin, Brent Harrison, and Mark O. Riedl. 2016. Improvisational computational storytelling in open worlds. In Interactive Storytelling, pages 73-84, Cham. Springer International Publishing.
Siobhan McHugh. 2016. How podcasting is changing the audio storytelling genre. Radio Journal: International Studies in Broadcast & Audio Media, 14(1):65-82.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311-318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
Nele Pöldvere, V Johansson, and C Paradis. 2017. The London-Lund corpus 2: A new corpus of spoken British English in the making. In the ICAME 38 Conference, Prague, Czech Republic.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. Technical report, OpenAI.
Iulian Serban, Alessandro Sordoni, Yoshua Bengio, Aaron C. Courville, and Joelle Pineau. 2015. Hierarchical neural network generative models for movie dialogues. *ArXiv*, abs/1507.04808.
Iulian Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron Courville, and Yoshua Bengio. 2017. A hierarchical latent variable encoder-decoder model for generating dialogues. In Thirty-First AAAI Conference on Artificial Intelligence.
Louis Shao, Stephan Gouws, Denny Britz, Anna Goldie, Brian Strope, and Ray Kurzweil. 2017. Generating long and diverse responses with neural conversation models. arXiv preprint arXiv:1701.03185.
Alessandro Sordoni, Michel Galley, Michael Auli, Chris Brockett, Yangfeng Ji, Margaret Mitchell, Jian-Yun Nie, Jianfeng Gao, and Bill Dolan. 2015. A neural network approach to context-sensitive generation of conversational responses. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 196-205, Denver, Colorado. Association for Computational Linguistics.
Viola Spolin, Arthur Morey, and Mary Ann Brandt. 1986. Theater Games for the Classroom: A Teacher's Handbook. Northwestern University Press.
David R Traum. 1999. Computational models of grounding in collaborative systems. In Psychological Models of Communication in Collaborative Systems-Papers from the AAAI Fall Symposium, pages 124-131.
Lauren Winston and Brian Magerko. 2017. Turn-taking with improvisational co-creative agents. In Thirteenth Artificial Intelligence and Interactive Digital Entertainment Conference.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumont, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R'emi Louf, Morgan Funtowicz, and Jamie Brew. 2019a. Huggingface's transformers: State-of-the-art natural language processing. ArXiv, abs/1910.03771.
Thomas Wolf, Victor Sanh, Julien Chaumond, and Clement Delangue. 2019b. Transfertransfo: A transfer learning approach for neural network based conversational agents. arXiv preprint arXiv:1901.08149.
Zhen Xu, Bingquan Liu, Baoxun Wang, Chengjie Sun, Xiaolong Wang, Zhuoran Wang, and Chao Qi. 2017. Neural response generation via GAN with an approximate embedding layer. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 617-626, Copenhagen, Denmark. Association for Computational Linguistics.
David Yarowsky. 1995. Unsupervised word sense disambiguation rivaling supervised methods. In 33rd Annual Meeting of the Association for Computational Linguistics, pages 189-196, Cambridge, Massachusetts, USA. Association for Computational Linguistics.
Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational
Linguistics (Volume 1: Long Papers), pages 2204-2213, Melbourne, Australia. Association for Computational Linguistics.
Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. 2017. Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 654-664, Vancouver, Canada. Association for Computational Linguistics.
Iteration	4	5	6	7
Spontaneanation +	10,459	10,459	10,459	10,459
Spontaneanation -	5,587	5,587	5,587	5,587
Cornell +	12,220	14,976	14,976	14,976
Cornell-	17,092	17,701	17,701	17,701
SubTle +	-	2,621	20,617	33,155
SubTle-	-	7,865	14,799	17,325
Total Training Samples	45,358	59,209	84,319	99,203
Dev Set Acc. (Spont)	73.0%	72.1%	68.4%	75.2%
Dev Set Acc. (Cornell)	64.5%	63.3%	63.3%	61.0%
Confidence Threshold	50% / 70%*	70%	70%	70%
New Extraction Volume	3,515 / 10,486*	36,608	15,424	14,979
New Proportion of yes-ands	78.4% / 25.0%*	58.4%	83.2%	76.0%
Table 6: Continuation of Table 1 with the extended version of SPOLIN that includes extracted yes-ands from SubTle. SubTle is collected from the fourth iteration onwards. *Statistics for Cornell/SubTle are shown separately. The same classifier is used for extracting candidates from Cornell and SubTle, but they are datasets with significantly different characteristics.
A.1 yes-and Guidelines for Turkers
We provide detailed annotation guidelines, shown in Figures 6-9, to the Turkers as a result of having continuous discussions with them and monitoring their submissions. Contrary to our expectations, it is difficult to make a binary decision on whether a dialogue turn is a yes-and or non-yes-and, and therefore these fine-grained details are crucial for collecting yes-ands in SPOLIN.
A.2 Iterative data collection results for SubTle
Due to SubTle's relatively large size, we split SubTle into 20 equal blocks that each contains 10,486 dialogue turns. Note that every successive iteration of SubTle was not performed on the same block but on the next block. This is different from Cornell, for which every iteration is on the same set of dialogue turns. This difference is not due to any characteristics in the dataset but because of practical reasons arising from the size of the SubTle corpus.