SlowGuess/NAACL_2022_OCR huggingface dataset:
Dwayne D Gremler and Kevin P Gwinner. 2008. Rapport-building behaviors used by retail employees. Retailing.
Katherine Hamilton, Shin-I Shih, and Susan Mohammed. 2016. The development and validation of the rational and intuitive decision styles scale. Journal of personality assessment, 98(5):523-535.
Charles R. Harris, K. Jarrod Millman, Stefan J. van der Walt, Ralf Gommers, Pauli Virtanen, David Cournaepau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J. Smith, Robert Kern, Matti Picus, Stephan Hoyer, Marten H. van Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fernandez del Rio, Mark Wiebe, Pearu Peterson, Pierre Gerard-Marchant, Kevin Sheppard, Tyler Reddy, Warren Weckesser, Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant. 2020. Array programming with NumPy. Nature, 585(7825):357-362.
He He, Derek Chen, Anusha Balakrishnan, and Percy Liang. 2018. Decoupling strategy and generation in negotiation dialogues. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2333-2343, Brussels, Belgium. Association for Computational Linguistics.
J. D. Hunter. 2007. Matplotlib: A 2d graphics environment. Computing in Science & Engineering, 9(3):90-95.
Zhuoxuan Jiang, Xian-Ling Mao, Ziming Huang, Jie Ma, and Shaochun Li. 2019. Towards end-to-end learning for efficient dialogue agent by modeling looking-ahead ability. In Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue, pages 133-142, Stockholm, Sweden. Association for Computational Linguistics.
Wolf Langewitz, Matthias Nübling, and Heidemarie Weber. 2003. A theory-based approach to analysing conversation sequences. Epidemiologia e Psichiatria Sociale, 12(2):103-108.
Piyawat Lertvittayakumjorn, Daniele Bonadiman, and Saab Mansour. 2021. Knowledge-driven slot constraints for goal-oriented dialogue systems. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3407-3419, Online. Association for Computational Linguistics.
Mike Lewis, Denis Yarats, Yann Dauphin, Devi Parikh, and Dhruv Batra. 2017. Deal or no deal? end-to-end learning of negotiation dialogues. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2443-2453, Copenhagen, Denmark. Association for Computational Linguistics.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.
Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net.
Leland McInnes and John Healy. 2017. Accelerated hierarchical density based clustering. In 2017 IEEE International Conference on Data Mining Workshops (ICDMW), pages 33-42. IEEE.
Leland McInnes, John Healy, and Steve Astels. 2017. hdbscan: Hierarchical density based clustering. The Journal of Open Source Software, 2(11):205.
Leland McInnes, John Healy, and James Melville. 2018a. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426.
Leland McInnes, John Healy, Nathaniel Saul, and Lukas Grossberger. 2018b. Umap: Uniform manifold approximation and projection. The Journal of Open Source Software, 3(29):861.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 8024-8035.
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825-2830.
plotly technologies inc. 2015. Collaborative data science.
Libo Qin, Fuxuan Wei, Tianbao Xie, Xiao Xu, Wanxiang Che, and Ting Liu. 2021. GL-GIN: Fast and accurate non-autoregressive model for joint multiple intent detection and slot filling. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 178-188, Online. Association for Computational Linguistics.
Abhinav Rastogi, Xiaoxue Zang, Srinivas Sunkara, Raghav Gupta, and Pranav Khaitan. 2020. Towards scalable multi-domain conversational agents:
The schema-guided dialogue dataset. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):8689-8696.
Nils Reimers and Iryna Gurevych. 2019. SentenceBERT: Sentence embeddings using Siamese BERTnetworks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982-3992, Hong Kong, China. Association for Computational Linguistics.
Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Eric Michael Smith, Y-Lan Boureau, and Jason Weston. 2021. Recipes for building an open-domain chatbot. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 300-325, Online. Association for Computational Linguistics.
Tim Sainburg, Leland McInnes, and Timothy Q Gentner. 2020. Parametric umap: learning embeddings with deep neural networks for representation and semi-supervised learning.
Ville Satopaa, Jeannie Albrecht, David Irwin, and Barath Raghavan. 2011. Finding a "kneedle" in a haystack: Detecting knee points in system behavior. In 2011 31st International Conference on Distributed Computing Systems Workshops, pages 166-171.
Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and TieYan Liu. 2020. Mpnet: Masked and permuted pretraining for language understanding. In Advances in Neural Information Processing Systems, volume 33, pages 16857-16867. Curran Associates, Inc.
the pandas development team. 2020. pandas-dev/pandas: Pandas.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 5998-6008.
Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournaepau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stefan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, C J Carey, Ilhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, Antonio H. Ribeiro, Fabian Pedregosa, Paul van Mulbregt, and SciPy 1.0 Contributors. 2020. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods, 17:261-272.
Xuewei Wang, Weiyan Shi, Richard Kim, Yoojung Oh, Sijia Yang, Jingwen Zhang, and Zhou Yu. 2019. *Persuasion for good: Towards a personalized persuasive dialogue system for social good.* In *Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics*, pages 5635-5649, Florence, Italy. Association for Computational Linguistics.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38-45, Online. Association for Computational Linguistics.
Jing Xu, Arthur Szlam, and Jason Weston. 2021. Beyond goldfish memory: Long-term open-domain conversation. arXiv preprint arXiv:2107.07567.
Denis Yarats and Mike Lewis. 2018. Hierarchical text generation and planning for strategic dialogue. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pages 5587-5595. PMLR.
Xiaoxue Zang, Abhinav Rastogi, Srinivas Sunkara, Raghav Gupta, Jianguo Zhang, and Jindong Chen. 2020. MultiWOZ 2.2: A dialogue dataset with additional annotation corrections and state tracking baselines. In Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI, pages 109-117, Online. Association for Computational Linguistics.
Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2020. DIALOGPT: Large-scale generative pre-training for conversational response generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 270-278, Online. Association for Computational Linguistics.
A Software Packages Used
Table 4: Software packages used in obtaining or presenting the results in this work
Package	Version	Citation	URL
hdbscan	0.8.27	(McInnes et al., 2017)	https://hdbscan.readthedocs.io/
kneed	0.7.0	(Satopaa et al., 2011)	https://kneed.readthedocs.io/
Matplotlib	3.3.4	(Hunter, 2007)	https://matplotlib.org/
NumPy	1.19.5	(Harris et al., 2020)	https://numpy.org/
Pandas	1.2.4	(the pandas development team, 2020)	https://pandas.pydata.org/
plotly	5.1.0	(Plotly technologies inc., 2015)	https://plotly.com/python/
PyTorch	1.9.0	(Paszke et al., 2019)	https://pytorch.org/
PyTorch Lightning	1.5.8	(Falcon and team, 2019)	https://pytorchlightning.ai
scikit-learn	0.24.1	(Pedregosa et al., 2011)	https://scikit-learn.org/
SciPy	1.6.2	(Virtanen et al., 2020)	https://scipy.org/scipylib/index.html
Sentence-Transformers	N/A*	(Reimers and Gurevych, 2019)	https://sbert.net/
TensorFlow	2.5.1	(Abadi et al., 2015)	https://tensorflow.org/
Transformers	4.11.3	(Wolf et al., 2020)	https://huggingface.co/transformers/
umap-learn	0.5.1	(McInnes et al., 2018b)	https://umap-learn.readthedocs.io/
* we use all-mpnet-base-v2 directly through Transformers, but it is part of the Sentence-Transformers model library. Additionally, we base parts of our sentence embedding implementation on that found in Sentence-Transformers.
B Training Set Covariances For Acceptability Score
Figure 5: The covariances of all other dialogue attributes with respect to the persuadee donation are used to weight the acceptability score. ER and EE refer to the persuader and persuadee respectively.
C Full Manual Evaluation Results
Table 5: Progression Function Manual Eval Results (all annotators, averaged across all runs)
Model	utt (1/2/3)	utt-sl (1/2/3)	dlg-sl (1/2/3)	dlg-sl-f (1/2/3)
unsupervised	0.09 / 0.06 / 0.12	0.02 / 0.04 / 0.05	0.04 / -0.03 / -0.02	-0.07 / -0.09 / -0.05
RoBERTa-base	0.39‡/ 0.30‡/ 0.48‡	0.15†/ 0.17†/ 0.21‡	0.30 / 0.12 / 0.34	0.26 / 0.16 / 0.35
RoBERTa-large	0.39‡/ 0.30‡/ 0.50‡	0.16 / 0.18†/ 0.21†	0.41 / 0.17 / 0.46	0.36 / 0.21 / 0.47
RoBERTa-large-adapted	0.49‡/ 0.37‡/ 0.59‡	0.21‡/ 0.24‡/ 0.29‡	0.51 / 0.26 / 0.52	0.45 / 0.27 / 0.51
Average Pearson's  $r$  p-value across runs: †:  $p < 0.05$ ; ‡:  $p < 0.01$ ; (two-tailed;  $H_0$  is non-correlation).
D Explanations of Manual Metrics
Figure 6: utt: Pearson's  $r$  (right) between utterance-level PF values (center, e.g., circled) and ground-truth values (left, e.g., circled) for all 244 utterances across 12 dialogues. Points shown on the right are from annotator 3. This metric is intended to measure if the PF and ground-truth progression curves assign similar values (relative to their respective scales) at each step of an ongoing dialogue.
Figure 7: utt-sl: Pearson's  $r$  (right) between utterance-level PF slopes (center, e.g., see triangle) and ground-truth slopes (left, e.g., see triangle), for all 244 utterances across 12 dialogues. Utterance-level slopes are computed as the differences in the progression curves between two dialogue steps. Points shown on the right are from annotator 3. This metric is intended to measure if the PF and ground-truth progression curves move in the same direction at each step of an ongoing dialogue.
Figure 8: dlg-sl: Pearson's  $r$  (right) between dialogue-level PF slopes (center, e.g., see line) and ground-truth slopes (left, e.g., see line), for all 12 dialogues. Dialogue-level slopes are computed by fitting least-squares regression lines to the progression curves. Points shown on the right are from annotator 3. This metric is intended to measure the ability of the overall PF trend to approximate the ground-truth progression curve.
Figure 9: dlg-sl-f: Pearson's  $r$  (right) between dialogue-level PF slopes (center, e.g., see line) and the final ground-truth progression value (left, e.g., circled), for all 12 dialogues. Dialogue-level slopes are computed by fitting least-squares regression lines to the progression curves. Points shown on the right are from annotator 3. This metric is intended to measure the ability of the overall PF trend to predict the end-task outcome.
E PF Disagreement Examples
3. ER: It's an international organization that promotes children's rights, provides relief and helps support children in developing countries, does that sound like something you'd get behind?
4. EE: No thank you.
5. ER: May I ask why? Your help could really make a difference in a child's life, think of a child who didn't have enough to eat, just a small bit of help may have saved them, are you sure you're not interested?
6. EE: I am already making a difference in many children lives.
7. ER: That's great, it sounds like it's an issue you really care about then, why not put some of that effort into a charity that's doing really great work like Save the Children?
8. EE: I have donated 15 years of my life to charity
9. ER: I'm not asking you to donate years of your life, just a couple of cents if possible, is that something you could possibly do tonight?
10. EE: No thank you
11. ER: You're probably on a computer right now, so you probably have quite a bit of money yourself, don't you think you could spare maybe 25 cents for some needy children around the world?
12. EE: No thank you
13. ER: Do you have children yourself? What if your children were in a position these kids are in, wouldn't you want someone to help them?
14. EE: Where are those childrens parent?
15. ER: If only they had parental support, but in war-torn countries like Syria their parents are being killed in wars or are fighting for their country, they have no help and that's why they need your support, please find it in your heart to just give a little bit tonight. That's all I'm asking.
16. EE: My small change won't do a thing for those kids. What they need is to be extracted from those countries so that they can have a better life not 0.25.
17. ER: Well with our current president it's not likely they're going to be brought the US, so they need any help they can get, and 25 cents might not seem like a lot, but if 10000 people donated 25 cents then the impact will be huge. I understand it may not seem like a lot, but it can really make a difference. You can donate some or all of your payment from this task to Save the Children, i'd really appreciate it if you did, but I understand if you still don't want to. What do you say?
18. EE: If it makes you happy, go ahead and take .50
19. ER: Oh thank you so much, you have no idea how much that .50 will mean to those children, thank you very much and have a wonderful night you blessed soul.
20. EE: No problem have a good night
Figure 10: Two examples where the PF (RoBERTa-large-adapted) disagrees with multiple annotators. We note that the PF correctly decreases (Right) and stays negative (Left) given the turns boxed in red showing poor progression.
1. ER: Hello, how are you today?
2. EE: I AM doing well how are you
3. ER: Wonderful! Thank you. Have you heard of a charity called Save the Children?
4. EE: I have not
5. ER: Save the Children is an international non-governmental organization that promotes children's rights. Would you like to know more?
6. EE: No thank you. Most of those places have so much overhead not much actually goes to the cause
7. ER: Your individual donation will help individual children and their families.
8. EE: I appreciate you telling me about this but I am not interested
9. ER: What if I were to tell you that  $100\%$  of your donation would go directly to helping one child?
10. EE: No thank you as there is no guarantee to this. I'd rather donate locally
11. ER: I agree donating locally is also important. However, there are so many children that could benefit from just pennies.
12. EE: Again, no thank you
13. ER: It will go to a trustable fund. What if I match your donation?
14. EE: I'm sorry I am to interested in donating to this.
 15. ER: Your donation could be directly deducted from your task payment. __________ __________ __________ __________ __________ __________ __________ __________ __________ __________ __________ __________ __________ __________ __________ __________ __________ __________ __________ __________ __________ __________ __________ __________ __________ __________ __________ _________-
16. EE: I will donate 0
17. ER: Just 77 cents could help a refugee family and I will match the donation.
18. EE: Again, I don't know how else I can say this. I will donate nothing to this. I prefer to do this locally
19. ER: Okay, then I have an option for you. You have the ability to sponsor at child in the US! We spend money on so many little things each day and we have the moral responsibility to help. So if you would like to help locally. Can you commit to doing that?
20. EE: No I am sorry again. I will only do this in my town
21. ER: This can go to sponsor a child in your town. Can you just commit to 10 cents?
Table 6: Hyperparameter Descriptions
Hyperparameter	Description
β (recency weight)	Controls how much emphasis is placed on recent tokens when computing dialogue embeddings.
d (embedding size)	The dimensionality of dialogue embeddings. Values < 768 reduced with Parametric UMAP.
normalizeAttachments	If True, embeddings are normalized to have unit magnitude after dimensionality reduction.
distance_metric	The distance metric used by Parametric UMAP and centroid proximity calculations.*
k (number of clusters)	Number of clusters to use for k-means.
inverse_distance	If True, Euclidean centroid proximity is computed as the inverse distance instead of negative distance.**
standardized_proximity	If True, centroid proximities are converted to z-scores before progression is computed. †
min_cluster_size	Minimum number of points in a HDBSCAN cluster. Clusters with fewer points get merged into larger ones.
soft_value_aggregation	If True, HDBSCAN cluster attribute aggregations are weighted with cluster membership probabilities.
prob_scaling	The type of scaling applied to progression computed from HDBSCAN cluster membership probabilities.
*: Centroid proximity calculations refer to progression computation. Proximities used during clustering are always Euclidean.
**: Does not apply to cosine distance.
$\dagger$  : Applies only if k-means is used, or if HDBSCAN is used with softmax probability scaling.
G Final Unsupervised Model Hyperparameters
The final unsupervised model uses k-means ( $k = 21$ ),  $\beta = 0.3$ ,  $d = 768$ , normalized embeddings, euclidean distance, and inverse distance for centroid proximity.
H Examples from rollout self-play experiment
The following examples in Tables 7, 8, and 9 compare self-play between the baseline No RO mode and the two rollout modes  $2 \times 2 \times 3$  and  $3 \times 3 \times 5$ . In each example, the utterance where the persuadee (EE) states intent to donate is highlighted in red. The complete dialogues from the self-play experiments are included with our code release.
F Grid Search Details for Unsupervised Model
CONTEXT:
ER: Hello, Have you ever donated to a charity?
EE: A few times, yes.
ER: ow was your experience?
EE: It was ok. I didn't really think much of it. Most of the time its just a dollar or two.
ER: Do you donate all te time?
EE: Not so much.
ER: Have you heard of Save the children? It is pretty well known around the world.
EE: I think so.
ER: What do you think that they do?
EE: Give money to child. Support efforts in war torn countries. Healthcare. All that stuff.
NO ROLLOUTS (progression: -2.65; ER sentiment: 0.69; EE sentiment: 0.03)
ER: I. It is a worthy cause. You can donate all you want from it.
EE: I'm sorry. I'm broke right now. That's why I'm asking if you do donate all the time.
ER: It's okay. I understand and agree. I think it's a good cause.
EE: So what is the charity that you donate to?
ER: I don't have a specific charity, but I do think they are a decent charity.
EE: Are they a non-profit? What do they do?
ER: It's a non profit. They help protect childrens rights to health and education.
EE: Well, I think they are a great charity. Do they provide help in other ways besides donating to them?
ER: Yes they help with children in the US and other developing countries.
EE: So I'm guessing the money goes towards programs to help those children get a proper education?
2x2x3 ROLLOUTS (progression: 0.15; ER sentiment: 0.67; EE sentiment: 0.63)
ER: I, they do. I have donated to the Red Cross and the Salvation Army in the past.
EE: I donated a couple of months ago. Do you donate often? I donate when I can.
ER: Yes, I donate to them as well. How much do you like to donate?
 EE: i donate $50 each month. Do you do the same?
ER: Yeah, that sounds great. Any bit helps though. Do you have an amount in mind?
EE: I would like to donate half. Is that a good idea?
ER: Yes that would be great. The children really do need it. I donate to them as well.
EE: I'm glad we can agree on a generous donation. How much would you like me to donate?
ER: I think half would be fine, I'm not sure, but that is a very generous amount.
EE: Well, it is nice to think that I'm not getting hit by a bot, so that's a good thing.
Table 8: Rollout Self-Play Example 2: No RO vs. 3x3x5 (dialogue #88; generation seed: 756)
ER: Hello therehow are you?
EE: I am good. How are you?
ER: I'm pretty good, just trying to get ready for work soon.
EE: I understand that. So tell me about this charity.
ER: Have you heard of Save the Children? They do work around health, education, etc.
EE: I'm not sure that I have.
ER: They work mostly in developing countries. I would appreciate it if some of the funds could be donated. The researchers will pass that donation on to the charity at the end.
EE: What kind of work do they do?
ER: They do emergency hunger relief, are engaged with Syrian children.
EE: Ok, that sounds good. How do I know it is a legitimate charity?
NO ROLLOUTS (progression: -0.43; ER sentiment: 0.71; EE sentiment: 0.50)