SlowGuess/NAACL_2021_OCR huggingface dataset:
Discovering the integrative potential: Integrative potential in a negotiation is based on how aligned the partner preferences are. Complete alignment leads to a distributive (or zero-sum) negotiation, having a low integrative potential where the benefit of one results in a high loss for the other. A negotiation is integrative if the preferences do not align, allowing for solutions that maximize mutual points. We assign each dialogue either 1, 2, or 3, depending on whether the integrative potential is low, medium, or high. The maximum joint points possible in these cases are 36, 39, and 42 respectively. We find that the participants are able to discover this integrativeness, thereby achieving significantly more joint points as the potential increases ( $r = 0.425$ ,  $p < 0.001$ ).
Use of negotiation strategies: Overall, we find that greater use of Prosocial strategies shows a general pattern to predict higher ratings for both subjective measures of satisfaction and likeness, for self as well as the partner. Engaging in small talk shows significant positive correlations ( $ps < 0.01$ ), confirming our hypothesis from prior work that it relates to healthier relationships among the negotiators. Similar effects are visible for No-Need ( $ps < 0.05$ ), where the participant decides to let go one of their low-priority items. Since this directly benefits the opponent, it is likely to improve the participant's perception. On the other hand, Proself strategies show a general pattern to predict lower satisfaction and likeness ratings for both self and the partner. We observe significant negative correlation for both Other-Need and Vouch-Fair
$(ps < 0.01)$ . Further, we find that these competitive strategies are also associated with lower points scored by the participant and the opponent, and hence, the joint points  $(ps < 0.01)$ . These correlations are not influenced by the integrative potential in the scenario, as when the integrated potential is controlled for, the effects generally remain unchanged and demonstrate the same patterns.
We further observe that the dialogue behavior of a negotiator significantly relates to the behavior of their opponent, where both tend to use similar negotiation strategies  $(ps < 0.01)$ . Our findings show that Prosocial strategies are more likely to be associated with Prosocial behavior in the opponents and achieve more favorable outcomes in our negotiation scenario as compared to Proself. These results suggest that an automated negotiator can benefit by employing different strategies based on Prosocial or Proself behaviors of the opponent, for instance, by matching Prosocial behaviors but not Proself. The first step in this direction is to recognize them in a given utterance, which is our focus in the next section.
5 Strategy Prediction
For building an automated dialogue system that incorporates the negotiation strategies discussed above, an important first step is to build computational models that recognize their usage in the observed utterances. Hence, we explore the task of strategy prediction, given an utterance and its previous dialogue context.
Pre-trained models have proved to be useful on a number of supervised tasks with limited in-domain datasets. Inspired by this success, we use BERT-base (Devlin et al., 2019) as the core encoding module. A natural way to use pre-trained models for our task is to fine-tune the model for every label independently in a binary classification setup, where the positive class represents the presence of a strategy, and the negative represents its absence. However, most of the utterances in the CaSiNo-Ann dataset are Non-strategic, resulting in a high imbalance where most of the data points belong to the negative class. As we later show, directly fine-tuning the BERT model fails to recognize the strategies for which the data is most skewed.
We instead propose a multi-task learning framework to allow parameter sharing between the dif
Figure 1: Architecture for multi-task strategy prediction. + represents element-wise summation.
ferent prediction tasks. Our architecture involves a common BERT-base encoder shared with all the tasks but uses task-specific self-attention to allow the model to focus on the most relevant parts of the input for each task separately. Consequently, this also enables interpretability by allowing us to visualize which parts of an utterance are attended for any given strategy. Our input consists of a finite size context window, which loses the turn index for a specific utterance. Hence, we also capture the turn position for each utterance using sinusoidal positional embeddings (Vaswani et al., 2017). We present the complete architecture in Figure 1.
In-Domain Pre-Training (IDPT): CaSiNoAnn is nearly  $40\%$  of the entire CaSiNo dataset. To incorporate the unannotated dialogues, we employ In-Domain Pre-training of the BERT encoder (Sun et al., 2019). For this purpose, we consider each unannotated dialogue as a separate sequence and fine-tune the BERT-base architecture on the Masked Language Modelling (MLM) objective (Devlin et al., 2019). This allows us to use the complete CaSiNo dataset in a principled way.
5.2 Experiment Design
Evaluation Metrics: We compare our methods for each strategy label on F1-score for positive class (presence of strategy label). To capture the overall performance, we report average F1 across all labels with uniform weights. Inspired by Joint Goal Accuracy from Dialog State Tracking (Kumar et al., 2020), we define another overall met
ric called Joint-A, which measures the percentage of utterances for which the model predicts all the strategies correctly.
Methods: Fine-tuning the pre-trained models has achieved state-of-the-art results across many supervised tasks. Hence, our primary baseline is BERT-FT, which fine-tunes the BERT-base architecture for binary classification of each strategy label separately. We consider a Majority baseline, where the model directly outputs the majority class in the training data. We also implement a Logistic Regression model for each label separately based on a bag-of-words feature representation of the input utterance. We refer to this model as LR-BoW. We refer to our complete architecture presented in Figure 1 as Full, and consider its ablations by freezing the BERT layer (Freeze), removing task-specific self-attention (No Attn), or removing the turn position embeddings (No Feats). We also implement a simple over-sampling strategy where every utterance with at least one strategy is considered twice while training (referred to as OS). For IDPT, we fine-tune BERT for 20 epochs using a masking probability of 0.3. We also tried a lower masking probability of 0.15, however, in that case, the model is unable to learn anything useful on our relatively small dataset.
Training Details: Our context window considers past 3 utterances and concatenates them using an EOS token. The embedding dimension is 768 for the encoder and the task-specific self-attention layers, each having only one attention head. We use the turn position embeddings of 32 dimensions. We train the models with Adam optimizer with a learning rate of  $5e^{-05}$  and weight decay of 0.01. We use ReLU activation for feed-forward layers, and a dropout of 0.1 to prevent overfitting. The models were trained for a maximum of 720 iterations with a batch size of 64 ( $\sim$  13 epochs). We checkpoint and evaluate the model after every 72 iterations and the best performing checkpoint on a held-out  $5\%$  validation set is used for evaluation. We provide further training details including specifics of the architecture design, computing infrastructure, and hyper-parameter tuning in Appendix F.
Results: Table 3 summarizes the results on 5-fold cross-validation. Majority baseline fails to recognize any of the strategies due to the data being skewed towards the negative class. It still achieves  $39.4\%$  Joint-A, indicating that these many utterances have none of the seven strategies present.
Model	Small-Talk F1	Self-Need F1	Other-Need F1	No-Need F1	Elicit-Pref F1	UV-Part F1	Vouch-Fair F1	Overall F1	Joint-A
Majority	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	39.6
LR-BoW	64.6	57.2	43.2	17.5	56.5	14.3	50.4	43.4	52.4
BERT-FT	81.6	72.3	76.7	16.4	80.5	20.4	61.9	58.5	64.0
Multi-task training
Freeze	81.0	69.1	69.5	14.8	77.6	9.2	66.3	55.4	65.8
No Attn	80.7	71.9	76.8	7.5	79.0	23.2	60.6	57.1	67.8
No Feats	82.7	75.1	78.8	37.8	82.4	46.2	66.8	67.1	69.9
Full	82.7	74.4	77.9	36.4	83.2	44.5	67.9	66.7	70.2
+OS	82.0	77.1	75.6	44.2	81.9	46.4	67.3	67.8	70.1
+IDPT	82.6	74.0	80.4	41.2	82.8	40.8	64.0	66.6	69.5
+IDPT+OS	82.6	75.2	78.8	46.2	81.8	47.3	66.1	68.3	70.2
Table 3: Performance on strategy prediction task for 5-fold cross validation. F1 score corresponds to the positive class.
Figure 2: Visualizing task-specific self-attention layers for two examples from the test dataset for the first cv fold. The heatmap shows the attention scores for each token in the utterance for corresponding strategy labels.
Incorporating the bag-of-words features, LR-BoW performs much better than Majority. BERT-FT highly improves the performance on all strategies except No-Need and UV-Part, for which the dataset is the most skewed. However, our Full multi-tasking framework is able to tackle the imbalance in these strategies through parameter sharing between all tasks. It achieves  $36.4\%$  F1 for No-Need and  $44.5\%$  F1 for UV-Part, indicating more than  $100\%$  improvements in both the cases. The model also improves F1 scores for all other metrics, but the improvement is not that substantial. Relatively lower scores for Freeze and No Attn suggest that both fine-tuning and task-specific attention layers are essential for the performance. Turn position embeddings, however, only help for a few strategies, indicating the diverse usage of strategies in CaSiNo-Ann. Overall, we find that using over-sampling and in-domain pre-training further helps the performance, especially for No-Need and UV-Part. Although there is no clear winner among OS and IDPT, our final model, Full+IDPT+OS, that combines both these strategies performs the best for us, achieving an overall F1 score of  $68.3\%$  and  $70.2\%$  Joint Accuracy.
Attention Visualization: To understand if the model learns meaningful representations, we vi
sualize the task-specific self-attention layers of the trained Full+IDPT+OS model. We consider two instances in Figure 2. For meaningful comparisons, the instances were picked randomly from the pool of all utterances that contain two strategies. As evident, the model is able to focus on the most relevant parts for each strategy label. For instance, in case of Other-Need, the scores are higher where the participant talks about their kids needing more food. The token we gets the most attention, which is commonly used by the participants when referring to group needs. We see similar trends in the second case as well. Remarkably, this suggests that although our annotations are at an utterance level, it might be possible to automatically retrieve the most relevant phrases for any given strategy – this requires further investigation which we aim to explore in the future.
Historically, negotiations have been widely studied across multiple disciplines, in game theory (Nash Jr, 1950), understanding human behaviour (Adair et al., 2001), and building automatic negotiation agents (Beam and Segev, 1997; Baarslag et al., 2016). Most efforts focused on agent-agent interactions (Williams et al., 2012;
Lin et al., 2014; Cao et al., 2018), although there is an increasing interest in human-agent negotiations (Mell and Gratch, 2017) as well. DeVault et al. (2015) used a multi-issue bargaining design similar to ours. However, they focus on face-to-face negotiations, including speech and virtual embodied systems, which can be interesting future extensions to our current focus in chat-based dialogue systems. Other datasets looked at negotiation dialogues such as game settings (Asher et al., 2016; Lewis et al., 2017), and buyer-seller negotiations (He et al., 2018). These datasets have fueled a number of efforts on developing negotiation systems (Cheng et al., 2019; Parvaneh et al., 2019) and building a negotiation coach (Zhou et al., 2019). Our focus is on campsite negotiations, targeting a realistic and a closed-domain environment.
Several other related efforts have explored problems between task-oriented and open-domain scenarios, such as persuasion for a charity (Wang et al., 2019), anti-scam (Li et al., 2020), collecting cards in a maze (Potts, 2012), and searching for a mutual friend (He et al., 2017). Instead, we focus on rich personal negotiations, which differ from these tasks in their ultimate goal and downstream applications.
We described the design and development of the CaSiNo dataset and the associated annotations. Our design is based on a relatable campsite scenario that promotes constrained, yet linguistically rich and personal conversations. We next plan to explore two main projects: first, extending the analysis to demographic and personality traits in the data, and second, using our insights towards the development of practical automated negotiation systems that engage in free-form dialogue and portray well-studied strategies from the prior negotiation literature. Our work fuels other tasks to advance the research in human-machine negotiations, such as predicting satisfaction and opponent perception from dialog behaviors, and building a feedback mechanism for skills training by identifying the use of pro-social versus pro-self strategies.
Finally, we note that there are many interesting extensions to our task design that make the scenario more complicated, but useful in specific realistic settings. For instance, incorporating more than two negotiating parties, and considering other modalities like facial expressions or embodied agents. In some realistic settings, the individual preferences
may change during the negotiation and our setup assumes a fixed set of preferences throughout. Further, in complex settings, it may be possible to break down an individual item and claim sub-parts, such as negotiating for who gets an orange, but one party ends up taking the husk and the other takes the pulp for their own purposes. This is again not considered in our work and opens up exciting avenues for future work.
8.1 Data Collection
Our study was approved by our Institutional Review Board (IRB). Each participant signed an Informed Consent document at the beginning of the study which covered the purpose of the study, warned about potential discomfort, and noted the collection of data and its later use. Further, the participants were informed that they can withdraw at any time. They were also instructed to not use any offensive or discriminative language. The compensation was determined in accordance with the fairness rules defined by our IRB approval process. Additionally, we release the anonymized version of the data for future work by the research community. All personally identifiable information such as MTurk Ids or HIT Ids was removed before releasing the data. Lastly, any mention of the demographics or the psychological personality of the participants is based on self-identified information in our pre-survey and standard procedures of collecting personality metrics in the literature.
8.2 Automatic Negotiation Systems
Students entering the modern workforce must have a number of interpersonal skills that are crucial across a wide range of jobs. One of the key interpersonal skills needed to address conflicts and work well with others is the ability to negotiate. Unfortunately, research shows that the average human is bad at negotiating. This can adversely impact work opportunities (Babcock and Laschever, 2009), legal settlements (Eisenberg and Lanvers, 2009), and cross-cultural border peace (Wunderle, 2007). The typical way to teach negotiation skills to students is by in-class simulations, which are expensive. Automated systems can dramatically reduce the costs of, and increase access to, negotiation training. Systems developed on CaSiNo would be useful in this context. Further, the techniques
developed find use-cases for advancing conversational AI and imparting the negotiation skills to existing AI assistants, making them more aware of our preferences and requirements. One such prototype is Google Duplex (Leviathan and Matias, 2018), where the AI system engages in a simple form of negotiation to book a haircut appointment over the phone.
How humans negotiate has been actively studied for decades in Economics, Psychology, and Affective Computing (Carnevale and Pruitt, 1992). With this huge progress in our understanding of human-human negotiations, ethics has also been a well-studied topic in the literature (Lewicki et al., 2016). Primary concerns include the acts of emotion manipulation, deception, bias, and misrepresentation. Naturally, these ethical concerns may creep into the automated systems, trained on a human-human negotiation dataset.
To mitigate these ethical impacts, we recommend that standard guidelines for deploying conversational AI assistants should be followed. It is essential to maintain transparency about the identity of the system. Ethical principles must be in place before the deployment of such systems with a regular update cycle. Our camping scenario is quite relatable to anyone who negotiates with the system, hence, it is important to be upfront about the potential behaviors of the deployed system. We recommend continuous monitoring by keeping humans in the loop, ensuring that the system is neither offensive nor discriminative. Further, it should be made easy for the users negotiating with the system to directly contact the team behind the deployment. Finally, any data which is collected during the deployment phase should be informed to the users and its future purpose should be properly laid out.
We would like to thank Shivam Lakhotia, along with colleagues at the Institute for Creative Technologies and Information Sciences Institute for their comments and helpful discussions. We further thank Mike Lewis, He He, Weiyan Shi, and Zhou Yu for their guidance. We also thank the anonymous reviewers for their valuable time and feedback. Our research was sponsored by the Army Research Office and was accomplished under Cooperative Agreement Number W911NF-20-2-0053. The views and conclusions contained in this document are those of the authors and should not be
interpreted as representing the official policies, either expressed or implied, of the Army Research Office or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.
Wendi L Adair, Tetsushi Okumura, and Jeanne M Brett. 2001. Negotiation behavior when cultures collide: the united states and japan. Journal of Applied Psychology, 86(3):371.
J Stacy Adams. 1976. The structure and dynamics of behavior in organizational boundary roles. Handbook of industrial and organizational psychology, 1175:1199.
Nicholas Asher, Julie Hunter, Mathieu Morey, Benamara Farah, and Stergos Afantenos. 2016. Discourse structure and dialogue acts in multiparty dialogue: the stac corpus. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16), pages 2721-2727.
Reyhan Aydogan, Tim Baarslag, Katsuhide Fujita, Johnathan Mell, Jonathan Gratch, Dave de Jonge, Yasser Mohammad, Shinji Nakadai, Satoshi Morinaga, Hirotaka Osawa, et al. 2020. Challenges and main results of the automated negotiating agents competition (anac) 2019. In Multi-Agent Systems and Agreement Technologies, pages 366-381. Springer.
Tim Baarslag, Mark JC Hendrikx, Koen V Hindriks, and Catholijn M Jonker. 2016. A survey of opponent modeling techniques in automated negotiation. In 15th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2016, pages 575-576. International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).
Linda Babcock and Sara Laschever. 2009. *Women don't ask: Negotiation and the gender divide*. Princeton University Press.
Carrie Beam and Arie Segev. 1997. Automated negotiations: A survey of the state of the art. Wirtschaftsinformatik, 39(3):263-268.
Sandy Bogaert, Christophe Boone, and Carolyn Declerck. 2008. Social value orientation and cooperation in social dilemmas: A review and conceptual model. *British Journal of Social Psychology*, 47(3):453-480.
Kris Cao, Angeliki Lazaridou, Marc Lanctot, Joel Z Leibo, Karl Tuyls, and Stephen Clark. 2018. Emergent communication through negotiation. In International Conference on Learning Representations.
Peter J Carnevale and Dean G Pruitt. 1992. Negotiation and mediation. Annual review of psychology, 43(1):531-582.
Minhao Cheng, Wei Wei, and Cho-Jui Hsieh. 2019. Evaluating and enhancing the robustness of dialogue systems: A case study on a negotiation agent. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3325-3335.
Stephen W Clopton. 1984. Seller and buying firm factors affecting industrial buyers' negotiation behavior and outcomes. Journal of Marketing Research, 21(1):39-53.
Rachel G Curtis, Tim D Windsor, and Andrea Soubelet. 2015. The relationship between big-5 personality traits and cognitive ability in older adults—a review. Aging, Neuropsychology, and Cognition, 22(1):42-71.
David DeVault, Johnathan Mell, and Jonathan Gratch. 2015. Toward natural turn-taking in a virtual human negotiation agent. In AAAI Spring Symposia. Cite-seer.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186.
Theodore Eisenberg and Charlotte Lanvers. 2009. What is the settlement rate and why should we care? Journal of Empirical Legal Studies, 6(1):111-146.
Chaim Fershtman. 1990. The importance of the agenda in bargaining. Games and Economic Behavior, 2(3):224-238.
Lewis R Goldberg. 1990. "An alternative" description of personality": the big-five factor structure. Journal of personality and social psychology, 59(6):1216.
He He, Anusha Balakrishnan, Mihail Eric, and Percy Liang. 2017. Learning symmetric collaborative dialogue agents with dynamic knowledge graph embeddings. In 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, pages 1766-1776. Association for Computational Linguistics (ACL).
He He, Derek Chen, Anusha Balakrishnan, and Percy Liang. 2018. Decoupling strategy and generation in negotiation dialogues. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2333-2343.