C a S i N o: A Corpus of Campsite Negotiation Dialogues for Automatic Negotiation Systems (2021)

Abstract: Automated systems that negotiate with humans have broad applications in pedagogy and conversational AI. To advance the development of practical negotiation systems, we present CaSiNo: a novel corpus of over a thousand negotiation dialogues in English. Participants take the role of campsite neighbors and negotiate for food, water, and firewood packages for their upcoming trip. Our design results in diverse and linguistically rich negotiations while maintaining a tractable, closed-domain environment. Inspired by the literature in human-human negotiations, we annotate persuasion strategies and perform correlation analysis to understand how the dialogue behaviors are associated with the negotiation performance. We further propose and evaluate a multi-task framework to recognize these strategies in a given utterance. We find that multi-task learning substantially improves the performance for all strategy labels, especially for the ones that are the most skewed. We release the dataset, annotations, and the code to propel future work in human-machine negotiations: https://github.com/kushalchawla/CaSiNo

## introduction negotiations are highly prevalent in our interactions, from deciding who performs the household chores to high-stake business deals to maintaining international peace. automatic negotiation systems are helpful in providing cost-effective social skills training @xcite and for advanced capabilities of ai assistants such as google duplex @xcite . a negotiation requires understanding the partner's motives along with effective reasoning and communication, which is challenging for an automated system. prior work in human-machine negotiations primarily uses strict communication protocols such as a pre-defined menu of options (mell * work done when authors were interns at usc ict @xcite . systems involving free-form dialogue are limited due to a lack of interdisciplinary efforts in nlp and computational social science in this direction. initial efforts in building dialogue systems for negotiations looked at game environments @xcite . dealornodeal @xcite involves two negotiators who split given quantities of three arbitrary items: books, balls, and hats. this provides a concrete structure to the negotiation, keeps the design tractable, and ensures a reliable evaluation based on final points scored. many practical solutions in negotiations follow similar closed-domain designs @xcite . however, most of the dialogues in these game settings reduce to merely an exchange of offers from both sides. for instance, 'i need the book and the balls you can have the hat' or 'i want the ball and 2 books' in dealornodeal. one reason for this lack of richness in language use is that the items are arbitrarily defined, that is, there is no semantic context around the items that the participants are negotiating for. hence, this setup fails to capture many realistic aspects of negotiations such as small talk, preference elicitation, emotion expression, and convincing strategies based on individual preferences and requirements. emulating real-world negotiations is desirable for developing practical systems for social skills training and robust ai assistants that are useful in realistic scenarios. on the other extreme, the cb dataset @xcite involves buyer-seller negotiations to finalize the price of a given product. targeting the collection of more open-ended dialogues, the participants are also encouraged to discuss side offers, such as free delivery or also selling other accessories at the same price. although this promotes diversity and rich natural conversations, unfortunately, such open-ended domains make the evaluation of negotiation performance non-trivial, which also inhibits the practical applicability of the systems developed on such datasets. for instance, in skills training, it is desirable to judge the performance and provide critical feedback @xcite . to address these shortcomings, we design a novel negotiation task. our design is based on a tractable closed-domain abstraction from the negotiation literature but is infused with a real-world camping scenario, resulting in rich dialogues for natural language research (section 2). the task involves two participants who take the role of campsite neighbors and negotiate for additional food, water, and firewood, based on individual preferences and requirements. based on this design, we collect casino: a corpus of 1030 camp site negotiation dialogues in english. the dialogues contain various aspects of a realistic negotiation, such as rapport building, discussing preferences, exchanging offers, emotion expression, and persuasion with personal and logical arguments. we also collect the participants' satisfaction from the outcome and how much they like their opponents, both being important metrics in negotiations @xcite . we annotate 9 persuasion strategies that span cooperative to selfish dialog behaviors (section 3). we perform an extensive correlational analysis to investigate the relationship among the final outcomes and explore how they relate to the use of negotiation strategies (section 4). further, we propose a multi-task framework with task-specific self-attention mechanisms to recognize these strategies in a given utterance (section 5). our insights form the foundation for the development of practical negotiation systems that engage in free-form natural conversations. we release the dataset along with the annotations to enable future work in this direction. ## the casino dataset our data was crowd-sourced on amazon mechanical turk. we describe our design by following the journey of a specific participant in our study. pre-survey: we start by collecting demographics and psychological personality traits of the participants which relate to their negotiation behaviors. for demographics, we gather age, gender, ethnicity, and the highest level of education. we consider two measures of individual personality differences: social value orientation or @xcite and big-5 personality (goldberg, 1990) that have been heavily studied in the context of negotiations @xcite . svo classifies the participants as prosocial, who tend to approach negotiations cooperatively, or proself, who tend to be more individualistic. big-5 personality test assesses the participants on five dimensions: extraversion, agreeableness, conscientiousness, emotional stability, and openness to experiences. our participants exhibit diverse demography and psychological personality. we provide aggregate statistics in appendix a. negotiation training: research shows that the average human is bad at negotiating @xcite , which can adversely impact the quality of the collected dialogues and consequently, the system trained on them. one way to mitigate this is by using reinforcement learning to optimize on a reward that measures the negotiation performance. rl training has proved to be challenging and often leads to degeneracy @xcite . further, this ignores prior work in human-human negotiations that provides guidelines for achieving favorable outcomes in realistic negotiations @xcite . to incorporate these best practices in a principled way, we design a training module. each participant is asked to watch a video tutorial before their negotiation. the tutorial takes an example of a negotiation between two art collectors to encourage them to follow some of the best practices in negotiations @xcite , including 1) starting with high offers, 2) discussing preferences, 3) appropriate emotion expression, and 4) discussing individual requirements to make convincing arguments. this results in a rich and diverse set of dialogues, as we explore further in later sections. we release the complete video tutorial publicly, with the hope that it promotes reproducibility and helps researchers to design similar data collection experiments in the future: https://youtu.be/7wly8qjjmty . ## strategy annotations after collecting the dataset, we developed an annotation schema to analyze the negotiation strategies used by the participants, and to facilitate future work. we follow the conceptual content analysis procedure @xcite to design the scheme. being a natural conversational dataset, we find several instances where a strategy spans multiple sentences in an utterance, as well as instances where the same sentence contains several strategies. hence, we define an utterance as the level of analysis. each utterance is annotated with one or more labels. if no strategy is evident, the utterance is labelled as non-strategic. although we label entire utterances, self-attention shows some promise as an automatic way to identify which part of an utterance corresponds to a given strategy, if desirable for a downstream application (section 5). human negotiation behaviors can be broadly cat-egorized as prosocial, which promote the interests of others or the common good, and proself, which tend to promote self-interest in the negotiations @xcite ). another important criterion is discussing preferences. prior work suggests that humans negotiate with a fixed-pie bias, assuming that the partner's preferences align, and hence achieving sub-optimal solutions @xcite . based on these distinctions and manual inspection, we define 9 strategies used in the casino dataset. the usage of these negotiation strategies correlates with both the objective and subjective metrics of negotiation performance. ## correlational analysis we next perform correlational analysis on casino-ann to understand how the points scored by a participant relate to their satisfaction from the outcome 1 researchers involved in the project. and their opponent perception. we further shed light on what kind of strategies are more likely to lead to better outcomes. such insights motivate our experiments on strategy prediction and would direct future efforts in building negotiation systems. we present complete results in appendix e and discuss the significant observations below. relationship among outcome variables: we consider the points scored, satisfaction from the outcome, and opponent likeness. we find that the points scored by a participant are positively correlated with their own satisfaction (@xmath0.376, @xmath1.01) and with their perception of the opponent (@xmath2.276, @xmath3.01). similar trends are visible with the corresponding variables of the negotiation partner as well, suggesting that the participants secured more points while still maintaining a positive perception in the eyes of their opponents. discovering the integrative potential: integrative potential in a negotiation is based on how aligned the partner preferences are. complete alignment leads to a distributive (or zero-sum) negotiation, having a low integrative potential where the benefit of one results in a high loss for the other. a negotiation is integrative if the preferences do not align, allowing for solutions that maximize mutual points. we assign each dialogue either 1, 2, or 3, depending on whether the integrative potential is low, medium, or high. the maximum joint points possible in these cases are 36, 39, and 42 respectively. we find that the participants are able to discover this integrativeness, thereby achieving significantly more joint points as the potential increases (@xmath4.425, @xmath5.001). use of negotiation strategies: overall, we find that greater use of prosocial strategies shows a general pattern to predict higher ratings for both subjective measures of satisfaction and likeness, for self as well as the partner. engaging in small talk shows significant positive correlations (@xmath6.01), confirming our hypothesis from prior work that it relates to healthier relationships among the negotiators. similar effects are visible for no-need (@xmath7.05), where the participant decides to let go one of their low-priority items. since this directly benefits the opponent, it is likely to improve the participant's perception. on the other hand, proself strategies show a general pattern to predict lower satisfaction and likeness ratings for both self and the partner. we observe significant negative correlation for both other-need and vouch-fair (@xmath8.01). further, we find that these competitive strategies are also associated with lower points scored by the participant and the opponent, and hence, the joint points (@xmath9.01). these correlations are not influenced by the integrative potential in the scenario, as when the integrated potential is controlled for, the effects generally remain unchanged and demonstrate the same patterns. we further observe that the dialogue behavior of a negotiator significantly relates to the behavior of their opponent, where both tend to use similar negotiation strategies (@xmath10.01). our findings show that prosocial strategies are more likely to be associated with prosocial behavior in the opponents and achieve more favorable outcomes in our negotiation scenario as compared to proself. these results suggest that an automated negotiator can benefit by employing different strategies based on prosocial or proself behaviors of the opponent, for instance, by matching prosocial behaviors but not proself. the first step in this direction is to recognize them in a given utterance, which is our focus in the next section. ## strategy prediction for building an automated dialogue system that incorporates the negotiation strategies discussed above, an important first step is to build computational models that recognize their usage in the observed utterances. hence, we explore the task of strategy prediction, given an utterance and its previous dialogue context. ## conclusions and future work we described the design and development of the casino dataset and the associated annotations. our design is based on a relatable campsite scenario that promotes constrained, yet linguistically rich and personal conversations. we next plan to explore two main projects: first, extending the analysis to demographic and personality traits in the data, and second, using our insights towards the development of practical automated negotiation systems that engage in free-form dialogue and portray well-studied strategies from the prior negotiation literature. our work fuels other tasks to advance the research in human-machine negotiations, such as predicting satisfaction and opponent perception from dialog behaviors, and building a feedback mechanism for skills training by identifying the use of pro-social versus pro-self strategies. finally, we note that there are many interesting extensions to our task design that make the scenario more complicated, but useful in specific realistic settings. for instance, incorporating more than two negotiating parties, and considering other modalities like facial expressions or embodied agents. in some realistic settings, the individual preferences may change during the negotiation and our setup assumes a fixed set of preferences throughout. further, in complex settings, it may be possible to break down an individual item and claim sub-parts, such as negotiating for who gets an orange, but one party ends up taking the husk and the other takes the pulp for their own purposes. this is again not considered in our work and opens up exciting avenues for future work.