Dialogue Act-based Breakdown Detection in Negotiation Dialogues (2021)

Abstract: Thanks to the success of goal-oriented negotiation dialogue systems, studies of negotiation dialogue have gained momentum in terms of both human-human negotiation support and dialogue systems. However, the field suffers from a paucity of available negotiation corpora, which hinders further development and makes it difficult to test new methodologies in novel negotiation settings. Here, we share a human-human negotiation dialogue dataset in a job interview scenario that features increased complexities in terms of the number of possible solutions and a utility function. We test the proposed corpus using a breakdown detection task for human-human negotiation support. We also introduce a dialogue act-based breakdown detection method, focusing on dialogue flow that is applicable to various corpora. Our results show that our proposed method features comparable detection performance to text-based approaches in existing corpora and better results in the proposed dataset.

## introduction negotiation is an essential task involved in our daily life. in negotiation, people work to maximize their profits by bargaining; however, negotiation sometimes breaks down due to conflicts between people's competing interests. to help them to reach rational agreement, previous studies of multiagent systems have proposed the use of negotiating agents @xcite @xcite . recently, several studies have succeeded in modeling a negotiating agent in natural language that can control both text generation and reasoning in the context of goaloriented dialogue systems, and such agents have produced better performance than human players in some cases @xcite @xcite . further, support for humanhuman negotiation in natural language has also been tackled, involving negotiation corpora developed for goal-oriented dialogue systems, such as a nash bargaining solution estimation @xcite , real-time negotiation coaching @xcite , and negotiation breakdown detection @xcite . although they have recently attracted additional attention, there are only few negotiation corpora, as the most recent follow-up studies @xcite @xcite have only utilized either the dealornodeal (dn) @xcite , craigslistbargain (cb) @xcite datasets or both. moreover, most existing corpora have simplified negotiation settings; for example, the dn dataset handles the negotiation of item division between humans with 22.5 possible solutions per dialogue and uses a standard linear additive utility function @xcite for scoring. the cb dataset is only concerned with price negotiation on a listed product between two human negotiators. these settings might make it easy for a machine learning (ml) model to reach optimal solution or fulfill its goal. finally, some existing corpora @xcite @xcite other than the dn and cb datasets have far smaller samples (scenarios), which makes it challenging to use them for goal-oriented dialogue systems or end-to-end human-human negotiation support. all of these factors inhibit further development in the field and its future applicability to real-world problems. furthermore, no effective breakdown detection method for negotiation dialogues has been proposed. negotiation features certain unique characteristics relative to other dialogues, such as offering proposals, accepting them, and making counter-offers @xcite . if the breakdown detection method can incorporate these characteristics, the quality of breakdown detection will be improved. this study proposes a new negotiation corpus in a job interview setting with increased complexities relative to a range of solutions and a utility function. we enact a breakdown detection task @xcite across three negotiation datasets including a proposed one with a novel dialogue act-based approach that can focus on dialogue flow. this task can support human-human negotiation by alerting negotiators to potential breakdowns, which prevents the loss of time and negotiator utility. we highlight the following contributions: ## task description task we formally define the task of breakdown detection in negotiation dialogues. let d be a negotiation dialogue between two negotiators, composed of @xmath0, s 2 , . . . , s n }, where each utterance s is a message from one of the negotiators and includes one or more sentences. given d, the task is to label d as either a success (reaching an agreement: 0) or a breakdown (failing to find an agreement: 1). evaluation metrics to evaluate the effectiveness of the different approaches, we employ area under curve (roc-auc) and confusion matrix (cm), both of which are based on @xcite . we also use average precision (ap) to consider the imbalanced nature of breakdown labels in negotiation datasets. ## methodology this section introduces our breakdown detection approach using a dialogue act-based feature and ml models, including linear and nn-based models. the intuition that guides this feature is that because a breakdown dialogue should have distinct flow (e.g., many disagreements), focusing on the dialogue flow can help detect this type of breakdown. ## conclusions and future work this study proposed a job interview negotiation dialogue dataset with 2639 dialogues and increased complexities compared to existing datasets to help propel development of the study of human-human negotiation support and goal-oriented dialogue systems. we also proposed a dialogue act-based breakdown detection model that can focus on negotiation flow. our approach (gru tag ) showed comparable results when used with existing datasets and better results for the proposed dataset than models trained with text-based features. in the future, we intend to explore another application of dialogue act-based features to related tasks, such as preference estimations. we will also utilize the proposed corpus in related tasks in human-human negotiation support and goal-oriented dialogue systems.