Are LLM s Effective Negotiators? Systematic Evaluation of the Multifaceted Capabilities of LLM s in Negotiation Dialogues (2024)

Abstract: A successful negotiation requires a range of capabilities, including comprehension of the conversation context, Theory-of-Mind (ToM) skills to infer the partner’s motives, strategic reasoning, and effective communication, making it challenging for automated systems. Despite the remarkable performance of LLMs in various NLP tasks, there is no systematic evaluation of their capabilities in negotiation. Such an evaluation is critical for advancing AI negotiation agents and negotiation research, ranging from designing dialogue systems to providing pedagogical feedback and scaling up data collection practices. This work aims to systematically analyze the multifaceted capabilities of LLMs across diverse dialogue scenarios throughout the stages of a typical negotiation interaction. Our analysis highlights GPT-4’s superior performance in many tasks while identifying specific challenges, such as making subjective assessments and generating contextually appropriate, strategically advantageous responses.

## introduction large language models (llms), known for their impressive understanding and reasoning capabilities, are increasingly being utilized as ai negotiation agents @xcite . however, prior research on llm-based negotiation agents lacks a systematic analysis and evaluation of the diverse abilities required for effective negotiation @xcite @xcite . to develop more advanced ai negotiating agents, it is crucial to assess the various capabilities necessary for negotiation. systematic evaluation of llms through diverse negotiation tasks can provide interpretable insights by breaking down the complex negotiation process into single tasks. this approach significantly enhances not visible to p1 … p1 p1 figure 1 : overview of the key capabilities (c1-c4) required for a successful negotiation. we design tasks aligned with these abilities to assess how llms can advance different aspects of negotiation research. the negotiation scenario is based on @xcite . the utility and value of llms for various purposes in negotiation dialogues (e.g., designing end-toend @xcite and modular negotiation dialogue systems @xcite , for providing pedagogical feedback through tom @xcite , and for scaling up data collection practices by automating the collection of utterance-level annotations @xcite ). for a sophisticated evaluation of the multifaceted capabilities of llms in negotiation, it is essential to consider the core competencies required by agents in a realistic yet challenging bilateral multi-issue negotiation scenario @xcite , as illustrated in figure1. players begin with predefined value preferences for certain issues (e.g., food, water, firewood) and engage in dialogue to divide all available items, aiming to maximize their total value. to succeed, a negotiation agent must understand the negotiation context and outcome (c1), grasp the dialogue's structure (e.g., intent) and semantics (e.g., linguistic strategies) (c2), infer the partner's preferences through conversation (c3), and generate coherent, strategically apt responses given negotiation context and conversation (c4). these challenges highlight negotiation as a rigorous ai research testbed. to this end, we propose a comprehensive framework that systematically analyzes state-of-the-art llms in negotiations across 35 tasks, considering the varied skills required by competent negotiation agents. specifically, successful negotiation requires the theory of mind (tom) @xcite -the ability to understand the opponent's perspective-and the capacity to infer subjective aspects like affective states, beyond objective information. our framework encompasses a range of partner modeling and subjective tasks, defined along three crucial axes: 1) objectivity (objective vs. subjective tasks), 2) time stage of the negotiation (start, during, and end), and 3) task type (comprehension, annotation, partner modeling, and generation), aligned with the core competencies necessary for successful negotiations (figure 1 ). our contributions are as follows: ## methodology we evaluate the multifaceted capabilities of llms in negotiations with a three-step framework (figure 2): 1) designing tasks from human-human dialogue datasets, 2) constructing task-specific prompts, and 3) evaluating and analyzing various proprietary and open-source llms. ## experiment design our primary goal is to analyze the effectiveness of state-of-the-art llms for strategic negotiation interactions. here, we discuss the compared models and evaluation metrics for the results presented in section 4 (overall zero-shot results) and section 5 (task-specific analysis to gain further insights). ## discussion we now analyze the impact of popular prompting strategies, focusing primarily on gpt-4, given its superior overall performance, and also evaluate prediction balance and robustness in llms. cot prompting: we focus on tasks requiring multi-hop arithmetic reasoning, such as calculating the maximum possible points and inferring actual total points in a negotiation. using cot with gpt-4 yields nearly 100% accuracy on the tasks we tested (figure 5 ), highlighting its effectiveness for llms in negotiation tasks. a sample cot prompt is provided in ## conclusion we devise a methodology to systematically analyze the multifaceted capabilities of llms in negotiations. when evaluated out-of-the-box, gpt-4 outperforms and is more robust than other llms but still leaves room for improvement in most tasks. cot and few-shot prompting help improve performance in several arithmetic reasoning and annotation tasks. in contrast, smaller open-source models struggle, performing comparably to the trivial majority baseline for key comprehension and partner modeling tasks. based on our results, we conclude that llms can indeed be helpful across a number of use cases in negotiation research. this is not only limited to designing dialogue systems but also includes scaling-up pedagogical and data collection practices. llms capable of partner modeling can help to provide feedback to students who fail to elicit and incorporate their partner's preferences @xcite . our results on annotation tasks indicate that llms can make the annotation process efficient (albeit with a human in the loop), aiding both linguistic strategy analysis and the design of modular dialogue systems. however, this is primarily true for proprietary llms, emphasizing the need to improve smaller open-source models. we also find that information in dialogue can confuse models about their own context, even when provided in the prompts. future work should focus on helping llms handle longer contexts by emphasizing relevant input. human evaluation shows all models struggle with generating strategically appropriate responses. recent efforts in structured reasoning @xcite may address this by breaking response generation into subgoals like comprehension, annotation, and partner modeling. we plan to combine llms with rl policies from prior negotiation work @xcite to control the dialogue agent's strategy or personality.