More Victories, Less Cooperation: Assessing Cicero’s Diplomacy Play (2024)

Abstract: The boardgame Diplomacy is a challenging setting for communicative and cooperative artificial intelligence. The most prominent communicative Diplomacy AI, Cicero, has excellent strategic abilities, exceeding human players. However, the best Diplomacy players master communication, not just tactics, which is why the game has received attention as an AI challenge. This work seeks to understand the degree to which Cicero succeeds at communication. First, we annotate in-game communication with abstract meaning representation to separate in-game tactics from general language. Second, we run two dozen games with humans and Cicero, totaling over 200 human-player hours of competition. While AI can consistently outplay human players, AI-Human communication is still limited because of AI’s difficulty with deception and persuasion. This shows that Cicero relies on strategy and has not yet reached the full promise of communicative and cooperative AI.

## diplomacy requires communication in a landmark paper, @xcite introduce cicero, an ai that plays the game diplomacy. the washington post claims "the model is adept at negotiation and trickery" @xcite , forbes asserts "cicero was able to pass as a human player" @xcite , and even the scientific publication's editor states ai "mastered diplomacy" @xcite . this work tests those popular perceptions to rigorously evaluate the communicative and strategic capabilities of cicero. our observations lead to insights about the current state of cooperation and communication in ai, highlighting its deceptive and persuasive characteristics. while cicero plays strategically and with a verisimilitude of human communication, the evaluation in @xcite focuses only on if cicero wins games. as the name implies, diplomacy is revered by its devotees as a game of nuanced negotiation @xcite , convincing persuasion, and judicious betrayal. we argue that mastering diplomacy requires these communicative skills (section 2). measuring persuasion, deception, and cooperation are open problems with no clear solution. a boardgame constrains the world of actions to make these measurements feasible. one contribution of this paper is to build measurements of these communicative skills and to evaluate the true state of ai play in diplomacy. however, a technical challenge to identifying persuasion and deception is mapping from communication to in-game actions (e.g., verifying that i follow through on a promise of helping you): persuasion uses words to convince someone to do something, and deception is saying something to alter another's belief @xcite . in both cases, we map and contrast communication to agents' actions. to enable this mapping, we annotate messages from human games with abstract meaning representation @xcite , which we use to train a grounded system to infer the goals of diplomacy players (section 3). after validating we can extract communicative intents, we use these representations to identify persuasion and deception (section 4). we then remove some of cicero's communicative ability: this does not impair its ability to win games. we then test cicero's skills in games against humans (section 5) while asking players to annotate if they think other players are an ai and if a message is a lie. confirming earlier work, cicero wins nearly every game, including against top players. however, our new annotations provide a counterargument to the prevailing view that cicero has mastered the communicative aspect of the game that is the priority of the nlp community. cicero plays "differently"; humans can reliably identify cicero and it is less deceptive and persuasive to hu-man players. communication from cicero is more transactional, relying on its optimal strategy rather than the alliance building which is the hallmark of top human players. cicero has yet to prove effective in the communication skills which are crucial to achieving goals in strategic games and real life. in section 7, we discuss what it would take for a computer to truly master diplomacy and how diplomacy's intrinsic persuasion and deception can improve computers' ability to not just talk like a human but to realistically tie words to actions. ## diplomacy as human-ai communication test bed diplomacy is a strategic board game that combines negotiation and strategy, where players take on the roles of various european powers (nations) on the eve of world war i. the essence of the game lies in forming and betraying alliances to control territories, requiring adept diplomacy (hence the name of the game) and strategic planning. some players focus on aggressive tactical decisions, while others focus on making alliances, communicating, and collaborating with others for better outcomes @xcite . the goal of the game is to capture territory, board regions called supply centers: once you capture enough of these supply centers, you win the game. the charm and challenge of diplomacy messages is that players are free to talk about anything, either strategy-related or not. most messages in diplomacy are about (1) past, current, and future turns, (2) alliance negotiations, and (3) acquiring and sharing third-party information such as, "did turkey talk to you?", "i can't believe you attacked russia in sev." a few messages are (4) non-diplomacy conversation, e.g. 'how are you today?" communication in diplomacy is attractive to academic researchers because it can be linguistically and cognitively complex, but is grounded in a constrained world with well-defined states and dynamics. 1 the diplomacy ai development environment @xcite is a structured syntax for ai agents to play diplomacy: create alliances, suggest moves, etc. several agents have used this stripped-down communica-tive environment: albert (van hal, 2009), sillyne-gobot @xcite , dipblue @xcite , inter alia. unlike these agents, cicero @xcite uses a large language model to enable free-form english communication, enabling "normal" play with humans. cicero excelled in an online diplomacy league, scoring over twice the average of human participants and ranking in the top 10% among those who played multiple games. in the original work, it is unclear if cicero's success is due to its use of natural language or its strategic model. in our human-cicero studies, we ask that participants annotate messages they perceive as deceptive, allowing us to more carefully study the communicative aspects of the game (more details are in section 5). ## grounding messages into intentions consider this in-game statement made by england to germany about a specific move-set (glosses added to locations): you can steal stp [st. petersburg] from russia if you're in swe [sweden] next turn. i will support you there. we want to be able to tell if the speaker is lying (e.g., they're going to do something else instead of what they claim they're going to do) and if the speaker has convinced the recipient to alter their actions. this is necessary to measure how effectively cicero communicates in the game. while we know the intended actions of players when they submit their moves, we need to see how those moves match up to their communications in the discussion period before they submit moves (figure 1 ). we use amr to build a machinereadable representation of the intent of actions in their communications. we are not starting from scratch: daide (section 2) provides a set of predicates (ally, move, etc.) critical to diplomacy communications. we thus focus on annotating these predicates that encode actions, allowing us to understand the communicative intent of messages, where speakers could say they will do something and follow through, or say they will do something and not follow through. because not all information needed for annotation is in the raw message text, we further show human annotators who wrote the text (e.g., france, germany), seasons (e.g., spring 1901), and the current game state. this information is necessary to annotate "you can steal stp from russia if you are in swe this turn. i will support you there" in the earlier example so that the annotators can assume what unit would support and what unit would move into sweden. in this case, england's fleet in norway supports a german fleet in skagerrak to move into sweden. ## promises made, promises kept, and finding dirty lies in the example from the previous section (figure 1 ), england says that it would support germany's move into sweden from the skagerrak sea, while germany agrees with the proposal from england. what does it mean for these to be deceptive or persuasive? the geopolitical definition of deception is to manipulate adversaries' perceptions to gain strategic advantage @xcite , e.g., germany alters england's belief so that england would do an action that benefits germany (i.e. germany has a better chance of ending up with higher score). however, evaluating deception is challenging because it requires estimating the differences in england's beliefs before and after germany deceives england. therefore, we break down a broader, amorphous concept into easier-to-handle concepts and leave broader deception to future work. the first subconcept is breaking of a commitment @xcite : someone saying they will do something and not following through. in the example, if germany commits to moving to sweden but later attacks england in norway, this will be detected as a broken commitment. the second subconcept is lying: human players in games with cicero annotate messages their messages as either truthful or deceptive. we build on @xcite , who define deception to players thusly: "typically, when [someone] lies @xcite say what [they] know to be false in an attempt to deceive the listener." again, deception is broader than lying (and there are top diplomacy players who intentionally deceive while never outright saying anything untrue), and our definition of deception is slightly broader than @xcite . in our more permissive annotations, human players mark interactions that are broken commitments, lies about other players, hedging about the state of alliances, and anything else that they feel is deceptive. despite this ontological uncertainty, for convenience, we refer to the process of humans annotating messages as "human lie annotations" for consistency with @xcite . to recap: a broken commitment can be a lie and an example of deception. 3 likewise, not all lies are broken commitments, but both breaking a commitment and lying are facets of deception. while we cannot capture all deception-because it is based on internal state-it is important to capture as much as we can to measure how important it is to playing diplomacy well. specifically, cicero's ability (or inability) to decieve or persuade has never been empirically measured, so we build on the amr parser of the previous section to detect broken commitment and persuasion. as we discuss in section 3.2, this is not perfect, but it has good coverage when players discuss their intentions. first, we parse english messages to amr structures, for which we define actions that the speaker intends to do, e.g. we can extract germany's communicative intent (f ska -swe) when germany agrees with england that they will move to sweden (middle, figure 1 ). we also define orders players submit before any communication as initial intents and final orders as orders that players submit when the turn ends. using initial intents, communicative intent, and final orders, we can now define broken commitment and persuasion. for a broken commitment, we say that germany violates a commitment with england if germany verbally agrees to move to sweden but actually attacks england in norway (deception by germany, figure 1 ). breaking a commitment may result when an intent changes but is not communicated. for example, germany agrees with england to move to sweden but instead moves to denmark to defend against france without informing england. although this may not involve deceptive intent, we still consider it deception because it alters the listener's beliefs and affects decision-making. for instance, england might decide to support germany based on their agreement. for persuasion, england's request is considered persuasive if germany moves to sweden, as england suggests, instead of germany's original plan to move to the north sea (persuasion by england, figure 1 ). we describe each of these more formally in this section. ## comparing cicero to humans cicero has strong strategic abilities and is relatively cooperative towards other players (section 3.3), but it is unclear whether cicero can achieve humanlevel gameplay in both tactics and communication. having defined the aspects of communication that we argue are important for mastering diplomacy, we want to investigate communication and cooperation between cicero and humans. specifically, we want to answer: in equation 2 ). following @xcite , humans annotate every message that they receive or send: they annotate each outgoing message for whether it is a lie (truth/lie/neutral options), and they annotate each incoming message for whether they perceive it as a lie (truth/lie options). while @xcite asked ex post facto if any opponents were a computer, we inform players before play that there is a computer and we ask human players their guess of the humanity of each opposing power. there are two to four human players per game, totaling 69 over all 24 games. 4 games typically finish after fourteen movement turns, where each movement turns is limited to a brisk ten minutes. there are two to four human players per game, and cicero fills any remaining slots. the game setup differs from meta's cicero study: players in this study know a priori that they are playing a bot. in total, we collect 27,665 messages from communication between humans and cicero ( cicero nearly always wins. of twenty-four games, cicero won twenty (84%), which strongly suggests that cicero has super-human strategy. on average, cicero has more supply centers than human players by the end of the game ( human players can reliably (but not perfectly) identify the bot. we calculate the average f -score of identification by turn (figure 3 ). by the 4 we recruit players from diplomacy forums and we pay at least $70 per game, which lasts approximately three hours. we do not collect demographic information. with smoothing via local regression @xcite . ## related work large language models are becoming ubiquitous in many tasks: fact-checking @xcite ), text generation @xcite @xcite including coding @xcite . all of these tasks require users to trust models' outputs. however, models are not always reliable; they could produce hallucinations or conflict with established facts @xcite @xcite . to mitigate this, their outputs often need to be verified against datasets @xcite @xcite . studies have used adversarial examples to expose weaknesses and to raise awareness @xcite @xcite to address the issue of unreliability, controllable lms been proposed by having steps to inject facts for better reasoning @xcite , or by prompting techniques, such as chainof-thought prompting, to enhance reasoning abilities @xcite . moreover, some studies focus on ai-generated misinformation @xcite , probing model to understand internal states when llm utters truthful or false information @xcite . deception and persuasion are studied within social contexts. huang and wang (2023)'s metaanalysis concludes ai can match humans in persuasion, and deck (2023) attributes some of the success to the ability to generate "bullshit", which are part of applications in marketing and public relations @xcite . part of what makes games like diplomacy as an object of study appealing is the ongoing race between humans and computers in games @xcite ; initial work on the language of diplomacy @xcite unlocked follow-on work both in diplomacy's agreements @xcite and in other games such as "the resistance: avalon" @xcite @xcite and "mafia" @xcite . ## conclusion and future work our research confirms that cicero can win most games of diplomacy, but has not mastered the nuances of communication and persuasion. truly mastering the game requires systems that (a) can maintain consistency between their communication and actions, (b) can communicate at a variety of levels, including tactics, strategy, and alliances, and (c) can use communication as a tool of persuasion, deception, and negotiation. diplomacy remains an attractive testbed for communication and strategic research. it offers the ability to build more comprehensive systems that understand relationship dynamics, can engage in realistic but hypothetical conversations, and that can be robust to the deceptions of others. because these are places where humans still outpace ai, it also offers synergies for developing human-computer collaboration. and while these tasks are important withing the silly game of diplomacy, they can help solve long-standing ai problems: helping users deal with llm-generated deception, collaborating with users on grounded planning, and understanding human norms of reciprocity, cooperation, and communication. this will help ai not just be fun for negotiation in board games but safer and more trustworthy when we negotiate everyday problems.