SlowGuess/ACL_2020_OCR huggingface dataset:
Denis Peskov, Benny Cheng
Ahmed Elgohary, Joe Barrow
Computer Science, University of Maryland
{dpeskov, bcheng96, elgohary, jdbarrow}@umd.edu
Jordan Boyd-Graber
iSchool, Language Science, UMIACS, LSC
jbg@umiacs.umd.edu
Trust is implicit in many online text conversations—striking up new friendships, or asking for tech support. But trust can be betrayed through deception. We study the language and dynamics of deception in the negotiation-based game Diplomacy, where seven players compete for world domination by forging and breaking alliances with each other. Our study with players from the Diplomacy community gathers 17,289 messages annotated by the sender for their intended truthfulness and by the receiver for their perceived truthfulness. Unlike existing datasets, this captures deception in long-lasting relationships, where the interlocutors strategically combine truth with lies to advance objectives. A model that uses power dynamics and conversational contexts can predict when a lie occurs nearly as well as human players.
A functioning society is impossible without trust. In online text interactions, users are typically trusting (Shneiderman, 2000), but this trust can be betrayed through false identities on dating sites (Toma and Hancock, 2012), spearphishing attacks (Dhamija et al., 2006), sockpuppetry (Kumar et al., 2017) and, more broadly, disinformation campaigns (Kumar and Shah, 2018). Beyond such one-off antisocial acts directed at strangers, deception can also occur in sustained relationships, where it can be strategically combined with truthfulness to advance a long-term objective (Cornwell and Lundgren, 2001; Kaplar and Gordon, 2004).
We introduce a dataset to study the strategic use of deception in long-lasting relationships. To collect reliable ground truth in this complex scenario, we design an interface for players to naturally generate and annotate conversational data while playing a negotiation-based game called Diplomacy.
Message	Sender's intention	Receiver's percep.
If I were lying to you, I'd smile and say “that sounds great.” I'm honest with you because I sincerely thought of us as partners.	Lie	Truth
You agreed to warn me of unexpected moves, then didn't ... You've revealed things to England without my permission, and then made up a story about it after the fact!	Truth	Truth
...I have a reputation in this hobby for being sincere. Not being duplicitous. It has always served me well. ... If you don't want to work with me, then I can understand that ...	Lie	Truth
(Germany attacks Italy)
Well this game just got less fun	Truth	Truth
For you, maybe	Truth	Truth
Table 1: An annotated conversation between Italy (white) and Germany (gray) at a moment when their relationship breaks down. Each message is annotated by the sender (and receiver) with its intended or perceived truthfulness; Italy is lying about ...lying. A full transcript of this dialog is available in Appendix, Table 9.
These annotations are done in real-time as the players send and receive messages. While this game setup might not directly translate to real-world situations, it enables computational frameworks for studying deception in a complex social context while avoiding privacy issues.
After providing background on the game of Diplomacy and our intended deception annotations (Section 2), we discuss our study (Section 3). To probe the value of the resulting dataset, we develop lie prediction models (Section 4) and analyze their results (Section 5).
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3811-3854
2 Diplomacy
The Diplomacy board game places a player in the role of one of seven European powers on the eve of World War I. The goal is to conquer a simplified map of Europe by ordering armies in the field against rivals. Victory points determine the success of a player and allow them to build additional armies; the player who can gain and maintain the highest number of points wins.<sup>1</sup> The mechanics of the game are simple and deterministic: armies, represented as figures on a given territory, can only move to adjacent spots and the side with the most armies always wins in a disputed move. The game movements become publicly available to all players after the end of a turn.
Because the game is deterministic and everyone begins with an equal amount of armies, a player cannot win the game without forming alliances with other players—hence the name of the game: Diplomacy. Conquering neighboring territories depends on support from another player's armies. After an alliance has outlived its usefulness, a player often dramatically breaks it to take advantage of their erstwhile ally's vulnerability. Table 1 shows the end of one such relationship. As in real life, to succeed a betrayal must be a surprise to the victim. Thus, players pride themselves on being able to lie and detect lies. Our study uses their skill and passion to build a dataset of deception created by battle-hardened diplomats. Senders annotate whether each message they write is an ACTUAL LIE and recipients annotate whether each message received is a SUSPECTED LIE. Further details on the annotation process are in Section 3.1.
2.1 A game walk-through
Figure 1 shows the raw counts of one game in our dataset. But numbers do not tell the whole story. We analyze this case study using rhetorical tactics (Cialdini and Goldstein, 2004), which Oliveira et al. (2017) use to dissect spear phishing e-mails and Anand et al. (2011) apply to persuasive blogs. Mentions of tactics are in italic (e.g., authority); context for quotes in Appendix, Table 7. For the rest of the paper, we will refer to players via the name of their assigned country.
Figure 1: Counts from one game featuring an Italy (green) adept at lying but who does not fall for others' lies. The player's successful lies allow them to gain an advantage in points over the duration of the game. In 1906, Italy lies to England before breaking their relationship. In 1907, Italy lies to everybody else about wanting to agree to a draw, leading to the large spike in successful lies.
Through two lie-intense strategies—convincing England to betray Germany and convincing all remaining countries to agree to a draw—Italy gains control of the board. Italy's first deception is a plan with Austria to dismantle Turkey. Turkey believes Italy's initial assurance of non-aggression in 1901. Italy begins by excusing his initial silence due to a rough day at work, evoking empathy and likability. While they do not fall for subsequent lies, Turkey's initial gullibility cements Italy's first-strike advantage. Meanwhile, Italy proposes a long-term alliance with England against France, packaging several small truths with a big lie. The strategy succeeds, eliminating Italy's greatest threat.
Local threats eliminated, Italy turns to rivals on the other end of the map. Italy persuades England to double-cross its long-time ally Germany in a moment of scarcity: if you do not act now, there will
<sup>1</sup>In the parlance of Diplomacy games, points are "supply centers" in specific territories (e.g., London). Having more supply centers allows a player to build more armies and win the game by capturing more than half of the 34 supply centers on the board.
be nowhere to expand. England accepts help from ascendant Italy, expecting reciprocity. However, Italy aggressively and successfully moves against England. The last year features a meta-game deception. After Italy becomes too powerful to contain, the remaining four players team up. Ingeniously, Italy feigns acquiescence to a five-way draw, individually lying to each player and establishing authority while brokering the deal. Despite Italy's record of deception, the other players believe the proposal (annotating received messages from Italy as truthful) and expect a 1907 endgame, the year with the most lies. Italy goes on the offensive and knocks out Austria. Italy's summary of the game in their own words is in the Appendix, Table 6.
Each game has relationships that are forged and then riven. In another game, an honest attempt by a strong Austria to woo an ascendant Germany backfires, knocking Austria from the game. Germany builds trust with Austria through a believed fictional experience as a Boy Scout in Maine (likability). In a third game, two consecutive unfulfilled promises by an ambitious Russia leads to a quick demise, as their subsequent excuses and apologies are perceived as lies (failed consistency). In another game, England, France, and Russia simultaneously attack Germany after offering duplicitous assurances. Game outcomes vary despite the identical, balanced starting board, as different players use unique strategies to persuade, and occasionally deceive, their opponents.
2.2 Defining a lie
Statements can be incorrect for a host of reasons: ignorance, misunderstanding, omission, exaggeration. Gokhman et al. (2012) highlight the difficulty of finding willful, honest, and skilled deception outside of short-term, artificial contexts (DePaulo et al., 2003). Crowdsourced and automatic datasets rely on simple negations (Pérez-Rosas et al., 2017) or completely implausible claims (e.g., "Tipper Gore was created in 1048" from Thorne et al. (2018)). While lawyers in depositions and users of dating sites will not willingly admit to their lies, the players of online games are more willing to revel in their deception.
We must first define what we mean by deception. Lying is a mischaracterization; it's thus no surprise that a definition may be divisive or the subject of academic debate (Gettier, 1963). We provide this definition to our users: "Typically, when [someone]
Figure 2: Every time they send a message, players say whether the message is truthful or intended to deceive. The receiver then labels whether incoming messages are a lie or not. Here Italy indicates they believe a message from England is truthful but that their reply is not.
lies [they] say what [they] know to be false in an attempt to deceive the listener" (Siegler, 1966). An orthodox definition requires the speaker to utter an explicit falsehood (Mahon, 2016); skilled liars can deceive with a patina of veracity. A similar definition is required for prosecution of perjury, leading to a paucity of convictions (Bogner et al., 1974). Indeed, when we ask participants what a lie looks like, they mention evasiveness, shorter messages, over-qualification, and creating false hypothetical scenarios (DePaulo et al., 2003).
2.3 Annotating truthfulness
Previous work on the language of Diplomacy (Niculae et al., 2015) lacked access to players' internal state and was limited to post-hoc analysis. We improve on this by designing our own interface that gathers players' intentions and perceptions in real-time (Section 3.1). As with other highly subjective phenomena like sarcasm (Gonzalez-Ibanez et al., 2011; Bamman and Smith, 2015), sentiment (Pang et al., 2008) and framing (Greene and Resnik, 2009), the intention to deceive is reflective on someone's internal state. Having individuals provide their own labels for their internal state is essential as third party annotators could not accurately access it (Chang et al., 2020).
Most importantly, our gracious players have allowed this language data to be released in accordance with IRB authorized anonymization, encouraging further work on the strategic use of deception in long-lasting relations.2
2Data available at http://go.umd.edu/diplomacy_data and as part of ConvoKit http://convokit.cornell.edu.
3 Engaging a Community of Liars
This dataset requires both a social and technical setup: finding a community that plays Diplomacy online and having them use a framework for annotating these messages.
3.1 Technical implementation
We need two technical components for our study: a game engine and a chat system. We choose Backstabbr<sup>3</sup> as an accessible game engine on desktop and mobile platforms: players input their moves and the site adjudicates game mechanics (Chiodini, 2020). Our communication framework is atypical. Thus, we create a server on Discord,<sup>4</sup> the group messaging platform most used for online gaming and by the online Diplomacy community (Coberly, 2019). The app is reliable on both desktop and mobile devices, free, and does not limit access to messages. Instead of direct communication, players communicate with a bot; the bot does not forward messages to the recipient until the player annotates the messages (Figure 2). In addition, the bot scrapes the game state from Backstabbr to sync game and language data.
Annotation of lies is a forced binary choice in our experiment. Explicitly calling a statement a lie is difficult, and people would prefer degrees of deception (Bavelas et al., 1990; Bell and DePaulo, 1996). Thus, we follow previous work that views linguistic deception as binary (Buller et al., 1996; Braun and Van Swol, 2016). Some studies make a more fine-grained distinction; for example, Swol et al. (2012) separate strategic omissions from blatant lies (we consider both deception). However, because we are asking the speakers themselves (and not trained annotators) to make the decision, we follow the advice from crowdsourcing to simplify the task as much as possible (Snow et al., 2008; Sabou et al., 2014). Long messages can contain both truths and lies, and we ask players to categorize these as lies since the truth can be a shroud for their aims.
3.2 Building a player base
The Diplomacy players maintain an active, vibrant community through real-life meetups and online play (Hill, 2014; Chiodini, 2020). We recruit top players alongside inexperienced but committed players in the interest of having a diverse pool.
Our experiments include top-ranked players and community leaders from online platforms, grizzled in-person tournament players with over 100 past games, and board game aficionados. These players serve as our foundation and during initial design helped us to create a minimally annoying interface and a definition of a lie that would be consistent with Diplomacy play. Good players—as determined by active participation, annotation and game outcome—are asked to play in future games.
In traditional crowdsourcing tasks compensation is tied to piecework that takes seconds to complete (Buhrmester et al., 2011). Diplomacy games are different in that they can last a month...and people already play the game for free. Thus, we do not want compensation to interfere with what these players already do well: lying. Even the obituary of the game's inventor explains
Diplomacy rewards all manner of mendacity: spying, lying, bribery, rumor mongering, psychological manipulation, outright intimidation, betrayal, vengeance and backstabbing (the use of actual cutlery is discouraged)" (Fox, 2013).
Thus, our goal is to have compensation mechanisms that get people to play this game as they normally would, finish their games, and put up with our (slightly) cumbersome interface. Part of the compensation is non-monetary: a game experience with players that are more engaged than the average online player.
To encourage complete games, most of the payment is conditioned on finishing a game, with rewards for doing well in the game. Players get at least  $40 upon finishing a game. Additionally, we provide bonuses for specific outcomes:$ 24 for winning the game (an evenly divisible amount that can be split among remaining players) and $10 for having the most successful lies, i.e., statements they marked as a lie that others believed. Diplomacy usually ends with a handful of players dividing the board among themselves and agreeing to a tie. In the game described in Section 2.1, the remaining four players shared the winner's pool with Italy after 10 in-game years, and Italy won the prize for most successful lies.
5 The lie incentive is relatively small (compared to incentives for participation and winning) to discourage an opportunistic player from marking everything as a lie. Games were monitored in real-time and no player was found abusing the system (marking more than  $\sim 20\%$  lies).
3 https://www.backstabbr.com
4 https://wwwdiscord.com
Category	Value
Message Count	13,132
ACTUAL LIE Count	591
SUSPECTED LIE Count	566
Average # of Words	20.79
Table 2: Summary statistics for our train data (nine of twelve games). Messages are long and only five percent are lies, creating a class imbalance.
Figure 3: Individual messages can be quite long, wrapping deception in pleasantries and obfuscation.
3.3 Data overview
Table 2 quantitatively summarizes our data. Messages vary in length and can be paragraphs long (Figure 3). Close to five percent of all messages in the dataset are marked as lies and almost the same percentage (but not necessarily the same messages) are perceived as lies, consistent with the "veracity effect" (Levine et al., 1999). In the game discussed above, eight percent of messages are marked as lies by the sender and three percent of messages are perceived as lies by the recipient; however, the messages perceived as lies are rarely lies (Figure 4).
3.4 Demographics and self-assessment
We collect anonymous demographic information from our study participants: the average player identifies as male, between 20 and 35 years old, speaks English as their primary language, and has played over fifty Diplomacy games.6 Players self-assess their lying ability before the study. The average player views themselves as better than average at lying and average or better than average at perceiving lies.
Figure 4: Most messages are truthful messages identified as the truth. Lies are often not caught. Table 3 provides an example from each quadrant.
In a post-game survey, players provide information on whom they betrayed and who betrayed them in a given game. This is a finer-grained determination than the post hoc analysis used in past work on Diplomacy (Niculae et al., 2015). We ask players to optionally provide linguistic cues to their lying and to summarize the game from their perspective (examples in Appendix, Table 6).
3.5 An ontology of deception
Four possible combinations of deception and perception can arise from our data. The sender can be lying or telling the truth. Additionally, the receiver can perceive the message as deceptive or truthful. We name the possible outcomes for lies as Deceived or Caught, and the outcomes for truthful messages as Straightforward or Cassandra,[7] based on the receiver's annotation (examples in Table 3, distribution in Figure 4).
4 Detecting Lies
We build computational models both to detect lies to better understand our dataset. The data from the user study provide a training corpus that maps language to annotations of truthfulness and deception. Our models progressively integrate information—conversational context and in-game power dynamics—to approach human parity in deception detection.
Our data skews  $80\%$  male and  $95\%$  of the players speak English as a primary language. Ages range from eighteen and sixty-four. Game experience is distributed across beginner, intermediate, and expert levels.
<sup>7</sup>In myth, Cassandra was cursed to utter true prophecies but never be believed. For a discussion of Cassandra's curse vis a vis personal and political oaths, see Torrance (2015).
Receiver's perception Lie
Sender's intention	Truth Lie	Straightforward Salut! Just checking in, letting you know the embassy is open, and if you decide to move in a direction I might be able to get involved in, we can probably come to a reasonable arrangement on cooperation. Bonne journee!	Cassandra I don't care if we target T first or A first. I'll let you decide. But I want to work as your partner. ...I literally will not message anyone else until you and I have a plan. I want it to be clear to you that you're the ally I want.
Deceived You, sir, are a terrific ally. This was more than you needed to do, but makes me feel like this is really a long term thing! Thank you.	Caught So, is it worth us having a discussion this turn? I sincerely wanted to work something out with you last turn, but I took silence to be an ominous sign.
Table 3: Examples of messages that were intended to be truthful or deceptive by the sender or receiver. Most messages occur in the top left quadrant (Straightforward). Figure 4 shows the full distribution. Both the intended and perceived properties of lies are of interest in our study.
4.1 Metric and data splits
We investigate two phenomena: detecting what is intended as a lie and what is perceived as a lie. However, this is complicated because most statements are not lies: less than five percent of the messages are labeled as lies in both the ACTUAL LIE and the SUSPECTED LIE tasks (Table 2). Our results use a weighted  $F_{1}$  feature across truth and lie prediction, as accuracy is an inflated metric given the class imbalance (Japkowicz and Stephen, 2002). We thus adopt an in-training approach (Zhou and Liu, 2005) where incorrect predictions of lies are penalized more than truthful statements. The relative penalty between the two classes is a hyperparameter tuned on  $F_{1}$ .
Before we move to computational models for lie detection, we first establish the human baseline. We know when senders were lying and when receivers spotted a lie. Humans spot  $88.3\%$  of lies. However, given the class imbalance, this sounds better than it is. Following the suggestion of Levine et al. (1999), we focus on the detection of lies, where humans have a 22.5 Lie  $F_{1}$ .
To prevent overfitting to specific games, nine games are used as training data, one is used for validation for tuning parameters, and two games are test data. Some players repeat between games.
4.2 Logistic regression
Logistic regression models have interpretable coefficients which show linguistic phenomena that correlate with lies. A word that occurs infrequently overall but often in lies, such as 'honest' and 'candidly', helps identify which messages are lies.
Niculae et al. (2015) propose linguistic Harbingers that can predict deception. These are word lists that cover topics often used in interpersonal communication—claims, subjectivity, premises, contingency, comparisons, expansion,
temporal language associated with the future, and all other temporal language (complete word list in Appendix, Table 8). The Harbingers word lists do not provide full coverage, as they focus on specific rhetorical areas. A logistic regression model with all word types as features further improves  $F_{1}$ .
Power dynamics influence the language and flow of conversation (Danescu-Niculescu-Mizil et al., 2012, 2013; Prabhakaran et al., 2013). These dynamics may influence the likeliness of lying; a stronger player may feel empowered to lie to their neighbor. Recall that victory points (Section 2) encode how well a player is doing (more is better). We represent the power differential as the difference between the two players. Peers will have a zero differential, while more powerful players will have a positive differential with their interlocutor. The differential changes throughout the game, so this feature encodes the difference in the season the message was sent. For example, a message sent by an Italy with seven points to a Germany with two points in a given season would have a value of five.
4.3 Neural
While less interpretable, neural models are often more accurate than logistic regression ones (Ribeiro et al., 2016; Belinkov and Glass, 2019). We build a standard long short-term memory network (Hochreiter and Schmidhuber, 1997, LSTM) to investigate if word sequences—ignored by logistic regression—can reveal lies.
Integrating message context and power dynamics improves on the neural baseline. A Hierarchical LSTM can help focus attention on specific phrases in long conversational contexts. In the same way it would be difficult for a human to determine prima facie if a statement is a lie without previous context, we posit that methods that operate at the level of a single message are limited in the types of cues they