SlowGuess/ACL_2020_OCR huggingface dataset:
Figure 5: Test set results for both our ACTUAL LIE and SUSPECTED LIE tasks. We provide baseline (Random, Majority Class), logistic (language features, bag of words), and neural (combinations of a LSTM with BERT) models. The neural model that integrates past messages and power dynamics approaches human  $F_{1}$  for ACTUAL LIE (top). For ACTUAL LIE, the human baseline is how often the receiver correctly detects senders' lies. The SUSPECTED LIE lacks such a baseline.
can extract. The hierarchical LSTM is given the context of previous messages when determining if a given message is a lie, which is akin to the labeling task humans do when annotating the data. The model does this by encoding a single message from the tokens, and then running a forward LSTM over all the messages. For each message, it looks at both the content and previous context to decide if the current message is a lie. Fine-tuning BERT (Devlin et al., 2019) embeddings to this model did not lead to notable improvement in  $F_{1}$ , likely due to the relative small size of our training data. Last, we incorporate information about power imbalance into this model. This model approaches human performance in terms of  $F_{1}$  score by combining content with conversational context and power imbalance.
5 Qualitative Analysis
This section examines specific messages where both players and machines are correctly identifying lies and when they make mistakes on our test set. Most messages are correctly predicted by both the model and players (2055 of 2475 messages); but this is because of the veracity effect. The picture is less rosy if we only look at messages the sender
marks as ACTUAL LIE: both players and models are generally wrong (Table 5).
Both models and players can detect lies when liars get into specifics. In Diplomacy, users must agree to help one another through orders that stipulate "I will help another player move from X to Y". The in-game term for this is "support"; half the messages where players and computers correctly identify lies contain this word, but it rarely occurs in the other quadrants.
Models seem to be better at not falling for vague excuses or fantastical promises in the future. Players miss lies that promise long-term alliances, involve extensive apologies, or attribute motivation as coming from other countries' disinformation (Model Correct). Unlike our models, players have access to conversations with other players and accordingly players can detect lies that can easily be verified through conversations with other players (Player Correct).
However, ultimately most lies are believable and fool both models and players (Both Wrong). For example, all messages that contain the word "true" are predicted as truthful by both models and play
Model Prediction
Correct	Wrong
Player Prediction	Correct	Both Correct Not sure what your plan is, but I might be able to support you to Munich.	Player Correct Don’t believe Turkey, I said nothing of the sort. I imagine he’s just trying to cause an upset between us.
Wrong	Model Correct Long time no see. Sorry for the stab earlier. I think we should try to work together to stop france from winning; if we work together we can stop france from getting 3 more centers, and then we will all win in a 3, 4, or 5 way draw when the game is hard-capped at 1910.	Both Wrong I’m considering playing fairly aggressive against England and cutting them off at the pass in 1901, your support for that would be very helpful.
Table 4: An example of an ACTUAL LIE detected (or not) by both players and our best computational model (Context LSTM + Power) from each quadrant. Both the model and the human recipient are mostly correct overall (Both Correct), but they are both mostly wrong when it comes to specifically predicting lies (Both Wrong).
Model Correct	Model Wrong
Player Correct	10	32
Player Wrong	28	137
Table 5: Conditioning on only lies, most messages are now identified incorrectly by both our best model (Context LSTM + Power) and players.
ers. Many of these messages are relatively tame;[8] confirming the Pinocchio effect found by Swol et al. (2012). If liars can be detected when they wax prolix, perhaps the best way to avoid detection is to be terse and to the point.
Sometimes additional contextual information helps models improve over player predictions. For example, when France tells Austria "I am worried about a steamroller Russia Turkey alliance", the message is incorrectly perceived as truthful by both the player and the single-message model. However, once the model has context—a preceding question asking if Austria and Turkey were cooperating—it can detect the lie.
Finally, we investigate categories from the Harbingers (Niculae et al., 2015) word lists. Lies are more likely to contain subjectivity and premises while true messages include expansion phrases ("later", "additionally"). We also use specific words in the bag of words logistic regression model. The coefficient weights of words that express sincerity (e.g., "sincerely", "frankly") and apology (e.g., "accusation", "fallout", "alternatives") skew toward ACTUAL LIE prediction in the logistic regression model. More laid back appella
tions (e.g., "dude", "man") skew towards truthfulness, as do words associated with reconnaissance (e.g., "fyi", "useful", "information") and time (e.g., "weekend", "morning"). Contested areas on the Diplomacy map, such as Budapest and Sevastopol, are more likely to be associated with lies, while more secure ones like Berlin, are more likely to be associated with truthful messages.
Early computational deception work focuses on single utterances (Newman et al., 2003), especially for product reviews (Ott et al., 2012). But deception is intrinsically a discursive phenomenon and thus the context in which it appears is essential. Our platform provides an opportunity to observe deception in the context in which it arises: goal-oriented conversations around in-game objectives. Gathering data through an interactive game has a cheaper per-lie cost than hiring workers to write deceptive statements (Jurgens and Navigli, 2014).
Other conversational datasets are mostly based on games that involve deception including Werewolf (Girlea et al., 2016), Box of Lies (Soldner et al., 2019), and tailor-made games (Ho et al., 2017). However, these games assign individuals roles that they maintain throughout the game (i.e., in a role that is supposed to deceive or in a role that is deceived). Thus, deception labels are coarse: an individual always lies or always tells the truth. In contrast, our platform better captures a more multifaceted reality about human nature: everyone can lie or be truthful with everyone else, and they use both strategically. Hence, players must think about every player lying at any moment: “given the evidence, do I think this person is lying to me now?”
Deception data with conversational labels is also available through interviews (Pérez-Rosas et al.,
Examples include "It's true—[Budapest] back to [Rumania] and [Serbia] on to [Albania] could position for more forward convoys without needing the rear fleet...” and "idk if it's true just letting u know since were allies".
2016), some of which allow for finer-grained deception spans (Levitan et al., 2018). Compared with game-sourced data, however, interviews provide shorter conversational context (often only a single exchange with a few follow-ups) and lack a strategic incentive—individuals lie because they are instructed to do so, not to strategically accomplish a larger goal. In Diplomacy, users have an intrinsic motivation to lie; they have entertainment-based and financial motivations to win the game. This leads to higher-quality, creative lies.
Real-world examples of lying include perjury (Louwerse et al., 2010), calumny (Fornaciari and Poesio, 2013), emails from malicious hackers (Dhamija et al., 2006), and surreptitious user recordings. But real-world data comes with real-world complications and privacy concerns. The artifice of Diplomacy allows us to gather pertinent language data with minimal risk and to access both sides of deception: intention and perception. Other avenues for less secure research include analyzing dating profiles for accuracy in self-presentation (Toma and Hancock, 2012) and classifying deceptive online spam (Ott et al., 2011).
In Dante's Inferno, the ninth circle of Hell—a fate worse even than that reserved for murderers—is for betrayers. Dante asks Count Ugolino to name his betrayer, which leads him to say:
but if my words can be the seed to bear  
the fruit of infamy for this betrayer  
who feeds my hunger, then I shall speak—in  
tears (Alighieri and Musa, 1995, Canto XXXIII)
Similarly, we ask victims to expose their betrayers in the game of Diplomacy. The seeds of players' negotiations and deceit could, we hope, yield fruit to help others: understanding multi-party negotiation and protecting Internet users.
While we ignore nuances of the game board to keep our work general, Diplomacy is also a rich, multi-agent strategic environment; Paquette et al. (2019) ignore Diplomacy's rich language to build bots that only move pieces around the board. An exciting synthesis would incorporate deception and language generation into an agent's policy; our data would help train such agents. Beyond playing against humans, playing with a human in the loop (HITL) resembles designs for cybersecurity threats (Cranor, 2008), annotation (Branson et al., 2010), and language alteration (Wallace et al.,
2019). Likewise, our lie-detection models can help a user in the moment better decide whether they are being deceived (Lai et al., 2020). Computers can meld their attention to detail and high infinite memory to humans' grasp of social interactions and nuance to forge a more discerning player.
Beyond a silly board game, humans often need help verifying claims are true when evaluating health information (Xie and Bugg, 2009), knowing when to take an e-mail at face value (Jagatic et al., 2007), or evaluating breaking news (Hassan et al., 2017). Building systems to help information consumers become more discerning and suspicious in low-stakes settings like online Diplomacy are the seeds that will bear the fruits of interfaces and machine learning tools necessary for a safer and more robust Internet ecosystem.
We thank Chris Martin for the introduction to the Diplomacy community and for study suggestions. Feedback from Philip Resnik, Alexander Fraser, Bill Ferguson, James Ryan, and Vinodkumar Prabhakaran helped shape the paper's structure.
The information provided in this document is derived from an effort sponsored by the Defense Advanced Research Projects Agency (DARPA) and Air Force Research Laboratory (AFRL), and awarded to Raytheon BBN Technologies under contract number FA865018-C-7885. Danescu-Niculescu-Mizil is supported by NSF award IIS-1750615 and by NSF grant IIS-1910147. Opinions, findings, conclusions, or recommendations expressed here are those of the authors and do not necessarily reflect views of the sponsors.
We thank Sebastian A., Joe Brelsford (TrustworthyWarMonger), Sam Brothers, Max Christie, Jordan Connors (Conq), Anna Conte, Bill Hackenbracht, Jack Henrichs, Melissa Lewis, Michael Lotfy (Blitzkrieg13), Joshua Lovett-Graff, Mitch McConeghey, Marko Papic, Christopher Rawles, David Van Slyke (happypopday), Reno Varghese, Tyler Waaler, Joseph Wheeler (Sloth), Phillip Wilcox, Jorge Zhang (Caped Baldy), Daniel Zhu, papa_k, questionmark, and the dozens of other players that made the games possible.
3819
Dante Alighieri and Mark Musa. 1995. Dante's Inferno: The Indiana Critical Edition. Indiana masterpiece editions. Indiana University Press.
Pranav Anand, Joseph King, Jordan Boyd-Graber, Earl Wagner, Craig Martell, Douglas W. Oard, and Philip Resnik. 2011. Believe me: We can do this! In The AAAI 2011 workshop on Computational Models of Natural Argument.
David Bamman and Noah A. Smith. 2015. Contextualized Sarcasm Detection on Twitter. In Proceedings of ICWSM.
Janet Beavin Bavelas, Alex Black, Nicole Chovil, and Jennifer Mullett. 1990. Truths, lies, and equivocations: The effects of conflicting goals on discourse. Journal of Language and Social Psychology, 9(1-2):135-161.
Yonatan Belinkov and James Glass. 2019. Analysis methods in neural language processing: A survey. Transactions of the Association for Computational Linguistics.
Kathy L Bell and Bella M DePaulo. 1996. Liking and lying. *Basic and Applied Social Psychology*, 18(3):243-266.
William E. Bogner, Margaret Edwards, Leon Zelechowski, Kevin J. Egan, William J. Rogers, Eloy Burciaga, and John Scott Arthur. 1974. Perjury: The forgotten offense. The Journal of Criminal Law and Criminology, 65(3):361-372.
Steve Branson, Catherine Wah, Florian Schroff, Boris Babenko, Peter Welinder, Pietro Perona, and Serge Belongie. 2010. Visual recognition with humans in the loop. In European Conference on Computer Vision.
Michael T. Braun and Lyn M. Van Swol. 2016. Justifications offered, questions asked, and linguistic patterns in deceptive and truthful monetary interactions. Group Decision and Negotiation, 25(3):641-661.
Michael Buhrmester, Tracy Kwang, and Samuel D Gosling. 2011. Amazon's mechanical Turk: A new source of inexpensive, yet high-quality data? Perspectives on psychological science: a journal of the Association for Psychological Science, 6 1:3-5.
David B. Buller, Judee K. Burgoon, Aileen Buslig, and James Roiger. 1996. Testing interpersonal deception theory: The language of interpersonal deception. Communication Theory, 6(3):268-289.
Jonathan P. Chang, Justin Cheng, and Cristian Danescu-Niculescu-Mizil. 2020. Don't let me be misunderstood: Comparing intentions and perceptions in online discussions. In Proceedings of the World Wide Web Conference.
Johnny Chiodini. 2020. Playing Diplomacy online transformed the infamously brutal board game from unbearable to brilliant. Dicebreaker.
Robert B Cialdini and Noah J Goldstein. 2004. Social influence: Compliance and conformity. Annual Review of Psychology, 55:591-621.
Cohen Coberly. 2019. Discord has surpassed 250 million registered users. Techspot.
B. Cornwell and D. C. Lundgren. 2001. Love on the internet: involvement and misrepresentation in romantic relationships in cyberspace vs. realspace. Computational Human Behavior, 17:197-211.
Lorrie F Cranor. 2008. A framework for reasoning about the human in the loop. In UPSEC.
Cristian Danescu-Niculescu-Mizil, Lillian Lee, Bo Pang, and Jon Kleinberg. 2012. Echoes of power: Language effects and power differences in social interaction. In Proceedings of the World Wide Web Conference.
Cristian Danescu-Niculescu-Mizil, Moritz Sudhof, Dan Jurafsky, Jure Leskovec, and Christopher Potts. 2013. A computational approach to politeness with application to social factors. In Proceedings of the Association for Computational Linguistics.
Bella M DePaulo, James J Lindsay, Brian E Malone, Laura Muhlenbruck, Kelly Charlton, and Harris Cooper. 2003. Cues to deception. Psychological bulletin, 129(1):74.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Conference of the North American Chapter of the Association for Computational Linguistics.
Rachna Dhamija, J. Doug Tygar, and Marti A. Hearst. 2006. Why phishing works. In International Conference on Human Factors in Computing Systems.
Tommaso Fornaciari and Massimo Poesio. 2013. Automatic deception detection in Italian court cases. Artificial intelligence and law, 21(3):303-340.
Margalit Fox. 2013. Allan Calhamer dies at 81; invented Diplomacy game. New York Times.
Edmund Gettier. 1963. Is justified true belief knowledge? Analysis, 23(6):121-123.
Codruta Girlea, Roxana Girju, and Eyal Amir. 2016. Psycholinguistic features for deceptive role detection in Werewolf. In Conference of the North American Chapter of the Association for Computational Linguistics.
Stephanie Gokhman, Jeff Hancock, Poornima Prabhu, Myle Ott, and Claire Cardie. 2012. In search of a gold standard in studies of deception. In Proceedings of the Workshop on Computational Approaches to Deception Detection.
Roberto González-Ibanez, Smaranda Muresan, and Nina Wacholder. 2011. Identifying sarcasm in Twitter: A closer look. In Proceedings of the Association for Computational Linguistics.
Stephan Greene and Philip Resnik. 2009. More than words: Syntactic packaging and implicit sentiment. In Conference of the North American Chapter of the Association for Computational Linguistics.
Naeemul Hassan, Fatma Arslan, Chengkai Li, and Mark Tremayne. 2017. Toward automated fact-checking: Detecting check-worthy factual claims by claimbuster. In Knowledge Discovery and Data Mining.
David Hill. 2014. Got your back. This American Life Podcast.
Shuyuan Mary Ho, Jeffrey T Hancock, and Cheryl Booth. 2017. Ethical dilemma: Deception dynamics in computer-mediated group communication. Journal of the Association for Information Science and Technology.
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735-1780.
Tom N Jagatic, Nathaniel A Johnson, Markus Jakobsson, and Filippo Menczer. 2007. Social phishing. Communications of the ACM, 50(10):94-100.
Nathalie Japkowicz and Shaju Stephen. 2002. The class imbalance problem: A systematic study. Intelligent data analysis, 6(5):429-449.
David Jurgens and Roberto Navigli. 2014. It's all fun and games until someone annotates: Video games with a purpose for linguistic annotation. In Transactions of the Association for Computational Linguistics.
Mary E Kaplar and Anne K Gordon. 2004. The enigma of altruistic lying: Perspective differences in what motivates and justifies lie telling within romantic relationships. Personal Relationships, 11(4):489-507.
Srijan Kumar, Justin Cheng, Jure Leskovec, and V.S. Subrahmanian. 2017. An Army of Me: Sockpuppets in Online Discussion Communities. In Proceedings of the World Wide Web Conference, Republic and Canton of Geneva, Switzerland.
Srijan Kumar and Neil Shah. 2018. False information on web and social media: A survey. In Social Media Analytics: Advances and Applications. CRC.
Vivian Lai, Han Liu, and Chenhao Tan. 2020. "why is 'chicago' deceptive?" Towards building model-driven tutorials for humans. In International Conference on Human Factors in Computing Systems.
Timothy R. Levine, Hee Sun Park, and Steven A. McCormack. 1999. Accuracy in detecting truths and lies: Documenting the "veracity effect". Communication Monographs, 66(2):125-144.
Sarah Ita Levitan, Angel Maredia, and Julia Hirschberg. 2018. Linguistic cues to deception and perceived deception in interview dialogues. In *Conference of the North American Chapter of the Association for Computational Linguistics*.
Max Louwerse, David Lin, Amanda Drescher, and Gun Semin. 2010. Linguistic cues predict fraudulent events in a corporate social network. In Proceedings of the Annual Meeting of the Cognitive Science Society.
James Edwin Mahon. 2016. The definition of lying and deception. In *The Stanford Encyclopedia of Philosophy*, winter 2016 edition. Metaphysics Research Lab, Stanford University.
Matthew L Newman, James W Pennebaker, Diane S Berry, and Jane M Richards. 2003. Lying words: Predicting deception from linguistic styles. *Personality and social psychology bulletin*, 29(5):665-675.
Vlad Niculae, Srijan Kumar, Jordan Boyd-Graber, and Cristian Danescu-Niculescu-Mizil. 2015. Linguistic harbingers of betrayal: A case study on an online strategy game. In Proceedings of the Association for Computational Linguistics.
Daniela Oliveira, Harold Rocha, Huizi Yang, Donovan Ellis, Sandeep Dommaraju, Melis Muradoglu, Devon Weir, Adam Soliman, Tian Lin, and Natalie Ebner. 2017. Dissecting spear phishing emails for older vs young adults: On the interplay of weapons of influence and life domains in predicting susceptibility to phishing. In International Conference on Human Factors in Computing Systems.
Myle Ott, Claire Cardie, and Jeff Hancock. 2012. Estimating the prevalence of deception in online review communities. In Proceedings of the World Wide Web Conference.
Myle Ott, Yejin Choi, Claire Cardie, and Jeffrey T Hancock. 2011. Finding deceptive opinion spam by any stretch of the imagination. In Proceedings of the Association for Computational Linguistics.
Bo Pang, Lillian Lee, et al. 2008. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2(1-2):1-135.
Philip Paquette, Yuchen Lu, Seton Steven Bocco, Max Smith, Satya Ortiz-Gagné, Jonathan K. Kummerfeld, Joelle Pineau, Satinder Singh, and Aaron C Courville. 2019. No-press diplomacy: Modeling multi-agent gameplay. In Proceedings of Advances in Neural Information Processing Systems. Curran Associates, Inc.
Verónica Pérez-Rosas, Mohamed Abouelenien, Rada Mihalcea, Yao Xiao, C. J. Linton, and Mihai Burzo. 2016. Verbal and nonverbal clues for real-life deception detection. In Proceedings of Empirical Methods in Natural Language Processing.
3821
Verónica Pérez-Rosas, Bennett Kleinberg, Alexandra Lefevre, and Rada Mihalcea. 2017. Automatic detection of fake news. Proceedings of International Conference on Computational Linguistics.
Vinodkumar Prabhakaran, Ajita John, and Doree D Seligmann. 2013. Power dynamics in spoken interactions: a case study on 2012 Republican primary debates. In Proceedings of the World Wide Web Conference.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why should i trust you?" explaining the predictions of any classifier. In Knowledge Discovery and Data Mining.
Marta Sabou, Kalina Bontcheva, Leon Derczynski, and Arno Scharl. 2014. Corpus annotation through crowdsourcing: Towards best practice guidelines. In Proceedings of the Language Resources and Evaluation Conference.
Ben Shneiderman. 2000. Designing trust into online experiences. Communications of the ACM, 43(12):57-59.
Frederick A Siegler. 1966. Lying. American Philosophical Quarterly, 3(2):128-136.
Rion Snow, Brendan O'Connor, Daniel Jurafsky, and Andrew Y Ng. 2008. Cheap and fast—but is it good?: Evaluating non-expert annotations for natural language tasks. In Proceedings of Empirical Methods in Natural Language Processing.
Felix Soldner, Verónica Pérez-Rosas, and Rada Mihalcea. 2019. Box of lies: Multimodal deception detection in dialogues. In Conference of the North American Chapter of the Association for Computational Linguistics.
Lyn M. Van Swol, Deepak Malhotra, and Michael T. Braun. 2012. Deception and its detection: Effects of monetary incentives and personal relationship history. Communication Research, 39(2):217-238.
James Thorne, Andreas Vlachos, Oana Cocarascu, Christos Christodoulopoulos, and Arpit Mittal, editors. 2018. Proceedings of the First Workshop on Fact Extraction and VERIFICATION (FEVER). Association for Computational Linguistics.
Catalina L Toma and Jeffrey T Hancock. 2012. What lies beneath: The linguistic traces of deception in online dating profiles. Journal of Communication, 62(1):78-97.
Isabelle Torrance. 2015. Distorted oaths in Aeschylus. Illinois Classical Studies, 40(2):281-295.
Eric Wallace, Pedro Rodriguez, Shi Feng, Ikuya Yamada, and Jordan Boyd-Graber. 2019. Trick me if you can: Human-in-the-loop generation of adversarial examples for question answering. Transactions of the Association for Computational Linguistics, 7:387-401.
Bo Xie and Julie M. Bugg. 2009. Public library computer training for older adults to access high-quality internet health information. Library and Information Science Research, 31(3).
Zhi-Hua Zhou and Xu-Ying Liu. 2005. Training cost-sensitive neural networks with methods addressing the class imbalance problem. IEEE Transactions on knowledge and data engineering, 18(1):63-77.
Table and Figure numbers continue from the main document. In the appendix are:
1. examples of game summaries written by players (Table 6);
2. the game engine view of the board (Figure 6);
3. examples of persuasion techniques (Table 7);
4. Harbingers word lists that are used as features in the logistic regression model (Table 8); and
5. A full transcript between two players, Germany and Italy (Table 9). Messages are long and carefully composed. This transcript is from the game described in Section 2.1 (Warning: it is twenty pages long).