SlowGuess/NAACL_2022_OCR huggingface dataset:
Samee Ibrahim*, Gaoyue Zhou*, and John DeNero
{sibraheem,zhougy_99,denero}@berkeley.edu
While neural networks demonstrate a remarkable ability to model linguistic content, capturing contextual information related to a speaker's conversational role is an open area of research. In this work, we analyze the effect of speaker role on language use through the game of Mafia, in which participants are assigned either an honest or a deceptive role. In addition to building a framework to collect a dataset of Mafia game records, we demonstrate that there are differences in the language produced by players with different roles. We confirm that classification models are able to rank deceptive players as more suspicious than honest ones based only on their use of language. Furthermore, we show that training models on two auxiliary tasks outperforms a standard BERT-based text classification approach. We also present methods for using our trained models to identify features that distinguish between player roles, which could be used to assist players during the Mafia game.
Correct interpretation of language must take into account not only the meaning of utterances, but also characteristics of the speaker and the context in which their utterances are produced. Modeling the impact of this context on language is still challenging for NLP systems. For example, differences in language identification accuracy, speech recognition word error rates, and translation quality have been observed on the basis of attributes such as a speaker's gender, race, dialect, or role (Blodgett and O'Connor, 2017; Tatman and Kasten, 2017; Tatman, 2017; Stanovsky et al., 2019). Moreover, these systems systematically underperform on data generated by those in the minority, having implications for the ethics and fairness of using these technologies.
This work explores language used for deception: a type of speaker context that is particularly challenging to model because it is intentionally hidden by the speaker. To do so, we collect and release a set of records for the game of Mafia, in which each player is assigned either an honest or a deceptive role. Then, we develop models that distinguish players' roles based only on the text of the players' dialog. We describe two auxiliary tasks that improve classification accuracy over a BERT-based text classifier.
The novel contributions of this paper include:
1. A methodology for collecting records of online Mafia games and a dataset collected from 460 human subjects,
2. Three classification models that can distinguish between honest and deceptive players,
3. An approach for identifying features of the game dialog text that can be used to help identify deceptive players during the game.
The task of identifying deception in dialog is far from solved. Our classification methods, while not accurate enough to reliably identify deceptive players in a game, do show that the text of a dialog in the setting we study does contain information about the roles of the participants, even when those participants are motivated to hide those characteristics by deceiving the listener. Although the models and results described in this work only apply to a particular game setting rather than dialog in general, the approaches we describe are general in character and therefore may inform future work on determining speaker roles from the contents of dialog.
The game of Mafia is particularly well-suited for the goal of determining whether the deceptive participants in a conversation can be identified from the contents of their utterances.
Human Language Technologies, pages 158 - 168
2.1 Deception in Language
Humans are a largely collaborative species. However, people sometimes have goals that incentivize them to deceive others. Understanding what cues and interaction styles people adopt when behaving deceptively will be crucial to both developing automated detection and a greater understanding of the complex interactions that people use in deception and revelation. Previous work indicates that people struggle with telling apart lies from truth, especially with deceptive statements (Bond Jr and DePaulo, 2006). This raises the question of what strategies deceptive actors use to avoid detection, as well as what strategies honest actors use to discover deceivers.
Deception is a difficult topic to study, however, because of its inherent complexity: multiple people with different motivations are trying to evaluate one another, while contending with group obligations and accusations, over a period of time that involves planning, taking actions, and responding to others' actions. Moreover, there is a distinction between a falsehood, which is a statement that is not true, a lie, which is a statement that the speaker does not believe, and deception, which is the act of convincing another person to hold a false belief. Whereas falsehoods and lies are properties of statements, deceptive intent is a characteristic of the speaker. Therefore, though deceptive speakers may tell falsehoods and lies, they might also provide truthful statements, and vice versa for honest speakers, thus rendering the truth conditions of individual utterances as unreliable indicators of deception. We are interested in how people solve these dual problems of deceiving and detecting deception, which requires a paradigm wherein we can observe all agents' actions and communication while simultaneously knowing agents' underlying incentives and goals. We thus turn to a game with a rich history of deception research: Mafia.
Previous work on detecting deception from linguistic cues has explored scenarios that either mimic or are taken directly from real-world investigations of potentially deceptive actors. Derrick et al. (2013) showed that deceptive parties take longer to formulate responses and use fewer words in the context of chat-based communication. Burgoon et al. (2003) similarly found that deceivers sent briefer chat messages. Fuller et al. (2011) demonstrated the effectiveness of training classifiers to identify deceptive language in relation to
crimes, and found that word quantity was a particularly useful feature. Fornaciari and Poesio (2013) also found surface-level features useful in detecting deceptive statements in a criminal context, specifically through the investigation of Italian court documents, while Mihalcea et al. (2013) found that written lies were easier to detect than transcripts of spoken ones. Abouelenien et al. (2014) took a multimodal approach to deception detection, finding that non-contact approaches were able to match or exceed the performance of those that were more invasive.
2.2 The Game of Mafia
Researchers have also examined deception in games, focusing on settings such as Diplomacy or negotiation over a set of items (Lewis et al., 2017; Niculae et al., 2015). In addition, there has been some work exploring the effects of biased voting on group decision making (Kearns et al., 2009). The game of Mafia specifically has attracted attention, and researchers have analyzed data from various online game communities. Zhou and Sung (2008) discovered differences between deception across cultural communities by analyzing data from an online Chinese Mafia game, Pak and Zhou (2011) used social network analysis to detect deceivers using the epicmafia.com website, and de Ruiter and Kachergis (2018) collected and trained models on a dataset from the online Mafiascum forum. Researchers have also studied the game of Werewolf, a variant of Mafia. Chittaranjan and Hung (2010) used audio information to classify deceptive parties, while Demyanov et al. (2015) used video information. Braverman et al. (2008) and Migdal (2010) developed a mathematical model of the Mafia game, assuming that all votes are cast at random, which allowed them to analyze how mafia and bystander win rates varied with role distribution in a highly controlled version of the game. Bi and Tanaka (2016) showed that under certain conditions, the strategy of mafia pretending to be bystanders is suboptimal.
Most of the deception-oriented games that have been studied in the natural language processing literature provided individual incentives to the players. Mafia allows for the study of patterns of deception that arise when incentives are only at the group level. In addition, whereas using datasets of online Mafia games presents a rich source of deceptive language, the complicated rule sets of games on
these forums makes it challenging to isolate specific strategies that participants use to engage in and detect deceptive behavior. In contrast to work using video or audio, we assume that players do not have access to any audiovisual clues about others' roles in order to focus on the role of language. This work takes these factors into account by studying a controlled environment that nonetheless supports the use of complex strategies for deceiving and detecting deceptive behavior.
A total of 460 English-speaking participants based in the United States were recruited from Amazon Mechanical Turk using the experiment platform Dallinger<sup>1</sup>. Between 4 and 10 participants were recruited for each Mafia game: 1 to 2 participants were designated mafia, and the rest were bystanders. Forty-four of these Mafia games are included in the final analysis. Participants were paid $2.50 for completing the task, plus bonuses for time spent waiting for other participants to arrive in a chatroom to begin the experiment. Waiting was paid at $5/hour.
Upon recruitment, participants were shown a consent form, per IRB approval, followed by an instructional video and accompanying transcript describing how to play the text-based Mafia game using an interface we developed (see Appendix). After they completed a quiz demonstrating they understood the information, they entered a waiting room until the desired number of participants was reached. Participants were then assigned a role (mafioso or bystander) and fake name, after which they began playing the game.
The game dynamics were as follows. Each mafia member was aware of the roles of their fellow mafia members and thus, by process of elimination, knew the roles of the bystanders. However, the bystanders did not know the true role of anyone else in the game. The goal of the mafia was to eliminate bystanders until the number of mafia was greater than or equal to that of the bystanders. The goal of the bystanders was to identify and eliminate all of the mafia members. Since the incentive structure was set up such that bystanders benefited from true beliefs about who the mafia members were, whereas mafia members benefited from false beliefs, bystanders were thus motivated to be honest actors, whereas mafia members were motivated to
M	B	T
Total #players	87	334	421
Avg #players per game	1.98	7.59	9.57
Std #players per game	0.15	1.21	1.28
Total #utt	770	1392	2162
Avg #utt per game	17.5	31.64	49.14
Std #utt per game	10.45	17.2	24.44
Total #players w/utt	84	265	349
Perc players w/outt	0.042	0.958	1
Table 1: Dataset statistics. # is short for number of.  $\mathbf{M}$  and  $\mathbf{B}$  denote the mafioso and bystander classes, respectively, while  $\mathbf{T}$  denotes the total number for both groups. The last row shows the distribution of roles among the players with no utterances throughout the game. Note that nearly all of the no-utterance players are bystanders.
be deceptive actors in the Mafia game. The game proceeded in phases, alternating between nighttime and daytime (Figure 1). During the nighttime, mafia members could secretly communicate to decide on who to eliminate, after which they discretely voted, and the person with the majority vote was eliminated from the game. If there was a tie, one of the people involved in the tie was randomly chosen to be eliminated. During the daytime, everyone was made aware of who was eliminated during the nighttime, and then all players could openly communicate to decide who to eliminate. All the players then voted publicly, and the person with the majority vote was eliminated and announced to be a bystander or mafioso. Thus, during the nighttime mafia could secretly communicate and eliminate anyone, whereas during the daytime mafia could participate in the voting and communication protocols in the same way as bystanders. The game proceeded until there was a winning faction according to the goals described above.
From these experiments, we collected a dataset consisting of both mafia and bystander utterances over the course of each game, as well as the participants' voting behavior. Dataset statistics appear in Table 1. Figure 2 displays a snippet of the daytime dialog from one Mafia game. As shown, many utterances are either social interactions (eg. "hi erybody") or discussions about what to do in the game, such as accusations or comments about voting (eg. "I bet it's Mandy...").
Upon further inspection of the data, we can observe several strategies used by mafia members to deceive bystanders:
http://github.com/dallinger/Dallinger
Figure 1: Mafia experiment screenshot during (left) first nighttime phase, with participant as a mafioso, and (right) first daytime phase, with participant as a bystander (note that mafia messages are not visible to the bystander).
1. Mafia members may suggest that there is not enough information to decide on who to eliminate, despite their knowledge of everyone's roles (eg. "Should we wait to eliminate someone?" / "It's a little early to tell." / "It's a shot in the dark."),
2. Mafia members may raise suspicion about another player, despite knowing that said player is a bystander (eg. hmm ok analyzing this conversation....I think bianca was a little to flippant in how she was like "sucks to be andrew" haha / I'm going to vote bianca. she's so casual with life and death),
3. Mafia members may invent a false motive and assign that motive to another player, despite knowing that the player is a bystander (eg. It might be Jonathan Kim... killing off Erin who accused him "yesterday").
Given our mafia dataset, there are several tasks that one might address, for example, predicting participants' daytime voting behavior or generating mafia members' nighttime dialog. As our aim is to identify deceptive actors, however, we focus on predicting participants' roles, i.e. bystander or mafioso. Due to the asymmetry in the knowledge available to each group and the goals which incentivize bystanders to increase true belief and mafia members to reduce it, the bystanders are said to take on an honest role in the game, whereas the mafia members take on a deceptive role. To focus on the relationship between language and
deception, we ignore voting behavior and consider just the daytime dialog in the game, as only the mafia members were able to converse during the nighttime. As shown in Table 1, since most of the players with no utterances are bystanders, we only consider players who make at least one utterance throughout the game.
To investigate whether linguistic information can be used to identify players' roles, we train and evaluate classifiers that predict the role of a particular player. Since we have a small dataset, we chose to fine-tune pre-trained Transformer models rather than train them from scratch (Vaswani et al., 2017). To predict the role for a player  $p$ , we construct an input representation  $r(C,p)$  of the full game dialog  $C$  that encodes the player of interest  $p$ . We develop three approaches which differ in both the dialog representation function  $r$  and the modeling approach.
4.1 Standard Classification
Our baseline approach uses a standard BERT-based text classifier (Devlin et al., 2018). To classify player  $p$  via the full record of the game  $C$ , let boolean variable  $M_p$  be true if  $p$  is a mafioso. Let  $T_p$  be the concatenation $^2$  of utterances made by  $p$ . We train BERT parameters  $\theta_M$  to predict  $P(M_p|T_p; \theta_M)$ .
This approach, which provides as input to the classifier only the utterances of the player to be classified, outperformed an alternative representation  $r(C,p)$  that included the entire record of all utterances by all players.
${}^{2}$  Utterances are concatenated with an end-of-sentence delimiter after each utterance.
Figure 2: Example messages (utterances) in a game. creation_time is the time at which the message was sent. The contents consists of the name of the sender, as well as the message, separated by a colon and space.
4.2 Auxiliary Tasks
Limiting the input representation  $r$  to contain only the speech of the player  $p$  being classified is not ideal; correctly interpreting a dialog requires considering all other players' statements as well. We introduce two auxiliary tasks that involve the entire game dialog  $C$ :
1. Given all of the prior utterances, is a bystander or a mafia member more likely to have produced the current utterance? (Utterance Classification)
2. Given all of the prior utterances, what current utterance would a player produce, given that they are a bystander or a mafia member? (Utterance Generation)
We develop a BERT-based classification model for task 1 and fine-tune the GPT-2 language model for task 2 (Radford et al., 2019). Then, we use each of these auxiliary models to classify the role of a particular player  $p$  in the game.
4.2.1 Utterance Classification
To classify player  $p$  using the auxiliary task of utterance classification, let boolean variable  $S_{i}$  be true if utterance  $C_i$  was made by a mafioso (rather than a bystander). Let  $C$  be the full record of utterances in the game and  $C_{\leq i}$  be the concatenation of all utterances  $C_1 \ldots C_i$ . We train BERT parameters  $\theta_S$  to predict  $P(S_{i}|C_{\leq i};\theta_{S})$ . Finally, let  $I_p$  be the set of indices of utterances by player  $p$ .  $M$  relates to  $S$  in that if  $M_p$  is true, then  $S_{i}$  is true for all  $i \in I_p$ . We thus calculate
$$
P (M _ {p} | C; \theta_ {S}) \propto \frac {\sum_ {i \in I _ {p}} P (S _ {i} | C _ {\leq i} ; \theta_ {S})}{N},
$$
where  $N = |I_p|$ .
Figure 3: Data processing for fine-tuning BERT. The original data is shown on the left-hand side, while the right-hand side shows the processed data containing two versions of each utterance, one assuming that the target player is a mafioso and one assuming that they are a bystander, with the prior conversation context preceding each and labels corresponding to whether the assumed role matches the actual role of the player.
4.2.2 Utterance Generation
To classify player  $p$  using the auxiliary task of utterance generation, we fine-tune GPT-2 to generate utterance  $C_i$  conditioned on prior utterances  $C_{<i}$  and the role  $S_i$  of the speaker that produced  $C_i$ . From Bayes' rule, we have  $P(M_p|C) \propto P(M_p)P(C|M_p)$ . To estimate  $P(C|M_p)$ , let  $C_p$  include all  $C_i$  for  $i \in I_p$ . We make the simplifying assumption that  $P(C|M_p) \propto P(C_p|M_p)$ , which assumes that the utterances made by players other than  $p$  are independent of the role of player  $p$ . Then, if  $M_p$  is true,  $S_i$  is true for all  $i \in I_p$ , and so,
$$
P \left(C _ {p} \mid M _ {p}; \theta_ {C}\right) = \prod_ {i \in I _ {p}} P \left(C _ {i} \mid C _ {<   i}, S _ {i}; \theta_ {C}\right).
$$
Using the full dialog  $C$ , the final probability of player  $p$  being mafioso is calculated as follows:
$$
P \left(M _ {p} \mid C\right) = \frac {P \left(M _ {p}\right) P \left(C _ {p} \mid M _ {p} ; \theta_ {C}\right)}{\sum_ {R \in \{M , \neg M \}} P \left(R _ {p}\right) P \left(C _ {p} \mid R _ {p} ; \theta_ {C}\right)} \tag {1}
$$
Figure 4: Data processing for fine-tuning GPT-2. The original data is shown on the left-hand side, while the right-hand side shows the processed data containing a version of the corresponding utterance with the prior conversation context preceding.
Figure 5: Prediction pipeline for our fine-tuned GPT-2 model. Similar to the pipeline used to produce the training utterances, for prediction, there are now two versions of each, one assuming that the target player is a mafioso and one assuming that they are a bystander. The losses for each utterance of the target player are summed together in order to calculate the mafia and bystander probabilities as described in Equation 1.
4.3 Data Processing
To train models for utterance classification (using BERT) and utterance generation (using GPT-2), we perform data processing procedures on the games' original dataset to create input representations  $r(C, p)$  for each player  $p$  and obtain our training datasets as shown in Figures 3 and 4. The left side of each figure shows a snippet of a game's data, where "Mafioso" and "Bystander" denote the true roles of the players. The utterances to the right of each figure are training examples used for finetuning the BERT and GPT-2 models. Structuring the data in this way provides both the prior context of utterances and the current utterance that happened within this context. This not only gives us the information needed for the auxiliary tasks, but also provides us with more training examples, as we only have 44 games and only 421 players in total, with only 2162 total utterances. Moreover, this mimics the real game scenario from the bystander view in that they can only confirm their own role, but no one else's, which is the appropriate setting for us in which to detect deception.
Figure 5 shows the pipeline for using the GPT
2 model to predict players' roles. Let us assume that the target player for whom we want to predict their role is Mafioso 1. From the original game log on the left, we first perform the data processing scheme from Figure 4 twice, assuming that the target player is a mafioso (top of Figure 5) and a bystander (bottom of Figure 5). Using our trained GPT-2 model, we then obtain a loss for each utterance denoted by L1 through L4. Summing all the losses for each role, as they denote log probabilities, we calculate  $P(M_{p}|C)$  and  $P(\neg M_p|C)$  via Equation 1. The target player's role as predicted by the model is finally given by comparing the two probabilities. A similar process is used to calculate  $P(M_{p}|C)$  and  $P(\neg M_p|C)$  for the utterance classification BERT model.
We train three fine-tuned models on the corpus of Mafia game records and compare their performance to a random baseline. The specifications for the baseline and models can be found below, and the results are shown in Table 2.
5.1 Random Baseline
This random classifier classifies each player as a mafioso or a bystander with probabilities equal to the prior distribution of each class, estimated as the ratio of roles across all training games. This serves as a baseline to be compared to for all other methods. In the game setting, this mimics a bystander player with only public information of how many mafia and bystanders are in the game.
5.2 Standard Classification
We initialize the model by loading a pre-trained BERT Base model (12 layers, 768 hidden dimension size, 12 attention heads). We train with a maximum sequence length of 256, which is sufficient for our post-processed dataset, setting the batch size to 16, the learning rate to 1e-5, and the maximum number of epochs to 25.
5.3 Utterance Classification
We initialize the model by loading a pre-trained BERT Base model (12 layers, 768 hidden dimension size, 12 attention heads). We train with a maximum sequence length of 512, which is sufficient for our post-processed dataset, setting the batch size to 5, the learning rate to 5e-5, and the maximum number of epochs to 25.
Avg Rank	Avg Rank/Game	Accuracy	Maf F1-score	Bys F1-score
Random	19.0	3.4	0.62	0.26	0.74
Std Class	17.9	3.0	0.69	0.4	0.79
Utt Class	14.5	1.8	0.74	0.50	0.83
Utt Gen	11.4	2.0	0.74	0.50	0.83
Table 2: Experiment results on the validation set for random baseline (Random), standard classification (Std Class), utterance classification (Utt Class), and utterance generation (Utt Gen) approaches. Methods that use auxiliary tasks (Utt Class and Utt Gen) outperform other methods in terms of average ranking overall and per game while also maintaining higher accuracy and F1-score for each class.
5.4 Utterance Generation