ISSN (Print) 0473-453X

Discussion Paper No. 1119

ISSN (Online) 2435-0982

BELIEFS IN REPEATED GAMES

Masaki Aoyagi
Guillaume R. Fréchette
Sevgi Yuksel

February 2021

The Institute of Social and Economic Research
Osaka University
6-1 Mihogaoka, Ibaraki, Osaka 567-0047, Japan

Beliefs in Repeated Games∗
Masaki Aoyagi
Osaka University

Guillaume R. Fréchette
NYU

Sevgi Yuksel
UCSB

February 12, 2021

Abstract
This paper uses a laboratory experiment to study beliefs and their relationshipto action and strategy choices in finitely and indefinitely repeated prisoners’ dilemma
games. We find subjects’ beliefs about the other player’s action are accurate despitesome systematic deviations corresponding to early pessimism in the indefinitely repeated game and late optimism in the finitely repeated game. The data reveals a close
link between beliefs and actions that differs between the two games. In particular,
the same history of play leads to different beliefs, and the same belief leads to different action choices in each game. Moreover, we find beliefs anticipate the evolutionof behavior within a supergame, changing in response to the history of play (in bothgames) and the number of rounds played (in the finitely repeated game). We then use
the subjects’ beliefs over actions in each round to identify their beliefs over supergame
strategies played by the other player. We find these beliefs correctly capture the different classes of strategies used in each game. Importantly, subjects using different
strategies have different beliefs, and for the most part, strategies are subjectively rational given beliefs. The results also suggest subjects tend to overestimate the likelihoodthat others use the same strategy as them, while underestimating the likelihood that
others use less cooperative strategies.
JEL classification: C72, C73, C92
Keywords: repeated game, belief, strategy, elicitation, prisoner’s dilemma.

∗

We wish to thank Hazem Alshaikhmubarak for excellent research assistance. We would further liketo thank the National Science Foundation (grants SES-1558857, SES-1225779 and SES-1558857), the CV
Starr Center and the Center for Experimental Social Science (CESS), the JSPS (grant numbers: 21653016,
24653048, 15K13006, 22330061, 15H03328 and 15H05728), the Joint Usage/Research Center at ISER,
Osaka University, and the International Joint Research Promotion Program of Osaka University for researchsupport. We are responsible for all errors. Aoyagi: aoyagi@iser.osaka-u.ac.jp; Fréchette: frechette@nyu.edu;
Yuksel: sevgi.yuksel@ucsb.edu.

11Introduction

Social dilemmas encompass a large class of situations of much interest in the social sciences.
Examples in economics are numerous, ranging from Cournot competition to natural resource extraction. Among them, the prisoner’s dilemma (PD) captures in its simplest forma tension between individual payoff maximization and social efficiency. How this tension
is resolved in a repeated setting as a function of environmental parameters—payoffs, monitoring technology, discounting, the game horizon, and so on—has been an active area oftheoretical research. However, our empirical understanding of behavior in repeated games
is much more limited. Although the number of experiments on repeated games is increasing,
the bulk of our knowledge concerns how the level of cooperation varies with the environmental parameters; and many facets of the documented behavior are still black boxes. For
example, why does a variety of strategies with different levels of cooperation coexist in
both finitely repeated and indefinitely repeated settings? Do these out-of-equilibrium phenomena (at least in light of simple standard models) stem from preferences, information,
incorrect beliefs, or bounded rationality? In this paper, we bring to light one key force tohelp us better understand behavior in such an environment.
A player’s beliefs about other players’ strategies form the foundation of equilibrium
analysis: beliefs are assumed to correctly identify the strategies played by other players,
and the strategies best respond to those beliefs. In repeated games, however, strategies
are complex because they are complete contingent plans that specify actions after every
history, and as in the case of repeated PD, many strategies can be rationalized as best
responses to some beliefs. A player of repeated games will have difficulty forming a belief
that correctly predicts other players’ strategies. This is potentially made even more difficult by multiplicity of equilibria. When repeated, the PD can generate diverse patternsof dynamic behavior rationalized by different beliefs, and thus provides an extremely informative framework for the joint study of beliefs and strategies. In this sense, making
beliefs observable can establish facts that speak to how people reason in repeated games.
For instance, we may find evidence pointing to non-standard preferences if strategies considered only from the perspective of payoffs do not best respond to beliefs, or to a failure oflearning if the strategies do best respond to beliefs but beliefs are incorrect. Such evidence
could explain the presence and persistence of cooperation in the finitely repeated PD andthe variety of strategies observed in the indefinitely repeated PD.
The theoretical contrast between the finitely and indefinitely repeated PD provides
a useful backdrop for the study of beliefs and their relationship to cooperation. The
unique equilibrium entails no cooperation in the finitely repeated PD, but a multitude ofoutcomes ranging from no cooperation to full cooperation are compatible with equilibrium
behavior in the infinitely repeated PD for sufficiently patient players. In contrast to thetheoretical predictions, experiments have shown certain game parameters can generate
early cooperation in both environments. Eliciting beliefs will allow us to explore whether2cooperation in these two canonical environments is sustained by similar forces, despite thetheoretically distinct nature of the two games.
There is a rapidly growing literature on experiments with belief elicitation, but a large
majority of these papers examine beliefs in individual decision making settings. (See Danzet al. (2020) for a recent review.) Those that study beliefs in games mostly use one-shotgames. The primary focus of these papers is the consistency of beliefs and actions, on which
there are mixed results. We review this literature in detail in Appendix A. However, thisliterature is very different from the current paper. In particular, to the extent that such
experiments have induced repeated games in the laboratory, they do so assuming thatincentives in static interactions remain unchanged in repeated play and do not analyze thedynamic incentives of the players.1 In contrast, our experiment elicits beliefs in repeatedgames where dynamic incentives are clearly important. As such, the only closely related
paper is Gill & Rosokha (2020), who elicit beliefs from subjects playing multiple indefinitely
repeated PD games. However, the paper differs from ours both in its focus and design.
In Gill & Rosokha (2020), subjects directly choose strategies (from a list of 10) and in
the first and last supergames also report beliefs over those strategies. Their design allows
them to study beliefs from the onset, and as such their focus is more directly on theevolution of beliefs in indefinitely repeated games and their connection to a level-k modelof rationality. They also link personality traits to strategies and beliefs. They find thatstrategy choices are broadly consistent with beliefs, which is also one of the findings in our
experiment despite the important differences in our designs. They show this for a variety
of indefinitely repeated games, we document this in both finitely and indefinitely repeatedgames.
In a first foray into beliefs in repeated PD games, many questions could be of interest.
However, given the challenges associated with implementing both repeated games andeliciting beliefs in the laboratory, we have opted for simplicity whenever possible. Most
importantly, we only elicit (first-order) beliefs about the other players’ stage actions andnot, for example, beliefs conditional on some action realization, beliefs over beliefs, or
beliefs over supergame strategies. We also use games with perfect monitoring where thepast actions of both players are observed without noise, instead of games with imperfect(public or private) monitoring.
Our experiment consists of two treatments: the Finite game and the Indefinite game.
In the former, subjects play eight rounds of a PD; in the latter, subjects play PDs overa random number of rounds with a continuation probability of 7/8.2 We selected these1
For example, some studies use games for which the equilibrium payoff set does not expand with repetition. In others, the repeated game is simply a byproduct of a design that uses fixed-pairing among subjects.
As such, analyses in these papers do not address dynamic strategies, and no scope exists for subjects tolearn over multiple supergames.
2
Indefinite repetition (first introduced by Murnighan & Roth (1983)) is the standard method of implementing infinitely repeated games in the laboratory. The continuation probability of the indefinite game is3parameters and the stage game based on prior results in the literature: they are expected togenerate not only significant levels of cooperation in both the Finite and Indefinite games,
but also similar levels of round-one cooperation in both environments. Finding parametersthat would generate very different initial cooperation rates between these two games iseasy (Dal Bó 2005). We intended to create two treatments where behavior was expected
to be similar despite the theoretical difference. The treatment variation hence permits acomparison of beliefs of subjects taking the same action in the same round (potentiallyalong the same history) across the Finite and the Indefinite games, and provides insight
into whether their strategic reasoning is similar or different across these two games.
We find beliefs covary with actions (on average and along many histories). However,
we also document some systematic deviations that can be tied to the slow unravelling ofcooperation in the Finite game. In addition, our results show beliefs are not simply the(weighted) empirical average of past observations and are forward looking. For example,
subjects correctly anticipate a decline in the likelihood that their opponent will cooperate
in later rounds of the Finite game.
The finding that beliefs are not simply the summary of past history implies the subjects
are aware (or at least behave as such) of the possibility that their opponent is playing a
supergame strategy that reacts to history in non-trivial ways. This suggests we need tolook beyond beliefs over stage actions and consider supergame beliefs, defined here as beliefs
over supergame strategies. In fact, the experimental literature hints at the possibility thatsupergame beliefs are a key driver of behavior in the repeated PD. For example, the datafrom previous repeated game experiments show evolution of behavior can be well describedby learning and evolutionary models over supergames (Dal Bó & Fréchette 2018, Embrey
et al. 2018, Proto et al. 2020).
Given that beliefs are elicited only on the realized path of play, supergame beliefs are
not directly observable and need to be estimated. We propose a novel method to recoversuch beliefs: we first type subjects according to the supergame strategy they are estimatedto be playing, and then estimate the supergame belief of each individual type separately.3
The results show that subjects who play different supergame strategies have different
supergame beliefs. In fact, for many types, their (supergame) strategy is subjectively rational in the sense that it best responds to their supergame beliefs (and most come close tobest responding).4 These observations suggest heterogeneity in behavior can be explained,
to a large extent, by heterogeneity in beliefs. More generally, in both the Finite and theassociated with the discount factor of the infinitely repeated game.
3
This formulation is close to that explored in Kalai & Lehrer (1993), who show that if players of an
infinitely repeated game start with subjective beliefs about the opponents’ strategies that place positive
probability on their true strategies, Bayesian updating will lead in the long run to the NE play of therepeated game.
4
Note supergame beliefs and supergame strategies are estimated independently: the former uses belief
reports and the latter uses action choices. Thus, the estimation does not necessarily imply such a relation.

4

Indefinite games, subjects’ supergame beliefs correctly anticipate the type of strategies
played in each environment, but are not necessarily well calibrated to the actual frequencyof strategies in the population. Furthermore, supergame beliefs reveal a general tendency
for subjects to overestimate the popularity of their own strategy and to underestimate thelikelihood that others adopt less cooperative strategies than they do.
The paper is organized as follows. The formal description of strategies and beliefs are
given in section 2. section 3 describes the experimental design. Results are presented in
section 4. We conclude with a discussion in section 5.

2Strategies and Beliefs

The stage game is the standard prisoners’ dilemma with two actions, C (cooperation) andD (defection). Let Ai = {C, D} be the set of (stage) actions, and let A = A1 × A2 be
the set of action profiles with a generic element a. The stage-game payoffs gi (a) are given
in Table 1. The horizon of the supergame (repeated game) is either finite or infinite. For
t = 1, 2, . . ., history ht of length t is a sequence of action profiles in rounds 1, . . . , t. Let
H t = At be the set of t-length histories. A player’s (behavioral) strategy σi = (σi1 , σi2 , . . .)is a mapping from the set of all possible histories to actions. σi1 (ai ) ∈ [0, 1] denotes theprobability of action ai in round 1, and for t ≥ 2 and history ht−1 , σit (ht−1 )(ai ) ∈ [0, 1]
denotes the probability of action ai in round t given history ht−1 . Let Σi denote the set ofstrategies of player i. In the supergame with finite horizon T < ∞, player i’s payoff under
the strategy profile is the simple average of stage payoffs:
ui (σ) = T −1

T
X



Eσ gi (at ) ,t=1

where Eσ is the expectation with respect to the probability distribution of hT = (a1 , . . . , aT )induced by σ. In the supergame with infinite horizon, the players have the common discountfactor δ < 1, and their payoff is the average discounted sum of stage-game payoffs:
ui (σ) = (1 − δ)

∞
X



δ t−1 Eσ gi (at ) .

t=1

We postulate that each subject i is endowed with a supergame strategy σi ∈ Σi and a
subjective belief about the supergame strategy played by the other player. Specifically,
we suppose player i believes j’s strategy is randomly chosen from some finite subset Zj
of Σj according to a probability distribution p̃i , which is referred to as player i’s (prior)
supergame belief.5 One interpretation of p̃i is that it represents i’s prior belief over the5

We use p̃ instead of p to denote beliefs. In later sections, we use p to denote the actual distribution ofstrategies in the population.

5

proportion of different strategies played by other subjects in that session.6
Note p̃i can be updated after each round of play conditional on realized history ofplay. For each t ≥ 2 and ht−1 ∈ H t−1 , we denote by p̃ti = p̃i (· | ht−1 ) player i’s updated
supergame belief about j’s strategy in round t given ht−1 . Associated with this is player i’sround t belief µti (ht−1 ), which describes his belief about j’s stage action in round t. Morespecifically, µti (ht−1 ) is the probability that i assigns to j’s choice of action C given ht−1 ,and is related to p̃ti throughXµti (ht−1 ) =p̃ti (σj ) σj (ht−1 )(C).
σj ∈ZjThe belief-elicitation task in this experiment involves beliefs over stage actions. That is,the design elicits from each subject i, in each round t (conditional on history of play),
his belief µti ≡ µti (ht−1 ). For simplicity, we often refer to µti as a “belief.” In section 4.3,
we recover the subjects’ supergame beliefs p̃i from the sequence of their elicited beliefsµ1i , µ2i , . . ..
Player i’s type refers to his supergame strategy σi . In our estimation of supergame
beliefs, we assume player i is Bayesian in the sense that his supergame belief p̃i (· | ht−1 )is updated according to Bayes rule after each history: for any t ≥ 1 and ht = (ht−1 , at ),t−1 t−1p̃t−1)(atj )i (σj ) σj (ht,p̃i (σj ) = P
t−1
t−1 t−1)(atj )σ̃j ∈Zj p̃i (σ̃j ) σ̃j (hwhere beliefs in the first round are p̃1i = p̃i . Player i is subjectively rational if his supergame
strategy σi best responds to his supergame belief p̃i :Xσi ∈ argmaxp̃i (σj ) ui (σ̃i , σj ).
σ̃i ∈Zi

σj ∈ZjSome of the key supergame strategies in our analysis are as follows. AC and AD are thestrategies that choose C and D, respectively, for every history. σi is Grim if σit (ht−1 ) = C if
and only if ht−1 = ((C, C), . . . , (C, C)). σi is TFT (resp. STFT) if σi1 = C (resp. σi1 = D)and σit (ht−1 ) = ajt−1 for every ht−1 and t ≥ 2. For k = 1, 2, . . . , σi is Tk, a thresholdstrategy with threshold k, if σi follows Grim for all t < k, and then switches to AD afterround k.7
6

With random matching, i’s belief about the strategy played by his opponent in each supergame is equal
to his belief about the proportion of strategies in the population.
7
All strategies considered in our analysis are listed and defined in Table 11 of Appendix B.

6

3

DesignThe experiment involves two (between-subjects) treatments, which we refer to as the Finite
and the Indefinite game. Three important considerations (besides the aforementioned aim
for simplicity) guided our experimental design.
1. Selecting parameters such that initial cooperation rates are high in both the Finite and theIndefinite games (and at similar rates). We can reasonably expect people to have different
beliefs in the Finite and the Indefinite games, especially when these games generate very
different behavior (cooperation rates). Although documenting this can have value, focusing
on the more puzzling case where these two types of games generate very similar behavior(specifically in terms of initial cooperation rates) is potentially more interesting. Given
the theoretical contrast between Finite and Indefinite games, beliefs can be particularly
informative in providing insights on whether cooperation is driven by similar considerations
across these two games. To achieve this, we based our parameter selections on previous
experiments and meta-analysis (Embrey et al. 2018, Dal Bó & Fréchette 2018).
2. Introducing belief elicitation while mitigating the impact the introduction might haveon how subjects play. One concern is that asking for beliefs from the onset of the experiment may alter how subjects approach the strategic interaction. To reduce this possibility,
we separate the experiment into two parts. First, subjects are presented with “standard”
repeated PD experimental instructions that do not mention beliefs. Second, after four
supergames, the experiment is paused, and instructions explaining the belief-elicitation
procedures are given. This two-part approach draws on Dal Bó & Fréchette (2019), who
do this for strategy elicitation.8 The results of the current experiment reproduce the qualitative features of previous experiments without belief elicitation (with similar parameters).
Although beliefs at the start of the experiment are not elicited in the two-part approach,the potential benefits of not distorting behavior outweigh the downside of not observing
beliefs in those early supergames.
3. Allowing subjects to gain ample experience. Prior research, both with finitely andindefinitely PD games, show the importance of experience (Embrey et al. 2018, Dal Bó &Fréchette 2018).9 For instance, for the parameters we use in the Finite game, Embrey et al.
(2018) find the average round of the last cooperation moves one round earlier for every
10 supergames. This desire to have subjects play as many supergames as possible is oneof the factors that increase the need for simplicity. Asking more complex belief questions
would necessarily slow down the experiment and reduce the number of supergames.
We now turn to the specifics of the experimental design.
8
They find that choices in their experiments with strategy elicitation are similar to those from experiments without strategy elicitation.
9
Whether experience should be defined in terms of the number of supergames or of the number of totalrounds across multiple supergames is not clear.

7

Table 1: Stage GameIn ECUC
D

C
51, 51
63, 22

D
22, 6339, 39

Normalized

C
D

C
1, 1
2, −1.416

D−1.416, 20, 0

The left panel of Table 1 shows the stage game used in the experiment (in experimental
currency units), whereas the right panel shows its normalized version.10 We use supergame
to refer to each repeated game played between two matched players, and round to referto each play of the stage game. In the Finite game, each supergame ends after eightrounds, T = 8.11 In the Indefinite game, after each round, there is a 87 probability thatthe supergame will continue for an additional round.12 To ensure the observation of atleast eight rounds of play, the indefinite treatment uses the block random design thatlets subjects play for eight rounds for sure, and then informs them of if and when the
supergame actually ended; if it has not ended, they subsequently make choices one roundat a time.13 In the Indefinite game, observation rounds refer to the rounds in which thesubjects actually made action choices, and game rounds refer to those rounds that werepart of the supergames. We denote by T the number of observation rounds in the Indefinite
game so that T = max {8, “No. of game rounds”}. For example, if an Indefinite game has
five rounds, T = 8 because we observe the subject make eight choices even though only
the first five mattered for payoffs, whereas if a supergame lasts 10 rounds, T = 10.
At the conclusion of each supergame, subjects are randomly re-matched to play a newsupergame. After four supergames are played, subjects are given new instructions on the
belief-elicitation task. From that point onward, each subject i is asked in every round t tostate their round t belief µti as an integer between 0 and 100.14 The task is incentivized
via the binarized scoring rule, which determines the likelihood that a subject wins 50
experimental currency units based on their response in this task and the realized action
10

The normalization facilitates comparison with prior studies. With normalization, we set the mutual
cooperation payoff equal to 1 and the mutual defection payoff equal to 0. The normalized temptation payoff
is hence 2 = (63 − 39)/(51 − 39) and the normalized sucker payoff is −1.41 = (22 − 39)/(51 − 39).
11
The parameters used in this paper are identical to those used in one treatment of Embrey et al. (2018).
12
The expected length of a supergame is hence eight rounds. The random termination is determined bya pseudo-random number generator whose seed is set arbitrarily at the beginning of the session.
13
This method was first introduced in Fréchette & Yuksel (2017) and has now been used in multiple
papers on a variety of topics, for example, Vespa & Wilson (2019) in dynamic games, Agranov et al. (2016)
in bargaining, and Weber et al. (2018) in a bond market.
14
Recall that µti is the probability assigned by i to j’s choice of action C in round t.

8

Table 2: Session Summary
No. of Game Rounds
No. ofNo. ofActions

SessionSubjects

Supergames

Only

Finite

1
2
3
4
5
6
7
8

20
20
20
20
20
20
20
18

12
12
13
11
13
13
12
12

8, 8, 8, 8

8, 8, 8

Indefinite

1
2
3
4
5
6
7
8

20
20
18
16
14
14
18
20

10
9
7
8
12
6
10
9

9, 7, 13, 7
8, 15, 7, 32
8, 2, 3, 14
9, 7, 10, 13
7, 22, 7, 3
1, 31, 4, 3
5, 6, 7, 14
11, 1, 4, 13

1, 2, 23,
2, 10,
25,
32,
2, 5, 8,24,
30, 8, 5,9, 5,Treatment

Total no. of

Actions and Beliefs
Early
8, 8,
8, 8,
8, 8, 8,8,
8, 8, 8,8, 8, 8,8, 8, 8,8, 8,4, 14,Obs.

Late

Rounds8, 8, 8

96
96
104
88
104
104
104
96

4, 1, 19
5, 1, 817, 107, 7, 69, 3, 1015
4, 9, 42, 4, 2

112
105
90
96
119
94
109
81

302 subjects in total.
Payment: $8 + choices from two supergames (pre/post) + beliefs in one.
Earnings from $22.00 to $63.75 (with an average of $35.30).

choice of the matched subject.15 The belief question is presented on a separate screen after
subjects have made their action decision for that round and before feedback is provided.
This process continues until the first supergame to terminate after at least one hour of play
has elapsed.
Although prior research on indefinite PDs has not found that risk aversion is an important determinant of choices (Dal Bó & Fréchette 2018), risk preferences could, in principle,
mediate the relation between beliefs and choices. For this reason, we also elicited subjects’risk preferences at the end of each session using the bomb task (Crosetto & Filippin 2013).
Instructions for this task were distributed after the completion of the last supergame.16
We conducted eight sessions per treatment and 16 sessions in total.17 Table 2 summarizes basic information about each session. The supergames for the part with belief
15

Unlike the classical quadratic scoring rule that is incentive compatible only under risk neutrality, incentive compatibility of the binarized scoring rule is independent of a subject’s risk attitude. See Hossain& Okui (2013) and Allen (1987) for an earlier formulation of the idea. We use the implementation outlined
in Wilson & Vespa (2018).
16
The maximum possible earning from this task is 99 experimental currency units.
17
This number of sessions per treatment is more than the typical number. The reason for having moresessions will become apparent in the section on beliefs over strategies, because the method we propose isdata intensive.

9

elicitation are separated into early and late. We use this categorization in the presentationof results, with most of the data analysis focusing on late supergames.18 We randomly
chose one supergame without belief elicitation and one supergame with elicitation for payment, and paid subjects for the outcomes of all game rounds for those two supergames.
We also paid subjects for the belief-elicitation task in one randomly selected round of one
randomly selected supergame.19

4

Results

The analysis of our data is separated into three sections. Section 4.1 provides an overview
of the qualitative features of observed behavior focusing on actions. Section 4.2 presents
results on beliefs (over actions), namely, their accuracy, how they are affected by history,
and their relation to actions. Finally, section 4.3 proposes a methodology to recover beliefs
over supergame strategies and uses this method to study how the strategy choice relates
to beliefs.

4.1

Actions

For any supergame, denote by xti the indicator of subject i’s choice of C in round t, and byx̄t , the round t cooperation rate averaged over subjects. As will be clear from the context,
the analysis in what follows sometimes aggregates x̄t over multiple supergames.
Figure 1 shows cooperation rates by supergame. Starting with the Finite game (theleft panel), we observe relatively high initial (round one) cooperation rates slightly above
80%. Focusing on rounds > 2, and dividing the sample into two cases, xti following theother player’s cooperation ajt−1 = C and those following other’s defection at−1
= D, wejobserve high cooperation rates following cooperation and low cooperation rates following
defection. We also observe that the difference between those two averages, referred to as
responsiveness, increases with experience. The cooperation rate in round eight is decreasing
with experience and is low by the end (below 20%).
The right panel of Figure 1 presents the same statistics for the Indefinite game. In thiscase, and as with the Finite game, round-one cooperation rates are high (start slightly below
18

We aimed for three supergames for both early and late when possible. When that was not possible, we
aimed for each group to have a division of total rounds that was as balanced as possible.
19
To address hedging concerns, we chose the supergame for the belief-elicitation task from the supergames
not used for the action task. Experimental currency units were translated into earning in dollars at an
exchange rate of 3 cents per point. All subjects also received a show-up fee of $8. Earnings from theexperiment varied from $22.00 to $63.75 (with an average of $35.30). All instructions (available in AppendixC) were read aloud. The computer interface was implemented using zTree (Fischbacher (2007)) and subjects
were recruited from UCSB students using the ORSEE software (Greiner (2015)).

10

.8
Cooperation Rate
.4
.6

Cooperation Rate
.4
.6

.8

1

Indefinite

1

FiniteRound 1Other Coop. at t-1
Other Defect at t-1
Round 85 6 7Supergames

L-2 L-1 Last

Late

5 6 7
Supergames

L-2 L-1 Last

0

0

.2

Late

.2

Early

Early

1

2

3

4

1

2

3

4

Figure 1: Cooperation Rate over Supergames

80% and increase to slightly above 80%). Cooperation rates following cooperation by theother are high, whereas cooperation rates following defection are low. Again, responsivenessincreases with experience. However, in contrast to the Finite game, cooperation rates inround eight are high and increasing with experience.20
Hence, consistent with prior experiments with similar parameters, the design successfully generates similar and high levels of round-one cooperation in both games. Also in linewith prior findings, subjects display responsiveness that increases with experience. Finally,
cooperation collapses at the end of the Finite game but persists in the Indefinite game. Insummary, behavior along key dimensions is qualitatively consistent with prior findings onthese two games, and we find no indication of important changes in the subjects’ behavior
caused by the belief-elicitation task in these environments.21
Result 1 We reproduce qualitative data patterns observed in previous experiments on
20

Instead of round eight, one might want to compare the round-eight behavior in the Finite game to thelast game round in the Indefinite game, or to the last observation round. Doing so does not qualitatively
change the results. These alternative figures are presented in Appendix B (Figure 13).
21
Table 6 in the Appendix B also shows no significant changes in round-one choices for supergames where
beliefs are elicited (compared with those where they are not).

11

Finite and Indefinite PD games. In particular, our results confirm cooperation is history
dependent in both games. Furthermore, cooperation evolves differently in both games: it
collapses at the end only in the Finite game.

4.2

BeliefsP
Let µ̄t = ni=1 µti denote the average of round t beliefs in any given supergame. Again, µ̄t
is aggregated over multiple supergames and/or over particular histories in what follows.

Round 41

Round 3
1

Round 2
1

1

Round 1

.8
0

.2

Cdf
.4 .6

.8
0

.2

Cdf
.4 .6

.8
.2
0

0

.2

Cdf
.4 .6

Indefinite

Cdf
.4 .6

Round 5

Round 6

Round 7

Round 80 .2 .4 .6 .8 1
Belief

.8
0

.2

Cdf
.4 .6

.8
.2

Cdf
.4 .6

.8
0

.2

Cdf
.4 .6

.8
Cdf
.4 .6
.2
0

0 .2 .4 .6 .8 1
Belief1

0 .2 .4 .6 .8 1
Belief1

0 .2 .4 .6 .8 1
Belief1

0 .2 .4 .6 .8 1
Belief1

0 .2 .4 .6 .8 1
Belief0.8

Finite0 .2 .4 .6 .8 1
Belief0 .2 .4 .6 .8 1
BeliefLate supergames.
Vertical lines indicate respective means.

Figure 2: Distribution of Beliefs by Round
Figure 2 displays the cumulative distributions of (unconditional) beliefs in rounds t =1, . . . , 8 of late supergames in the Finite and Indefinite games.22 As the figure clearly22

The reader interested in an equivalent to Figure 1 from the previous section (but focusing on beliefs
instead of actions) is referred to Appendix B (Figure 14).

12

shows, beliefs evolve very differently over rounds across these two types of games.23 Beliefs
become comparatively more pessimistic in the Finite game as the supergame unfolds. The
difference in the average belief is statistically significant in rounds six, seven, and eight.24
A few more observations are worth making. First, beliefs are varied and do not concentrate on a few values. Second, subjects do report beliefs of 0 and 1, not just interior values.
In fact, in round eight, more than 40% of subjects place probability one on defection by
the other player in the Finite game, whereas more than 40% of subjects place probability
one on cooperation by the other player in the Indefinite game.
Result 2 Beliefs are different in Finite and Indefinite games. The main difference is that
beliefs about cooperation collapse toward the end in the Finite game.
4.2.1

Actions and Beliefs

Putting beliefs and actions together reveals beliefs–on average–track cooperation rates
closely. Figure 3 shows for late supergames that the point estimate for average belief µ̄t isclose to that for the average cooperation rate x̄t in each round t and that their confidence
intervals display substantial overlap. When aggregated over all rounds, the differences
between action frequencies and beliefs are small, at less than one percentage point for
Finite and two percentage points over the first eight rounds of Indefinite. This difference
is not statistically different from 0 for the Finite game, but it is for the Indefinite game(even though the difference is small in magnitude).25
However, when we look at each round separately, both in the Finite and the Indefinite
games, we see a statistical difference between action frequencies and beliefs for rounds one
through three. The difference is about four percentage points for each of the three rounds ofthe Finite game, whereas it is 11, 5.8, and 0.2 percentage points for the same rounds of the
Indefinite game. In rounds seven and eight, we also see statistically significant differences
between action frequencies and average beliefs for the Finite game. The difference is 9.5 and3.1 percentage points for rounds seven and eight, respectively. (The corresponding values
are 0.1 and 1.1 in the Indefinite game.) In other rounds (rounds 4-6 of the Finite game23
Throughout, results over rounds will focus on the first eight rounds. For the Indefinite game, we have
many more rounds, but sample sizes are substantially smaller for rounds nine and above.
24
Respectively, p < 0.05, p < 0.01, and p < 0.01. Throughout, when statistically significant is usedwithout a qualifier, it refers to the 10% level. Here and elsewhere, unless noted otherwise, statistical tests
involve subject-level random effects and session-level clustering (see Fréchette (2012) and Online Appendix
A.4. of Embrey et al. (2018) for a discussion of issues related to hypothesis testing for experimental data).
In the case of beliefs, as here, we use a tobit specification allowing for truncation. For tests of cooperation,
we use a probit specification.
25
We perform the test on the difference between the opponent’s action (coded as 1 for cooperate and 0
for defect) and the reported belief. Results are robust to including all observation rounds or only the firsteight rounds.

13

.8
Cooperation Rate
.4
.6
.2
0

0

.2

Cooperation Rate
.4
.6

.8

1

Indefinite

1

Finite

1

2

3

4
5
Round

6

7

8

1

2

3

4
5
Round

Cooperation Rate

Belief

95% Conf. Int. Coop.

95% Conf. Int. Belief

6

7

8

Late supergames.

Figure 3: Choices and Beliefs by Roundand rounds 4-8 of the Indefinite game), beliefs and cooperation rates are not statisticallydifferent at the 10% level. In summary, to the extent that action frequencies and beliefs
differ, the deviations are most prominent for late rounds in the Finite game and earlyrounds for the Indefinite game.
One natural question is whether, with experience, subjects learn to correct their mispredictions. Figure 4 displays the error in key rounds for early versus late supergames. As
the figure shows, in many cases where more substantial error occurs in early supergames,
improvement is observed in late supergames, but not for round seven of the Finite gameand round one of the Indefinite game. Even in these cases, however, subjects’ beliefs domove in the right direction. As seen in Figure 15 in Appendix B, which reports average
cooperation rates and average beliefs for rounds one and seven over supergames, beliefs
move in the correct direction with experience, but not fast enough to catch up with thechanges in actions. We should note, however, that the changing behavior over the course of
the session does not always imply beliefs are systematically off. For instance, in that same
figure, one can see cooperation rates in round seven of the Indefinite game are changing
with experience, but subjects correctly anticipate this change, as reflected in their beliefs.
Although determining exactly how beliefs are formed is not the goal of this study,
understanding what allows subjects to predict actions relatively well is of clear interest.
One conjecture is that subjects are simply reporting back their observations about others’
behavior from previous supergames. Alternatively, subjects may form beliefs relying on
14

.1
.05
Belief error
0
-.05
-.1

.05
-.05
-.1

Belief error
0

.1

.15

Indefinite.15

Finite

1

6

7

-.15

-.15

Late supergames

8

Round

Early supergames

1

6

7

8

Round

Belief error denotes average difference between beliefs and actions.

Figure 4: Belief Errors in Early vs. Late Supergames

introspection alone, or some combination of learning and introspection.26 The data suggeststhat although experiences matter in shaping beliefs, they are not the sole determinant.
Figure 16 in Appendix B shows the kernel density estimates of the differences between
beliefs and the subject-specific experienced frequencies for the fifth (the first with belief
elicitation) and last supergames of any given session. Although each panel displays a peakclose to 0, many are relatively flat and some are not centered at zero.
So far in Figures 3 and 4, we considered only unconditional beliefs, but what about
the subjects’ ability to anticipate actions following specific histories? To consider histories
with a sufficient number of observations, we examine this question for round two. Figures 5
and 6 present the relevant data conditional on round-one histories (labeled with one’s own
action first followed by the opponent’s action). In both the Finite and Indefinite games, weobserve that beliefs quickly adjust in response to the other’s action.27 Interestingly, note thedownward adjustments following a unilateral choice of D by either player in round one are
26
The earlier observation about the Finite game—although behavior is changing in round seven, beliefs
track action frequencies closely—already suggests subjects cannot be basing their beliefs only on empiricalfrequencies.
27
They should not be correct in round one, because beliefs are unconditional, whereas by construction,
the figures present specific action frequencies in round one.

15

14%
Cooperation Rate
0 .2 .4 .6 .8 1

Coop-Defect in Round 1

70%
Cooperation Rate
0 .2 .4 .6 .8 1

Coop-Coop in Round 1

Actions
Beliefs

2

3

4
5
Round

6

7

8

1

2

3

4
5
Round

6

7

Defect-Defect in Round 1

14%

2%

8

Cooperation Rate
0 .2 .4 .6 .8 1

Defect-Coop in Round 1
Cooperation Rate
0 .2 .4 .6 .8 1

1

1

2

3

4
5
Round

6

7

8

1

2

3

4
5
Round

6

7

8

Own choice listed first.
Late supergames.
Percentage of cases under the title.

Figure 5: Conditional Round-Two Beliefs, Finite Gamesof the correct magnitude even though the required adjustment is quite large. Comparing
the two figures, we see action frequencies and beliefs evolve in a similar fashion in all panels
except for the top-left panel, which shows clear differences across the two treatments. In
the Finite game, most of the initially cooperative interactions eventually break down, andbreakdown is mirrored by beliefs. In the Indefinite game, on the other hand, beliefs about
cooperation are sustained if they survive the second round.
These results showing beliefs that are fairly accurate, both averaged over histories andalong specific histories, do not speak directly to whether many or few subjects correctly
anticipate actions at the individual level. One way to answer this question in a simple but
structured way is to look at whether subjects are accurate in at least assessing whether
cooperation by their opponent is a relatively likely or unlikely event. Specifically, we denotecooperation (by one’s opponent) conditional on a history to be unlikely if the empiricalfrequency of cooperation is less than one third, likely if the empirical frequency is morethan two thirds, and uncertain if the empirical frequency is between these values. Then,we identify the share of observations for which a subject’s belief is accurate relative tothis categorization; that is, we look at whether the belief lies in the same tercile (unlikely/likely/uncertain) as the observed average cooperation rate. We do so for rounds one
and two.
16

12%
Cooperation Rate
0 .2 .4 .6 .8 1

Coop-Defect in Round 1

71%
Cooperation Rate
0 .2 .4 .6 .8 1

Coop-Coop in Round 1

Actions
Beliefs

2

3

4
5
Round

6

7

8

1

2

3

4
5
Round

6

7

Defect-Defect in Round 1

12%

5%

8

Cooperation Rate
0 .2 .4 .6 .8 1

Defect-Coop in Round 1
Cooperation Rate
0 .2 .4 .6 .8 1

1

1

2

3

4
5
Round

6

7

8

1

2

3

4
5
Round

6

7

8

Own choice listed first.
Late supergames.
Percentage of cases under the title.

Figure 6: Conditional Round-Two Beliefs, Indefinite Games

Table 8 in Appendix B shows that accuracy of beliefs at the individual level, as defined
above, is high both for round one (73% in the Finite game, 67% in the Indefinite game)and round two (83% in the Finite game, 80% in the Indefinite game). The accuracy rate is
substantially above 33% (the benchmark if beliefs were random) even in early supergames.
However, after one history, accuracy is low: in round two of the Indefinite game alongh1 = (C, D) (cooperation by oneself and defection by the other), beliefs fall in the correct
tercile only 29% of the time. Interestingly, the opposite is not true: round-two beliefs alongh1 = (D, C) (defection by oneself and cooperation by the other) fall in the correct tercile
79% of the time. Table 8 also considers more demanding tests of accuracy by reporting thefraction of times the empirical frequencies of cooperation are within ±5 and 10 percentage
points of reported beliefs. Beliefs are fairly accurate along some histories (especially themore common ones, e.g., h1 = (C, C)), but less so along other histories that are lesscommon (particularly along h1 = (C, D) and (D, C) in the Indefinite game).
As Figures 5 and 6 above show, supergames starting with joint cooperation are the mostcommon. How do beliefs evolve on a mutual cooperation path? Figure 7 shows the average
cooperation rates x̄t and average beliefs µ̄t along the history ht−1 = ((C, C), . . . , (C, C)).28
28

Note t = 2 corresponds to cases presented in Figures 5 and 6.

17

.8

.8

Cooperation Rate
.4
.6

Cooperation Rate
.4
.6

.2

.2

1

Indefinite

1

FiniteBeliefs: Cooperative Path00Actions: Cooperative Path

2

3

4
5
Round

6

7

8

2

3

4
5
Round

6

7

8

Cooperative path: joint cooperation up to that round.
Late supergames.

Figure 7: Cooperative Path (First Eight Rounds)

For example, a solid circle at round five indicates the empirical cooperation rate after four
rounds of joint cooperation (close to 100% in both games). The most striking observation is
the sharp decline in beliefs toward the end in the Finite game. That is, subjects (correctly)
anticipate the increasing likelihood of defection from their opponent despite the fact that
all choices up to that point were cooperative for both players.29 Nonetheless, we see clear
evidence that subjects underestimate the degree to which cooperation drops from round
6 to 7: whereas beliefs are well calibrated in round 6 (within 1 percentage points of theempirical frequency), they show optimism (13 percentage points higher than the empiricalfrequency) in round 7.30 In summary, these findings suggest that although subjects anticipate the decline in cooperation, they underestimate the magnitude and foresee only 60%of the actual drop in cooperation. In the Indefinite game, on the other hand, beliefs andcooperation rates remain high as the supergames unfold. We note also these patterns are
already visible in early supergames (see Figure 17 in Appendix B).
The last observation suggests, in particular, that the evolution of beliefs in the Finite
game cannot simply be explained by heuristic models based on past action choices (within
29

The decline in beliefs is not driven by selection: conditioning on subjects who remain on a cooperative
path until the eighth round, beliefs decline from 89% in round 2 to 49% in round 8.
30
By round 8, the error declines to less than 4 percentage points.

18

a supergame). For example, if a subject always set his belief equal to his opponent’s actionin the previous round, he would report beliefs for round 7 (in the Finite game) that arealmost three times more over-optimistic and less than half as accurate than the ones weobserve in the data.31 Clearly, beliefs in the Finite game change on a cooperative pathwith the length of the interaction, and hence are non-stationary.
Result 3 (1) Beliefs are accurate, on average, but show some systematic and persistentdeviations: they are optimistic late in the Finite game and pessimistic early in the Indefinite
game. (2) Beliefs respond to the history of play. (3) However, differences exist across games
even along the same history. In particular, subjects correctly anticipate cooperation will
break down despite a history of joint cooperation in the Finite game.
FiniteIndefinite
3

3

Defect2
Density1001

Density

2

Coop.

0

.2

.4

.6

.8

1

0

Belief

.2

.4

.6

.8

1

Belief

Vertical lines indicate respective averages.
Late supergames.

Figure 8: Beliefs of Defectors vs. Cooperators in Round One

We now turn to the question of whether different actions are supported by different
beliefs. Figure 8 shows kernel density estimates of the distribution of round-one beliefsµ1i by treatment and by the subject’s own action a1i in round one. At a broad level, we
can easily see that the beliefs of cooperators and defectors are more different from one
another in the Indefinite game than in the Finite game. The average beliefs of cooperators
31

P
P
7
7
6
7
1
For the first exercise, we compare n1m(i) is the subject
i (µi −xm(i) ) with ni (xm(i) −xm(i) ), where P
P
7
7
6
7
1
matched with subject i. For the second exercise, we compare n ( i |µi − xm(i) | with n1
i |xm(i) − xm(i) |.

19

and defectors are statistically different in the Indefinite game (p < 0.01) but not in the
Finite game.32 Of those subjects who reported a belief of less than 50% in round one,
only 39% cooperated in the Indefinite game, in contrast to 55% who cooperated in the
Finite game. Subjects with optimistic beliefs cooperated in both treatments: of thosesubjects who reported a belief greater than 50% in round one, 94% cooperated in the
Indefinite game and 87% cooperated in the Finite game. In other words, round-one beliefs
were more predictive of round-one actions in the Indefinite game than in the Finite game,and subjects with higher beliefs tend to defect more often in the Finite game than in the
Indefinite game.33.8

1Round 4

0

.2

CDF
.4 .6

.8
CDF
.4 .6
.2
0

.2

CDF
.4 .6

.8

1

Round 3

1

Round 2

0

0

.2

CDF
.4 .6

.8

1

Round 1Round 5

Round 6

Round 7

Round 80 .2 .4 .6 .8 1
Belief

.8
0

.2

CDF
.4 .6

.8
0

.2

CDF
.4 .6

.8
0

.2

CDF
.4 .6

.8
CDF
.4 .6
.2
0

0 .2 .4 .6 .8 1
Belief1

0 .2 .4 .6 .8 1
Belief1

0 .2 .4 .6 .8 1
Belief1

0 .2 .4 .6 .8 1
Belief1

0 .2 .4 .6 .8 1
Belief0 .2 .4 .6 .8 1
BeliefCoop.--Finite

Coop--Indefinite

Defect--Finite

Defect--Indefinite

0 .2 .4 .6 .8 1
BeliefLate supergames.

Figure 9: Beliefs by Action and Treatment: Rounds One through Eight

Figure 9 plots the CDF of beliefs by action and treatment for each round. It clearly
32

However, a Kolmogorov-Smirnov test rejects that the distributions are the same in both treatments at
the 1% level, but we cannot account for the panel structure of the data with this test.
33
Appendix B reports additional results with respect to the determinants of cooperation using regression
analysis. These results support the patterns reported in the paper. Specifically, we find beliefs are predictiveof actions in both the Finite and Indefinite games. Focusing on round one, although beliefs are significant inboth games, we find they have more predictive power in the Indefinite game. These results also suggest risk
preferences have some limited predictive power for round-one choice in the Finite game (with the likelihoodof cooperation decreasing with risk aversion). Recently, Proto et al. (2019) provide some evidence consistent
with this finding.

20

shows cooperation and defection are associated with different beliefs. Except for round
one in the Finite game, in every other comparison—every round for each treatment—the
average belief is statistically different between those who cooperate versus those who defect(all p-values < 0.01). Higher cooperation rates are associated with more optimistic beliefs
more generally. Table 9 in Appendix B shows that in all rounds, the marginal impactsof beliefs on the likelihood of cooperation are positive in both the Finite and Indefinite
games. In addition, the round number has a significant negative impact on cooperationin the Finite game but is insignificant in the Indefinite game.34 More specifically, subjects
are more likely to defect later in the Finite game, even if their beliefs are the same.
Perhaps more surprisingly, cooperation and defection in certain rounds are associatedwith different beliefs for Finite versus Indefinite games. In round eight, beliefs of thesubjects who cooperate are statistically different across treatments (p < 0.01), as are thoseof the subjects who defect (p < 0.1). Subjects who defect in round eight of the Finite gameare more pessimistic (on average) than those who do so in the Indefinite game. Similarly,
subjects who cooperate are more optimistic in the Indefinite game than those in the Finite
game. On the other hand, subjects who defect in round one of the Finite game are more
optimistic than those who do so in the Indefinite game (p < 0.01). Hence, the same action
can be supported by different beliefs in those two games.
Result 4 Beliefs correlate to actions, and more optimistic subjects are more likely tocooperate. The same-round belief can generate different actions in each game.

4.3

Beliefs over Supergame Strategies

The preceding section finds a link between beliefs and actions, and also that beliefs are
not just the summary of past action choices in a supergame. These patterns lead us tothe consideration of beliefs over strategies.35 The estimation method we develop has threestages and treats separately data on actions and beliefs without imposing any structure
between them, thus allowing meaningful questions about whether strategies best respondto beliefs.
Plan for the Estimation Strategy:
1. Estimate strategies at the population level.
2. Use these estimates and each subject’s choices to classify them into types.
3. Estimate beliefs over supergame strategies separately for each type.
34

Specifications with our measure of risk attitude find that it is not statistically significant.
We do not make the claim that subjects reason in terms of strategies per se, but that we can represent
their behavior as such.
35

21

Details are provided in sections 4.3.1, 4.3.2, and 4.3.3 below. Here, we outline theintuition for the approach using a simplified example. Suppose we want to recover beliefs
over strategies for one player (referred to as player 1) when the data available to us areround beliefs over actions elicited in one supergame (against player 2). For the purposeof the example, assume we know player 1 considers that player 2 uses one of only threestrategies: AD, AC, or Grim. In round one, we observe player 1’s unconditional belief
that his opponent will start by cooperating: µ11 = 0.6. From this belief, we can alreadyinfer the probability player 1 associates with player 2 playing AD, because that strategy
is the one considered that starts by defection. That is, we can infer p̃(AD) = 0.4 andp̃(AC) + p̃(Grim) = 0.6. However, we cannot determine p̃(AC) or p̃(Grim) separately. To
do so, we look at beliefs elicited in other rounds of the supergame. Assume that in roundone, player 1 plays D and player 2 plays C. After observing this history, player 1 reports
his round-two belief: µ21 = 0.1. Because player 2 started by playing C, player 1 now knows
she is not playing AD. However, player 1’s belief about whether player 2 will cooperate
in round two can reveal information about whether he believes player 2’s strategy is more
likely to be AC or Grim. Note that after such a history of (D, C), the two strategies indeedprescribe different actions: D for Grim and C for AC. Given µ21 , we can recover (via Bayes’rule) that p̃(AC) = 0.06 and p̃(Grim) = 0.54. This method provides us with a roadmap for
how we can recover ex-ante beliefs over strategies using data on beliefs over stage actions
elicited in each round of a supergame. In addition, we allow for players to believe others
implement their strategies with error and that subjects may report their belief with someerror.
The example above lays out the intuition behind our methodology as well as highlightingsome of the challenges it presents. We outline below how we address these challenges.
(1) Belief estimation in the example above relies on the assumption that the relevant
strategies (over which subjects have beliefs) are known. How do we specify therelevant set of strategies for our data set? By now, a significant body of literature
documents which strategies are used in repeated PD games. We use results from thisliterature to determine which strategies to include in our consideration set.
(2) The example was constructed such that the data can easily separate the strategies
considered; but in some cases, this can require specific histories that are not common
and thus call for more data. This forces us to pool data from multiple subjects.
However, assuming all subjects share the same beliefs seems unreasonable. Instead,
we group subjects according to the strategy that best describes how they play, referred
to as their type. We assume subjects of the same type share the same beliefs.36
36
To explore whether subjects of the same type have similar beliefs, we do the following exercise. We
compute the spread of beliefs defined as the difference between the 25th and 75th percentiles of beliefs
averaged over rounds and histories. We test whether the spread of beliefs is less among subjects that areof the same type relative to all others in the population. Out of the 10 types (to be defined later) observed

224.3.1

Population-Level Estimates of Strategies

We first use the Strategy Frequency Estimation Method (SFEM) introduced in Dal Bó &Fréchette (2011) to estimate the distribution of strategies used.37 The method first specifies
the set of candidate strategies and then estimates their frequencies in a finite-mixture modelallowing for the possibility of implementation errors. Formally, the SFEM results provide
two outputs p and β, both at the population level: p is a probability distribution over theset of strategies, and β is the probability that the choice corresponds to what the strategy
prescribes. We identify the values of p and β that maximize the likelihood of the observed
sequences of action choices.
We use a two-step procedure to determine the set of strategies in our analysis. First werely on prior evidence to construct a consideration set of 16 strategies. Then, we use resultson this set to narrow our focus to 10 strategies. The larger consideration set includes all
strategies that Fudenberg et al. (2012) report have a statistically significant SFEM estimate
in at least one indefinitely repeated game with perfect monitoring.38 Motivated by the
results of Embrey et al. (2018), who document the prevalent use of threshold strategies
with experience in finitely repeated PD games, we also add to the consideration set all
threshold strategies up to T8.39 Results on this consideration set are reported in Appendix
B. However, because our primary goal is to estimate beliefs over strategies, focusing onsuch a large set is more costly than it is typically with SFEM: having more strategies canmake identifying beliefs over different strategies difficult; it can also reduce the number ofobservations per type in the belief estimation. For these reasons, we use results from thelarger consideration set to focus our analysis on the 10 strategies that are most importantin terms of choices as well as beliefs. This set consists of AD, AC, Grim, TFT, STFT,
Grim2, and TF2T, as well as threshold strategies T8, T7, and T6.40
Table 3 presents the estimation results (in columns 2 and 5) sorted by prevalence. The
results are consistent with prior evidence on strategy choice in repeated PD: Thresholdstrategies are important in the Finite game (Embrey et al. 2018), and AD, Grim, and TFT
in the Finite game and the eight types in the Indefinite game; only three of the 18 paired comparisons is
not in line with the assumption that the spread in beliefs is less among subjects of the same type.
37
See Dal Bó & Fréchette (2018) for results across a variety of experiments applying the SFEM toindefinite PDs with perfect monitoring. Results from direct elicitation of strategies (Dal Bó & Fréchette(2019) and Romero & Rosokha (2019)) are in line with SFEM results and indicate that for indefinite PDs
with perfect monitoring, the majority of subjects use strategies with a TFT, Grim, or AD structure.
38
Our aim was to be inclusive in the first step of the selection process. A strategy is included if it has beenshown to have predictive power in any setting among a variety of environments covering a large parameter
set.
39
Thus, the consideration set is AD, AC, Grim, TFT, STFT, Grim2, Grim3, TF2T, 2TFT, and T2–T8.
Appendix B provides a detailed description of each of these strategies.
40
From the original set, we eliminate T2–T5, which our estimates indicate are not relevant in the Finite
game, as well as 2TFT and Grim3, which are not popular enough in the Indefinite game to generate reliable
belief estimates.

23

Table 3: Strategy Prevalence and Typing

Type
T7
T8
AD
TFT
T6
Grim
TF2T
STFT
AC
Grim2

FiniteShare
SFEM Typing
0.30
0.35
0.22
0.20
0.12
0.12
0.09
0.12
0.08
0.08
0.08
0.02
0.04
0.04
0.03
0.03
0.03
0.03
0.00
0.01

Indefinite
Share
Type SFEM Typing
TFT
0.36
0.59
Grim
0.18
0.09
Grim2
0.11
0.11
AC
0.11
0.05
TF2T
0.10
0.01
AD
0.09
0.10
STFT
0.04
0.04
T8
0.01
0.01
T7
0.00
0.00
T6
0.00
0.00

Estimation using late supergames.
SFEM estimate for β are 0.94 for both.

account for a majority of the strategies in the Indefinite game (Dal Bó & Fréchette 2018).41
More specifically, in the Finite game, T7 and T8 account for a little over half of thestrategies, and they, along with AD, make up two thirds of the choices. Another thresholdstrategy, T6, is also in the top 5 at 8%. Additionally, TFT and Grim are commonly usedstrategies (at the 4th and 6th positions).
In the Indefinite game, conditionally cooperative strategies dominate, with TFT and
Grim representing more than half of the choices. The lenient versions of Grim and TFT are
also among the popular strategies, accounting together for 21% of the choices. Together
these four account for more than two thirds of the strategies. Other prominent strategies are AC and AD, two unconditional strategies, representing 20% of the choices. Allother strategies are at most 4% each, and the threshold strategies are almost completely
irrelevant. Together, conditionally cooperative strategies account for 75% of the data (bycontrast, these strategies represent only 21% of the data in the Finite game).
Result 5 We reproduce results about strategy choices observed in previous finitely andindefinitely repeated PD games.42 In particular, our results confirm strategic heterogeneity
exists within and across treatments. In the Finite game, subjects mostly use threshold
41

The Appendix also reports SFEM results for early supergames (the changes are presented in Figure 19).
Consistent with Embrey et al. (2018) those results show that threshold strategies increase with experiencein the Finite game.
42
To our knowledge, this study is the first to compare strategies in Finite and Indefinite games within
the same experimental paradigm.

24strategies, whereas in the Indefinite game, they mostly rely on conditionally cooperative
strategies.
4.3.2

Typing of Subjects

We use the SFEM results to type subjects according to the strategy that they are most likely
playing. Recall the SFEM yields the probability distribution over supergame strategies (p)and the probability of implementation errors (1 − β). These probabilities can be used tocompute the Bayesian posterior that a subject is playing each of the candidate supergame
strategies given the sequence of his actions. Each subject is associated with the supergame
strategy that has the highest likelihood according to this posterior.43
To demonstrate how this works, consider a simpler setup where the set Z of candidate
strategies consists only of AD and AC. Assume the SFEM yields p = (pAD , pAC ) = (0.7, 0.3)
d and AC,d whereand β = 0.9. The corresponding behavioral strategies are then given by AD
t−1
for every h ,d t−1 ) = 0.9 ◦ D + 0.1 ◦ C,AD(hd t−1 ) = 0.9 ◦ C + 0.1 ◦ D.
AC(hb = {AD,d AC}
d using the
We suppose the strategy of each subject is chosen from the set Z
44
prior distribution p. Assume now that a subject exists who, over multiple supergames
consisting of 24 rounds in total, cooperates in 20 rounds and defects in four rounds. Givend versus
p and β, we can calculate the Bayesian posterior that this subject is playing AD
420pAD
ˆ β (1−β)d In fact, the posterior that the subject is playing AD
d isAC.
4 (1−β)20 +p
20 (1−β)4 ,
pAD
β
β
ˆ
ˆ
AC
d is close to 1. Consequently,
which is close to 0, whereas the posterior that he is playing AC
this subject would be typed as playing AC. Note that in the actual typing exercise, mostof the strategies are history dependent. This finding implies that calculating the Bayesianposterior requires comparing for each round the actual action choice of the subject withthe action implied by each strategy given the history up to that point.
The results of the typing exercise are reported in the third and sixth columns of Table
3. The type shares are largely similar to the population estimates from SFEM. However,
we also observe some differences. In particular, in the Indefinite game, the fraction ofsubjects typed as TFT is greater than the fraction of TFT in the population.45 Clearly,
43

A unique strategy exists within the consideration set for each subject in our data set that achieves thehighest posterior (given the SFEM results).
44
One could use a different prior. We have explored using a uniform prior in simulations, and the results
are far worse than with the SFEM estimates.
45
Two potential sources for such differences are possible. First, and simply mechanically, some subjects
play more supergames than others; thus, the fraction of subjects corresponding to a type can differ from the

25

the smaller the fraction of subjects of a given type, the less reliable their belief estimates
will be.
4.3.3

Estimating Supergame Beliefs

For each type in our data, we estimate their supergame beliefs over strategies p̃, as well as
parameters β̃ and ν.46 Specifically, p̃ is a probability distribution over the set Z̃ β̃ , which
has one-to-one correspondence with the set Z of candidate strategies used in the SFEM as
follows: for each σj ∈ Z, σ̃j ∈ Z̃ β̃ is a stochastic version of σj in the sense that at eachhistory, σ̃j chooses the same action as σj with probability β̃, but chooses the other action
by error with probability 1 − β̃. For every ht−1 ,((β̃) ◦ C + (1 − β̃) ◦ D if σjt (ht−1 ) = C,t t−1
σ̃j (h ) =(β̃) ◦ D + (1 − β̃) ◦ C if σjt (ht−1 ) = D.
Note p̃ and β̃ jointly pin down beliefs over stage actions given each history. For illustration,
suppose again that the set Z of candidate strategies consists only of AD and AC so thatg and ACf for β̃ = 0.9. It then follows that theZ̃ β̃ consists of their randomized versions AD
1
1
round-one belief µi equals p̃AD
g × 0.1 + p̃AC
f × 0.9. If the subject observes aj = C in thefirst round, by Bayes’ rule, his belief in round two will increase to




p̃AD
p̃AC
g × 0.1
f × 0.9
0.1 +
0.9.
p̃AD
p̃AD
g × 0.1 + p̃AC
f × 0.9g × 0.1 + p̃AC
f × 0.9
The third parameter ν represents potential errors in the reporting of beliefs. Formally, if
a subject’s belief in any round t (implied by p̃ and β̃) is µti , we assume his reported belief
is distributed according to the logistic distribution with mean µti and variance ν truncatedto the unit interval. For each type, we identify the values of p̃, β̃, and ν that maximize thelikelihood of the sequence of elicited beliefs in all rounds of late supergames. A summary ofthese estimation results are reported in Tables 4 and 5, with the complete results provided
in the Online Appendix. Note some types are not observed frequently enough to allow
for estimation, which is the case whenever only 1% of subjects are of a certain type. In
addition, there is sometimes insufficient variation to separate the beliefs with respect tosome of the strategies. In those cases, we set the least popular strategies (according toSFEM) to zero and “assign” the belief to the more popular strategy. This applies to only
population (over supergames) fraction of that strategy. Second, imagine a data set where a large fraction ofsubjects play TFT, and a small fraction plays Grim. However, for some of the subjects playing Grim, thenumber of observations that distinguishes Grim from TFT is very small. When computing the posterior at
the subject level, the few observations of difference for a given subject may not be enough to generate thehighest posterior on Grim given the strong prior in favor of TFT.
46
The variables with tilde are estimates about beliefs and distinguished from the corresponding SFEM
estimates of strategies.

26

three of the 84 estimates reported in Tables 4 and 5. The rows are sorted by frequency ofthe strategy, and the columns are sorted by average belief (i.e., the first strategy for which
we report beliefs is the one that subjects put the most weight on, on average).
Table 4: Beliefs over Strategies in the Finite Game

Share

Estimated Beliefs - p̃

Type

SFEM

Typing

T7

T8

Grim

TFT

AD

TF2T

Grim2

Other

ν

β̃

T7
T8
AD
TFT
T6
Grim
Other

0.30
0.22
0.12
0.09
0.08
0.08
0.11

0.35
0.20
0.12
0.12
0.08
0.02
0.11

0.43
0.00
0.75
0.00
0.99
0.00
0.01

0.39
0.50
[0.00]
0.33
0.00
0.22
0.16

0.18
0.04
[0.00]
0.00[0.00]
0.17
0.35

0.00
0.01
0.00
0.53
0.00
0.16
0.30

0.00
0.09
0.07
0.11
0.00
0.34
0.01

0.00
0.15
0.00
0.00
0.00
0.01
0.11

0.00
0.21
0.00
0.00
0.00
0.00
0.00

0.00
0.00
0.18
0.03
0.00
0.10
0.06

0.03
0.01
0.03
0.01
0.01
0.02

1.00
1.00
1.00
1.00
1.00
1.00

0.30

0.29

0.11

0.09

0.07

0.05

0.05

0.04

All

Estimation on late supergames out of 10 strategies: AD, AC, Grim, TFT, STFT, T8-T6, Grim2, and TF2T.
Rows, top 6 played strategies. Columns, top 7 believed strategies.
Estimates in [square brackets] are not estimated due to collinearity.
SFEM estimate for β is 0.94. Complete results in Table 15.

Table 5: Beliefs over Strategies in the Indefinite Game

Share

Estimated Beliefs - p̃

Type

SFEM

Typing

Grim

TFT

TF2T

AC

AD

Grim2

STFT

Other

ν

β̃

TFT
Grim
Grim2
AC
TF2T
AD
Other

0.36
0.18
0.11
0.11
0.10
0.09
0.05

0.59
0.09
0.11
0.05
0.01
0.10
0.05

0.28
0.80
0.22
0.00
0.33
0.00
0.35

0.25
0.13
0.00
0.20
0.00
0.00
0.00

0.19
0.02
0.23
0.00
0.40
0.00
0.00

0.00
0.00
0.23
0.80
0.27
0.00
0.00

0.08
0.00
0.00
0.00
0.00
1.00
0.48

0.14
0.05
0.31
0.00
0.00
0.00
0.16

0.05
0.00
0.00
0.00
0.00
0.00
0.00

0.00
0.00
0.00
0.00
0.00
0.00
0.00

0.02
0.02
0.02
0.03
0.01
0.05

1.00
1.00
1.00
1.00
1.00
1.00

0.32

0.14

0.14

0.14

0.14

0.10

0.02

0.00

All

Estimation on late supergames out of 10 strategies: AD, AC, Grim, TFT, STFT, T8-T6, Grim2, andTF2T.
Rows, top 6 played strategies. Columns, top 7 believed strategies.
Estimates in [square brackets] are not estimated due to collinearity.
SFEM estimate for β is 0.94. Complete results in Table 15.

Tables 4 and 5 reveal important differences in beliefs between the Finite and the Indefinite games. The bottom row of each table presents (weighted) average beliefs over27

strategies. In the Finite game, subjects believe others are most likely to use thresholdstrategies (T7 and T8 account for 59%), whereas in the Indefinite game, they believe
others are most likely to play conditionally cooperative strategies (Grim and TFT havetogether 46%). That is, at least in this respect, we observe subjects’ beliefs to be in linewith actual behavior in both games: subjects correctly anticipate the most popular classof strategies to be different between the games (threshold vs. conditionally cooperative).
Furthermore, looking at the first two rows of each table, and focussing on the two mostcommon strategies, we see evidence of substantial heterogeneity in beliefs between types(in the same game). For instance, T8 types in the Finite game put 0 weight on T7, whereas
the T7 types believe 43% of others play T7. In the Indefinite game, TFT types believe only
28% of subjects play Grim, whereas Grim types expect 80% to be Grim players. Theseestimates are from late supergames; hence, our results indicate heterogeneity in beliefs
across types can be persistent.
Result 6 Beliefs are different between the Finite and the Indefinite games: subjects correctly anticipate the most popular class of strategies to be different between the games(threshold vs. conditionally cooperative).
FiniteT7
T8

TFTGrim
T6

T7
T8
AD
T6 Grim
TF2T
AC
STFT

TFT

-.9

-.6

-.3
0
.3
.6
Belief Overestimate

.9

T7
T8
AD
TFTGrimT6

0

TF2T
STFT

0

TF2T
AC
STFT

0

SFEM Share
.1
.2
.3

SFEM Share
.1
.2
.3

SFEM Share
.1
.2
.3

.4

More Cooperative.4

Own Strategy.4Less Cooperative

-.9

-.6

-.3
0
.3
.6
Belief Overestimate

.9

-.9

-.6

-.3
0
.3
.6
Belief Overestimate.9

Indefinite.4

More Cooperative.4

Own Strategy.4Less Cooperative

Grim
AC Grim2
TF2T

TFT

Grim
Grim2
TF2T

AD

-.6

-.3
0
.3
.6
Belief Overestimate

Grim2
TF2T

.9

0

STFT

0

-.9

Grim
AD

STFT

0

STFT

AC

SFEM Share
.1
.2
.3

TFT

SFEM Share
.1
.2
.3

SFEM Share
.1
.2
.3

TFT

-.9

-.6

-.3
0
.3
.6
Belief Overestimate

.9

-.9

-.6

-.3
0
.3
.6
Belief Overestimate.9

Late supergames. Vertical lines indicate the average weighted by SFEM shares.
Belief Overestimate represents the difference between beliefs and estimated frequency of strategies.

Figure 10: Belief vs. Estimated Frequencies
Tables 4 and 5 report detailed information on how beliefs differ by type in each game.
28

However, the richness of these data make identifying general patterns on how beliefs overstrategies are connected to subjects’ strategy choice difficult. To simplify our analysis, weorder strategies in terms of cooperativeness and use this order to study the degree to which
subjects believe others are less, as, or more cooperative than they are. Formally, we definea strategy to be more cooperative than another one if, as the probability of implementation
errors goes to zero (i.e. as β → 1), the expected payoff associated with playing the former
strategy against itself is higher than the expected payoff of playing the later strategy
against itself.47 This generates the following order of cooperativeness (from least to most):
AD, STFT, T6, T7, T8, Grim, TFT, Grim2, TF2T, and AC. Note that for the strategies
considered, the cooperativeness order is strict; as such, subjects’ beliefs about others being
as cooperative also correspond to their beliefs about others using the same strategy as them.
Figure 10 makes use of this categorization to study the accuracy of beliefs about others
being less, as, or more cooperative. Each of these cases is presented in a separate panel. In
each graph, the x-coordinate for each type represents the difference between beliefs and theestimated frequency of strategies. Consider, for example, T8 in the lop left panel. First, we
compute T8’s subjective belief about the likelihood that others are using a less cooperative
strategy than T8. This involves summing over T8’s beliefs about others using AD, STFT,
T6 or T7, which are the only strategies less cooperative than T8. From this we subtract
the objective likelihood (obtained from the SFEM) of others using AD, STFT, T6 or T7.
For T8, we see that this difference is negative. This means that T8 types underestimate thelikelihood that others are less cooperative than they are. On these graphs, the y-coordinate
for each type represents the estimated frequency of the type, allowing us to evaluate theprevalence of such deviations. The figure suggests a tendency shared in both games: (1)Subjects underestimate (relative to actual frequencies) the likelihood that others use lesscooperative strategies than their own. (2) Subjects overestimate the likelihood that others
use the same strategy as their own. The most pronounced between these distortions is thefirst for the Finite game and the second for the Indefinite game. With respect to beliefs onothers being more cooperative, we observe opposite patterns in the Finite and the Indefinite
games. In the Finite game, subjects overestimate the likelihood of such an event; in the
Indefinite game, subjects underestimate it. However, in both cases, this is the smallest of
the three distortions. Note also that the most popular type in the Finite game (T7—aswell as the second most popular T8) displays the tendency observed, on average, whereas
for the Indefinite game, TFT (the most popular type) is well calibrated.
The accuracy of beliefs over strategies can be studied more directly without relying
on the cooperativeness order. In Appendix B, we compute, for each type, the Euclidean
distance between beliefs and the estimated frequency of strategies. To study whether beliefs
47
Analytical derivation of the cooperativeness order for an infinitely repeated PD with a general specification of the payoffs and discount factor is presented in Appendix B. It also describes ways to numericallyverify that this order is preserved in the Finite game. On the subset of strategies considered by Proto et al.
(2020), our cooperativeness order coincides with the inverse of their harshness ranking.

29

become more accurate with experience, we also look at how this distance changes from early
to late supergames.48 We find that, in aggregate, beliefs are becoming more accurate withexperience in the Finite game, whereas accuracy changes little in the Indefinite game. In
both cases, the most popular strategy types (T7 in Finite and TFT in Indefinite) have themost accurate beliefs in late supergames.49
Result 7 Substantial heterogeneity exists in beliefs within each game: subjects using
different strategies hold different beliefs. The results also suggest subjects tend to overestimate the likelihood that others use the same strategy as their own, while underestimating
the likelihood that others use less cooperative strategies.
The observation that subjects using different strategies hold different beliefs raises thequestion of how they are connected. To shed light on this connection, we explore the extent
to which subjects are subjectively rational. That is, we study how close their strategy choice
is to best responding to their beliefs. Our analysis poses no restrictions on the link between
the strategies and beliefs: the strategy estimation is based on the subjects’ actions and isdone separately from the belief estimation, which is based on their round belief reports.
For the purposes of our discussion in this section, we call a type subjectively rational(given preferences induced in the experiment) if her strategy choice is a best response toher supergame beliefs among those strategies Z in the consideration set.50
The results, presented in Figures 11 and 12, suggest most subjects’ strategy choices areeither exact or approximate best responses given their supergame beliefs.51 In the Finite
game, T7 and T6 types (38% of the population) exactly best respond to their supergame
beliefs, and T8, TFT, and Grim types (39% of the population) approximately best respondto their supergame beliefs by obtaining 90%, 86%, and 89% of their best-response payoff,
respectively. Of the most common six types, the only type whose strategy is far from a
best response is AD (12%). In fact, their strategy choice is close to being the worst given
the stated beliefs.52
48
Additionally, in the same Appendix, we document in detail how the distribution of strategies, types,
and beliefs for each type change from early to late supergames.
49
In the Finite game, early beliefs overestimate the likelihood of T8 and underestimate the likelihood of
T7. Both of these errors are reduced (or eliminated) with experience. For the Indefinite game, early beliefs
overestimate the likelihood of Grim and underestimate the likelihood of TFT; however, these errors (which
are less costly than those observed in the Finite game) are not corrected with experience.
50
For consistency, the best-response analysis incorporates beliefs over implementation noise in how others
carry out their intended strategy (captured by 1 − β̃). However, because estimated values for β̃ are very
close to 1, incorporating β̃ does not affect the results. To calculate the expected payoff of each strategy, we
simulate play in 1,000 supergames given β̃.
51
Table 18 in the Online Appendix provides detailed best-response analysis for each of the six common
types in both the Finite and Indefinite games.
52
Note subjects playing AD receive weakly higher payoffs in any supergame than their opponent, andthese subjects have little chance to observe what would happen along alternative histories. This may
contribute to why ‘they fail to optimize given their beliefs.

30

Finite

TF2T
TF2T

T6

GRIM2
GRIM2

T7

T8

TFT

STFT

AC

GRIM

T6

T7

T8

TFT

STFT

GRIM

AC

AD

Expected payoff per round
0
.5
1
TF2T

GRIM2

T6

T7

T8

TFT

STFT

GRIM

AC

AD

TF2T

GRIM2

T6

T7

T8

TFT

STFT

GRIM

AC

AD

AD

TF2T

T6

T7

T8

TFT

STFT

AC

GRIM2

Typed as Grim

Expected payoff per round0.5
1

Typed as T6

Expected payoff per round0.5
1

Typed as TFT

GRIM

AD

TF2T

T6

GRIM2

T7

T8

TFT

STFT

AC

GRIM

AD

Expected payoff per round0.5
1

Typed as AD

Expected payoff per round0.5
1

Typed as T8

Expected payoff per round0.5
1

Typed as T7

The strategy corresponding to the type is higlighted in dark grey.
Analysis uses normalized stage-game payoffs.

Figure 11: Best Response for Top 6 Types in the Finite Game

In the Indefinite game, a similar pattern emerges. Most common types (TFT, Grim,
Grim2, TF2T, and AD—84% of subjects) almost exactly best respond to their beliefs.53
One “major” type far from best responding to their belief is AC (11%), who selects theworst strategy given their beliefs. Indeed, given their beliefs, the best-response strategy is
AD. For these subjects, however, some form of other-regarding preferences could reconcile
strategy choices and beliefs.54 Hence, overall, the majority of subjects appear subjectively
rational or close to subjectively rational.
Result 8 Most types are close to best responding to their beliefs: they are subjectively
rational.
53

For TFT, the strict best response is TF2T or Grim2, but TFT achieves 99% of the best-response payoff.
The other type for which strategy choice is far from best response is STFT (4%). Given beliefs, thebest response is TFT.
54

31

Indefinite

TF2T
TF2T

T6

GRIM2
GRIM2

T7

T8

TFT

STFT

AC

GRIM

T6

T7

T8

TFT

STFT

GRIM

AC

AD

Expected payoff per round
0 .5 1
TF2T

GRIM2

T6

T7

T8

TFT

STFT

GRIM

AC

AD

TF2T

GRIM2

T6

T7

T8

TFT

STFT

GRIM

AC

AD

AD

TF2T

T6

T7

T8

TFT

STFT

AC

GRIM2

Typed as AD

Expected payoff per round0 .2 .4 .6 .8 1

Typed as TF2T

Expected payoff per round0 .2 .4 .6 .8 1

Typed as AC

GRIM

AD

TF2T

T6

GRIM2

T7

T8

TFT

STFT

AC

GRIM

AD

Expected payoff per round0 .2 .4 .6 .8 1

Typed as Grim2

Expected payoff per round0 .2 .4 .6 .8 1

Typed as Grim

Expected payoff per round0 .2 .4 .6 .8 1

Typed as TFT

The strategy corresponding to the type is higlighted in dark grey.
Analysis uses normalized stage-game payoffs.

Figure 12: Best Response for Top 6 Types in the Indefinite Game

Note the best-response analysis reported so far is subjective in the sense that it is basedon the expected payoffs given the subjective beliefs of each type. To provide a contrast,
we replicate the best-response analysis using objective expected payoffs computed from thestrategy distribution estimated at the population level by SFEM. This analysis reveals T6
is the best response to the population in the Finite game, and Grim2 is the best responseto the population in the Indefinite game. In the Finite game, the most frequent T7 typeachieves 97% of the best-response payoff from T6. In the Indefinite game, we see the mostfrequent TFT type achieves 94% of the best-response payoff from Grim2. However, somestrategy-types are further away from best responding to the population. For example, theAD type in the Finite game only achieves 64% of the best-response payoff.

32

5Conclusion

Beliefs play a central role in equilibrium theory, and increasing evidence suggests they arealso key to understanding behavior observed in repeated settings. This study elicits beliefs
in finitely and indefinitely repeated PD games with the main goal of providing a novel
data set to inform our views on how beliefs, actions, and strategy choices are linked in thisimportant class of games.
Our first key finding is that beliefs are, in aggregate, remarkably accurate. In both the
Finite and Indefinite games, beliefs averaged over all rounds are less than three percentage
points away from the empirical action frequencies. To the extent that average beliefs do
not correspond to the empirical action frequencies, the key deviations are over-optimism inlate rounds of the Finite game and over-pessimism in early rounds of the Indefinite game.
Beliefs also adjust appropriately to the history of play even when these adjustments are
not small: in some histories, they move by almost 60 percentage points between rounds
one and two. On the individual level, although beliefs are heterogeneous with a varying
degree of accuracy, a vast majority of subjects correctly anticipate the likelihood of theiropponent cooperating.
Importantly, results show beliefs over stage actions are forward looking. Most notably,
beliefs along the history of mutual cooperation evolve very differently in the Finite andthe Indefinite games. Persistence of cooperation in the Indefinite game and its collapse in
the Finite game are again correctly anticipated along such histories. In general, beliefs aredifferent across the two games even when they lead to the same action.
Our second category of findings is based on the development of a novel method to recover beliefs over supergame strategies from beliefs over stage actions in each round. First,
we find subjects in the Finite and the Indefinite games correctly anticipate the different
class of strategies used in these games: threshold strategies in the former and conditionallycooperative strategies in the later. Second, we study the connection between the choice ofsupergame strategies and beliefs over them. We observe that, in both the Finite and theIndefinite games, subjects playing different strategies have strikingly heterogeneous beliefs
over the strategy choice of the other player. In both games, the results also suggest atendency by subjects to overestimate the likelihood that others use the same strategy as
they do, while underestimating the likelihood that others use less cooperative strategies.
Third, when we study the link between strategies and beliefs, we find most types are close
to being subjectively rational; that is, we observe the expected payoff associated with theirstrategy choice to be close to their best-response payoff given their beliefs. This, in particular, suggests the observed heterogeneity in strategy choice is related to the heterogeneity
in beliefs.
These results also provide insights into the forces that underlie some of the key behavioral patterns observed in these games. Our results illustrate how small but systematic
33

departures from accurate beliefs (at key points in the supergame) can sustain long-run
cooperation in finitely repeated PD. Although beliefs are fairly accurate, for 80% of subjects, best responding to their beliefs involves cooperating more than it is optimal (given
the observed strategy distribution in the population). In the indefinitely repeated PD, ourresults highlight the difficulty of resolving equilibrium selection. Different subjects holdpersistently different beliefs about others, and in environments conducive to cooperation,such as ours, they experience few histories where those beliefs are revealed to be incorrect.
As a consequence, a variety of conditionally cooperative strategies remain popular despite
many repetitions.
In summary, our results on beliefs suggest subjects understand the difference between
finitely and indefinitely repeated environments even when their observed behavior in termsof actions is identical. In other words, subjects have a refined awareness of the rules of
the game and the implications of these rules for the dynamics of cooperative behavior.
They also suggest the calculus underpinning choices are very different across finitely andindefinitely repeated environments.

References
Agranov, M., Cotton, C. & Tergiman, C. (2016), ‘Persistence of power: Repeated multilateral bargaining’, Journal of Public Economics . forthcoming.
Allen, F. (1987), ‘Discovering personal probabilities when utility functions are unknown’,Management Science 33(4), 542–544.
Aoyagi, M., Bhaskar, V. & Fréchette, G. R. (2019), ‘The impact of monitoring in infinitely
repeated games: Perfect, public, and private’, American Economic Journal: Microeconomics 11(1), 1–43.
Chaudhuri, A., Paichayontvijit, T. & Smith, A. (2017), ‘Belief heterogeneity and contributions decay among conditional cooperators in public goods games’, Journal of Economic
Psychology 58, 15 – 30.
Cheung, Y.-W. & Friedman, D. (1997), ‘Individual learning in normal form games: Somelaboratory results’, Games and Economic Behavior 19(1), 46 – 76.
Costa-Gomes, M. A., Huck, S. & Weizsäcker, G. (2014), ‘Beliefs and actions in the trust
game: Creating instrumental variables to estimate the causal effect’, Games and Economic Behavior 88, 298 – 309.
Costa-Gomes, M. A. & Weizsäcker, G. (2008), ‘Stated Beliefs and Play in Normal-Form
Games’, The Review of Economic Studies 75(3), 729–762.

34

Crosetto, P. & Filippin, A. (2013), ‘The “bomb” risk elicitation task’, Journal of Risk andUncertainty 47(1), 31–65.
Dal Bó, P. (2005), ‘Cooperation under the shadow of the future: experimental evidence
from infinitely repeated games’, American Economics Review 95(5), 1591–1604.
Dal Bó, P. & Fréchette, G. (2011), ‘The evolution of cooperation in infinitely repeatedgames: Experimental evidence’, American Economic Review 101(1), 411–429.
Dal Bó, P. & Fréchette, G. R. (2018), ‘On the determinants of cooperation in infinitely
repeated games: A survey’, Journal of Economic Literature 56(1), 60–114.
Dal Bó, P. & Fréchette, G. R. (2019), ‘Strategy choice in the infinitely repeated prisoners
dilemma’, American Economic Review . Forthcoming.
Danz, D. N., Fehr, D. & Kübler, D. (2012), ‘Information and beliefs in a repeated normalform game’, Experimental Economics 15(4), 622–640.
Danz, D., Vesterlund, L. & Wilson, A. J. (2020), Belief elicitation: Limiting truth tellingwith information on incentives, Technical report, National Bureau of Economic Research.
Embrey, M., Fréchette, G. R. & Yuksel, S. (2018), ‘Cooperation in the finitely repeatedprisoner’s dilemma’, The Quarterly Journal of Economics 133(1), 509–551.
Fischbacher, U. (2007), ‘z-Tree: Zurich toolbox for readymade economic experiments’,
Experimental Economics 10(2), 171–178.
Fischbacher, U. & Gächter, S. (2010), ‘Social preferences, beliefs, and the dynamics of freeriding in public goods experiments’, American Economic Review 100(1), 541–56.
Flood, M. M. (1952), ‘Some experimental games’, Research Memorandum RM-789, The
Rand Corporation, Santa Monica .
Fréchette, G. R. (2012), ‘Session-effects in the laboratory’, Experimental Economics
15(3), 485–498.
Fréchette, G. R. & Yuksel, S. (2017), ‘Infinitely repeated games in the laboratory: Four perspectives on discounting and random termination’, Experimental Economics 20(2), 279–
308.
Fudenberg, D., Rand, D. G. & Dreber, A. (2012), ‘Slow to anger and fast to forget: Leniency
and forgiveness in an uncertain world’, American Economic Review 102(2), 720–749.
Gächter, S. & Renner, E. (2010), ‘The effects of (incentivized) belief elicitation in public
goods experiments’, Experimental Economics 13(3), 364–377.

35Gill, D. & Rosokha, Y. (2020), Beliefs, learning, and personality in the indefinitely repeatedprisoner’s dilemma. Working Paper.
Greiner, B. (2015), ‘Subject pool recruitment procedures: organizing experiments with
orsee’, Journal of the Economic Science Association 1(1), 114–125.
Hossain, T. & Okui, R. (2013), ‘The binarized scoring rule’, Review of Economic Studies
80(3), 984–1001.
Hyndman, K., Özbay, E. Y., Schotter, A. & Ehrblatt, W. (2012a), ‘Belief formation: an
experiment with outside observers’, Experimental Economics 15(1), 176–203.
Hyndman, K., Ozbay, E. Y., Schotter, A. & Ehrblatt, W. Z. (2012b), ‘Convergence: An
experimental study of teaching and learning in repeated games’, Journal of the European
Economic Association 10(3), 573–604.
Hyndman, K., Terracol, A. & Vaksmann, J. (2010), ‘Strategic interactions and belief formation: an experiment’, Applied Economics Letters 17(17), 1681–1685.
Kalai, E. & Lehrer, E. (1993), ‘Rational learning leads to nash equilibrium’, Econometrica
61(5), 1019–1045.
Kocher, M. G., Martinsson, P., Matzat, D. & Wollbrant, C. (2015), ‘The role of beliefs,
trust, and risk in contributions to a public good’, Journal of Economic Psychology 51, 236– 244.
Murnighan, J. K. & Roth, A. E. (1983), ‘Expecting continued play in prisoner’s dilemma
games’, Journal of Conflict Resolution 27(2), 279–300.
Neugebauer, T., Perote, J., Schmidt, U. & Loos, M. (2009), ‘Selfish-biased conditional
cooperation: On the decline of contributions in repeated public goods experiments’,
Journal of Economic Psychology 30(1), 52 – 60.
Nyarko, Y. & Schotter, A. (2002), ‘An experimental study of belief learning using elicited
beliefs’, Econometrica 70(3), 971–1005.
Proto, E., Rustichini, A. & Sofianos, A. (2019), ‘Intelligence personality and gains fromcooperation in repeated interactions’, Journal of Political Economy 127(3), 1351–1390.
Proto, E., Rustichini, A. & Sofianos, A. (2020), Intelligence, errors and strategic choices in
the repeated prisoners’ dilemma. Working Paper.
Rey-Biel, P. (2009), ‘Equilibrium play and best response to (stated) beliefs in normal form
games’, Games and Economic Behavior 65(2), 572 – 585.

36

Romero, J. & Rosokha, Y. (2019), Mixed strategies in the indefinitely repeated prisoner’s
dilemma. Working Paper.
Smith, A. (2013), ‘Estimating the causal effect of beliefs on contributions in repeated public
good games’, Experimental Economics 16(3), 414–425.
Smith, A. (2015), ‘Modeling the dynamics of contributions and beliefs in repeated public
good games’, Economics Bulletin 35, 1501–1509.
Vespa, E. & Wilson, A. (2019), ‘Experimenting with the transition rule in dynamic games’,
Quantitative Economics 10, 1825–1849.
Weber, M., Duffy, J. & Schram, A. (2018), ‘An experimental study of bond market pricing’,
The Journal of Finance 73(4), 1857–1892.
Wilson, A. J. & Vespa, E. (2018), Paired-uniform scoring: Implementing a binarized scoring
rule with non-mathematical language. Working Paper.

37

Online Appendix forBeliefs in Repeated Games
Masaki Aoyagi

Guillaume R. Fréchette

CONTENTS:
A. Related Literature
B. Additional Figures and Analysis
C. Instructions

Sevgi Yuksel

A

Related Literature

Nyarko & Schotter (2002) are among the first to study elicited beliefs in repeated games.
Studying a finite repetition of a 2×2 game with a unique mixed Nash equilibrium (NE)played in fixed and random pairing, Nyarko & Schotter (2002) find the subjects’ beliefs overactions are not empirical in the sense that they cannot be approximated by the weighted
average of the opponent’s past actions.55 Hyndman et al. (2010) study beliefs when astage game with a unique mixed NE is repeated 20 times, and find subjects’ beliefs about
the other’s action in the present round do take into account the effect of their own action
choice in the preceding rounds, and hence cannot be expressed by the weighted average ofthe other player’s actions in the past. Hyndman et al. (2012b) advance this observationin an experiment in which subjects play a finite repetition of 3×3 and 4×4 normal form
games with and without dominance-solvable NE. Hyndman et al. (2012b) note some players
attempt to influence the beliefs of other players through their own actions, and thus help
the process converge to an NE.56
The experimental literature on beliefs examines the question of whether actions are best
responses to beliefs with no definite answers. Nyarko & Schotter (2002) find the actions
in each round mostly best respond to the stated beliefs, but also find fictitious-play beliefs
better predict the opponents’ action than the stated beliefs. Costa-Gomes & Weizsäcker(2008) use 14 3×3 games to investigate the relationship between subjects’ elicited beliefs
and their strategy choice. Regardless of whether belief elicitation precedes strategy choice,
Costa-Gomes & Weizsäcker (2008) find the strategies are not best responses to the beliefs
in a half of the games, and attribute this finding to the difference in the perception of
the game in the two situations. Danz et al. (2012) use a dominance-solvable 3×3 gamerepeated 20 times to study beliefs under various combinations of feedback and matchingconditions. Danz et al. (2012) find feedback of past actions helps advance the iterative
elimination process both in terms of actions and beliefs. Using a series of 3×3 games eachwith a unique NE, Rey-Biel (2009) find more than two-thirds of subjects choose actions
that best respond to their elicited beliefs.
The literature on voluntary-contribution games often finds conditional cooperation,
which refers to the fact that subjects make higher contributions if they believe other members of their group make higher contributions. This relationship is observed, for example
by Gächter & Renner (2010), Fischbacher & Gächter (2010) and Kocher et al. (2015).57
Neugebauer et al. (2009) confirm this relationship in their experiment on a finitely repeated55

Nyarko & Schotter (2002) specifically consider a generalization of fictitious play called the γ-empirical
average as proposed by Cheung & Friedman (1997).
56
Hyndman et al. (2012a) have outside observers predict the actions of the subjects in Hyndman et al.
(2012b), and find a large variance in their beliefs both in terms of accuracy and updating.
57
Costa-Gomes et al. (2014) analyze the relationship in the trust game. Smith (2013, 2015) note thebeliefs are endogenous, and hence that the effect on contribution, if interpreted as causal, is overestimated.

1

voluntary contribution game, and further observe that both contribution levels and beliefs
about others’ contribution levels decline toward the end. Chaudhuri et al. (2017) observe
similar joint dynamics of contribution and beliefs, allowing for heterogeneity across subjects
and classifying them into types according to their initial beliefs about others’ contributions.
On cooperation and strategies in finitely and infinitely repeated PD, Dal Bó & Fréchette(2018) and Embrey et al. (2018) find some key patterns by re-analyzing data from a collection of laboratory experiments.58 First, in finitely repeated PD, the fraction of thresholdstrategies increases with experience.59 By the end, threshold strategies always account forthe majority of the data, and use of the threshold strategies with lower thresholds increases
with experience. This contributes to a (sometimes very) slow aggregate movement towardearlier defection.60 In the finitely repeated PD, if the parameters are conducive to cooperation, round-one cooperation increases with experience, whereas last-round cooperation
decreases with it.61 Otherwise, cooperation remains low in all rounds. In indefinitely repeated PD, on the other hand, experience leads cooperation (in the first or last round)to almost any level, depending on how conducive the parameters are to cooperation. Experience also amplifies the magnitude of the effects of the parameters, although it does
not change the direction of those effects. In most experiments with perfect monitoring, a
few simple strategies account for more than 50% of the strategies used. They are “always
defect” (AD), “always cooperate” (AC), “grim trigger” (Grim), “tit-for-tat” (TFT), and“suspicious-tit-for-tat” (STFT).62 AD, Grim, and TFT are generally the most popular,
and Grim becomes more popular with experience and appears to be a counterpart to thethreshold strategies in finite games. The implementation error term in Grim also decreases
with experience.63 Experience also increases responsiveness, which is measured as the difference between the probability of cooperative action after cooperation by the other player
and that after defection by the other player. This is documented in Aoyagi et al. (2019)and confirmed by Dal Bó & Fréchette (2018) in their analysis of the meta-data and new experiments: according to a simple regression, experience has a significant positive impact on
responsiveness in 11 paper-treatments, whereas it is insignificant in 20 paper-treatments.64
58

Experimental research on the subject goes as far back as Flood (1952).
A threshold strategy (with threshold k ≥ 2) starts with C and plays like grim-trigger before round k,
but reverts to the unconditional play of D from round k on.
60
Embrey et al. (2018) find that in the treatment most conducive to cooperation (replicated by the finite
treatment of this study), the modal round at which cooperation breaks down moves earlier approximately
by one round every 10 supergames.
61
A longer horizon T , a higher discount factor δ, a lower temptation payoff 1 + g, or a higher sucker
payoff −` all induce more cooperation.
62
Grim cooperates until a defection is observed, at which point it defects forever; TFT starts by cooperating and thereafter matches what the other did in the previous round; STFT starts by defecting andthereafter matches what the other did in the previous round.
63
See Dal Bó & Fréchette (2019), Tables 8 and A10.
64
This analysis eliminates all data in within-subjects designs after a change in treatment and only preserves the initial treatment. Most of the insignificant cases have a small number of observations. One
59

2

treatment sees a negatively significant impact, perhaps because of relatively low round-one cooperation at
0.36.

3

B

Additional Figures and Analysis

B.1

The Indefinite Game: Round Eight, Last Game Round, Last Observation Round.8.6

Cooperation Rate
.4
.6

.8

1

Beliefs

1

Actions

Beliefs

Last Game Round
Last Observation Round.4

Round 8

Early

Late

5 6 7
Supergames

L-2 L-1 Last

0

0

.2

Late

.2

Early

1

2

3

4

5 6 7
Supergames

L-2 L-1 Last

1

2

3

4

Figure 13: The Indefinite Game

4

B.2

Effect of Belief Elicitation on Actions

Table 6 shows no statistically significant differences in the probability of cooperation inround one x̄1 for supergames where beliefs are elicited. The other regressors are variables
that have been considered in similar analysis.
Table 6: Correlated Random Effects Probit
Determinants of Cooperation in Round One

FiniteIndefinite

Beliefs Are Elicited

0.0654(0.294)0.188(0.280)

Other Cooperated in Previous Supergame

0.250∗∗∗(0.0661)0.624∗∗∗(0.181)Supergame

0.0131(0.0431)0.187∗∗∗(0.0532)

Length of Previous Supergame-0.00119(0.00807)

Cooperated in Supergame 1

2.571∗∗∗(0.754)2.830∗∗∗(0.649)Risk Measure

0.0189∗∗∗(0.00691)

0.00534(0.00663)

Constant-0.509(0.334)-1.461∗∗∗(0.567)

Observations

1778

1126

Standard errors clustered (at the session level) in parentheses. ∗∗∗ 1%, ∗∗ 5%, ∗ 10% significance.
All variables refer to behavior in Round 1.
Risk Measure is equal to the number of boxes collected in the bomb task.

5

Table 7: Correlated Random Effects Probit (Marginal Effects)
Determinants of Cooperation in Round One

FiniteIndefinite

Beliefs Are Elicited

0.00660(0.0295)

0.0193(0.0288)

Other Cooperated in Previous Supergame

0.0252∗∗∗(0.00559)

0.0642∗∗∗(0.0188)

Supergame

0.00132(0.00439)

0.0193∗∗∗(0.00497)

Length of Previous Supergame-0.000122(0.000827)

Cooperated in Supergame 1

0.259∗∗∗(0.0790)0.291∗∗∗(0.0482)

Risk Measure0.00191∗∗∗(0.000615)

0.000549(0.000676)

Observations

1778

1126

Standard errors clustered (at the session level) in parentheses. ∗∗∗ 1%, ∗∗ 5%, ∗ 10% significance.
All variables refer to behavior in Round 1.
Risk Measure is equal to the number of boxes collected in the bomb task.

6

B.3

Beliefs–Comparative Statics.8.6

Round 1

Beliefs.8

Other Coop. at t-1
Other Defect at t-1
Round 8.4

.4

Beliefs.6

1

Indefinite

1

Finite

Early

Late

5 6 7
Supergames

L-2 L-1 Last

0

0

.2

Late

.2

Early

1

2

3

4

5 6 7
Supergames

L-2 L-1 Last

1

2

3

4

Figure 14: Beliefs Over Supergames

The evolution of beliefs depicted in Figure 14 mirrors the patterns observed for cooperation in Figure 1. µ̄1 are high in both games. Beliefs are responsive in both games:µ̄ti (∗, at−1
= C, ∗)− µ̄ti (∗, ajt−1 = D, ∗) > 0. Beliefs µ̄T in the last round are low in the Finite
jgame, but are high in the Indefinite game.

7

B.4

Actions and Beliefs in Rounds One and Seven.8.8

Cooperation Rate
.4
.6

Cooperation Rate
.4
.6

1

Indefinite

1

Finite

Early

.2

Late

.2

Early

Late

Coop-Round 1
Belief-Round 1

Belief-Round 700

Coop-Round 7

1

2

3

4

5 6 7
Supergames

L-2 L-1 Last

1

2

3

4

5 6 7
Supergames

Figure 15: Average Cooperation and Belief

8

L-2 L-1 Last

How Beliefs Relate to Experiences

Supergame 5

Round twoDensity0 1 2 3Round One
Density0 1 2 3Round twoDensity0 1 2 3

Density0 1 2 3Round One

Last Supergame

Round ThreeRound FourRound ThreeRound Four
Density0 1 2 3

-1 -.8-.6-.4-.2 0 .2 .4 .6 .8 1
Belief - Experienced Frequency

Density0 1 2 3

-1 -.8-.6-.4-.2 0 .2 .4 .6 .8 1
Belief - Experienced Frequency

Density0 1 2 3

-1 -.8-.6-.4-.2 0 .2 .4 .6 .8 1
Belief - Experienced Frequency

Density0 1 2 3

-1 -.8-.6-.4-.2 0 .2 .4 .6 .8 1
Belief - Experienced Frequency

Round Five

Round SixRound Five

Round Six
Density0 1 2 3

-1 -.8-.6-.4-.2 0 .2 .4 .6 .8 1
Belief - Experienced Frequency

Density0 1 2 3

-1 -.8-.6-.4-.2 0 .2 .4 .6 .8 1
Belief - Experienced Frequency

Density0 1 2 3

-1 -.8-.6-.4-.2 0 .2 .4 .6 .8 1
Belief - Experienced Frequency

Density0 1 2 3

-1 -.8-.6-.4-.2 0 .2 .4 .6 .8 1
Belief - Experienced Frequency-1 -.8-.6-.4-.2 0 .2 .4 .6 .8 1
Belief - Experienced Frequency

Round Seven

Round Eight

Round Seven

Round Eight-1 -.8-.6-.4-.2 0 .2 .4 .6 .8 1
Belief - Experienced Frequency

Density0 1 2 3

-1 -.8-.6-.4-.2 0 .2 .4 .6 .8 1
Belief - Experienced Frequency

Density0 1 2 3

-1 -.8-.6-.4-.2 0 .2 .4 .6 .8 1
Belief - Experienced Frequency

Density0 1 2 3

-1 -.8-.6-.4-.2 0 .2 .4 .6 .8 1
Belief - Experienced Frequency-1 -.8-.6-.4-.2 0 .2 .4 .6 .8 1
Belief - Experienced Frequency

Density0 1 2 3B.5

-1 -.8-.6-.4-.2 0 .2 .4 .6 .8 1
Belief - Experienced Frequency-1 -.8-.6-.4-.2 0 .2 .4 .6 .8 1
Belief - Experienced Frequency

Solid = Finite, Dashed = Indefinite.

Figure 16: Difference Between Stated Beliefs and Experienced Frequencies of Cooperation
by Subject

9

Cooperative Path For Early Supergames.8.8

Cooperation Rate
.4
.6

Cooperation Rate
.4
.6

.2

.2

1

Indefinite

1

FiniteActions: Cooperative Path
Beliefs: Cooperative Path

0

0

B.6

2

3

4
5
Round

6

7

8

2

3

4
5
Round

6

Cooperative path: joint cooperation up to that round.
Early supergames.

Figure 17: Cooperative Path (First Eight Rounds)

10

7

8

B.7

Accuracy Across Games and Experience
Table 8: Accuracy (numbers are percentages)
FiniteEarly

Round 1
Actions

Round 1Round 2CCCD
DC
DD
Average

Late

Correct

WithinCorrect

WithinTercile10%

5%

Tercile

10%

5%

69

14

8

73

14

7

87
63
67
67
80

60
16
11
0
44

7
8
4
0
7

91
67
66
67
83

60
16
7
8
45

9
9
7
8
9

Indefinite
Early

Round 1
Actions

Round 1Round 2CCCD
DC
DD
Average

Late

Correct

WithinCorrect

WithinTercile10%

5%

Tercile

10%

5%

65

13

7

67

10

5

86
35
65
11
73

52
24
6
0
40

5
12
6
0
6

91
29
56
79
80

66
10
17
0
52

58
2
12
0
45

Round 1 actions are listed own action first, other’s action second: i.e. (ai , aj ).
Average is weighted by the number of observations.
Note: the number of observations following DD is small, with 2% and 5%, forfinite and indefinite respectively, of observations for late supergames.

11

B.8

Relation Between Beliefs and Actions by Treatment
Table 9: Correlated Random Effects Probit (Marginal Effects)
Dependent Variable: Cooperation
FiniteIndefinite

0.462∗∗∗(0.0176)

0.395∗∗∗(0.0146)

Round-0.0336∗∗∗(0.00339)-0.00238(0.00282)

Coop. in Round 1, Supergames 1-4

0.244∗∗∗(0.0477)0.0805∗∗∗(0.0244)

Coop. in Last Round, Supergames 1-4

0.126∗∗∗(0.0164)0.111∗∗∗(0.0321)

Risk Measure-0.0000121(0.000771)

0.000105(0.000633)

Observations

3792

3628

BeliefStandard errors clustered (at the session level) in parentheses. ∗∗∗ 1%, ∗∗ 5%, ∗ 10% significance.
Late supergames.
Risk Measure is equal to the number of boxes collected in the bomb task.

We use round to make the regressions succinct, but a specification with round indicatorvariables gives similar estimates.

12

Table 10: Correlated Random Effects Probit (Marginal Effects)
Dependent Variable: Cooperation in Round One
FiniteIndefinite

Belief0.0938∗∗∗(0.0272)

0.258∗∗∗(0.0192)

Other Cooperated in Previous Supergame-0.0382(0.0379)

0.0274(0.0340)

Supergame

0.00143(0.00925)

0.00798(0.00572)

Length of Previous Supergame-0.00161(0.00121)

Cooperated in Supergame 1

0.413∗∗∗(0.0810)0.0493∗∗∗(0.0164)

Risk Measure0.00163∗(0.000848)-0.000351(0.000561)

Observations

474

Standard errors clustered (at the session level) in parentheses.

378
∗∗∗

1%,

∗∗

5%, 10% significance.

All variables refer to behavior in Round 1.
Late supergames.
Risk Measure is equal to the number of boxes collected in the bomb task.

13∗

Table 11: Description of Strategies EstimatedName of Strategy

Code

Description

Always Defect
Always Cooperate
Grim
Tit-For-Tat
Suspicious Tit-For-Tat
Threshold 8Threshold 7Threshold 6
Threshold 5Threshold 4Threshold 3
Threshold 2
Lenient Grim 2
Tit-For-2 Tats
2Tits-For-Tat
Lenient Grim 3

AD
AC
GRIM
TFT
STFT
T8
T7
T6
T5
T4
T3
T2
GRIM2
TF2T
2TFT
GRIM3

always play D.
always play C.
play C until either player plays D, then play D forever.
play C unless partner played D last round.
play D in the first round, then TFT.
play Grim until round 8 (last round) then switch to AD.
play Grim until round 7 then switch to AD.
play Grim until round 6 then switch to AD.
play Grim until round 5 then switch to AD.
play Grim until round 4 then switch to AD.
play Grim until round 3 then switch to AD.
play C in round 1 then switch to AD.
play C until 2 consecutive rounds occur in which either player played D, then play D forever.
play C unless partner played D in both of the last rounds.
play C unless partner played D in either of the last 2 rounds.
play C until 3 consecutive rounds occur in which either player played D, then play D forever.

14

B.9

Proof of the Cooperativeness Order

When each strategy is denoted by a finite automaton, we assume that an implementation
error is made in the choice of an action in each state, and not in transition from thecurrent state to the next. We also assume that the errors are independent and identically
distributed between the players and across rounds. Denote by ε ∈ [0, 12 ] the probabilityof such an error.65 For the analytical comparison of cooperative levels, we assume that ε
is small. In some cases considered below, this implies that we treat ε2 as negligible. In
other cases, however, we need to consider the difference in the order of ε2 and treat ε3 as
negligible. Let p = (1 − ε)2 , q = ε(1 − ε) and r = ε2 . The normalized stage payoffs with
implementation errors are given by
gCC = p + q(1 + g − `),gCD = p(−`) + q + r(1 + g),
gCD = p(1 + g) + q + r(−`), gDD = q(1 + g − `) + r,
where g = 1 and ` = 17/12 ≈ 1.416 in our implementation. Define
gCC
gCD g=
gDC  .
gDD
We consider a Markov process induced by a pair of the same strategy implemented witherrors ε. Let Θ be the set of states of this Markov process. For each strategy that can be
expressed as an S-state automaton, Θ can have up to S × S elements. The Markov process
is defined over the set ∆Θ of distributions over those states. Let ω 1 ∈ ∆Θ be the row
vector representing the initial distribution and A = (ast )s,t∈Θ be the transition matrix: ast
is the probability that the next state is t when the current state is s. The distribution ω 2over round 2 states is given by ω 2 = ω 1 A, and the distribution ω t over round t states isgiven by ω t = ω 1 At−1 . With the distribution ω over states, the expected stage payoff to aplayer is given by ωg. In the case of the finite games, the average payoff over eight rounds
can be computed as
8

1 X t
1
ω g = ω 1 I + A1 + · · · + A7 g.
(1)
8
8
t=1

In the case of the indefinite games, the average discounted payoff can be computed as(1 − δ)∞Xω t δ t−1 g = (1 − δ) ω 1 I + δA1 + · · · + δ t At + · · · g

t=1
1= (1 − δ) ω (I − δA)
65

Hence, ε = 1 − β for the parameter β in SFEM.

15

−1

g,

(2)

where δ = 7/8 in our implementation. If we denote by vθ the average discounted payoffin the indefinite games along the Markov process with the initial state θ (i.e., the initial
distribution ω 1 places probability one on state θ), and by v = (vθ )θ∈Θ the corresponding
column vector, then (2) implies the recursive equationv = (1 − δ) (I − δA)−1 g
B.9.1

⇔

v = (1 − δ) g + δAv.

(3)

Indefinite games with small implementation errors

1. TFT and STFT: These strategies have two states 0 and 1. Both strategies play C
in state 0, and D in state 1. Because the implementation errors occur independently
between the two players, state transitions do not synchronize between them. Accordingly, the Markov process has four states Θ = {(0, 0), (0, 1), (1, 0), (1, 1)}. The initial
distribution is ω 1 = (1, 0, 0, 0) if both play TFT and ω 1 = (0, 0, 0, 1) if both play
STFT. We hence have v TFT = v00 and v STFT = v11 . The transition matrix is givenby
p q q r
q r p q A=q p r q  .
r q q pIgnoring the terms of order ε2 , we can write (3) as
 
 



v00
gCC
v00
1 − 2εεε0v01 
 v01 gCD 
 ε
0
1
−
2ε
ε
  = (1 − δ) 
 .


v10 
gDC  + δ  ε
1 − 2ε0ε  v10 0εε
1 − 2εgDD
v11
v11
It follows from the second and third rows of (4) that



 

 
gCD
v00 + v11 − 2v10v01v10
= (1 − δ)+ δε+δv00 + v11 − 2v01
v10
gDC
v01

 

g
v
= (1 − δ) CD + δ 10 + O(ε),
gDC
v01
where O(ε) is the term of order ε. Hence,

 


1 −δ v01
gCD= (1 − δ)+ O(ε).
−δ 1
v10
gDC
Solving this, we get
 



1
v01
1 δ gCD=+ O(ε).
v10
1 + δ δ 1 gDC
16(4)

Substituting this into the first and fourth rows of (4), we obtain
 


 


v00
gCC
v00
v01 + v10
= (1 − δ)+ δ(1 − 2ε)+ δε(1 + δ)
v11
gDD
v11
v01 + v10


 


g
v
g
+ gDC
= (1 − δ) CC + δ(1 − 2ε) 00 + δε CD+ O(ε2 ).
gDD
v11
gCD + gDC
This can be rewritten as 
1 − δ + 2δε0v000
1 − δ + 2δε v11




gCC
gCD + gDC
= (1 − δ)+ δε+ O(ε2 ).
gDD
gCD + gDC
Ignoring the terms involving ε2 , we hence obtain TFT   

1
v00
v
(1 − δ) gCC + δε(gCD + gDC )=.
=v STFTv11
1 − δ + 2δε (1 − δ) gDD + δε(gCD + gDC )2. Grim: The strategy has two states 0 and 1 where it chooses C and D, respectively.
State transitions are synchronized between the two players when they both play
Grim so that the Markov process has only two states Θ = {(0, 0), (1, 1)}. We have
ω 1 = (1, 0) so that v Grim = v00 . The transition matrix is given by


p 1−p
.
A=
0
1
Ignoring the terms of order ε2 , we can write (3) as
 



 
v00
gCC
1 − 2ε 2ε v00= (1 − δ)
+δ
v11
gDD
0
1 v11
This yields
v Grim = v00 =(1 − δ)gCC + 2δεgDD.
1 − δ + 2δε

3. Grim2: The strategy has three states 0, 1 and 2, where it chooses C, C, and D,
respectively. State transitions are synchronized between the two players so that theMarkov process has three states Θ = {(0, 0), (1, 1), (2, 2)}. We have ω 1 = (1, 0, 0) sothat v Grim2 = v00 . The transition matrix is given by


p 1−p
0
0
1 − p .
A = p001
17

We can write (3) as


 

 
(1 − ε)2 ε(2 − ε)
0
v00
gCC
v00


v11  = (1 − δ)  gCC  + δ (1 − ε)20ε(2 − ε)v11  .
0
0
1
v22
gDD
v22
Solving this, we obtainv Grim2 = v00 =(1 − δ){1 + δε(2 − ε)}gCC + 4δ 2 ε2 gDD.
(1 − δ){1 + δε(2 − ε)} + 4δ 2 ε2

4. TF2T: The strategy has three states 0, 1 and 2, where the action choices are C,
C, and D, respectively. Since state transitions are not synchronized, the Markov
process has 3 × 3 = 9 states Θ = {(0, 0), . . . , (2, 2)}. We have ω 1 = (1, 0, . . . , 0) sothat v TF2T = v00 . The transition matrix is given by
p q 0 q r 0 0 0 0
p 0 q q 0 r 0 0 0
 q 0 r p 0 q 0 0 0
p q 0 0 0 0 q r 0


.
p
0
q
0
0
0
q
0
r
A=


q 0 r 0 0 0 p 0 q  q p 0 0 0 0 r q 0
q 0 p 0 0 0 r 0 q 
r 0 q 0 0 0 q 0 p
Using (3), we have
v11 = (1 − δ)gCC + δv00 + O(ε)
v02 = (1 − δ)gCD + δv10 + O(ε)
v20 = (1 − δ)gDC + δv01 + O(ε).
Substituting these into the recursive equations for v01 and v10 , we obtain
 


 


v01
gCC
v00
gCD
= (1 − δ)+ δ(1 − 2ε)+ δ(1 − δ)ε
v10
gCC
v00
gDC

 
0
1 + δ v01
+ δε
+ O(ε2 ),1+δ
0v10which yields

 


1−δ
v01
1
δε(1 + δ) gCC + δεgCD=
v10
1gCC + δεgDC
1 − δ 2 ε2 (1 + δ)2 δε(1 + δ) 
δ(1 − 2ε)v00
1
δε(1 + δ) 1+
+ O(ε2 ).
1
1
1 − δ 2 ε2 (1 + δ)2 δε(1 + δ)
18

(5)

It then follows that
(1 − δ){1 + δε(1 + δ)}{2gCC + δε(gCD + gDC )}1 − δ 2 ε2 (1 + δ)22δ(1 − 2ε){1 + δε(1 + δ)}+
v00 + O(ε2 )1 − δ 2 ε2 (1 + δ)2(1 − δ)={2gCC + δε(gCD + gDC )}1 − δε(1 + δ)
2δ(1 − 2ε)+v00 + O(ε2 ).
1 − δε(1 + δ)

v01 + v10 =(6)

On the other hand, the recursive equation for v00 yields
v00 =(1 − δ)gCC + δε(1 − ε)(v01 + v10 ) + δε2 v11.
1 − δ(1 − ε)2(7)

Substituting (5) and (6) into (7) and ignoring the terms of order ε3 , we obtainv TF2T = v00 ={1 + δ(1 − δ)ε − δε2 }gCC + δ 2 ε2 (gCD + gDC ).
1 + δ(1 − δ)ε − δ(1 − 2δ)ε2

As for the strategies AC, AD, and T6-T8, it can be readily verified that their cooperativeness is given as follows.
5. AD: v AD = gDD .
6. AC: v AC = gCC .
7. T8: v T8 = (1 − δ 7 ) gCC + δ 7 gDD + O(ε).
8. T7: v T7 = (1 − δ 6 ) gCC + δ 6 gDD + O(ε).
9. T6: v T6 = (1 − δ 5 ) gCC + δ 5 gDD + O(ε).
Combining the above cases, we can rank the ten strategies from the least cooperative tothe most cooperative in the indefinite games as follows:
AD  STFT ≪ T6 ≪ T7 ≪ T8
≪ Grim  TFT  Grim2 < TF2T < AC,where ≪,  and < represent domination in the orders of ε0 (= 1), ε, and ε2 , respectively.

19B.9.2

General implementation errors

When the probability ε ∈ [0, 21 ] of implementation errors is not necessarily small, thecooperativeness of the strategies TFT, STFT, Grim, Grim2, and TF2T can be computed
numerically using (1) for the finite games and by (2) for the indefinite games, whereas thecooperativeness of AC and AD equals gCC and gDD , respectively, as above. Consider now
the strategy Tk (k = 6, 7, 8). In the indefinite games, its cooperativeness can be computedas
1 − (δp)k−1
1 − (δp)k−2
Tkk−2
v = (1 − δ)
gCC + δ (1 − p)
+ (δp)gDD .
1 − δp1 − δpIn the finite games, suppose that t < k and let vt denote the sum of stage payoffs in roundst, t + 1, . . . , 8 when Tk still specifies action C in round t. We have the following recursive
equations:vk−1 = gCC + (9 − k)gDD ,vk−2 = gCC + pvk−1 + (1 − p)(10 − k)gDD ,..
.
v2 = gCC + pv3 + (1 − p) · 6gDD ,v1 = gCC + pv2 + (1 − p) · 7gDD .
The cooperativeness of Tk then equals v Tk = v81 .

20

B.10

Complete Estimation Results
Table 12: Estimates for the Finite Game on Late Supergames
Share
SFEM TYPING

21

AD

0.12

0.12

AC

0.03

0.03

GRIM

0.08

0.02

TFT

0.09

0.12

STFT

0.03

0.03

T8

0.22

0.20

T7

0.30

0.35

T6

0.08

0.08

GRIM2

0.02

0.01

TF2T

0.04

0.04

ALL

AD

AC

GRIM

TFT

Estimated Beliefs - p̃
STFT
T8
T7
T6

0.07
(0.09)
0.04(0.04)
0.34
(0.2)
0.11
(0.07)
0.00(0.02)
0.09(0.08)
0.00(0.01)
0.00(0.07)-

0.00(0.02)
0.00(0.05)
0.10(0.05)
0.00(0.05)
0.00(0.13)
0.00(0.06)
0.00
(0)
0.00
(0)
-

[0.00]

0.18(0.09)
0.03(0.04)
0.00(0.12)
0.03(0.04)
0.00(0.02)
0.00(0.02)
0.00
(0)
0.00
(0.02)-[0.00]

-

0.00
(0.1)
0.30
(0.16)
0.16(0.08)
0.53(0.22)
0.65
(0.4)
0.01
(0.14)
0.00(0.09)
0.00(0.21)-

0.00(0.09)
0.07

0.14
(0.1)
0.01

0.83
(0.35)
0.11

0.00
(0.1)
0.09

0.00
(0.01)
0.03

0.16(0.13)
0.17(0.34)
0.00(0.06)[0.00]
0.04
(0.1)
0.18
(0.14)[0.00]

Estimation on late supergames. SFEM estimate for β is 0.94.
Estimates in [square brackets] are not estimated due to collinearity.
Estimates in (brackets) show bootstrapped standard deviation.

GRIM2

TF2T

ν

β̃

0.00
(0.06)
0.00(0.12)
0.01(0.04)
0.00(0.13)
0.35(0.38)
0.15(0.11)
0.00(0.01)
0.00(0.01)
-

0.06

1.00

0.07

1.00

0.07

1.00

0.05

1.00

0.11

1.00

0.04

1.00

0.04

1.00

0.03

1.00

-

-

0.00
(0.07)
0.05

0.04

1.00

[0.00]

0.50(0.12)
0.39(0.16)
0.00(0.15)-

0.75(0.17)
0.00(0.03)
0.00(0.09)
0.00(0.03)
0.00(0.03)
0.00(0.07)
0.43(0.15)
0.99(0.31)-

0.00(0.03)
0.00(0.01)
0.00
(0)
0.00
(0.02)
0.00
(0)
0.00
(0)
0.00
(0.09)-

0.00(0.04)
0.00(0.04)
0.00(0.03)
0.00(0.05)
0.00(0.07)
0.21(0.11)
0.00
(0)
0.00
(0)
-

0.00
(0.13)
0.29

0.03(0.11)
0.30

0.00(0.03)
0.00

0.00(0.17)
0.05

0.46
(0.2)
0.22
(0.13)
0.33(0.12)[0.00]

Table 13: Estimates for the Indefinite Game on Late Supergames
Share
SFEM TYPING

AD

AC

GRIM

TFT

Estimated Beliefs - p̃
STFT
T8
T7
T6

GRIM2

TF2T

ν

β̃

0.00
(0.02)
0.80(0.33)
0.00(0.09)
0.00(0.06)
0.00(0.07)
-

0.00
(0.1)
0.00
(0.03)
0.80(0.24)
0.28(0.14)
0.35(0.24)-

0.00(0.05)
0.20(0.26)
0.13(0.17)
0.25(0.13)
0.00(0.12)-

0.00(0.17)
0.00(0.05)
0.00(0.04)
0.05(0.03)
0.00(0.11)-[0.00][0.00][0.00]

1.00

0.00(0.02)
0.00(0.02)
0.00
(0)
0.00
(0.04)-

0.00(0.02)
0.00(0.01)
0.00
(0)
0.00
(0.06)-

0.00(0.01)
0.00(0.11)
0.02(0.11)
0.19(0.12)
0.00(0.09)
-

0.04

0.00
(0.02)
0.00(0.03)
0.00
(0)
0.00
(0.05)-

0.00(0.01)
0.00(0.14)
0.05(0.06)
0.14(0.12)
0.16(0.09)
-

0.11

1.00

0.06

1.00

0.01

1.00

0.08

1.00

-

-

22

AD

0.09

0.10

AC

0.11

0.05

GRIM

0.18

0.09

TFT

0.36

0.59

STFT

0.04

0.04

T8

0.01

0.01

1.00
(0.24)
0.00(0.04)
0.00(0.06)
0.08(0.04)
0.48(0.27)
-

T7

0.00

0.00

-

-

-

-

-

-

-

-

-

-

-

-

T6

0.00

0.00

-

-

-

-

-

-

-

-

-

-

-

-

GRIM2

0.11

0.11
0.01

0.22
(0.16)
0.33
(0.2)
0.32

0.00
(0.07)
0.00(0.04)
0.14

0.00(0.02)
0.00
(0)
0.02

0.00
(0)
0.00
(0.01)
0.00

0.00
(0)
0.00
(0)
0.00

0.00
(0)
0.00
(0)
0.00

0.31
(0.2)
0.00
(0.04)
0.10

0.23(0.14)
0.40(0.18)
0.14

1.00

0.10

0.23
(0.12)
0.27(0.13)
0.14

0.02

TF2T

0.00
(0.03)
0.00
(0)
0.14

0.01

1.00

ALL

Estimation on late supergames. SFEM estimate for β is 0.94.
Estimates in [square brackets] are not estimated due to collinearity.
Estimates in (brackets) show bootstrapped standard deviation.

Table 14: Estimates for the Finite Game on Early Supergames
Share
SFEM TYPING

23

AD

0.12

0.13

AC

0.03

0.02

GRIM

0.01

0.00

TFT

0.17

0.15

STFT

0.03

0.04

T8

0.30

0.36

T7

0.25

0.20

T6

0.00

0.00

GRIM2

0.04

0.03

TF2T

0.05

0.08

ALL

AD

AC

GRIM

TFT

Estimated Beliefs - p̃
STFT
T8
T7
T6

GRIM2

TF2T

ν

β̃

0.25
(0.14)
0.03(0.26)-

0.00(0.04)
0.78(0.37)-[0.00]

0.21(0.16)
0.16(0.09)
-

0.00
(0.1)
0.00
(0.08)-

0.55(0.21)
0.03(0.07)-[0.00][0.00]

1.00[0.00]

0.16

0.92

-

-

0.00
(0.09)
0.00
(0.1)
-

0.11

[0.00]

0.00(0.05)
0.00
(0.1)
-

-

-

0.19
(0.09)
0.00(0.09)
0.01(0.06)
0.00(0.03)-

0.00(0.02)
0.00(0.12)
0.00(0.02)
0.00
(0)
-

0.03
(0.05)
0.00(0.12)
0.00(0.01)
0.00(0.02)-

0.26(0.14)
0.00
(0.1)
0.58
(0.1)
0.75
(0.17)-

0.00(0.04)[0.00]

0.00(0)[0.00]

0.15

1.00

0.00
(0.01)
0.00(0.01)-

0.00(0.07)
0.00(0.18)
0.00(0.04)
0.00(0.02)
-

1.00

0.00
(0.04)
0.25(0.12)-

0.02(0.05)
0.58(0.33)
0.00(0.05)
0.00(0.01)
-

0.05

0.05

1.00

0.03

1.00

-

0.00
(0.15)
0.42(0.33)
0.00(0.07)
0.00(0.11)
-

-

-

0.00
(0.02)
0.15(0.19)
0.07

0.30(0.19)
0.00(0.05)
0.04

0.00
(0.2)
0.06
(0.12)
0.21

0.43(0.18)
0.54(0.19)
0.09

0.00(0.04)
0.15(0.16)
0.01

0.27(0.19)
0.11(0.09)
0.49

0.00(0.02)
0.00(0.01)
0.06

0.00(0.03)
0.00(0.02)
0.00

0.00
(0.1)
0.00
(0.04)
0.02

0.00
(0.1)
0.00
(0.06)
0.00

0.02

1.00

0.05

1.00

[0.00]
0.50(0.21)[0.00]
0.40(0.13)[0.00]

Estimation on early supergames. SFEM estimate for β is 0.92.
Estimates in [square brackets] are not estimated due to collinearity.
Estimates in (brackets) show bootstrapped standard deviation.

Table 15: Estimates for the Indefinite Game on Early Supergames
Share
SFEM TYPING

AD

AC

GRIM

TFT

Estimated Beliefs - p̃
STFT
T8
T7
T6

GRIM2

TF2T

ν

β̃

0.03
(0.03)
0.00(0.09)
0.11
(0.2)
0.19
(0.09)
0.02(0.02)-

0.20(0.11)
0.00(0.01)
0.45(0.22)
0.40(0.13)
0.15(0.08)-

0.00(0.05)
1.00(0.44)
0.19(0.14)
0.16(0.09)
0.16(0.08)-

0.14(0.14)
0.00(0.01)
0.14(0.09)
0.05(0.03)
0.53(0.31)-[0.00][0.00][0.00]

1.00

0.00(0.01)
0.00(0.11)
0.00(0)[0.00]

0.00(0.01)
0.00(0.08)
0.00(0)[0.00]

0.08

1.00

0.10

1.00

0.01

1.00

0.05

1.00

-

-

-

0.00
(0.05)
0.00(0.12)
0.00(0.16)
0.12(0.06)
0.03(0.02)
-

0.05

0.00
(0.01)
0.00(0.04)
0.00(0)[0.00]

0.04(0.04)
0.00(0.05)
0.00(0.12)
0.00(0.04)
0.10(0.06)
-

-

-

24

AD

0.13

0.13

AC

0.05

0.02

GRIM

0.21

0.09

TFT

0.36

0.60

STFT

0.02

0.01

T8

0.00

0.00

0.59
(0.22)
0.00(0.01)
0.11(0.11)
0.08(0.03)
0.00(0.03)
-

T7

0.00

0.00

-

-

-

-

-

-

-

-

-

-

-

-

T6

0.00

0.00

-

-

-

-

-

-

-

-

-

-

-

-

GRIM2

0.10

0.05
0.10

0.31
(0.2)
0.25
(0.12)
0.32

0.00(0.05)
0.37(0.11)
0.14

0.00(0.07)
0.11(0.08)
0.02

0.00(0.05)
0.00(0.01)
0.00

0.00(0.02)
0.00(0.01)
0.00

0.00(0.02)
0.00(0.01)
0.00

0.26(0.23)
0.06(0.06)
0.10

0.00(0.07)
0.09(0.08)
0.14

1.00

0.14

0.00
(0.13)
0.00(0.05)
0.14

0.06

TF2T

0.42
(0.23)
0.12(0.09)
0.14

0.02

1.00

ALL

Estimation on early supergames. SFEM estimate for β is 0.94.
Estimates in [square brackets] are not estimated due to collinearity.
Estimates in (brackets) show bootstrapped standard deviation.

Table 16: Estimates for the Finite Game on Late Supergames
Type
AD
AC
GRIM
TFT
STFT
T8
T7
T6
T5
T4
T3
T2
GRIM2
TF2T
2TFT
GRIM3
ALL

Share
SFEM Typing
0.12
0.02
0.07
0.09
0.02
0.22
0.30
0.08
0.00
0.00
0.00
0.01
0.01
0.03
0.00
0.03

0.12
0.01
0.02
0.12
0.02
0.20
0.35
0.08
0.00
0.00
0.00
0.01
0.01
0.04
0.00
0.03

AD

AC

GRIM

TFT

STFT

T8

T7

T6

Estimated Beliefs - p̃
T5
T4
T3

T2

GRIM2

TF2T

2TFT

GRIM3

ν

β̃

0.12
0.06
0.11
0.00
0.05
0.00
0.00
0.00
0.00
0.04

0.00
0.06
0.00
0.00
0.00
0.00
0.00
0.17
0.00
0.01

[0.00]
0.11
0.00[0.00]
0.00
0.00[0.00]
0.83[0.00]
0.04

0.00
0.20
0.55
0.81
0.00
0.36
0.00
0.00
0.78
0.22

0.13
0.27
0.03
0.00
0.00
0.00
0.00
0.00
0.00
0.04

[0.00]
0.23
0.30[0.00]
0.52
0.23
0.00
0.00
0.00
0.23

0.00
0.00
0.00
0.00
0.00
0.41
1.00
0.00
0.00
0.21

[0.00]
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00

[0.00]
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.22
0.01

0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00

0.00
0.00
0.00
0.00
0.15
0.00
0.00
0.00
0.00
0.03

0.00
0.06
0.00
0.19
0.15
0.00
0.00
0.00
0.00
0.04

0.75
[0.00]
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.09

0.00
0.02
0.00
0.00
0.13
0.00
0.00
0.00
0.00
0.09

0.06
0.07
0.05
0.14
0.04
0.04
0.03
0.03
0.07

1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00

[0.00]
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00

[0.00]
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00

Estimation on late supergames. SFEM estimate for β is 0.94.
Estimates in [square brackets] are not estimated due to collinearity.

25

Table 17: Estimates for the Indefinite Game on Late Supergames
Type
AD
AC
GRIM
TFT
STFT
T8
T7
T6
T5
T4
T3
T2
GRIM2
TF2T
2TFT
GRIM3
ALL

Share
SFEM Typing
0.09
0.10
0.15
0.34
0.04
0.01
0.00
0.00
0.00
0.00
0.02
0.00
0.07
0.09
0.05
0.06

0.10
0.10
0.07
0.58
0.04
0.01
0.00
0.00
0.00
0.00
0.03
0.00
0.02
0.03
0.01
0.02

AD

AC

GRIM

TFT

STFT

T8

T7

T6

Estimated Beliefs - p̃
T5
T4
T3

T2

GRIM2

TF2T

2TFT

GRIM3

ν

β̃

0.90
0.00
0.00
0.08
0.48
0.00
0.00
0.00
0.00
0.14

0.01
0.85
0.00
0.12
0.00
0.00
0.24
0.97
0.01
0.24

0.07
0.00
1.00
0.08
0.15
0.16
0.00
0.00
0.24
0.21

0.01
0.15
0.00
0.28
0.00
0.30
0.00
0.02
0.01
0.13

0.00
0.00
0.00
0.04
0.00
0.00
0.00
0.00
0.00
0.02

[0.00]
0.00
0.00
0.00
0.00
[0.00]
0.00
0.00
0.00
0.00

[0.00]
0.00
0.00
0.00
0.00
[0.00]
0.00
0.00
0.00
0.00

[0.00]
0.00
0.00
0.00
0.00
[0.00]
0.00
0.00
0.00
0.00

[0.00]
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00

[0.00]
0.00
0.00
0.00
0.21
0.00
0.00
0.00
0.00
0.01

0.01
0.00
0.00
0.00
0.14
0.14
0.52
0.00
0.72
0.09

0.00
0.00
0.00
0.26
0.00
0.03
0.24
0.00
0.01
0.11

0.00
0.00
0.00
0.14
0.00
0.23
[0.00]
0.01[0.00]
0.06

0.01
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.01
0.06

0.04
0.05
0.06
0.01
0.07
0.08
0.05
0.00
0.01

1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00

Estimation on late supergames. SFEM estimate for β is 0.94.
Estimates in [square brackets] are not estimated due to collinearity.

[0.00]
0.00
0.00
0.00
0.01
0.07
0.00
0.00
0.00
0.00

[0.00]
0.00
0.00
0.00
0.00
0.07
0.00
0.00
0.00
0.00

Table 18: Best Response Analysis

Type
T7
T8
AD
TFT
T6
GRIM
Other
All

FiniteShare
SFEM Typing
0.30
0.22
0.12
0.09
0.08
0.07
0.12

0.35
0.20
0.12
0.12
0.08
0.02
0.11

Best Response
BRS Rs
Ro

Type

T7
T7
T8
T8
T6
T7
T6
T7

TFT
GRIM
AC
AD
TF2T
GRIM2
Other
All

1
0.89
0.23
0.87
1
0.84

0.97
0.87
0.6
0.77
1
0.82

Indefinite
Share
Best Response
SFEM Typing
BRS
Rs
0.34
0.15
0.10
0.09
0.09
0.07
0.16

0.58
0.07
0.10
0.10
0.03
0.02
0.10

TF2T/GRIM2
GRIM
AD
AD
STFT
STFT
TFT
TFT

Estimation on late supergames out of 16 strategies: AD, AC, Grim, Grim2, Grim3, TFT, TF2T, 2TFT, STFT, T2-T8.
Rows represent top 6 played strategies; BRS: Best Response strategy given beliefs.
In Finite games the best response strategy to the actual distribution (SFEM) is T6; in Indefinite games it is GRIM2.
Rs : Expected payoff from strategy/Best response payoff given beliefs.
Ro : Expected payoff from strategy/Best response payoff given actual distribution (SFEM).

26

0.99
1
0.78
1
0.96
0.89

Ro
0.93
0.92
0.74
0.76
0.95
1

Expected payoff given population estimates

Expected payoff per round
.2
.4
.6

TF2T

GRIM2

T6

T7

T8

STFT

TFT

GRIM

AC

AD

0
TF2T

GRIM2

T6

T7

T8

STFT

TFT

GRIM

AC

AD

0

Expected payoff per round
.2
.4
.6

.8

Indefinite.8

FiniteBest response strategy is higlighted in a darker color.
Analysis uses normalized stage-game payoffs.

Figure 18: Normalized Expected Payoff by Type Given Strategy Distribution

27

28

Y-axis for Early and Late panels denotes distance between weigted average subjective beliefs and SFEM values.

Figure 19: Strategy Changes and Belief Accuracy
Late

T8

STFT

TFT

GRIM

AC

AD

TF2T

GRIM2

T6

T7

T8

STFT

TFT

GRIM

AC

AD

T6
GRIM2
TF2T

T6
GRIM2
TF2T

T7

.2

Indefinite

T7

Belief-SFEM
-.1
0
.1

Early

T8

-.2

.2

TF2T

GRIM2

T6

T7

T8

STFT

TFT

GRIM

AC

AD

-.2

-.2

-.1

Belief-SFEM
-.1
0
.1

Belief-SFEM
-.1
0
.1

.2

.2

Change in Frequency
-.05
0
.05
.1

Early

STFT

TFT

GRIM

AC

AD

TF2T

GRIM2

T6

T7

Belief-SFEM
-.1
0
.1

SFEM: Late - Early

T8

-.2

Change in Frequency
-.05
0
.05
.1

SFEM: Late - Early

STFT

TFT

GRIM

AC

AD

TF2T

GRIM2

T6

T7

T8

STFT

TFT

GRIM

AC

AD

-.1

FiniteLate

Finite

T8
T7
TFT
AD
TF2T
Grim2
Grim
T6

0

STFT AC

.2
.4
.6
.8
Distance of Belief from SFEM

1

Late - Early(Late) Popularity of Strategy
0
.1
.2
.3
.4

Late
(Late) Popularity of Strategy
0
.1
.2
.3
.4

(Early) Popularity of Strategy
0
.1
.2
.3
.4

Early
T7

T8

AD
Grim TFT
Grim2

0

T6
TF2T
STFT

AC

.2
.4
.6
.8
Distance of Belief from SFEM

1

T7
T8

AD
TFT
AC
Grim2

STFT

Grim

T6

TF2T

-.8 -.6 -.4 -.2 0 .2 .4 .6
Difference in Distance.8

Indefinite
TFT

Grim
TF2T

AD
Grim2
AC

T6
T7
T8

0

STFT

.2
.4
.6
.8
Distance of Belief from SFEM

1

Late - Early

TFT

Grim
Grim2
TF2T

AC

AD

STFT
T8
T6
T7

0

.2
.4
.6
.8
Distance of Belief from SFEM

1

(Late) Popularity of Strategy
0
.1
.2
.3
.4

Late
(Late) Popularity of Strategy
0
.1
.2
.3
.4

(Late) Popularity of Strategy
0
.1
.2
.3
.4

Early

TFT

Grim
Grim2
AC

-.8 -.6 -.4 -.2 0 .2 .4 .6
Difference in Distance

Vertical lines indicate the average weighted by popularity of strategies.

Figure 20: Change in Accuracy

29

TF2T
AD

STFT
T8
T6
T7

.8

Table 19: Type Evolution: Finite

Early Types
Number that Change
No. 1 Change (%)
No. 2 Change (%)

T7
32
8
T6 (50%)T8
57
39
T7 (62%)
T6 (15%)

AD
20
4

TFT
24
15
T8 (47%)T7 (20%)

T6
0

Grim
0

TF2T
12
11
TFT (27%)

AC
3
3

STFT
6
5
T7 (40%)Grim2
4
4
T8 (75%)
TF2T (25%)

AD
19
10
STFT (40%)

STFT
2
2
AC (100%)

T8

Sorted by late frequency.
Last two rows provided if no. 1 and 2 are unique.

Table 20: Type Evolution: Indefinite

Early Types
Number that Change
No. 1 Change (%)
No. 2 Change (%)

TFT
86
25
TF2T (44%)
Grim (32%)

Grim
13
11
TFT (64%)
AD (18%)Grim2
7
5
TFT (80%)
Grim (20%)

Sorted by late frequency.
Last two rows provided if no. 1 and 2 are unique.

30

AC
3
3

TF2T
14
14
TFT (79%)

T6

T7

B.11

Belief-Estimation Method

In this section, we document properties of our belief-estimation method using simulations.
First, we show that the method recovers the correct beliefs in a simple stylized example
of a population that consists of only AD (25%) , Grim (40%) and TFT types (35%). We
simulate data—including both actions and round-by-round beliefs—based on the model ofbelief formation described in the paper assuming the following supergame beliefs for thedifferent types. AD types believe others are playing AD with 40% probability, Grim with
10% probability, and TFT with 50% probability. Grim types believe others are playing AD
with 10% probability, Grim with 30% probability, and TFT with 60% probability. TFT
believe others are playing AD with 20% probability, Grim with 50% probability, and TFT
with 30% probability.66
Figure 21 plots how well these input parameters are recovered by our belief estimation
method from the simulated data. Note that this involves all three steps of our method:
applying SFEM to the simulated data, estimating strategy type of each simulated subject
taking population level SFEM estimates as a prior, and, finally, for each strategy typeestimating beliefs over strategies given simulated round-by-round beliefs. We contrast
results from experiments with two sessions, four sessions and eight sessions.
Next we consider a similar exercise but for conditions similar to the ones in our data
set. Namely, the data generating process is assumed to correspond to the one we report inTables 16 and 17. The sample size is assumed to be the same as the one we have collectedin the experiment. Figures 22 and 23 show that the input parameters are recovered quite
well for the most common types.67

66
We set β = 0.9365, the average estimated value for this parameter in the experiment (using values fromthe Finite and Indefinite games). We set β̃ = 1.00 and ν = 0.05 which are the median estimated values forthese parameters from the experiment (including all types in the Finite and Indefinite games).
67
One notable exception is the supergame beliefs of the AC in the Indefinite game, which are not recovered
as well as other types. However, it is useful to note the nature of the discrepancy in this case: input values
are such that the AC type puts 80% probability on others playing AC; the recovered values are such thatsome of this weight is shifted to TF2T. Thus, the discrepancy between the input and output values areamong the most cooperative two strategies.

31

2 sessions per experiment

0

.2

.4 .6
Truth

.8

1

.8
0

.2

Belief.4 .6.8Belief
.4 .6
.2
0

0

.2

Belief
.4 .6

.8

1

TFT (35%)

1

Grim (40%)

1

AD (25%)

0

.2

.4 .6
Truth

.8

1

0

.2

.4 .6
Truth

.8

1

4 sessions per experiment

0

.2

.4 .6
Truth

.8

1

.8
0

.2

Belief.4 .6.8Belief
.4 .6
.2
0

0

.2

Belief
.4 .6

.8

1

TFT (35%)

1

Grim (40%)

1

AD (25%)

0

.2

.4 .6
Truth

.8

1

0

.2

.4 .6
Truth

.8

1

8 sessions per experiment

0

.2

.4 .6
Truth

.8

1

.8
0

.2

Belief.4 .6.8Belief
.4 .6
.2
0

0

.2

Belief
.4 .6

.8

1

TFT (35%)

1

Grim (40%)

1

AD (25%)

0

.2

.4 .6
Truth

.8

1

0

.2

.4 .6
Truth

.8

1

Estimation results from 100 simulated experiments with 18 subjects in each session.
Truth refers to input values.
Solid dots represent median estimate, hollow bubbles represent 25th and 75th percentile estimates.
Input values for all other parameters (not depicted in graphs) correspond to median values from belief
estimates reported in the paper.

Figure 21: Estimation results using simulations
32

Finite

0

.2

.4
.6
Truth

.8

1

.8
0

.2

Belief.4 .6.8Belief
.4 .6
.2
0

0

.2

Belief
.4 .6

.8

1

AD

1

T8

1

T7

0

.2

.8

1

.4
.6
Truth

.8

1

.4
.6
Truth

.8

1

.8

1

1
0

.2

Belief
.4 .6

.8

1
.8
.2
0

.2

.2

OthersBelief.4 .6.8Belief
.4 .6
.2
0

0

0

T6

1

TFT

.4
.6
Truth

0

.2

.4
.6
Truth

.8

1

0

.2

.4
.6
Truth

Truth refers to input values.
Solid dots represent median estimate, hollow bubbles represent 25th and 75th percentile estimates from 150 simulations.
We report simulation results for which identification matches original experiment (96% of data).

Figure 22: Estimation results using simulations

33

Indefinite

0

.2

.4
.6
Truth

.8

1

.8
0

.2

Belief.4 .6.8Belief
.4 .6
.2
0

0

.2

Belief
.4 .6

.8

1

Grim 2

1

Grim

1

TFT

0

.2

.8

1

.4
.6
Truth

.8

1

.4
.6
Truth

.8

1

.8

1

1
0

.2

Belief
.4 .6

.8

1
.8
.2
0

.2

.2

OthersBelief.4 .6.8Belief
.4 .6
.2
0

0

0

TF2T

1

AC

.4
.6
Truth

0

.2

.4
.6
Truth

.8

1

0

.2

.4
.6
Truth

Truth refers to input values.
Solid dots represent median estimate, hollow bubbles represent 25th and 75th percentile estimates from 150 simulations.
We report simulation results for which identification matches original experiment (99% of data).

Figure 23: Estimation results using simulations

34

C

Instructions

F8	
  INSTRUCTIONS	
  
	
  
You	
  are	
  about	
  to	
  participate	
  in	
  an	
  experiment	
  on	
  decision-­‐‑making.	
  What	  you	
  earn	  depends	
  partly	
  on	
  your	  decisions,	
  partly	
  on	
  the	
  decisions	
  of	
  others,	
  and	  partly	
  on	
  chance.	
  Please	
  turn	
  off	
  cell	  phones	  and	
  similar	
  devices	  now.	
  Please	
  do	
  not	
  talk	
  or	
  in	
  
any	
  way	
  try	  to	
  communicate	
  with	  other	  participants.	
  	
  
We	
  will	  start	  with	  a	
  brief	
  instruction	
  period.	
  If	
  you	  have	  any	
  questions	
  during	  this	  period,	  raise	  your	  hand	  and	
  your	  question	
  will	  be	
  answered	
  so	
  everyone	
  can	
  hear.	
  This	
  experiment	
  has	
  three	  parts;	
  these	  instructions	  are	
  for	  the	
  first	  part.	
  Once	  this	
  
part	
  is	
  over,	  instructions	
  for	  the	
  next	  part	  will	  be	
  given	  to	
  you.	
  Your	  decisions	
  in	
  this	  part	
  have	  no	
  influence	
  on	
  the	
  other	  parts.	
  General	
  Instructions	  	
  1.   In	
  this	
  experiment	
  you	
  will	  be	
  repeatedly	
  matched	
  with	  a	
  randomly	
  selected	
  person	  in	
  the	
  room.	
  	
  During	
  each	  match,	
  you	
  will	  be	
  asked	  to	
  make	  decisions	
  
over	
  a	
  sequence	
  of	
  rounds.	
  	
  2.   The	  points	
  you	
  can	
  obtain	  in	
  each	  round	  of	  a	
  match	
  depend	
  on	
  your	  choice	  and	
  the	
  choice	
  of	
  the	  person	  you	  are	
  paired	
  with.	
  The	
  table	  below	
  represents	
  all	
  the	  possible	
  outcomes:	
  
	
  
Your	
  	
  
Other’s	
  Choice	  Choice	
  
1	
  
2	
  
1	
  
51,	
  51	
  
22	
  63	
  
2	
  

63,	
  22	
  

39,	
  39	
  

	
  
The	
  table	
  shows	
  the	  points	
  associated	
  with	  each	  combination	
  of	
  your	  choice	  and	
  choice	
  of	
  the	  person	  you	  are	
  paired	
  with.	
  The	
  first	
  entry	  in	
  each	  cell	  represents	
  the	  points	
  you	
  obtain	
  for	
  that	  round,	
  while	  the	
  second	
  entry	  (in	  italics)	
  represents	
  the	  points	
  obtained	
  by	
  the	  person	  you	  are	
  paired	
  with.	
  
	
  
That	
  is,	  in	
  each	  round	  of	  a	
  match,	
  if:	  -­‐‑   (1,1):	
  Your	  choice	
  is	
  1	
  and	  the	
  other’s	
  choice	
  is	
  1,	
  you	
  each	  make	  51.	  -­‐‑   (1,2):	
  Your	  choice	
  is	
  1	
  and	  the	
  other’s	
  choice	
  is	
  2,	
  you	
  make	
  22	
  while	  the	
  other	
  makes	
  63.	  -­‐‑   (2,1):	
  Your	  choice	
  is	
  2	
  and	  the	
  other’s	
  choice	
  is	
  1,	
  you	
  make	
  63	
  while	  the	
  other	
  makes	  22.	  -­‐‑   (2,2):	
  Your	  choice	
  is	
  2	
  and	  the	
  other’s	
  choice	
  is	
  2,	
  you	
  each	  make	  39.	  	
  3.   At	
  the	
  end	  of	
  each	  round,	
  you	
  will	  see	
  your	  choice	
  (1	
  or	
  2)	
  and	  the	
  choice	
  of	
  
the	
  person	  you	
  were	  paired	
  with	  (1	  or	
  2).	
  	
  

35

F8	
  

4.   Each	
  match	
  will	  last	  for	  8	
  rounds.	
  	
  
	
  
5.   Once	  a	
  match	
  ends,	  you	
  will	  be	
  paired	
  randomly	
  with	  someone	
  for	  a	
  new	
  match.	
  You	
  will	  not	  be	
  able	  to	
  identify	
  who	
  you've	
  interacted	
  with	  in	
  previous	  or	
  future	
  matches.	
  	
  
	
  
6.   Each	
  part	  of	
  the	
  experiment	
  will	
  generate	
  points	
  that	  count	
  towards	
  your	  final	
  payoff.	
  In	
  this	  part,	
  one	
  match	
  will	  be	
  randomly	
  selected	
  to	
  count	  towards	
  your	  final	
  payoff.	
  Points	
  earned	
  in	
  this	  match	
  will	  be	
  converted	
  to	
  dollars	
  at	
  a	
  rate	  of	
  3	
  cents	
  per	  point.	
  You	
  will	  receive	
  an	
  additional	
  $8	
  show	
  up	
  fee	
  for	
  your	  participation.	
  You	
  will	  only	  be	
  informed	
  of	
  your	  payoffs	
  at	
  the	
  
end	
  of	
  the	
  experiment.	
  	
  7.   This	  part	  will	  last	  for	
  four	  matches.	
  	
  
	
  

	
  
	
  
	
  
	
  
Are	
  there	  any	
  questions?	
  	
  Before	  we	
  start,	
  let	  me	
  remind	
  you	
  that:	  	
  •   Each	
  match	
  will	  last	  for	  8	
  rounds.	
  You	
  will	  interact	
  with	  the	
  same	  person	
  for	  the	
  entire	
  match.	
  	
  •   Your	
  choice	
  and	  the	
  choice	
  of	
  the	  person	  you	  are	
  paired	
  with	  will	  be	
  shown	  to	
  both	  of	
  you	  at	  the	
  end	  of	
  the	
  round.	
  	
  •   Points	
  obtained	
  in	
  each	  round	
  depend	
  on	
  these	
  choices.	
  	
  
	
  
•   After	
  a	
  match	  is	
  finished,	
  you	
  will	  be	
  randomly	
  paired	
  with	  someone	
  for	  a	  new	
  match.	
  
	
  

36

F8	
  

General	
  Instructions	
  for	  Part	
  2	
  
	
  
The	
  basic	
  structure	
  of	
  this	  part	  is	
  very	  similar	
  to	
  part	
  1.	
  How	  the	
  match	  proceeds	
  and	
  how	
  you	  are	
  paired	
  with	  others	
  will	  remain	
  the	
  same.	
  	
  
	
  
However,	
  in	
  this	  part,	  you	
  will	  have	  one	  more	  task.	
  In	
  each	  round	  of	  a	
  match,	
  after	  you	
  make	  a	
  choice,	
  we	
  will	  ask	
  you	  to	
  submit	
  your	  belief	
  about	  the	
  choice	
  of	
  the	
  person	
  you	  are	
  paired	
  with.	
  	  To	  indicate	
  your	  beliefs,	
  you	
  will	  use	  a	
  slider.	
  Where	
  you	  move	  the	  slider	
  will	  represent	
  your	  best	  assessment	
  of	
  the	
  likelihood	
  (expressed	
  as	
  chance	
  out	  of	
  100)	
  that	
  the	  person	  you	  are	
  paired	
  with	  chose	
  1	
  or	
  2.	
  	
  
Two	
  different	
  matches	
  from	  this	  part	  will	  be	
  randomly	
  selected	
  to	
  count	
  towards	
  payment.	
  For	
  one	  of	
  these,	  you	
  will	  receive	  the	  points	
  associated	
  with	  your	  choices	
  
as	
  in	
  part	
  1.	
  For	  the	
  other,	
  the	
  computer	
  will	  randomly	
  choose	
  one	
  round	
  from	  that	  match	
  for	
  payment	
  for	
  beliefs.	
  The	
  belief	
  that	  you	
  report	
  in	
  that	  round	
  will	  determine	
  your	  chance	
  of	
  winning	
  a	
  prize	
  of	
  50	
  points.	
  To	
  determine	
  your	  payment,	
  the	
  computer	
  will	  randomly	
  draw	
  two	
  numbers.	
  For	  each	
  draw,	
  all	
  numbers	
  between	
  0	
  and	
  100	  (including	
  decimal	
  numbers)	
  are	
  equally	
  likely	  to	
  be	
  selected.	
  Draws	
  are	  independent	
  in	
  the	
  sense	
  that	  the	
  outcome	
  of	
  the	
  first	
  draw	
  in	
  no	
  way	
  affects	
  the	
  outcome	
  of	
  the	
  second	
  draw.	
  	
  If	
  the	  person	  you	  are	
  paired	
  with	  chose	  1	
  in	
  that	  round	  and	  the	
  number	  you	
  indicated	
  as	
  the	
  likelihood	
  that	  the	
  other	
  chose	  1	
  is	
  larger	
  than	
  either	
  of	
  the	
  two	
  draws,	
  you	
  will	  win	  the	
  prize.	
  	
  
If	
  the	  person	  you	  are	
  paired	
  with	  chose	  2	
  in	
  that	  round	  and	  the	
  number	  you	
  indicated	
  as	
  the	
  likelihood	
  that	  the	
  other	
  chose	  2	
  is	
  larger	
  than	
  either	
  of	
  the	
  two	
  draws,	
  you	
  will	  win	  the	
  prize.	
  	
  
The	
  rules	
  that	  determine	
  your	  chance	
  of	
  winning	
  this	  prize	
  were	  purposefully	
  designed	
  so	
  that	  you	  have	  the	
  greatest	
  chance	
  of	
  winning	
  the	
  prize	
  when	
  you	  answer	  the	
  question	
  with	  your	  true	  assessment	
  on	
  how	
  likely	
  the	  person	  you	  are	  paired	  with	  chose	
  1	
  or	
  2.	
  
The	
  first	
  match	  to	
  end	
  after	
  60	
  minutes	
  of	
  play	  (including	
  the	
  first	  part)	
  will	  mark	  the	
  end	  of	
  the	
  experiment.	
  
	
  
	
  

37

F8	
  

General	
  Instructions	
  for	  Part	
  3	
  
	
  
On	
  the	
  screen,	
  you	
  see	  a	
  field	
  composed	
  of	
  100	
  boxes,	
  as	
  shown	
  below	  (the	  numbers	
  on	
  each	  box	
  will	  not	  be	
  visible):	
  	
  

	
  

1
11
21
31
41
51
61
71
81
91

2
12
22
32
42
52
62
72
82
92

3
13
23
33
43
53
63
73
83
93

4
14
24
34
44
54
64
74
84
94

5
15
25
35
45
55
65
75
85
95

6
16
26
36
46
56
66
76
86
96

7
17
27
37
47
57
67
77
87
97

8
18
28
38
48
58
68
78
88
98

9 10
19 20
29 30
39 40
49 50
59 60
69 70
79 80
89 90
99 100	
  There	  is	
  also	
  a	
  Start	  button—please	
  do	
  not	
  click	
  on	
  this	  button	
  until	  we	
  finish	
  reading	  the	
  instructions.	
  Once	  the	
  Start	  button	
  is	
  clicked,	
  the	
  experiment	
  begins.	
  Every	
  two	
  seconds,	
  a	
  box	
  will	  be	
  collected,	
  beginning	
  with	  Box	  #1	  (top	  left)	
  and	
  ending	  with	  Box	
  #100	  (bottom	
  right).	
  You	
  earn	
  3	
  cents	
  for	
  every	
  box	
  that	  is	
  collected.	
  	
  Once	  collected,	
  the	
  box	
  changes	
  from	
  dark	  grey	  to	
  light	
  grey,	  and	
  your	  earnings	
  are	
  updated	
  accordingly.	
  At	
  any	  moment,	  on	
  the	
  information	
  box,	
  you	
  can	
  see	  the	
  number	
  of	
  boxes	  collected	
  so	
  far	  and	
  the	
  amount	
  earned	
  up	
  to	
  that	  point.	
  Such	
  earnings	
  are	
  only	  potential,	
  however,	
  because	
  behind	
  one	  of	
  these	  boxes	
  a	
  
bomb	
  is	
  hidden	
  that	  destroys	
  everything	
  that	  has	  been	  collected	
  in	
  this	  part	  of	
  the	  experiment.	
  You	  do	
  not	
  know	
  the	
  location	
  of	
  the	
  bomb.	
  Moreover,	
  even	
  if	
  you	  collect	  the	
  bomb,	
  you	
  will	  not	
  know	
  it	
  until	  the	
  end	  of	
  the	
  experiment.	
  Your	  task	
  is	
  to	
  
choose	  when	  to	
  stop	  the	  collecting	
  process.	
  You	
  stop	  the	
  process	
  by	
  hitting	
  ‘Stop’	
  at	
  
any	
  time.	
  Payoffs:	
  If	
  at	
  the	
  moment	
  you	
  hit	
  ‘Stop’	
  none	  of	
  the	
  boxes	  you	  have	  collected	
  contain	  the	
  bomb,	
  you	
  will	  receive	  the	
  amount	
  of	
  money	  you	  have	  accumulated.	
  If	
  at	
  the	  moment	  you	
  hit	
  ‘Stop’	
  you	
  happen	
  to	
  have	  collected	
  the	
  box	
  with	  the	
  bomb,	
  then	
  you	  will	
  earn	
  $0.	
  Remember	
  that	  you	
  will	  not	  be	
  told	
  if	
  a	
  box	
  that	  you	  have	  collected	
  has	  or	
  does	  not	  have	  the	
  bomb	
  until	
  after	  you	
  hit	  the	
  ‘Stop’	
  button.	
  So	
  the	
  earnings	
  you	
  
see	
  on	
  the	
  screen	  are	
  only	  potential	
  earnings,	
  and	
  you	
  will	  earn	
  those	
  earnings	
  only	  if	
  none	  of	
  the	
  boxes	  you	  have	  collected	
  had	  the	
  bomb.	
  Location	
  of	
  the	  Bomb:	
  The	
  interface	
  will	  randomly	
  choose	  a	
  number	  between	
  1	
  
and	
  100.	
  All	
  numbers	
  are	
  equally	
  likely.	
  The	
  interface	
  will	  then	  place	
  the	
  bomb	
  in	
  
the	
  box	
  with	  the	
  randomly	
  chosen	
  number.	
  38