ISSN (Print) 0473-453X

Discussion Paper No. 1117

ISSN (Online) 2435-0982

STRATEGIC UNCERTAINTYANDPROBABILISTIC SOPHISTICATION

Masaki Aoyagi
Takehito Masuda
Naoko Nishimura

February 2021

The Institute of Social and Economic Research
Osaka University
6-1 Mihogaoka, Ibaraki, Osaka 567-0047, Japan

Strategic Uncertainty and Probabilistic Sophistication∗
Masaki Aoyagi† Takehito Masuda‡ Naoko Nishimura§
March 22, 2021

Abstract
This paper uses laboratory experiments to study subjects’ assessment of uncertainty resulting from strategic and non-strategic decisions of other players. Nonstrategic events are defined by the colors of balls drawn from urns, whereas strategic
events are defined by the action choice in Stag Hunt (SH) and Prisoners’ Dilemma(PD) games. We elicit subjects’ matching probabilities and examine if they satisfy
the law of probability including monotonicity and additivity. Violations from thelaw are observed for both uncertainty sources, but are more substantial for strategic
uncertainty. In particular, we observe a coordination fallacy, a violation of monotonicity whereby the probability weight placed on a symmetric coordination profileof the games exceeds that placed on the corresponding action choice. The violation
is found to be severer for an eﬀicient coordination profile.
Key words: matching probability, ambiguity, uncertainty, coordination, conjunction
JEL codes: C91, D01, D81, D91∗

We are particularly grateful to Aurélien Baillon, Chen Li, Peter Wakker, and Songfa Zhong forproviding very detailed comments. We also thank Yoichiro Fujii, Nobuyuki Hanaki, Takashi Hayashi,
PJ Healy, Keigo Inukai, Asen Kochov, Yukio Koriyama, Hiroaki Masuhara, Yusuke Osaki, Yoshi Saijo,
Tridib Sharma, Eli Spiegelman, and the participants of SAET 2018 Taiwan and 2019 North Americanand European ESA meetings as well as seminars at Osaka, Dijon, Erasmus, Waseda, Kyoto, Kochi Tech,
Shinshu, and Hitotsubashi. Shuya Abe, Yuki Hamada and Yoshitaka Okano provided excellent research
assistance. Financial support from the JSPS (grant numbers: 22330061, 23530216, 24330064, 24653048,
15K13006, 15H03328, 16H03597, 17K13701, 15H05728, 20H05631) and the Joint Usage/Research Center
at ISER, Osaka University, and the International Joint Research Promotion Program of Osaka University
is gratefully acknowledged.
†

ISER, Osaka University, 6-1 Mihogaoka, Ibaraki, Osaka 567-0047, Japan. E-mail: aoyagi@iser.osakau.ac.jp.
‡

Faculty of Economics and Law, Shinshu University, 3-1-1 Asahi, Matsumoto, Nagano 390-8621,
Japan. E-mail: tmasuda@shinshu-u.ac.jp.
§

Department of Gastronomy Management, Ritsumeikan University, 1-1-1 Noji-higashi, Kusatsu,
Shiga 525-8577, Japan. E-mail: nnaoko@fc.ritsumei.ac.jp.

11Introduction

While there is by now an extensive literature on the theory of ambiguity preferences,the experimental work on the subject has focused almost exclusively on artificial andnon-strategic uncertainty based on the colors of balls drawn from urns.1 In contrast,
uncertainty in the real world is often associated with the strategic decision making of
human beings. For example, uncertainty in the exchange rate results from governments’strategic trade policies or central banks’ market intervention. Uncertainty associatedwith climate change is not purely in the domain of natural sciences, but a large partof it is associated with how we respond to the change, and the responses are based on
strategic interactions among different parties involved.
A natural question then concerns how humans perceive uncertainty when it resultsfrom strategic interactions of others. Would individuals facing strategic uncertainty
behave in much the same way as they would in the presence of artificial uncertainty suchas the colors of balls drawn from urns?
This paper uses a laboratory experiment to investigate if and how individuals’ perception of uncertainty changes when uncertainty is of strategic nature compared withwhen it is not. Specifically, we have a group of subjects create strategic and non-strategic
uncertainty in the first set of experiments, and then have a new group of subjects predict
the outcome of the first experiments to measure their perception of uncertainty. Subjects in the first set of the experiments are called players, and they fill balls into urns
and then play one-shot games against each other. Non-strategic events are defined bythe (combinations of) colors of balls drawn from urns filled by these subjects, whereas
strategic events are defined by individual action choices and joint action profiles in thegames played by them. In the second set of the experiments, a new group of subjects
called observers face uncertain bets whose outcome is determined by the events from thefirst set of experiments. Specifically, for each event E, the observers make a choice between an uncertain bet which yields a positive reward if and only if the event E occurs,
and another risky bet which yields the same positive reward with an objectively known
probability. For each observer i and for each event E, we elicit their matching probabilitymi (E) of E which equals the probability of the risky bet that they state is indifferent
to the uncertain bet based on E.
For the observers’ choices to be consistent with the theory of subjective expectedutility, the matching probabilities thus elicited must satisfy the law of probability.
Specifically, they must satisfy monotonicity: E ⊂ F ⇒ mi (E) ≤ mi (F ), and additivity:
E ∩ F = ∅ ⇒ mi (E) + mi (F ) = mi (E ∪ F ). We follow the convention in the literatureand call observer i probabilistically sophisticated (PS) if his matching probability misatisfies the law of probability including these conditions. Our empirical strategy is to1

See Trautmann & van de Kuilen (2015a) for a comprehensive survey of the literature.

2

examine if our observer subjects are PS by constructing various sophistication indicesand computing the values of those indices from the stated matching probabilities.
The player subjects in our experiments play Stag-Hunt (SH) and Prisoners’ Dilemma(PD) games. SH represents a coordination game with two pure Nash equilibria (NE).
One of the NE yields each player a strictly higher payoff than the other NE, and hence
is payoff dominant. On the other hand, which of the two NE is risk dominant depends
on the payoff parameter.2 We have the observer subjects predict the outcome of two
SH games. The payoff dominant NE and the risk dominant NE coincide in one SH
game, and they correspond to different action profiles in the other SH game. Theinteraction between the two notions of dominance in these games creates non-trivial
uncertainty for the observers. In PD, on the other hand, there exists a unique NE that
corresponds to the strictly dominant action of each player. Despite this, however, thechoice of the dominated action is commonly observed in the laboratory, and is known tobe more frequent when the temptation payoff is smaller. We have the observers predict
the outcome of two PD games with different temptation parameters. The expecteddifference in the deviation from the theoretical prediction in these PD games also creates
substantial uncertainty for the observers. Importantly, the player subjects in this first
set of experiments play these games in perfect stranger format with no interim feedback
so that they have no chance to coordinate their action choices.
The observers’ assessment of uncertainty consistently violates the law of probability
for both strategic and non-strategic events, but the patterns of violation are substantially different between the two uncertainty sources. First, regardless of the uncertainty
sources, the binary complementarity (BC) index, which equals one minus the sum of thematching probabilities of complementary events, is negative on average. This implies
that the observers are on average ambiguity seeking in line with some recent findings in
the literature. However, this is almost the only similarity between the two sources, andsubstantial difference exists in the distributions of matching probabilities themselves:
While a majority of observers place almost the same probability weights on two ballcolors for the urns, they have little agreement on the likelihood of each action choice forthe games. Interestingly, we confirm a version of the familiarity hypothesis whereby theobservers who state in the post-experimental questionnaire that the games are easier topredict than the urns tend to be more ambiguity seeking in their prediction of the game
events.
The most striking difference between the uncertainty sources arises in the matchingprobabilities of conjunctional events, which correspond to the color combination of balls
from two urns, or the action profile of two players in games. Specifically, for the strate2

A NE of a symmetric 2 × 2 coordination game is risk dominant when the action corresponding to theequilibrium yields a higher expected payoff when the other player chooses each action with probability
one-half.

3

gic uncertainty, the matching probabilities of the coordination profiles of the games are
extremely large, and in many cases violate monotonicity (MN) in the sense that theyexceed the matching probabilities of the corresponding individual action choices. We
call this phenomenon a coordination fallacy. Overall, nearly half of the observers display the coordination fallacy for at least one of the two symmetric profiles, and more
frequently for the Pareto eﬀicient coordination profile in both SH and PD. Such violationof monotonicity is not observed for the conjunctional urn events.3
To further evaluate the probability weight placed on the symmetric coordination
profiles, we introduce quasi-complementarity (QC) and coordination premium (CP) indeices. QC equals one minus the sum of the matching probabilities of two conjunctional
events, and is non-negative under PS. We however find that the index takes negativevalues for half of observations for the games. CP equals the matching probability of thecoordination profile minus the product of the matching probabilities of the corresponding action choices of the two players, and equals zero if the observers believe that thetwo players make their action choices independently as in our experiments. We observethat CP is indistinguishable from zero for many of the conjunctional urn events, but issignificantly positive for all conjunctional game events. The positive CP implies that theobservers believe in the presence of some mechanism that helps the players coordinate
their actions.
In order to account for the possibility that the observed violation of PS is caused bytrembling hand rather than some fundamental deviation from probabilistic thinking, we
examine k-PS whereby the values of the indices are allowed to deviate by a small amountk > 0 from what is implied by PS. Through this exercise, we again find substantial
difference between the urns and games. In the case of the urns, BC is “binding” in thesense that observer i is k-PS whenever he satisfies BC by the margin k (i.e., |BCi | ≤ k),whereas in the case of the games, the proportion of k-PS observers is substantially lower
for each value of k, and there is no single index that is binding in such a sense.
To the best of our knowledge, our experiments are the first to confirm the violationof monotonicity for strategic uncertainty using an incentivized elicitation of matchingprobabilities. While the literature usually takes monotonicity for granted in economics,
the violation of monotonicity in various forms is discussed in the psychological literature.4 The best known among them is the conjunction fallacy, which states that anindividual, when informed of the occurrence of event X, often reports a higher value for3

Relatedly, Baillon et al. (2018) find violation of monotonicity to be infrequent even when the subjects
had to make predictions on stock market indices under time pressure.
4

Monotonicity is the bottom line for models of decision under uncertainty (Offerman et al. 2009).
For example, the capacity function in Choquet expected utility is assumed monotone (Schmeidler 1989).
Monotonicity is also assumed for the theories of maxmin preferences and their generalizations.

4the probability Pr(A ∩ B | X) than for the probability Pr(A | X).5 The coordination
fallacy identified in this paper is different from the conjunction fallacy in that it involves
no conditioning event X.
This paper is organized as follows. We discuss the related literature in Section 2,
and describe the experimental design in Section 3. Section 4 presents the sophistication
indices used in our analysis, and Section 5 describes results on the violation of PS wellas the difference between strategic and non-strategic uncertainty. Approximate PS is
examined in Section 6. We offer one possible model which accommodates the violationof monotonicity in Section 7, and then conclude in Section 8.

2

Related Literature

We contribute to the growing experimental literature on descriptive and empirically
relevant models of attitudes toward uncertainty. The experimental literature, especiallysince Halevy (2007), recognizes that decision making under uncertainty has much morediverse patterns than universal aversion typically assumed in theory: Attitudes toward
uncertainty are found to depend on such factors as event likelihood, the sign of thepayoff associated with the event, and familiarity with uncertainty sources (Trautmann& van de Kuilen 2015a, Kocher et al. 2018, Abdellaoui et al. 2011, Chew et al. 2012,
Baillon & Bleichrodt 2015, Baillon & Emirmahmutoglu 2018).6
Uncertainty attitudes are studied most commonly in the context of ambiguity preferences. As mentioned in the Introduction, much of the experimental literature on
ambiguity preferences (Halevy 2007, Chew et al. 2017, Kocher et al. 2018) follows theoriginal implementation by Ellsberg and formulates uncertainty in terms of artificial andnon-strategic events defined by the colors of balls drawn from urns.7 Attempts to mea5

The conjunction fallacy is usually explained by representativeness: The description of the conditioning event X is chosen to be representative of the properties of B, but not of A so that when evaluating
Pr(A ∩ B | X), attention is focused on B, but when evaluating Pr(A | X), A is discounted. In the famous Linda example in Tversky & Kahneman (1982), X is the statement “Linda is 31 years old, single,
outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned withissues of discrimination and social justice, and also participated in anti-nuclear demonstrations,” A isthe statement “Linda is a bank teller,” and B is the statement “Linda is active in a feminist movement.”
See also Gigerenzer (1996). Some authors use incentivized experiments to examine monotonicity in thecontext of the conjunction fallacy (e.g., Zizzo et al. 2000, Charness et al. 2010), but do not quantify the
degree of violation unlike in the present experiments.
6

The proportion of subjects whose behavior conforms to a major decision model varies across studies.
For example, the proportion of subjects whose behavior conforms to subjective expected utility is 15-20%
in Halevy (2007) and about 60% in Ahn et al. (2014). Chew et al. (2017) and Yang & Yao (2017) reportthat a substantial proportion of subjects violate subjective expected utility and other decision models
including maxmin and smooth ambiguity models.
7

See Trautmann & van de Kuilen (2015a) for a comprehensive survey of the extensive literature.

5sure ambiguity preferences when uncertainty comes from natural sources include Baillon& Bleichrodt (2015), and Baillon et al. (2018), who use uncertainty over stock market
indices and observe that familiarity with the source of uncertainty and/or time pressure
influence its perception, and Li et al. (2017), who use uncertainty over how other subjects
rank three alternatives.8
On the perception of uncertainty involved in strategic decision making, Camerer& Karjalainen (1994) observe that subjects prefer objectively risky bets to the play ofgames against another subject, and conclude that strategic uncertainty entails ambiguity. Eichberger et al. (2008) change the characteristics of players that subjects playgames against, and observe through their action choices that the opponents’ characteristics influence their perception of strategic uncertainty. Heinemann et al. (2009) elicit
from subjects playing threshold public good games the certainty equivalents of strategic
uncertainty that the threshold is achieved.
Our experiments are characterized by the following design choices. First, we use
multiple price lists and solicit matching probabilities as in Baillon & Bleichrodt (2015)
and Dimmock et al. (2016).9 This requires no assumption on risk or uncertainty attitudes
and allows for a direct and quantifiable measurement of the perception of uncertainty.
Second, the non-strategic uncertainty is created by subjects who fill balls into urns andreceive a flat payment for the task. The process hence is in line with the creation ofstrategic uncertainty, which results from the same subjects’ action choices in games.
This also addresses the concern that the observers may think that an experimenterattempts to control the payment when they fill urns themselves.10 Third, in contrast
to the majority of belief elicitation experiments which elicit beliefs from the subjects
who subsequently make decisions, observers in our experiments do not make decisions
themselves. This not only removes restrictions on the type of events whose matchingprobabilities can be elicited, but also eliminates concerns over the possible effect of belief
8Empirical evidence of source preferences includes a home bias in stock holdings (French & Poterba1991), and a trading volume bias based on the proximity of countries (Epstein & Schneider 2010). Ritov& Baron (1990) find public aversion to receiving vaccination which involves uncertainty over its intended
as well as side effects.
9

The concept of matching probability dates back to Raiffa (1968). See also Wakker (2010), Baillon
et al. (2018) and Li et al. (2019) for the measurement of uncertainty attitudes using matching probabilities.
10

See for example Al-Najjar & Weinstein (2009) and Dominiak & Duersch (2019).

6elicitation on the decisions.11,12

3

Experimental Design

3.1

OutlineThe experiments were conducted at Osaka University with subjects recruited from thestudent body through ORSEE (Greiner 2015) and the program coded using z-Tree (Fischbacher 2007).13 The experiments consist of two types of sessions as follows: The first
type is player sessions where subjects fill urns with colored balls and then play 2×2
games against each other. The second type is observer sessions where a new group of
subjects make “predictions” about the outcome of the player sessions. There are twoplayer sessions and sixteen observer sessions, and no subjects participated in more than
one session. In both types of sessions, no communication is allowed among the subjects.
The following is the description of the player and observer sessions in more detail.

3.2

Player Sessions

The player sessions took place in March 2018.14 Each player session had 22 subjects andconsisted of two parts: In Part 1, a subject was shown two urns on their screen, andasked to put one-hundred colored balls in each urn. Specifically, they can put red (R)and green (G) balls in Urn 1, and yellow (Y) and blue (B) balls in Urn 2. Any color
composition was allowed in each urn as long as the total number equaled one hundred.
A subject received a fixed payment of 500 JPY, or approximately 5USD, for this task.
Non-strategic uncertainty is hence created by the subjects themselves just as strategic
uncertainty is created by their action choices.
In Part 2, subjects formed pairs and played a sequence of 21 different 2×2 games.
These are seven parameterizations each of Prisoner’s Dilemma games (PD), Stag-Huntgames (SH), and Battle of the Sexes games (BoS). All payoffs are denominated in JPY.
The 21 games appeared in random order, but the order was the same for all pairs in
11

See Schlag et al. (2015) for a survey on the subject. On the effect of elicitation on the subsequentaction choice, belief elicitation is found to increase cooperative behavior in public good games (Gächter& Renner 2010), and reduce the choice of dominated actions (Hoffmann 2014). Hedging behavior is
discussed in Blanco et al. (2010).
12

Among those experiments that have players and observers as two separate groups of subjects are
Hyndman et al. (2009), Palfrey & Wang (2009), and Cason et al. (2020). See Section 5.5.
13

The subject pool contains more than 3,000 students of various majors. Recruitment advertisement
states that there is an opportunity to earn money in a research experiment.
14

Upon arrival in the lab, the subjects received an instruction sheet and a record sheet which they can
use to record their response. The instructions were read aloud by the same experimenter.

7

each session. After each round of play, random re-matching took place by the perfectstranger format. There was no feedback until after the session was over. At the end ofeach player session, one game was randomly chosen for payment, and the subjects were
informed of their earnings and the choice of their matched partner in the chosen game.
Payment was made in cash at the conclusion of each session, which lasted for about 90
minutes. Average earnings were 2617.4 JPY. We also elicited demographic informationof the participants of these sessions. The instructions for the player sessions stated thatthe outcome of these experiments would be used for the decision making of other subjects
in future experiments, but did not explicitly mention the nature of the observer sessions
so that their choices would not be affected by the existence of observers.
SHx
P1 \ P2

A2

A1
B1

x
0

PDx
P1 \ P2

B2
x
x

x
1

0
1

A1
B1

A2
1
1+x

B2
1
−x

−x
0

1+x
0

Table 1: Standardized SH and PD Games
Table 1 shows the SH and PD games with standardized payoffs (g1 (·), g2 (·)). SHx
and PDx refer to the SH and PD games, respectively, with parameter x. For SH, x is
the payoff associated with the safe action Ai , and the larger is its value, the lower isthe relative attractiveness of the payoff dominant NE (B1 , B2 ).15 In fact, (B1 , B2 ) is
the risk dominant equilibrium if x < 12 , and (A1 , A2 ) is the risk dominant equilibrium
if x > 21 . For PD, x is the temptation parameter, and we expect the lower cooperation
rate when x is larger as shown by Dal Bó & Fréchette (2018). The actual payoffs in
the experiments are the aﬀine transformations of these standardized payoffs: They are
given by 1600gi (·) + 100 for the SH games, and by 700gi (·) + 1000 for the PD games.
The player sessions had seven SH game with x = 0.2, . . . , 0.8 and seven PD games withx = 0.1, . . . , 0.7. Tables 2 and 3 depict four games SH0.2, SH0.8, PD0.1, and PD0.5
which were used in the observer sessions.
SH0.2
P1 \ P2
A1
B1

A2
420
100

SH0.8
P1 \ P2

B2
420
420

420
1700

100
1700

A1
B1

A2
1380
100

B2
1380
1380

Table 2: SH Games in Observer Sessions

15

Our choice of SH games is based on Rankin et al. (2000).

8

1380
1700

100
1700

PD0.1P1 \ P2
A1
B1

A2
1700
1770

PD0.5
P1 \ P2

B2
1700
930

930
1000

1770
1000

A1
B1

A2
1700
2050

B2
1700
650

650
1000

2050
1000

Table 3: PD Games in Observer Sessions

3.3

Observer Sessions

In the observer sessions which took place in April and November 2019, we elicit froma new group of subjects their predictions about the outcome of the player sessions. A
participant in these sessions is referred to as an observer in what follows.
The observers’ predictions are formally elicited in the form of matching probabilities(Baillon & Bleichrodt 2015, Baillon & Emirmahmutoglu 2018) defined as follows. Let y
be a real number and E be an event corresponding to the set of outcomes in the player
sessions. For any monetary reward y, denote by ym 0 the risky bet that pays out y withprobability m and 0 with probability 1 − m, and by yE 0 the uncertain bet that pays out
y when event E takes place and 0 otherwise. The matching probability m = mi (E) of
event E for observer i is a probability that makes him indifferent between ym 0 and yE 0:m = mi (E) ⇔ ym 0 ∼ yE 0. We say that observer i is probabilistically sophisticated (PS)if his matching probability mi (E) satisfies the law of probability. Let Ω be the universal
event so that mi (∅) = 0, and mi (Ω) = 1. For any events E, F ⊂ Ω,E ⊂ F ⇒ mi (E) ≤ mi (F ), and E ∩ F = ∅ ⇒ mi (E ∪ F ) = mi (E) + mi (F ).
With some abuse of notation, we denote by R the event in which a Red ball is drawn
from the first urn, and by G the event in which a Green ball is drawn from it. Events
Y (Yellow) and B (Blue) for the second urn are similarly defined. Likewise, we denoteby Ai and Bi the events in which player i chooses actions Ai and Bi , respectively. To
simplify notation, we also use EE ′ to denote the intersection E ∩ E ′ of two events E andE ′ so that RG = R ∩ G, A1 A2 = A1 ∩ A2 , etc.
Each observer session consists of three parts. Questions in Parts 1 and 2 of theobserver sessions elicit matching probabilities of events from the player sessions using
multiple price lists as shown in Figure 1, and described in more detail below. Part 1
has eight questions on urn events E ⊂ {R, G} × {Y, B}, and Part 2 has eight questions
on game events E ⊂ {A1 , B1 } × {A2 , B2 }.16 Table 4 shows the correspondence between
the events and the questions in Parts 1 and 2. Events are classified into three different
types as follows. Events are simple if they correspond to the color of a ball drawn from
16

The two urns used for Part 1 (i.e., one with R and G and the other with Y and B) are filled by two
different player subjects, and this is stated in the instructions of the observer sessions.

9Part 1

Part 2

Event Type

Question #Event

Question #Event

Simple

1
2
3
4

R
G
Y
B

1
2
3
4

A1
B1
A2
B2

Conjunctional

5
6

RY
GB

5
6

A1 A2
B1 B2

Diagonal

78

RY ∪ GBRB ∪ GY

78

A1 A2 ∪ B1 B2
A1 B 2 ∪ B 1 A2Table 4: Correspondence between questions and events in observer sessions
Treatment#Sessions(#Observers)

Action choices
by players (%A, %B)

Average total earnings
of observers in JPY (USD)

SH0.2
SH0.8
PD0.1
PD0.5

4 (24, 23, 23, 22)
4 (23, 24, 23, 20)
4 (21, 24, 22, 24)
4 (23, 23, 22, 24)(13.6, 86.4)(56.8, 43.2)(45.4, 54.6)(20.4, 79.6)

2096.3 (19.23)
1689.6 (15.50)
1641.2 (15.06)
1977.2 (18.14)

Notes. a) The male-to-female ratio among subjects (2.50) is close to that (2.58) of the undergraduate student body of Osaka University as of 1 May 2019. b) Conversion rate: 1 USD=109 JPY.

Table 5: Treatment summary for observer sessions.
a single urn or the action choice of a single player, conjunctional if they correspond tothe combination of colors of balls drawn from the two urns, or the action profile of thetwo players, and diagonal if they correspond to the union of two (disjoint) conjunctional
events.
For Parts 1 and 2, the reward y equals 1000 JPY.17 Section 3 has five questions thatelicit certainty equivalents of urn and game events and other measures of uncertainty
attitudes.18
There are four sessions each for the four treatments that correspond to the four games
SH0.2, SH0.8, PD0.1, and PD0.5 with a total of 365 observers participating. Table 5
presents the summary statistics of the observer sessions.
At the beginning of each observer session, an instruction sheet for each part was17

y = 9.17 USD according to the average exchange rate 1 USD=109 JPY in 2019.

18

The questions in Part 3 are as follows. Q1 elicits the certainty equivalent (CE) of a bet 100050 0; Q2
elicits n ∈ {−350, −300, ..., +150} which makes 100050 0 equivalent to a bet (25, 1500; 25, 500 + n); Q3
elicits the CE of a bet 1000G 0; Q4 elicits the CE of a bet 1000B1 0; Q5 elicits the CE of a bet 1000B1 B2 0.

10

Figure 1: Urn part decision screen for observer sessions.
distributed along with a record sheet that the subjects used to write down their responses.
These instructions included the reproduction of the instructions for the player sessions
so that there would be clear understanding among our observer subjects on how theuncertainties had been generated. All instructions were read aloud by a text-to-speech
software so as to control the time and tone of the articulation of the instructions acrosssessions. In addition, a snapshot of one of the player sessions was shown to informthat the questions asked would be about the behavior of other students from the same
university. Payments ranged from 500 JPY to 4000 JPY, including the show-up fee of
500 JPY. A typical observer session lasted for approximately 100 minutes.
Every question of the observer session has a similar structure. For illustration, Figure
1 shows Question 1 in Part 1 of the observer sessions. It elicits a matching probabilityof event R by presenting a choice between “Bet R”, which pays out 1000 JPY if R takes
place and zero otherwise, and “Bet z”, which pays out 1000 JPY with probability z/100,
where z = 0, 1, . . . , 100. The first screen shows a list of z’s with 10% increments, and thesecond screen presents a list of z’s with 1% increments based on the choice in the firstscreen. For example, if a switch from Bet R to Bet z takes place at z = 40 in the firstscreen, then the second screen zooms in to the list of z’s with z = 30, 31, . . . , 39, 40.19
We call this switching value of z an observer’s choice and identify it with his matchingprobability of the corresponding event.20 After the choice is made, a confirmation screen
19

In this case, the choices for z = 30 and z = 40 in the second screen are determined by the choicesin the first screen. If the choice is either z = 0 or z = 100 in the first screen, then the second screen isskipped and the confirmation screen appears.
20

More specifically, we follow Baillon & Emirmahmutoglu (2018) and convert the switching point to
the matching probability as follows. For any uncertain event E, if observer i switches from yE 0 to yz 0

11

appears and prompts the subjects to either confirm their choice or return to the firstscreen to reenter their choice for the question. The experimenter makes sure that no one
proceeds to the next question before everyone confirms his choice.

4

Sophistication IndicesWe begin by the description of probabilistic sophistication (PS) measures used in our
analysis. Application of the indices below to our data will be based on the considerationthat there is in principle no reason for the observers to distinguish between the two urns
or the two players. For this reason, we take the average of the matching probabilities ofsimple urn events R and Y which are the first color of each urn, and the average of thematching probabilities of G and B which are the second color of each urn. Likewise, we
use the average of the matching probabilities of simple game events A1 and A2 , and theaverage of the matching probabilities of B1 and B2 .21
miG+B = 12 (mi (G) + mi (B)),
mR+Y= 12 (mi (R) + mi (Y )),i
A1 +A21
1 +B2mi= 2 (mi (A1 ) + mi (A2 )), mB= 21 (mi (B1 ) + mi (B2 )).
i

4.1

(1)

Monotonicity

Monotonicity (MN) holds if the matching probability of an event is larger than that ofits sub-event. Specifically, for any events E and E ′ such that E ′ ⊂ E, define
M Ni (E, E ′ ) = mi (E) − mi (E ′ ).

(2)

Clearly, PS implies M Ni (E, E ′ ) ≥ 0. Our special focus is on the case when E is simple
and E ′ is conjunctional. Based on the combined matching probabilities introduced inB(1), we define the indices M N UiRY and M N UiGB for the urns and M N GA
i and M N Gi
as in Table 6. We also define M N Ui and M N Gi to be the combined measures as in
Table 6.
at z = 1, . . . , 100, then his matching probability is set equal to the mid-point between z − 1 and z. For
example, when i switches at z = 68, then we set mi (E) = 0.675. On the other hand, if subject i prefers
y0 0 to yE 0 for z = 0, or prefers yE 0 to y100 0 for z = 100, we censor it from data as an inconsistent choice.
Out of 5,840 (=365 subjects × 16 questions) elicited switching points, 5,397 responses (=92.4% of theinitial) survived this process. The proportion of censored data (7.6%) is comparable to that in Baillon &Emirmahmutoglu (2018) (6.2%) who also use the matching probability elicitation method. Our analysis
in what follows computes each index for a subsample of observations in which the matching probabilities
required for the computation of the index are available. See Table 15 in Appendix A for the size of thesubsample for each index.
21

See Section 5.1 for the examination of this point.

12IndexDefinition

PS implies≥0

M N UiM N GimR+Y− mi (RY )imG+B− mi (GB)i
miA1 +A2 − mi (A1 A2 )1 +B2
− mi (B1 B2 )mBimin {M N UiRY , M N UiGB }B
min {M N GA
i , M N Gi }

BC

BCUi
BCGi
BCUid
BCGdi

1 − mR+Y− mG+Bii
A1 +A2
1 +B2
1 − mi− mB
i
1 − mi (RY ∪ GB) − mi (RB ∪ GY )1 − mi (A1 A2 ∪ B1 B2 ) − mi (A1 B2 ∪ B1 A2 )

=0

QC

QCUi
QCGi

1 − mi (RY ) − mi (GB)1 − mi (A1 A2 ) − mi (B1 B2 )≥0CPCP UiRY
CP UiGB
CP GA
i
CP GB
i

mi (RY ) − mi (R)mi (Y )mi (GB) − mi (G)mi (B)mi (A1 A2 ) − mi (A1 )mi (A2 )mi (B1 B2 ) − mi (B1 )mi (B2 )=0MN

M N UiRY
M N UiGBM N GAi
M N GBi

1 +B2
Notes. mR+Y
, mG+B, miA1 +A2 , and mB
are defined in (1).
iiiTable 6: Definitions of indices

4.2

Binary Complementarity

Binary complementarity (BC) as proposed by Baillon & Bleichrodt (2015) measuresthe deviation from unity of the sum of the matching probabilities of an event and its
complement: Let E c = Ω \ E denote the complement of event E, and define the BC of
event E by
BCi (E) = 1 − mi (E) − mi (E c ).
(3)
Clearly, PS implies BCi (E) = 0 for any event E. On the other hand, BCi (E) > 0
corresponds to ambiguity aversion, and BCi (E) < 0 corresponds to ambiguity seeking.22
Define BCUi and BCGi for the combined simple events and BCUid and BCGdi for the
diagonal events as in Table 6.

4.3

Quasi-Complementarity

Quasi-complementarity (QC) is a weaker version of BC in the sense that it measures thedifference between unity and the sum of matching probabilities of two disjoint events(whose union may not be universal):
QCi (E, E ′ ) = 1 − mi (E) − mi (E ′ )22

See for example Baillon & Bleichrodt (2015).

13

for E, E ′ with E ∩ E ′ = ∅.

Clearly, PS implies QCi (E, E ′ ) ≥ 0. We will use QC to evaluate the weights placed on
the two conjunctional events and define QCUi and QCGi as in Table 6.

4.4

Coordination Premium

In the player sessions, the paired subjects make their choices simultaneously with nocommunication between them. As such, their choices are independent.23 We test if thematching probabilities observe the independence of choices. Specifically, for non-empty
subsets E and E ′ , we consider the coordination premium (CP) index defined byCPi (E, E ′ ) = mi (EE ′ ) − mi (E)mi (E ′ ).

(4)

CPi = 0 when E and E ′ are independent according to mi (mi (EE ′ ) = mi (E)mi (E ′ )),and is positive when E and E ′ are positively correlated according to mi . We will test
this when E and E ′ are both simple and EE ′ is conjunctional, and define CP UiRY andB
CP UiGB for the urns and CP GA
i and CP Gi for the games as in Table 6.
For each index introduced above, our aggregate analysis is based on the sample
average expressed without the superscript i. That is, when we denote by N the number∑∑
RY , BCG = 1
of observers, we let M N U RY = N1
i M N Ui
i BCGi and so on.
N

5

Results

5.1

Distribution of Matching Probabilities

Table 7 shows the frequency of each action profile realized in the player sessions foreach treatment along with a few statistics. As seen, the action choice is concentratedon (B1 , B2 ) in SH0.2 and PD0.5, whereas it is evenly spread over four profiles in SH0.8
and PD0.1.
Table 8 lists the mean matching probabilities of each event along with the standard
deviations. Table 16 in Appendix A shows that no statistical difference is found except
in a few cases between the matching probabilities of two events which differ only in theidentities of the urns or the players.24
23

However, even when the choices in each pair are independent, the choices may appear correlated
when they are aggregated over all pairs. This point is discussed in the next section.
24

Based on the two-sided t-test, the difference is significant in one case at p < 0.05, and in two cases
at p < 0.10.

14Treatment

SH0.2

SH0.8

PD0.1

PD0.5

Action

A2

B2

A2

B2

A2

B2

A2

B2

A1
B1

1
3

1
17

8
3

6
5

4
4

8
6

1
2

5
14

Coord. ratep(A1 A2 )p(B1 B2 )p(A1 A2 ) + p(B1 B2 )

0.045
0.773
0.818

0.364
0.227
0.591

0.182
0.273
0.455

0.045
0.636
0.682

Entropy
H

1.085

1.917

1.936

1.418

Correlation
ρ

0.261

0.189

-0.0690.054#obs. of x
#player pairs .

Notes. For each action profile x, p(x) =Entropy∑
H = − x p(x)log2 p(x). A higher entropy implies more dispersion.
p(A1 A2 )−p(A1 )p(A2 ).
Correlation ρ = √2
2{p(A1 )−p(A1 ) }{p(A2 )−p(A2 ) }Table 7: Realized action profile distributions by treatment
Treatment

Urn events# Observers

m(R)

m(G)

m(Y )

m(B)

m(RY )m(GB)m(RY ∪ GB)

m(RB ∪ GY )

SH0.2
92
SH0.8
90
PD0.1
91
PD0.5
92

0.54
(0.12)
0.54
(0.1)
0.53
(0.1)
0.53
(0.13)

0.53(0.12)
0.51(0.09)
0.53
(0.1)
0.52
(0.14)

0.52(0.12)
0.52(0.11)
0.52(0.11)
0.51(0.14)

0.55(0.11)
0.53(0.12)
0.52(0.11)
0.53(0.14)

0.31(0.13)
0.3(0.12)
0.3(0.11)
0.32(0.13)

0.31(0.13)
0.32(0.12)
0.31(0.12)
0.32(0.13)

0.5(0.14)
0.5(0.13)
0.47(0.14)
0.48(0.13)

0.5(0.14)
0.5(0.14)
0.47(0.14)
0.48(0.13)

Treatment

Game events# Observersm(A1 )

m(B1 )m(A2 )

m(B2 )m(A1 A2 )m(B1 B2 )m(A1 A2 ∪ B1 B2 )m(A1 B2 ∪ B1 A2 )

SH0.2
92
SH0.8
90
PD0.1
91
PD0.5
92

0.27
(0.26)
0.5(0.36)
0.45(0.33)
0.43(0.29)

0.75(0.24)
0.47(0.33)
0.55(0.31)
0.66(0.25)

0.33
(0.3)
0.5
(0.34)
0.46(0.32)
0.41(0.26)

0.73(0.25)
0.54(0.32)
0.53(0.31)
0.65(0.24)

0.27(0.26)
0.45(0.32)
0.45(0.31)
0.38(0.28)

0.71(0.27)
0.49(0.33)
0.44(0.29)
0.59(0.27)

0.78(0.21)
0.79(0.17)
0.7(0.22)
0.69
(0.2)

0.32
(0.24)
0.31(0.21)
0.38(0.19)
0.4(0.22)

Notes. Standard deviation in parentheses.

Table 8: Average matching probabilities by treatment and by event
Figures 2, 3, and 4, and 5 display the distributions of matching probabilities of simple

15

and conjunctional events.25 As immediately seen, there is a clear difference between
the distributions for the urns and those for the games. In the case of the urns, thedistributions have spread along the 45-degree line, suggesting that the observers agree
that each color or color combination is equally likely. In the case of the games, on theother hand, the distributions have large two-dimensional spread in each treatment.
25

Figures 12 and 13 in Appendix A show the distributions of (mi (RY ∪ GB), mi (RB ∪ GY )) and(mi (A1 A2 ∪ B1 B2 ), mi (A1 B2 ∪ B1 A2 )), respectively.

16

Notes. The red square represents the average point.

Figure 2: (mR+Y, mG+B): all treatments combinediiNotes. The red square in each panel represents theaverage point.
1 +A2
1 +B2
Figure 3: (mA, mB) by treatmentii

17

The red square represents theaverage point.

Figure 4: (mi (RY ), mi (GB)): all treatments combined

The red square in each panel represents the average
point.

Figure 5: (mi (A1 A2 ), mi (B1 B2 )) by treatment

5.2

Binary Complementarity

Observations on BC are summarized as follows.
Result 1i) (Ambiguity seeking attitudes) BCU < 0 in every treatment, whereas
18

BCG < 0 in three out of four treatments.
ii) (Familiarity effect) BCGi tends to be negative when observer i perceives the game
to be easier to predict than the urn.
Ambiguity seeking attitudes The negative 45-degree line in Figures 2 and 3 corresponds to PS BCUi = 0 and BCGi = 0, respectively. As seen, PS is violated forthe majority of the observations. Furthermore, the patterns of violation are surprisingly
different between the two uncertainty sources: While strong positive correlation exists
between the two matching probabilities for the urns, negative correlation exists between
them for the games.26 More specifically, the spread along the 45-degree line for theurns suggests that the observers consider each color equally likely but do not necessarily
think that their probability equals 0.5. In contrast, the large spread orthogonal to the
45-degree line for the games suggests that the observers differ substantially in their assessment of the likelihood of the two actions. On the other hand, Table 17 in Appendix
A checks whether or not the observers who are nearly PS with |BCUi | ≤ 0.1 for theurns are more likely to have |BCGi | ≤ 0.1 for the games, and find correlation for SH0.8
and PD0.5 (p < 0.05, χ2 test). On average, the observers are ambiguity seeking for theboth uncertainty sources with BCU < 0 in every treatment and BCG < 0 except in
PD0.1 (p < 0.05) as seen in Table 9. This observation corroborates findings in the recent
experimental studies using matching probability elicitation (e.g., Baillon & Bleichrodt
2015, Baillon et al. 2018, Trautmann & van de Kuilen 2015b, Li et al. 2019).27
Kocher et al. (2018) survey the literature to point out that ambiguity seeking isparticularly prevalent for low likelihood events. The diverse matching probabilities ofthe simple game events reported by our observers allow us to test this hypothesis forstrategic uncertainty. Specifically, we take a small δ > 0 and identify observer i with
1 +A2
1 +B2
min {mA, mB} < δ as having low likelihood assessment of either simple gameiievent. We then test if their average BCG is different from that of the observers with
1 +A2
1 +B2
min {mA, mB} ≥ δ. The result is at odds with Kocher et al. (2018), and
i
i
1 +A2
1 +B2
indicates that the observers with min {mA, mB} < δ are less ambiguity seekingii
A1 +A2B1 +B2than the observers with min {mi, mi} ≥ δ for different values of δ ≤ 0.25.
Familiarity effect Heath & Tversky (1991), Fox & Tversky (1995) and subsequentstudies show that decision makers are relatively more ambiguity-seeking when they face
26

The Pearson correlation equals 0.640 for the urns and −0.784 for the games.

27

Trautmann & van de Kuilen (2015b) use various elicitation methods to conclude that the sum of
elicited probabilities of complementary events exceeds 1 on average. Table 1 of Oechssler & Roomets(2015) displays heterogeneity in ambiguity attitudes reported in the literature.

19SH0.2Treatment
SH0.8
PD0.1

PD0.5

ANOVA for difference in
BC among treatments

BCU-0.080***(0.188)-0.044***(0.136)-0.053***(0.173)-0.047**(0.206)p = 0.508

BCG-0.044**(0.167)-0.068***(0.190)-0.038(0.194)-0.104***(0.198)p = 0.156

BCU − BCG
#Obs.

-0.054*
65

0.026
62

-0.02164

0.061**
70

Notes. a) * p < 0.10, ** p < 0.05, and *** p < 0.01. b) BC = 0 is tested using two-sided ttest. c) Standard deviation in parentheses. d) The difference is computed for the subsample
in which both terms are available, is tested using two-sided t-test.

Table 9: BC by treatment and by uncertainty source
uncertainty from sources they feel more familiar or competent with.28 In order to determine which uncertainty source each observer finds more familiar, we use their response to
the post-experimental questionnaire and suppose that observer i is more familiar with thegames if he states that the games are easier to predict, and vice versa. Our hypothesis isthat BCGi < BCUi if observer i finds the game more familiar, and BCGi > BCUi otherwise.29 Formally, let Fi ∈ {u, g} denote observer i’s response to the post-experimental
questionnaire: Fi = u if observer i states that the urn was easier to predict, and Fi = gotherwise. Let BCU u and BCU g be the mean BCU ’s for the urns of the observers withresponses Fi = u and g, respectively, in the questionnaire. Define BCGu and BCGg forthe game similarly. Our hypotheses are
H0BCU : BCU u = BCU gand

H1BCU : BCU u < BCU gand

H1BCG : BCGg < BCGu

for the urns and
H0BCG : BCGg = BCGu
for the games.30
28

The literature uses words familiar, knowledgeable, and competent interchangeably. A classic field
example is the so-called home bias in stock markets (French & Poterba 1991): Investors’ portfolio has
too much weight on a domestic market even when a higher return is expected in overseas markets.
29

The specification of familiarity resembles that by Heath & Tversky (1991), who classify subjects basedon their stated familiarity with football and politics. Subsequent literature uses exogenous specificationof familiarity according to the subjects’ background. For example, Chew et al. (2012) and Baillon &Bleichrodt (2015) use geographical proximity as a proxy of familiarity. Fox & Weber (2002) show thatwhen a pair of uncertain bets are presented simultaneously, subjects’ attitudes toward the same bet
change depending on whether or not they are more familiar with the uncertainty source of the other bet.
30

The numbers of available observations are as follows: 106 (BCU u ), 235 (BCU g ), 92 (BCGu ), and
170 (BCGg ).

20

The results corroborate the finding in the literature. More precisely, H0BCG is rejected(by two-sided t-test) when data from the four treatments are combined (t = −2.751,
p = 0.006), and also for SH0.2 (t = −1.695, p = 0.0949) and SH0.8 (t = −3.3438,
p = 0.001) when data for each treatment are tested separately. In contrast, H0BCU is not
rejected (t = −1.028, p = 0.305) when all urn data are combined.31 In Appendix B, wepresent further examination on the effect of familiarity in conjunction with the entropyof play.

5.3

Monotonicity

Our primary findings on MN can be summarized as follows:
Result 2i) (Coordination fallacy) M N U RY , M N U GB > 0 in every treatment,
while M N GA and M N GB are often indistinguishable from zero, and nearly oneB
half of the observers have M N Gi = min {M N GA
i , M N Gi } < 0.
ii) (Association between urns and games) There is no correlation between M N Ui and
M N Gi in every treatment.
iii) (Eﬀicient versus ineﬀicient profile) M N G is indistinguishable from zero more often
for the Pareto eﬀicient profile than for the ineﬀicient profile.

Coordination fallacy Figures 6 and 7 present scatter plots of matching probabilities
of simple and conjunctional events for the urns and games, respectively. The region
below the 45-degree line corresponds to PS: M Ni ≥ 0. The difference between the urns
and games is clear: While a predominant majority of observers lie below the 45-degreeline for the urns, a substantial proportion lies above the line for the games. In fact,
M N Gi < 0 in 48.5% (= 126/260) of the cases in the subsample in which both M N GAi
32 (Figure 7). Table 10 shows that the average M N G for
and M N GB
canbecomputed.
ithe games is indistinguishable from zero in many cases.33
31

On the other hand, perceived familiarity creates no difference between M N U and M N G, or CP U
and CP G.
32

See Footnote 20.

33

Appendix C uses data from Part 3 to confirm that the violation of monotonicity is also substantial interms of certainty equivalents for some pair of game events: For many observers, the certainty equivalentof the bet 1000B1 B2 0 is larger than that of the bet 1000B1 0.

21

Notes. The red square in each panel represents theaverage point.

Figure 6: (mR+Y, mi (RY )) (left) and (mG+B, mi (GB)) (right): all treatments combinediiNotes. The red square in each panel represents the average point.
1 +A2
Figure 7: (mA, mi (A1 A2 )) (left) and (miB1 +B2 , mi (B1 B2 )) (right) by treatmenti

Association between M N Ui and M N Gi We may suspect that the violation ofmonotonicity is due to some hidden characteristics of individual observers, and thatobservers with those characteristics violate monotonicity whether the event is strategic
or not. An observer’s cognitive ability is one possible candidate for such characteristics.
22SH0.2Treatment
SH0.8
PD0.1

PD0.5

0.223***(0.116)0.235***(0.114)0.222***(0.108)0.198***(0.127)0.229***(0.103)0.220***(0.110)0.206***(0.123)0.205***(0.137)

M N U RY − M N U GB
#Obs.

-0.01090

0.021**
87

0.012
85

0.008
88

M N GA

0.026(0.140)0.037**(0.151)

0.068**(0.173)0.009(0.189)0.005(0.185)0.102***(0.182)0.038*(0.187)0.074***(0.145)-0.01366

0.078**
62-0.101***61-0.03471

M N U RYM N U GBM N GBM N GA − M N GB
#Obs.

Notes. a) * p < 0.10, ** p < 0.05, *** p < 0.01. b) M N = 0 is
tested using two-sided t-test. c) Standard deviation in parentheses.
d) The difference is computed for the subsample in which both terms
are available (Footnote 20, Table 15), and is tested using two-sided
t-test. d) Bold indicates the Pareto action profile (B1 B2 in SH andA1 A2 in PD).

Table 10: M N by treatment and by uncertainty source
For example, Chew et al. (2018) observe that subjects with better math skills are more
ambiguity averse, suggesting correlation between the subjects’ cognitive abilities andtheir BC values. Likewise, Oechssler et al. (2009) find that subjects with high CRT
scores (≥ 2) are less likely to exhibit the conjunction fallacy. In a similar vein, we
may conjecture that an observer with a low cognitive ability may violate monotonicity
both for the urns and games.34 Table 11 tests this conjecture by classifying observers
based on the signs of their M N Ui and M N Gi as defined in Table 6. While M N Ui ≥ 0
for 93.4% (= 327/350) of observers, M N Gi ≥ 0 for only 51.5% (= 134/260) of them.
Furthermore, approximately one half of those observers with M N Ui ≥ 0 have M N Gi < 0in every treatment. As shown in the bottom line of Table 11, we cannot reject the null
hypothesis that the likelihood of violation for one uncertainty source is independentof that of the other source (Fisher’s exact test). In other words, this shows that theviolation of monotonicity is strongly specific to strategic uncertainty and not specific tothe observers.35
34

The 742 subjects who were recruited from the same subject pool at Osaka University for an (unrelated) online experiment conducted in August 2020 scored on average 2.51 points for the three-item
CRT. We should note that this score is higher than the average (2.05 points) of the high-score group of
Oechssler et al. (2009).
35

We find no clear evidence on the relationship between BCGi and M N Gi either. For example, theaverage BCG value of the subjects with M N Gi ≥ 0 is not statistically lower than the corresponding

23SH0.2

M N Ui

+
−

#Obs.
p

+
33
1

−
29
2

SH0.8
+
27
1

65
0.602

PD0.1

M N Gi
− +
29 31
3
0

60
0.616

−
28
1

60
0.483

PD0.5
+
37
1

−
27
4

69
0.166

Notes. a) Two-sided Fisher’s exact test for independence of observations in a 2 × 2 contingent table.36b) M N Ui and M N Gi are computed for the subsamples in which every matching probability involved isavailable (Footnote 20, Table 15).

Table 11: Source dependence of monotonicity
Eﬀicient versus ineﬀicient profiles A closer look at Table 10 reveals that M N G for
the Pareto eﬀicient action profile (indicated by bold) is indistinguishable from zero more
often: M N GA = 0 cannot be rejected for PD0.1, and M N GB = 0 cannot be rejected
for SH0.8. To examine this point more closely, we classify observers i based on the signs
Bof M N GA
i and M N Gi in Table 12. In line with the above observation, it shows thatthe violation of monotonicity is more likely when it involves the Pareto action profile inboth SH and PD. Formally, let νA be the number of subjects who violates monotonicity
for A1 A2 only, and νB be the number of subjects who violate monotonicity for B1 B2
only:BνA = #{i : M N GA
i < 0 and M N Gi ≥ 0},B
νB = #{i : M N GA
i ≥ 0 and M N Gi < 0}.

The relative degree of violation of monotonicity between the two profiles is captured by
η=

νB
.
νA

We see from Table 12 that η < 1 in both PD games whereas η > 1 in both SH games.
In fact, the hypothesis νA = νB is rejected for SH0.8 and PD0.5 (p = 0.065 for SH0.8
and p = 0.089 for PD0.5, one-sided McNemar test).37
value of the subjects with M N Gi < 0.
36

Chi-square test is inappropriate given that the number of observations is less than five in some cell.

37

The alternative hypothesis is νB > νA for SH and νA > νB for PD. The McNemar test is a versionof chi-square test for binary matched pairs. See Moffatt (2015).

24

SH0.2

M N GA
#Obs
χ2
p

+
−

+
34
11

−
12
9

66
0.04
0.417

SH0.8

PD0.1

M N GB+ − +29 18 32
10
5 16
62
2.29
0.065

−
10
3

61
1.38
0.120

PD0.5
+
40
17

−
10
4

71
1.81
0.089

Notes. p values are based on the one-sided McNemar
test.
Table 12: M N values for symmetric profiles

5.4

Quasi-Complementarity

Result 3 QCGi < 0 for about a half of the observations for the games, whereas QCUi <0 is rare for the urns.
In Figures 4 and 5, the negative 45-degree line corresponds to QCUi = 0 or QCGi = 0
so that any point above this line presents a violation of QCUi , QCGi ≥ 0. According to
Figure 4, only 5.0% (=18/361) of the observations fall in the region QCUi < 0 for theurns. We also see strong positive correlation between mi (RY ) and mi (GB) (Pearson
correlation 0.737, p < 0.01). We have a completely different picture when it comes togame prediction. In Figure 5, 44.9% (=132/294) of the observations fall into the region
QCGi < 0 for the games, and strong negative correlation exists between mi (A1 A2 ) andmi (B1 B2 ) (Pearson correlation: −0.594 (SH0.2), −0.782 (SH0.8), −0.666 (PD0.1), and−0.400 (PD0.5), p < 0.01 for each treatment.)

5.5

Coordination Premium and Correlation

Result 4i) (Positive CP for the games) CP GA , CP GB > 0 in every treatment.
Moreover, CP G tends to be larger for the Pareto eﬀicient profile.
ii) (Ambiguous correlation) Based on comparison between the simple and diagonal
events, approximately half of observers have assessment incompatible with independence of the two players’ actions.
Positive CP for the games Figures 8 and 9 plot the matching probability of asymmetric profile against the product of the matching probabilities of the corresponding
elements. The vertical distance between each point and the 45-degree line in Figure 9
Bequals the coordination premium CP Gi (CP GA
i in the upper panels and CP Gi in the25lower panels). As seen, the predominant majority of observations lie above the 45-degreeline in most panels. Table 13 presents the mean CP values for both games and urns,
and rejects the null hypotheses CP GA = 0 and CP GB = 0 in all four treatments (twosided). Despite the fact that the observers were shown the instructions of the player
sessions which stated that it adopted perfect stranger matching, hence, they believe inthe presence of some mechanism that helps the players coordinate their actions. We
also see that this tendency is stronger for the Pareto eﬀicient profile in three out of
the four treatments (SH0.8, PD0.1 and PD0.5). A closely related observation is madeby Cason et al. (2020), who elicit observers’ likelihood rankings over the four outcomesof 2 × 2 games played by others, and conclude that the result is consistent only withpositive correlation of the action choices by the two players. In contrast, CP U RY andCP U GB for the urns are positive in only four out of eight cases, and are much smaller
in magnitude than CP GA and CP GB for the games.

26

Notes. The red square in each panel represents the average
point.

Figure 8: (mi (R)mi (Y ), mi (RY )) and (mi (G)mi (B), mi (GB)): all treatments combined

Notes. The red square in each panel represents the average point.

Figure 9: (mi (A1 )mi (A2 ), mi (A1 A2 )) and (mi (B1 )mi (B2 ), mi (B1 B2 )) by treatment27

SH0.2Treatment
SH0.8
PD0.1

PD0.5

0.018
(0.115)0.006(0.122)0.024**(0.110)0.050***(0.130)0.0131(0.107)0.023*(0.117)0.0346**(0.137)0.034**(0.148)CP U RY − CP U GB
#Obs.

0.011
90

-0.023**87-0.01485-0.00888

CP GA

0.122***(0.140)0.074***(0.186)0.152***(0.193)0.154***(0.185)CP GB

0.106***(0.179)0.171***(0.240)0.066***(0.183)0.109***(0.151)CP GA − CP GB
#Obs.

0.021
66

-0.112**62

0.088**
61

0.048*
71

CP U RY
CP U GB

Notes. a) * p < 0.10, ** p < 0.05, *** p < 0.01. Standard deviations
in parentheses. b) CP = 0 tested using two-sided t-test. c) The difference is computed for the subsample in which both terms are available(Footnote 20, Table 15), and is tested using two-sided t-test. d) Bold
indicates the Pareto action profile (B1 B2 in SH and A1 A2 in PD).

Table 13: CP by treatment and by uncertainty source
Ambiguous correlation Epstein & Halevy (2019) examine preferences over bets on
the colors of balls drawn from two urns, and find violation from PS in terms of theassessment of the likelihood of simple and diagonal events. Define
1 +B2
OGmax = max {miA1 +A2 , mB},i
A1 +A2B1 +B2
OGmin = min {mi, mi},
T Gmax = max {mi (A1 A2 ∪ B1 B2 ), mi (A1 B2 ∪ B1 A2 )},T Gmin = min {mi (A1 A2 ∪ B1 B2 ), mi (A1 B2 ∪ B1 A2 )}.

If an observer’s assessment satisfies binary complementarity for both the simple anddiagonal events, we should have BC = BC d (= 0), or equivalently, OGmax + OGmin =T Gmax + T Gmin . This implies that under PS, if OGmax < T Gmax , then OGmin > T Gmin ,and if OGmax > T Gmax , then OGmin < T Gmin . Furthermore, under PS, if an observer
believes that the two players’ action choices are independent, then OGmax ≥ T Gmax
and OGmin ≤ T Gmin should hold although the reverse implication is not true. In ournotation, the two non-PS preference patterns reported by Epstein & Halevy (2019)
are: “two ≻ one” with OGmax < T Gmax and OGmin < T Gmin , and “one ≻ two” with
OGmax > T Gmax and OGmin > T Gmin .38 Similar definitions apply to the preference
38“two ≻ one” implies BCGd < BCG and “one ≻ two” implies BCG < BCGd .

28

patterns for the urns.39
For the urns, “one ≻ two” holds for the average matching probabilities (p < 0.05,
paired t-test), and for about 40% of observers as seen in Table 18 in Appendix A inline with the findings of Epstein & Halevy (2019). The proportion of observers withassessment compatible with independence is slightly less for the urns. For the games,on the other hand, neither “one ≻ two” nor “two ≻ one” holds for the average matchingprobabilities when data from all the treatments are combined. This is mostly consistent with the decomposition in Table 18. The proportion of observers with assessment
compatible with independence is approximately one half, and the other one half shows
various patterns with the proportion of “two ≻ one” the highest among them in threetreatments.

6

Approximate Sophistication

Even when the matching probabilities of an observer exhibit violation of PS, the degreeof violation may be relatively small. In such a case, we may interpret the violation asa result of trembling in the choice of a response in the multiple price list rather thanfundamental deviations from probabilistic thinking. We investigate this possibility byrelaxing the (in)equality defining sophistication for each index by a fixed constant k, andcall observer i k-probabilistically sophisticated (k-PS) if his matching probabilities misatisfy the modified inequality. More specifically, define
kiBCU = |BCUi |, kiM N U = max {0, −M N Ui }, kiQCU = max {0, −QCUi },kiBCG = |BCGi |, kiM N G = max {0, −M N Gi }, kiQCG = max {0, −QCGi }.
kiBCU and kiBCG measure the extents to which observer i deviates from BCUi = 0 and
BCGi = 0, respectively, in either direction. Likewise, kiM N U , kiM N G , kiQCU and kiQCG
equal zero if observer i is PS in terms of the respective indices, but take positive values
otherwise and hence measure the extent of deviation from PS. For k ≥ 0, we say thatan observer i is
BCU ≤ k


k-BC

 ki
kiM N U ≤ k .
k-MN for the urns if k-QC k QCU ≤ ki
An observer is k-PS for the urns if he is k-BC, k-MN, and k-QC for the urns. k-PS forthe games is similarly defined. When k = 0, 0-PS is equivalent to PS elsewhere in theanalysis. Figure 10 shows the proportions of observers who are k-PS for the urns andfor the games as we vary k. As seen, the patterns are strikingly distinct between the
39

The definitions are based on OUmax = max {mR+Y, mG+B}, OUmin = min {miR+Y , mG+B},iii
T Umax = max {mi (RY ∪ GB), mi (RB ∪ GY )}, and T Umin = min {mi (RY ∪ GB), mi (RB ∪ GY )}.

29two uncertainty sources. In particular, the k-PS curve coincides with the k-BC curve forthe urns, implying that k-BC is the defining condition for k-PS. For the games, on theother hand, the k-PS curve is below the other three curves by a significant margin for
k = 0.01, . . . , 0.2 (p < 0.10 against each of k-BC, k-MN and k-QC, t-test), implying thatthe three indices bind for different observers. Among them, k-BC is least likely satisfied.
Between the two uncertainty sources, the proportion of k-PS observers is significantly
lower for the games than for the urns for each value of k (k = 0.01, 0.02, . . . , 0.2, p < 0.05,
t-test).
Figure 11 shows the proportion of k-PS for the games by index and by treatment.
While the difference across treatments is rarely significant, we see that the treatments
are ordered remarkably consistently across indices with PD0.5 at the bottom and either
SH0.2 or PD0.1 at the top for the most part. We also notice that for most values of k,
PD0.1 lies above PD0.5 for every index, and SH0.2 lies above SH0.8 except QCG.

(a) Urns(b) Games

Notes. The horizontal axis is marked by 100k.

Figure 10: k-PS by source
How does the degree of PS for the urns of a given observer correlate with the degreeof his PS for the games? Table 19 in Appendix A presents OLS regressions of the degrees
of deviation from PS for the games on those for the urns.40 As seen, correlation tends
to be positive although significance is weak (if any) except for MN.41
40

The three models on the right include the dummy 1{Fi =g} which equals one if an observer statesthat the game was easier to predict.
41

In contrast with the finding in Section 5.3 that the tendency of an observer to violate MN for the30Notes. The horizontal axis is marked by 100k.
Figure 11: k-PS by treatment

31

7

A Model of the Observers’ Matching Probabilities

It is diﬀicult to give a definitive account for the difference in the patterns of violationof PS, and monotonicity in particular, between the urns and games. As one attempt,
we hypothesize here that an observer’s matching probability mi (E) of any event E iscomputed in reference to some event which varies with E. In the terminology of thetheory of ambiguity preferences, we suppose that the predictive priors have moving
support as event E varies. Formally, for any event E, the reference event of E for i,denoted Λi (E), is the event which observer i uses as a reference when computing thematching probability of E. We assume that E ⊂ Λi (E) so that a reference event always
contains the event in question.
Given event E and its reference event Λi (E), we assume that i’s matching probabilitymi (E) is given byσi (E)mi (E) =,σi (Λi (E))
where σi is the support function defined over the collection of events: σi (E) > 0 is thesignificance of the episodes that observer i associates with event E, and is similar to
the “support value” of event E as assumed by Tversky & Koehler (1994). We assume
that σi is monotone if σi (E) ≥ σi (E ′ ) if E ⊃ E ′ , but do not necessarily assume that it
is additive. It follows that we may have mi (E) + mi (E ′ ) ̸= 1 even when E and E ′ arecomplementary to each other.42
We hypothesize that the reference event of each event is given as follows:a) (simple) Λi (E) = Ω if E is a simple urn or game event.
b) (conjunctional) Λi (RY ) = Λi (GB) = Ω, Λi (A1 A2 ) = Λi (B1 B2 ) = A1 A2 ∪ B1 B2 .
c) (diagonal) Λi (E) = Ω if E is a diagonal urn or game event.
The only difference between the games and urns is hence the reference events of the
conjunctional events (b): While the observers take into consideration all four possiblecombinations of ball colors when contemplating on a conjunctional urn event, they take
into consideration only the two coordination profiles when contemplating on a conjunctional game event. In this formulation, the violation of monotonicity for the games is notgames is independent of whether or not he violates MN for the urns, the positive correlation between
kiM N U and kiM N G in Table 19 shows that an observer who violates MN by a large margin for the urns
tends to do so for the games as well. The positive correlation between kiBCU and kiBCG in Model (1) of
Table 19 corresponds to the finding in Table 17.
42

For example, even if the reference set of both A1 and B1 equals Ω (Λi (A1 ) = Λi (B1 ) = Ω), we have
BCi (A1 ) = 1 − mi (A1 ) − mi (B1 ) < 0 if σi is sub-additive (σi (A1 ) + σi (B1 ) < σi (Ω)), and BCi (A1 ) > 0
if σi is super-additive (σi (A1 ) + σi (B1 ) > σi (Ω)).

32

inconsistent with the monotonicity of σi . Furthermore, it is consistent with the negative
correlation between mi (A1 A2 ) and mi (B1 B2 ) found in Figure 5 in Section 5.4.43
Under the above hypothesis, quasi-complementarity QCGi = 1−mi (A1 A2 )−mi (B1 B2 )is similar to binary complementarity BCGi in the sense that A1 A2 and B1 B2 together
constitute the reference event of each conjunctional event just like A1 and B1 together
constitute the reference event of each simple event. In this sense, we would obtain indirect
and partial support for our hypothesis if QCGi and BCGi correlate with each other at
the individual level. Table 14 confirms this by classifying observers based on their valuesof QCGi and BCGi as well as BCGdi for the diagonal events. The top panel shows that
correlation is positive between the signs of BCGi and QCGi in every treatment (p < 0.10,
Fisher’s exact test). Under the above hypothesis, this implies that an observer’s support
function σi tends to be sub-additive for simple events (σi (Ω) < σi (A1 ) + σi (B1 )) if andonly if it is so for conjunctional events (σi (A1 A2 ∪B1 B2 ) < σi (A1 A2 )+σi (B1 B2 )).44 The
middle panel in Table 14 uses BCGdi for diagonal events and finds the same tendency at
p = 0.05 except in PD0.5. The bottom panel of Table 14 checks if the observers tend tohave QCGi < 0 if they found the outcome of the games easier to predict (i.e., Fi = g) as
in the case of BCGi in Result 1(ii) on binary complementarity in Section 5.2. It showsthat this is true in SH0.2 (p < 0.05) and marginally so in SH0.8 (p = 0.126).45 This is
in sharp contrast with Table 20 in Appendix A, where no correlation is found between
the signs of BCUi and QCUi .

8

Conclusion

The elicited matching probabilities violate the law of probability including monotonicity
and additivity for both strategic and non-strategic events, but the patterns of deviations
are substantially different between the two uncertainty sources. The most stark difference
between the non-strategic and strategic events is the coordination fallacy in terms of themonotonicity (MN) index. The violation of monotonicity (i.e., strictly negative MN
43

1 A2 )
1)
Note that mi (A1 A2 ) > mi (A1 ) ⇔ σi (Aσ1i (A> σσii(Acan hold even when σi (Ω) > σi (A1 ) >A2 ∪B1 B2 )(Ω)
σi (A1 A2 ) and σi (Ω) > σi (A1 A2 ∪ B1 B2 ) > σi (A1 A2 ).

44

By (b), we havemi (A1 A2 ) =σi (A1 A2 )σi (A1 A2 ∪ B1 B2 )and mi (B1 B2 ) =σi (B1 B2 ).
σi (A1 A2 ∪ B1 B2 )(5)

Hence, sub-additivity σi (A1 A2 ∪ B1 B2 ) < σi (A1 A2 ) + σi (B1 B2 ) implies QCi = 1 − mi (A1 A2 ) −mi (B1 B2 ) < 0. Likewise, sub-additivity σi (Ω) < σi (Ai ) + σi (Bi ) for i = 1, 2 implies BCi (A1 ) < 0.
45

The positive impact of the dummy variable 1{Fi =g} on kiQCG in Table 19 also supports the conjecturethat those observers who find the games easier to predict are more likely to use the symmetric profiles
A1 A2 and B1 B2 as their reference events when asked about the matching probability of one of those
events.

33

SH0.2

SH0.8

PD0.1

PD0.5

QCGi
BCGi
+
−
p
BCGdi

+
20
12

−
6
28

0.000
−
5
32

+
−

+
13
21

p

0.028

Fi
g
u

+
21
14

p

0.015

−
34
5

+
19
16

−
6
21

0.018
+
19
19

−
2
28

0.000
+
22
17

−
22
7

0.126

+
19
20

−
5
17

0.059
+
18
29

−
3
19

0.050
+
25
21

−
14
8

0.602

+
15
19

−
6
31

0.018
+
17
21

−
11
28

0.159
+
21
16

−
27
10

0.223

Table 14: Association between QCGi and BCGi , BCGdi , and Fi
values) is severe for the strategic events, and is caused by the excessively large matchingprobabilities placed on the symmetric coordination profiles of the games: Approximately
one half of observers exhibit the coordination fallacy for at least one of the two symmetricprofiles. The average MN values tend to be smaller for the Pareto eﬀicient profile. In
contrast, the violation of monotonicity is rare for the non-strategic events.
The weights placed on the two symmetric profiles of the games are so large that
their sum exceeds one in half of the cases. This, along with the observation on thenegative correlation between the matching probabilities of the two symmetric profiles,
leads us to the conjecture that the subjects focus only on the two symmetric profiles whencontemplating on the matching probability of such a profile. Under this hypothesis, thequasi-complementarity (QC) index, defined to be one minus the sum of the two matchingprobabilities of the two symmetric profiles, is similar in its functional form to the BC
index of individual action choices. We find data to be consistent with this implication.
While the difference between the two types of uncertainty sources is substantial,
the difference across the four game treatments is less clear. While Pareto dominance isfound to induce the violation of monotonicity more often, risk dominance in the case of
the SH games seems to have little impact on the predictions. There is also no obvious
relationship between the predictions and the empirical statistics of how the games wereplayed in the players sessions. One possibility is that the across-treatment variation is
hidden behind the extremely large within-treatment variations in the predictions.
This is a first exploration into the perception of strategic uncertainty, and there are anumber of possible extensions. We used stag-hunt and prisoners’ dilemma games for thecreation of strategic uncertainty. Given that both these games have symmetric profiles
34

as focal outcomes, they may induce violation of monotonicity more strongly than someother games. The perception of strategic uncertainty in other games is the most obvious
extension of the present analysis. In the sense that many of the strategic uncertainties
in the reality involve more than two decision makers, it will be of practical importanceto use games with a larger number of players.
Although we have focused attention on bets with a positive reward, the literatureobserves that uncertainty attitudes are often different in the positive and negative domains. Whether violation of PS is severer or not in the negative domain is an open
question. In our experiments, the observers make predictions based on the informationthat the player subjects are students of the same university. An interesting extensionthat corresponds to the testing of the familiarity hypothesis would involve players andobservers from totally different social backgrounds.

References
Abdellaoui, M., Baillon, A., Placido, L. & Wakker, P. P. (2011), ‘The rich domainof uncertainty: Source functions and their experimental implementation’, American
Economic Review 101(2), 695–723. 2
Ahn, D., Choi, S., Gale, D. & Kariv, S. (2014), ‘Estimating ambiguity aversion in a
portfolio choice experiment’, Quantitative Economics 5, 195–223. 6
Al-Najjar, N. I. & Weinstein, J. (2009), ‘The ambiguity aversion literature: A criticalassessment’, Economics and Philosophy 25(3), 249–284. 10
Baillon, A. & Bleichrodt, H. (2015), ‘Testing ambiguity models through the measurementof probabilities for gains and losses’, American Economic Journal: Microeconomics
7(2), 77–100. 2, 3.3, 4.2, 22, 5.2, 29
Baillon, A. & Emirmahmutoglu, A. (2018), ‘Zooming in on ambiguity attitudes’, International Economic Review 59(4), 2107–2131. 2, 3.3, 20
Baillon, A., Huang, Z., Selim, A. & Wakker, P. P. (2018), ‘Measuring ambiguity attitudes
for all (natural) events’, Econometrica 86(5), 1839–1858. 3, 2, 9, 5.2
Blanco, M., Engelmann, D., Koch, A. K. & Normann, H.-T. (2010), ‘Belief elicitation inexperiments: is there a hedging problem?’, Experimental Economics 13(4), 412–438.
11
Camerer, C. F. & Karjalainen, R. (1994), Ambiguity-aversion and non-additive beliefs
in non-cooperative games: Experimental evidence, Springer Netherlands, Dordrecht,
pp. 325–358. 2
35

Cason, T. N., Sharma, T. & Vadovič, R. (2020), ‘Correlated beliefs: Predicting outcomesin 2×2 games’, Games and Economic Behavior 122, 256–276. 12, 5.5
Charness, G., Karni, E. & Levin, D. (2010), ‘On the conjunction fallacy in probability judgment: New experimental evidence regarding Linda’, Games and Economic
Behavior 68(2), 551–556. 5
Chew, S. H., Ebstein, R. P. & Zhong, S. (2012), ‘Ambiguity aversion and familiarity
bias: Evidence from behavioral and gene association studies’, Journal of Risk andUncertainty 44(1), 1–18. 2, 29
Chew, S. H., Miao, B. & Zhong, S. (2017), ‘Partial ambiguity’, Econometrica
85(4), 1239–1260. 2, 6
Chew, S. H., Ratchford, M. & Sagi, J. S. (2018), ‘You need to recognise ambiguity to
avoid it’, Economic Journal 128(614), 2480–2506. 5.3
Dal Bó, P. & Fréchette, G. R. (2018), ‘On the determinants of cooperation in infinitely
repeated games: A survey’, Journal of Economic Literature 56(1), 60–114. 3.2
Dimmock, S. G., Kouwenberg, R. & Wakker, P. P. (2016), ‘Ambiguity attitudes in alarge representative sample’, Management Science 62(5), 1363–1380. 2
Dominiak, A. & Duersch, P. (2019), ‘Interactive Ellsberg tasks: An experiment’, Journalof Economic Behavior & Organization 161, 145–157. 10
Eichberger, J., Kelsey, D. & Schipper, B. C. (2008), ‘Granny versus game theorist:
Ambiguity in experimental games’, Theory and Decision 64(2-3), 333–362. 2
Epstein, L. G. & Halevy, Y. (2019), ‘Ambiguous correlation’, Review of Economic Studies
86(2), 668–693. 5.5
Epstein, L. G. & Schneider, M. (2010), ‘Ambiguity and asset markets’, Annual Review
of Financial Economics 2(1), 315–346. 8
Fischbacher, U. (2007), ‘z-Tree: Zurich toolbox for ready-made economic experiments’,
Experimental Economics 10(2), 171–178. 3.1
Fox, C. R. & Tversky, A. (1995), ‘Ambiguity aversion and comparative ignorance’, Quarterly Journal of Economics 110(3), 585–603. 5.2
Fox, C. R. & Weber, M. (2002), ‘Ambiguity aversion, comparative ignorance, and decision context’, Organizational Behavior and Human Decision Processes 88(1), 476–498.
29

36

French, K. R. & Poterba, J. M. (1991), Investor diversification and international equity
markets, Working Paper 3609, National Bureau of Economic Research. 8, 28
Gächter, S. & Renner, E. (2010), ‘The effects of (incentivized) belief elicitation in public
goods experiments’, Experimental Economics 13(3), 364–377. 11
Gigerenzer, G. (1996), ‘On narrow norms and vague heuristics: A reply to Kahneman
and Tversky.’. 5
Greiner, B. (2015), ‘Subject pool recruitment procedures: Organizing experiments with
ORSEE’, Journal of the Economic Science Association 1(1), 114–125. 3.1
Halevy, Y. (2007), ‘Ellsberg revisited: An experimental study’, Econometrica 75(2), 503–
536. 2, 6
Heath, C. & Tversky, A. (1991), ‘Preference and belief: Ambiguity and competence inchoice under uncertainty’, Journal of Risk and Uncertainty 4(1), 5–28. 5.2, 29
Heinemann, F., Nagel, R. & Ockenfels, P. (2009), ‘Measuring strategic uncertainty incoordination games’, Review of Economic Studies 76(1), 181–221. 2
Hoffmann, T. (2014), The effect of belief elicitation game play, Annual Conference
2014 (Hamburg): Evidence-based Economic Policy 100483, Verein für Socialpolitik/German Economic Association. 11
Hyndman, K., Terracol, A. & Vaksmann, J. (2009), ‘Learning and sophistication incoordination games’, Experimental Economics 12(4), 450–472. 12
Kocher, M. G., Lahno, A. M. & Trautmann, S. T. (2018), ‘Ambiguity aversion is notuniversal’, European Economic Review 101, 268–283. 2, 5.2
Li, C., Turmunkh, U. & Wakker, P. P. (2019), ‘Trust as a decision under ambiguity’,
Experimental Economics 22(1), 51–75. 9, 5.2
Li, Z., Loomes, G. & Pogrebna, G. (2017), ‘Attitudes to uncertainty in a strategic
setting’, Economic Journal 127(601), 809–826. 2
Moffatt, P. G. (2015), Experimetrics: Econometrics for Experimental Economics,
Macmillan International Higher Education. 37
Oechssler, J., Roider, A. & Schmitz, P. W. (2009), ‘Cognitive abilities and behavioral
biases’, Journal of Economic Behavior & Organization 72(1), 147–152. 5.3, 34
Oechssler, J. & Roomets, A. (2015), ‘A test of mechanical ambiguity’, Journal of Economic Behavior & Organization 119, 153–162. 27
37

Offerman, T., Sonnemans, J., Van de Kuilen, G. & Wakker, P. P. (2009), ‘A truth
serum for non-bayesians: Correcting proper scoring rules for risk attitudes’, Review of
Economic Studies 76(4), 1461–1489. 4
Palfrey, T. R. & Wang, S. W. (2009), ‘On eliciting beliefs in strategic games’, Journalof Economic Behavior & Organization 71(2), 98–109. 12
Raiffa, H. (1968), Decision Analysis: Introductory Lectures on Choices under Uncertainty, Addison-Wesley. 9
Rankin, F. W., Huyck, J. B. V. & Battalio, R. C. (2000), ‘Strategic similarity andemergent conventions: Evidence from similar stag hunt games’, Games and Economic
Behavior 32(2), 315–337. 15
Ritov, I. & Baron, J. (1990), ‘Reluctance to vaccinate: Omission bias and ambiguity’,
Journal of Behavioral Decision Making 3(4), 263–277. 8
Schlag, K. H., Tremewan, J. & Van der Weele, J. J. (2015), ‘A penny for your thoughts:
A survey of methods for eliciting beliefs’, Experimental Economics 18(3), 457–490. 11
Schmeidler, D. (1989), ‘Subjective probability and expected utility without additivity’,
Econometrica 57(3), 571–587. 4
Trautmann, S. T. & van de Kuilen, G. (2015a), Ambiguity Attitudes, John Wiley & Sons,
Ltd, chapter 3, pp. 89–116. 1, 2, 7
Trautmann, S. T. & van de Kuilen, G. (2015b), ‘Belief elicitation: A horse race among
truth serums’, Economic Journal 125(589), 2116–2135. 5.2, 27
Tversky, A. & Kahneman, D. (1982), Judgments of and by Representativeness, Cambridge University Press, pp. 84–98. 5
Tversky, A. & Koehler, D. J. (1994), ‘Support theory: A nonextensional representationof subjective probability’, Psychological Review 101(4), 547–567. 7
Wakker, P. P. (2010), Prospect Theory: For Risk and Ambiguity, Cambridge University
Press. 9
Yang, C.-L. & Yao, L. (2017), ‘Testing ambiguity theories with a mean-preserving design’, Quantitative Economics 8(1), 219–238. 6
Zizzo, D. J., Stolarz-Fantino, S., Wen, J. & Fantino, E. (2000), ‘A violation of themonotonicity axiom: Experimental evidence on the conjunction fallacy’, Journal of
Economic Behavior & Organization 41(3), 263–276. 5

38

AppendicesA

Tables and Figures#Observations after censoring
Index

SH0.2

SH0.8

PD0.1

PD0.5

total

MN

M N UiRY
M N UiGB
M N UiRY − M N UiGBM N GAi
M N GBiB
M N GAi − M N Gi
M N UiM N GiM N U i − M N Gi

92
90
90
75
69
66
90
66
65

87
88
87
73
66
62
87
62
60

86
88
85
70
69
61
85
61
60

88
91
88
77
76
71
88
71
69

353
357
350
295
280
260
350
260
254

BC

BCUi
BCGi
BCUi − BCGi
BCUid
BCGdi
BCUid − BCGdi

90
66
65
92
75
75

88
64
62
90
72
72

85
66
64
89
75
74

89
73
70
90
80
78

352
269
261
361
302
269

QC

QCUi
QCGi
QCUi − QCGi

92
74
74

89
70
70

90
72
72

90
78
77

361
294
293

CP

CP UiRY
CP UiGBCP UiRY − CP UiGB
CP GA
i
CP GB
i
B
CP GA
i − CP Gi

92
90
90
75
69
66

87
88
87
73
66
62

86
88
85
70
69
61

88
91
88
77
76
71

353
357
350
295
280
260

Notes. See Footnote 20 for the description of the censoring criterion.

Table 15: Size of subsamples after censoring

39

Figure 12: (mi (RY ∪ GB), mi (RB ∪ GY )): all treatments combined

Figure 13: (mi (A1 A2 ∪ B1 B2 ), mi (A1 B2 ∪ B1 A2 )) by treatment
Table 16 summarizes p-values for paired t-test for the difference in the average matchingprobabilities of pairs of events that are symmetric with respect to the urns or the players.
It finds no statistical difference in the matching probabilities of these pairs of events
except in one case (p < 0.05).

40

Hypothesis

Treatment
SH0.2

SH0.8

PD0.1

PD0.5

m(R) = m(Y )m(G) = m(B)m(RY ) = m(GB)m(RY ∪ GB) = m(RB ∪ GY )0.1399 (92)0.1615 (90)0.5586 (92)0.7863 (92)0.1485 (88)0.2910 (89)0.0601 (89)0.9208 (90)0.9233 (86)0.5329 (88)0.4987 (90)0.5330 (89)0.3135 (89)0.1909 (91)0.9230 (90)0.5548 (90)m(A1 ) = m(A2 )m(B1 ) = m(B2 )0.0079 (75)0.7010 (70)0.6801 (75)0.1409 (68)0.7415 (72)0.7567 (73)0.2906 (77)0.6629 (79)

Notes. The number in the parentheses is the size of the subsample in which both matching probabilities are available after data censoring described in Footnote 20.

Table 16: Paired t-test of event exchangeability.

SH0.2|BCUi | ≤ 0.1|BCUi | > 0.1
total
p-value|BCGi | ≤ 0.1
33
18
51

SH0.8

|BCGi | > 0.1
8
6
14

total
41
24
65

|BCGi | ≤ 0.1
33
6
39

0.603

PD0.1

|BCGi | > 0.1
12
11
23

total
45
17
62

|BCGi | ≤ 0.1
30
13
43

0.006

PD0.5

|BCGi | > 0.1
11
10
21

total
41
23
64

|BCGi | ≤ 0.1
28
9
37

0.173

|BCGi | > 0.1
15
18
33

total
43
27
70

0.010

Table 17: Association between BCUi and BCGi

Urns
SH0.2
SH0.8
PD0.1
PD0.5

Omax < Tmax
Omin > Tmin

Omax ≥ Tmax
Omin ≤ Tmin

Omax > Tmax
Omin > Tmin

Omax < Tmax
Omin < Tmin

total

0.05
0.08
0.16
0.13
0.17

0.37 (0.06)
0.54 (0.31)
0.40 (0.25)
0.49 (0.36)
0.49 (0.32)

0.38
0.05
0.13
0.13
0.18

0.17
0.26
0.22
0.18
0.11

0.97
0.92
0.91
0.93
0.95

Notes. O = OU and T = T U for the urns (all treatments combined), andO = OG and T = T G for the games. The independence of the two players’ action choices implies, but is not implied by, the second case with Omax ≥ Tmax
and Omin ≤ Tmin . The numbers in the parentheses represent the proportionsof observations satisfying strict inequalities.

Table 18: Types based on the assessment of simple and diagonal events

B

Entropy and Familiarity

We may conjecture that more dispersion in play makes prediction more diﬀicult, andhence implies less confidence in prediction. This in turn leads to the conjecture thata higher entropy implies a higher degree of ambiguity aversion. We hence hypothesizethat BCGi is larger if the game has a higher entropy. Figure 14 shows the proportion41

Models
Dept. var.

(1)
kiBCG

kiBCU

0.089*(0.048)kiM N U(2)
kiM N G

(3)
kiQCG

(4)
kiBCG

(5)
kiM N G

0.224
(0.213)0.261**(0.120)1.907***(0.185)

kiQCU

0.123
(0.146)2.924(2.307)-0.162(0.240)1{Fi =g}kiBCU * 1{Fi =g}-0.154(1.924)

kiM N U * 1{Fi =g}

Observations
R-squared-0.013(0.039)4.151***(1.354)-1.783***(0.287)

kiQCU * 1{Fi =g}
Constant

(6)
kiQCG

11.14***(0.668)7.329***(0.736)

7.381***(0.943)8.952***(1.449)7.311***(1.800)0.332(0.225)4.412***(1.278)

261
0.007

254
0.002

293
0.006

254
0.017

247
0.011

283
0.041

Notes. a) * p < 0.10, ** p < 0.05, *** p < 0.01. b) Robust standard errors clustered by session
in parentheses. c) 1{Fi =g} = 1 if observer i states that the game was easier to predict than theurns.

Table 19: Association of deviations from PS between the urns and games: OLS regressions
SH0.2

SH0.8

PD0.1

PD0.5

QCUi
BCUi
+
−
p
BCUid

+
38
50

−
1
3

0.635
−
0
4

+
−

+
55
33

p

0.024

Fi
g
u

+
66
22

p

1.000

−
3
1

+
45
41

−
1
2

0.608
+
49
37

−
0
3

0.087
+
59
24

−
1
2

0.216

+
48
35

−
2
5

0.235
+
55
27

−
2
5

0.093
+
47
33

−
4
2

1.000

+
43
43

−
2
2

1.000
+
52
33

−
1
3

0.299
+
56
26

−
4
0

0.310

Table 20: Association between QCUi and BCUi , BCUid , and Fi
42

Figure 14: Entropy and the proportion of observers with BCi < −0.10
of observers with BCi < −0.10 plotted against the realized entropy of each game. The
red line corresponds to the observers with Fi = g, the green line to the observers with
Fi = u, and the blue line to both types. While we find no trend when both types are
combined, there exist a positive trend for Fi = g, and a negative trend for Fi = u. In
other words, among those observers who perceive game prediction to be easier, a higher
entropy implies a higher proportion of BCGi < −0.10 contrary to our conjecture.

C Matching Probabilities and Certainty Equivalents
One important question concerns whether violation of PS is observed even when we use
measures other than matching probabilities. To answer this question, we use data fromPart 3 of the observer sessions to check if monotonicity is violated also in terms of certainty equivalents. For event E, let ci (E) denote observer i’s certainty equivalent of thebet 1000E 0. Part 3 of the observer experiments elicits ci (E) for the urn events E = G
as well as the game events E = B1 and B1 B2 .46 Given this limited set of questions, violation of MN is defined by M N Gi (B1 , B1 B2 ) ≡ mi (B1 ) − mi (B1 B2 ) < 0, and violationof monotonicity in terms of certainty equivalents is defined by M N GCi (B1 , B1 B2 ) ≡ci (B1 ) − ci (B1 B2 ) < 0. As seen in Table 21, 28.5% (= 81/284) of observers exhibit violation of monotonicity in terms of certainty equivalents of these events, and the proportionis higher than that in terms of matching probabilities (18.4% = 52/284).47
The scatter plots in Figure 15 show positive correlation between the difference in46

See Footnote 18.

47

We should however note that there is little correlation between those who violate MN in terms ofmatching probabilities and those who violate MN in terms of certainty equivalents. On the other hand,the predominant majority are PS among those who show consistent responses between the two measuresof MN.

43

M N GCi (B1 , B1 B2 )
<0
≥0
total

M N Gi (B1 , B1 B2 )
<0
≥0

total

18
34
52

81
203
284

63
169
232
0.282

p
Notes. p-value for χ2 test.

Table 21: Association between M N Gi (B1 , B1 B2 ) and M N GCi (B1 , B1 B2 )certainty equivalents against the difference in matching probabilities. This implies thefirst-order stochastic dominance (FOSD) property of observers’ preferences, which justifies the use of multiple price list in the elicitation of their matching probabilities.48

Notes. Left panel: (ci (B1 ) − ci (G), mi (B1 ) − mi (G)), middle panel:(ci (G) − ci (B1 B2 ), mi (G) − mi (B1 B2 )), right panel: (M N GCi (B1 , B1 B2 ), M N Gi (B1 , B1 B2 )).
Pearson correlation = 0.59, 0.67, and 0.27, respectively, with p = 0.000 for all three cases.

Figure 15: Matching probabilities and certainty equivalents: All treatments combined48

Observer i’s preferences ⪰i over bets satisfy satisfy first-order stochastic dominance (FOSD) if for
any two events E and E ′ , mi (E ′ ) ≥ mi (E) implies 1000E ′ 0 ⪰i 1000E 0. Since i is indifferent between
receiving ci (E) for sure and 1000E 0, FOSD holds if for any two events E and E ′ , mi (E ′ ) ≥ mi (E)implies ci (E ′ ) ≥ ci (E). It follows that for any E and E ′ , positive correlation between mi (E ′ ) − mi (E)and ci (E ′ ) − ci (E) implies consistency with FOSD.

44