ISSN (Print) 0473-453X

Discussion Paper No. 1140

ISSN (Online) 2435-0982

PREFERENCE FOR RANDOMIZATION
AND VALIDITY OF
RANDOM INCENTIVE SYSTEMUNDER AMBIGUITY:
AN EXPERIMENTTomohito Aoyama
Nobuyuki Hanaki

August 2021

The Institute of Social and Economic Research
Osaka University
6-1 Mihogaoka, Ibaraki, Osaka 567-0047, Japan

Preference for randomization and validity of random
incentive system under ambiguity: An experiment∗
Tomohito Aoyama†and Nobuyuki Hanaki‡
August 16, 2021

Abstract
The random Incentive System (RIS) is a standard method to incentivize participants ineconomic experiments. However, recent theoretical studies point out the possibility of its
failure under ambiguity. We propose a modiﬁcation of RIS, named independent RIS (I-RIS),
to improve its reliability. We conducted an experiment to evaluate the performances of thestandard RIS and I-RIS in direct and indirect manners. Whereas a nonnegligible fractionof participants are not consistent with the reversal-of-order axiom, a majority of ambiguity
averse and seeking participants are. This implies that participants with nonneutral ambiguity
attitudes may not report truthful preferences when RIS is used. However, randomization
attitudes do not explain inconsistent choices under RIS. In addition, we did not ﬁnd signiﬁcantdiﬀerences in performance between RIS and I-RIS. These results suggest that preferences for
randomization, which is driven by nonneutral ambiguity attitudes, do not cause the failureof RIS.
Keywords: Random Incentive System, Ambiguity, Incentive Compatibility
JEL Code: C91, D81∗

We gratefully acknowledge ﬁnancial support from Joint Usage/Research Center at ISER, Osaka University, andJapan Society for the Promotion of Science (18K19954, 20H05631). The experiment reported in this paper has
been approved by IRB of ISER, Osaka University.
†
Hitotsubashi Institute for Advanced Study, Hitotsubashi University‡
Institute of Social and Economic Research, Osaka University

1

1

Introduction

The random Incentive System (RIS) is a standard method to incentivize participants ineconomic experiments (Azrieli et al. 2018). However, recent theoretical studies point out
the possibility of its failure when participants are concerned about ambiguity (Bade 2015,
Kuzmics 2017, and Oechssler and Roomets 2014). They argue that participants may view
the whole experiment as a single decision problem and exploit randomness in RIS to hedge
ambiguity. If their conjecture holds, it is diﬃcult to experimentally investigate choice behavior
under ambiguity.
We propose a variation of RIS, named independent RIS (I-RIS) in which choice alternatives in diﬀerent choice situations suﬀer ambiguity from independent ambiguity sources.
In addition, we posit a preference condition, which is a variation of the compound independence axiom of Segal (1990), which guarantees that I-RIS is incentive compatibility. Thispreference condition is satisﬁed by, for example, maxmin expected utility with independent
multiple priors (Gilboa and Schmeidler, 1989).
To test the performance of the standard RIS and I-RIS empirically, we conducted anexperiment in which we elicit participants’ preferences under ambiguity and relate them tobehavior under RIS. Our experiment consists of two parts, which are conducted two weeks
apart. In the ﬁrst part, participants face a series of lotteries and report willingness-to-pay(WTP) for each, which allows us to elicit participants’ attitudes toward ambiguity, randomization, and reversal of uncertainty resolution order. In the second part, participants face
two identical binary choice problems under ambiguity. If RIS is incentive compatible, participants should choose the same lottery twice in the second part. If a participant chooses
diﬀerent lotteries (we call such choices, inconsistent choices), instead, we conclude s/he does
not report his or her preferences truthfully.
There are two treatments in the second part, correlated (COR) and independent (IND),
corresponding to the two versions of RIS. In the COR treatment, just one ball is drawn at2

the end of the experiment to determine participants’ earnings as is normally done in RIS.
The IND treatment, instead, employs I-RIS. Namely, in the IND treatment, one ball is drawn
independently for each lottery. We are interested in whether the observed frequencies ofinconsistent choices are the same in the COR and IND treatments, as well as the relationshipbetween participants’ randomization attitudes and frequencies of inconsistent choices.
Based on an online experiment conducted with 192 students of Osaka University in February 2021, we did not ﬁnd a statistically signiﬁcant diﬀerence (we employed a 5% signiﬁcancelevel throughout the paper) in the frequencies of inconsistent choices between the IND and
COR treatments. Thus, at least, in the simple setting considered in our experiment, thetwo versions of RIS deliver practically the same outcome. Moreover, preferences for randomization, which are elicited in the ﬁrst part, do not explain the inconsistent choices. These
results suggest that the interaction of the randomness of RIS and ambiguity aversion is not
the source of inconsistent choices.
In Section 2, we review the theoretical and experimental literature that are related to theperformance of RIS. In Section 3, we propose I-RIS and argue that it is incentive compatible
if an appealing preference condition is satisﬁed. In Section 4, we describe our hypotheses andthe experiment, and we report the results. In Section 5, we discuss our experimental results.
Section 6 concludes.

2

Theoretical and experimental backgroundIn this section, we review the theories that motivate our experiment and related experimental
studies.

3

2.1

Random incentive system

Usually, experimenters want to elicit participants’ preferences multiple times in one experiment. One simple method to incentivize participants to report truthfully is paying all chosenalternatives. However, if the experiment involves complementary goods, and if participants
view the whole experiment as a single decision problem, the choice in each choice situation
may be diﬀerent from the one that would be observed when that choice situation was the
only one presented.
The random incentive system is a more common incentive scheme that works as follows:1
1. The experimenter randomly and privately select a choice situation that will be used forcompensation.
2. A participant reports his/her best-preferred alternative in each choice situation.
3. The alternative that the participant reported as his/her best-preferred one in the choicesituation selected by the experimenter are paid to him/her.
The remarkable part of this scheme is that only one choice situation is used for compensation. False reports are expected to decrease under this scheme compared with the casewhen an experimenter pays all chosen alternatives because participants cannot obtain a setof complementary goods.
Even under RIS, participants may view the whole experiment as a single decision problem.
When a participant chooses f1 , . . . , fN in N choice situations, the RIS generates a lottery over
f1 , . . . , fN . Holt (1986) points out that when a participant’s preferences do not satisfy the
independence axiom, RIS is not incentive compatible.2 Karni and Safra (1987) also conducts
a similar analysis in the context of the preference reversal phenomenon. Many authors, forexample, Cubitt et al. (1998), Cox et al. (2014), Harrison and Swarthout (2014), Cox et al.
1

Baillon et al. (2019) recommend determining the choice situation used for compensation at the beginning of
the experiment, which we follow throughout this paper.
2
He implicitly assumes that the participant reduces two-stage lotteries to one-shot ones.

4

(2015), Brown and Healy (2018), and Freeman et al. (2019) conduct experiments to determine whether such a concern exists. All of these authors consider the case in which choicealternatives are objective lotteries.
The arguments of Holt (1986) and Karni and Safra (1987) also apply to preferences for
ambiguity because they are typically nonlinear. Recently, Oechssler and Roomets (2014),
Bade (2015), and Kuzmics (2017) prove it is impossible to elicit ambiguity preferences using
RIS in diﬀerent settings. A common assumption Bade (2015) and Kuzmics (2017) posit is that
all acts are deﬁned on a common state-space. Oechssler and Roomets (2014), who focus on
the α-maxmin expected utility model, consider the case where the state-space has a productstructure. However, their speciﬁc assumption on multiple priors makes the situation the sameas those of Bade (2015) and Kuzmics (2017). As we see in Section 3, if their assumptions are
violated, our modiﬁcation of RIS may be incentive compatible even if participants’ preferences
are ambiguity nonneutral.3

2.2

Preference conditions

Azrieli et al. (2018) show that RIS is essentially the only incentive compatible preference
elicitation scheme when a preference condition, called monotonicity, is the only assumption
the experimenter imposes. The monotonicity condition of Azrieli et al. (2018) states that if
f1 , . . . , fN are the best-preferred choices in each choice situation, then the lottery generated
from these choices through RIS is the best one that is given in this manner. As Azrieli et al.
(2018) show, monotonicity is violated when a participant’s preferences are nonlinear andsatisﬁes the reversal-of-order axiom. The reversal-of-order axiom, which was proposed by
Anscombe and Aumann (1963), states that the participant ignores the order of uncertainty
resolutions of multistage lotteries.4 To introduce the reversal-of-order axiom formally, letΩ be a state space and call f : Ω → ∆R an act, which stands for a lottery that gives a3
4

Oechssler and Roomets (2014) recognize this point. See footnote 9 of that paper.
See also Seo (2009) for implication of this axiom for ambiguity attitudes.

5

random monetary outcome conditional on states.5 Let F be the set of all acts. For acts f
and g, and α ∈ [0, 1], let αf ⊕ (1 − α)g be the state-wise mixture of f and g.6 Next, foracts f and g, let αf + (1 − α)g be an objective lottery that gives f with probability α andgives g with probability 1 − α. A preference relation ⪰ satisﬁes the reversal-of-order axiom if
αf ⊕ (1 − α)g ∼ αf + (1 − α)g for all f, g and α.
To see that nonlinearity of preferences and the reversal-of-order axiom imply failure of
RIS, consider an Ellsberg-type experiment. Let Ω = {R, B} be the state-space that represents Ellsberg urn with red and black balls. An experimenter is interested in a participant’s
preferences for bets on ball colors R and B, and bets on the results of a toss of a fair coin,
H (for heads) and T (for tails). The experimenter asks the participant which of R or H,
and B or T s/he prefers, using the RIS that assigns a probability of 1/2 to each choice situation. Typical ambiguity averse choices in this setting are H ≻ R and T ≻ B. Suppose
the participant’s preferences are such ones. By reporting these preferences, RIS generates alottery 12 H + 12 T , which reduces to an objective lottery that wins with a probability of 1/2.
However, choosing R and B in each situation, the randomness of RIS generates the lottery
11
2 R + 2 B, which provides exposure to R or B, each with probability 1/2. If the participant’s

preferences satisfy the reversal-of-order axiom, 12 R + 12 B ∼ 12 R ⊕ 12 B holds. But 21 R ⊕ 12 B isan objective lottery that wins with probability 1/2, which is the same as the one that is given
when reporting the true preferences. Then, the participant may as well report preferences
R ⪰ H and B ⪰ T . That is, the randomization provided by RIS hedges ambiguity. As thisexample shows, the reversal-of-order axiom is related to the performance of RIS. Therefore,
empirically testing this axiom is important. Recently, Oechssler et al. (2019) conﬁrm thatthe reversal-of-order axiom holds at the population level.
In the example above, the preferences 12 R ⊕ 12 B ≻ R, B, together with the reversal-ofHere, ∆R denotes for the set of Borel probability measures over R.
For p, q ∈ ∆R, let αp⊕(1−α)q ∈ ∆R be the objective lottery deﬁned by [αp⊕(1−α)q](B) = αp(B)+(1−α)(B)for B ⊂ R. The act αf ⊕ (1 − α)g is deﬁned by [αf ⊕ (1 − α)g](ω) = αf (ω) ⊕ (1 − α)g(ω).
566order axiom, imply 12 R + 12 B ≻ R, B. The latter nonlinear preferences for lotteries overambiguous bets are the causes of the failure of RIS in the example. In this paper, we say⪰ is randomization seeking if f ∼ g and λf + (1 − λ)g ≻ f is observed for some f and g.
Similarly, we say ⪰ is randomization averse if f ∼ g and pf + (1 − p)g ≺ f for some f andg. If λf + (1 − λ)g ∼ f for any f ∼ g for any f and g, we say ⪰ is randomization neutral.7
Even if participants’ preferences do not satisfy the reversal-of-order axiom, they may berandomization nonneutral. In such a case, again, RIS fails. Saito (2015) and Ke and Zhang(2020) study utility models that accomodate decision makers who believe that ex ante randomization hedges ambiguity to some extent, but not as much as a state-wise mixture. Their
models predict the correlation of ambiguity aversion and randomization seeking. However,Dominiak and Schnedler (2011) do not ﬁnd a signiﬁcant correlation of this kind.

2.3

Random choice

Nonneutral ambiguity attitudes are not the only potential cause of preferences for randomization. Wheres the models of Saito (2015) and Ke and Zhang (2020) assume that a decision
maker has stable beliefs and random choices under ambiguity are driven by the motivation ofhedging ambiguity, Lu (2019) views belief itself as random. The preferences of the participantsof our experiment may not be stable, as in the model of Lu (2019). In this paper, however,
we insist on the view that they have stable beliefs and their seemingly random choices are
driven by the purpose of hedging ambiguity. This is because the theoretical papers that pointout the possible failure of RIS under ambiguity share this view.

7

The deﬁnitions of randomization attitudes are essentially the same as those of Eichberger et al. (2016), although
they use a diﬀerent setup.

7

3

Independent random incentive system

In this section, we propose a variant of RIS, in which choice alternatives in diﬀerent choicesituations depend on diﬀerent state spaces. We provide a preference condition that guarantees
the incentive compatibility of our incentive scheme. It states that participants do not hedge
ambiguity by stochastically choosing alternatives that depend on diﬀerent state spaces.

3.1

Setup

First, we formally introduce choice alternatives that appear in experiments on ambiguity. For
any topological space Y , let ∆Y denote for the set of Borel probabilities over Y . Let X be
the outcome space, which is a compact metrizable set. Let S1 , . . . , SK be ﬁnite state spaces∏
and let Ω = K
k=1 Sk . Each Sk is interpreted as a diﬀerent ambiguity source. An act is amapping f : Ω → ∆X and we denote the set of all the acts as F.8 A random act P ∈ ∆F
is a probability over F. Let ⪰ be a continuous weak order over ∆F, which is interpreted as
preferences of a decision maker (DM, henceforth). A second-order random act P ∈ ∆(∆F)
is a probability over ∆F. Each x ∈ X is identiﬁed with fx ∈ F such that fx (ω) = x forall ω. Each f ∈ F is identiﬁed with the degenerate probability δf ∈ ∆F . Each P ∈ ∆F is
identiﬁed with the degenerate probability δP ∈ ∆(∆F).

3.2

Incentive scheme

Next, we develop a framework to describe experiments on ambiguity and incentive schemes
following Azrieli et al. (2020). Let D = {D1 , . . . , DN } be a list of choice situations in theexperiment, where Dn is a compact subset of ∆F. That is, participants report his/her
best-preferred alternatives from each Dl . This setup covers several kinds of experiments.
For example, the Becker–DeGroot–Marschak (BDM) method (Becker et al., 1964) to elicit

8

The set F is endowed with the product topology.

8the WTP for an act generates a distribution over the act and some amounts of money,
conditional on a reported WTP. In this case, a participant chooses random acts. Anotherexample is an experiment in which a participant faces a ﬁnite set of acts. Such an experimentis also accommodated in this setup because an act is identiﬁed with the random act thatgives the former with probability one.
Under ambiguity, RIS may fail partly because choice alternatives in diﬀerent choice situations depend on a common ambiguity source. In order to avoid this pitfall, we consider∏
the choice situations in which this is not the case. For I ⊂ {1, . . . , K}, let SI = k∈I Sk∏
and denote typical elements of SI as sI , s′I , etc. Let S−I = k∈I/ Sk . We denote FI = {f ∈F|f (sI , s−I ) = f (sI , s′−I ) for any sI ∈ SI , s−I , s′−I ∈ S−I }.
Deﬁnition 1. D = {D1 , . . . , DN } is independent if there exists a partition {I1 , . . . , IN } of
{1, . . . , K} and Dn ⊂ ∆FIn for each n.
Given D, the DM reports her choices m = (m1 , . . . , mN ) ∈ M :=∏n Dn . The mechanism

we consider randomly pays one of m1 , . . . , mN . If a participant views the whole experiment
as a single decision problem, s/he would manipulate reports to obtain better outcomes. To
capture such underlying preferences, assume ⪰ extends to a relation ⪰∗ over ∆(∆F), whichis also a continuous weak order. Whereas ⪰ represents the true preferences the experimenteris concerned about, ⪰∗ dictates the ovserved choices.
Deﬁnition 2 (Random incentive system). An RIS is a mapping φ : M → ∆(∆F) such that
there exists a full-support probability λ ∈ ∆D and∑φ(m)(P ) =λ(Dn )

n; mn =Pholds for any m ∈ M and P ∈∪N

n=1 Dn .

For Dn and ⪰, let arg max⪰ Dn = {P ∈ Dn | P ⪰ Q for all Q ∈ Dn }. Deﬁne arg max⪰∗ φ(M ) =9

{φ(m)|φ(m) ⪰ φ(m′ ) for all m′ ∈ M }.9 We say RIS is incentive compatible if the reported
best-preferred items are actually the best-preferred ones in each choice situation. That is,Deﬁnition 3 (Incentive compatibility). RIS is incentive compatible if φ(m) ∈ arg max⪰∗ φ(M )implies mn ∈ arg max⪰ Dn for all n.

3.3

Preference condition

Next, we propose a condition that restricts the relationship between ⪰ and ⪰∗ . For second∑Lorder random acts P1 , . . . , Pl and α1 , . . . , αL ≥ 0 with
l=1 αl = 1, deﬁne the mixture∑L∑L∑L
10 Thisl=1 αl Pl ∈ ∆(∆F) by ( l=1 αl Pl )(G) =l=1 αl Pl (G) for any Borel sets G ⊂ ∆F.
operation also applies to ﬁrst-order random acts because they are identiﬁed with degeneratesecond-order random acts.
Condition 1. For any I, I ′ ⊂ {1, . . . , K}, P, Q ∈ ∆FI , R ∈ ∆FI ′ , and α ∈ (0, 1], if I ∩I ′ = ∅,P ⪰ Q ⇔ αP + (1 − α)R ⪰∗ αQ + (1 − α)R.
This condition has three implications. First, ⪰ and ⪰∗ , restricted to ∆F, coincide (for thecase α = 1). Second, for any P, Q ∈ ∆FI , the preference P ⪰∗ Q does not reverse whenambiguity-free lottery R ∈ ∆X is mixed (for the case I ′ = ∅); αP +(1−α)R ⪰∗ αQ+(1−α)R.
Third, the DM does not hedge ambiguity by randomly choosing random acts that depend on
diﬀerent ambiguity sources. The third implication is related to the DM’s perception about
how ambiguity sources correlate.
To clarify, consider an experiment in which the experimenter uses two Ellsberg urnsthat contain red and black balls. Here, the two urns are diﬀerent ambiguity sources; Sk ={Rk , Bk }, k = 1, 2. Typical participants are indiﬀerent between betting on R1 or B1 ; R1 ∼ B1 .
9
The continuity of ⪰ and ⪰∗ , the compactness of each Dl , and the continuity of φ guarantees that these two
sets are nonempty.
∑2
10
For the case L = 2, we write l=1 αl Pl also as α1 P1 + (1 − α1 )P2 .

10

Under Condition 1, in this case, 12 R1 + 12 R2 ∼∗ 12 B1 + 12 R2 would hold. However, this prediction is not convincing if the participant believes that ball-draws from two urns correlate.
Suppose she believes that R2 is more likely than B2 conditional on R1 , and B2 is more likely
than R2 conditional on B1 . Then, both components of 21 R1 + 12 R2 are good options conditional on R1 , but they are bad options conditional on B1 . However, two components of
1
1
2 B1 + 2 R2 are good options conditional on B1 , and in R1 , respectively.

So the participant

may view 21 R1 + 21 R2 as suﬀering more ambiguity than 12 B1 + 12 R2 does. Then, dependingon her ambiguity attitude, the indiﬀerence 12 R1 + 12 R1 ∼∗ 12 B1 + 21 R2 may well be violated.
Condition 1 excludes such behavior coming from perceptions of correlation. However, if the
DM is conﬁdent that the ambiguity sources are independent, it is natural that she does nothedge ambiguity by mixing bets on diﬀerent ambiguity sources.
Condition 1 contains two preference relations, ⪰ and ⪰∗ . As noted above, whereas ⪰∗dictates the observed data, ⪰ is not directly observed. Thus, this condition is what to assume,rather than to test. It guarantees the incentive compatibility of RIS with independent choicesituations.
Proposition 1. Any RIS is incentive compatible if D is independent and (⪰, ⪰∗ ) satisﬁes
Condition 1.
Proof. En route to a contradiction, suppose φ(P1 , . . . , PL ) ∈ arg max⪰∗ φ(M ), and Ql ≻ Pl
for some l and Ql ∈ Dl . Then, by Condition 1,

φ(P1 , . . . , PL ) =L∑

λ(Dl′ )Pl′l′ =1

= (1 − λ(Dl ))∑l′ ̸=l≺∗ (1 − λ(Dl ))λ(Dl′ )Pl′ + λ(Dl )Pl
1 − λ(Dl )∑l′ ̸=lλ(Dl′ )Pl′ + λ(Dl )Ql
1 − λ(Dl )= φ(P1 , . . . , Pl−1 , Ql , Pl+1 , . . . , PL ).

11

The strict preference follows from Il ∩∪l′ ̸=l Il′ = ∅ and Condition 1. A contradiction.

In our experiment, participants are instructed that outcomes of acts or random acts are
determined by the sampling of balls with replacement from a ﬁxed ambiguous urn, whichguarantees the independence of choice situations. In addition, we assume Condition 1 for the
incentive compatibility of our incentive scheme in Part 1. To distinguish RIS with independent
choice situations from the usual RIS in which acts depend on a common state-space, we callthe former as independent RIS (I-RIS).

3.4

ExampleMaximin expected utility with independent multiple priors is an example of models thatsatisfy Condition 1 and have a nonneutral ambiguity attitude.11 Consider the case of K = 2.
Let Ck ⊂ ∆Sk (k = 1, 2) be closed convex sets. Gilboa and Schmeidler (1989) deﬁne theproduct of C1 and C2 as C1 ⊗ C2 = co{p1 ⊗ p2 | pk ∈ Ck , k = 1, 2}, where p1 ⊗ p2 is the
product measure of p1 and p2 over Ω = S1 × S2 .12,13
Let ⪰ over F be deﬁned by∫
V (f ) =min

π∈C1 ⊗C2

u(f )dπ.

(1)

S1 ×S2

Assume that the DM ignores the order of uncertainty resolutions, and reduces multistage
objective uncertainty as the corresponding one-shot uncertainty.14 Then, ⪰ and ⪰∗ satisfy
Condition 1. Because the DM identiﬁes any second-order random acts as some act that11

Maximin expected utility itself represents preferences for acts. See footnote 14 for how we extend this model
to represent ⪰∗ .
12
The symbol co(·) denotes for closed convex hull.
13
These multiple priors are what Oechssler and Roomets(2014) regard )as superstitious.
(∫∫
14
Formally, deﬁne Φ : ∆(∆F)) → F by Φ(P)(ω) = ∆F F f (ω)dP (f ) dP(P ) and assume P ⪰∗ Q ⇔ Φ(P) ⪰
Φ(Q).

12

depends on the same state-space as the former, it is suﬃcient to see

f ⪰ g ⇔ αf ⊕ (1 − α)h ⪰ αg ⊕ (1 − α)h(2)

for any f, g ∈ F1 , h ∈ F2 , and α ∈ (0, 1]. Because∫
V (αf ⊕ (1 − α)h) =min

π1 ∈C1 ,π2 ∈C2∫= α minπ1 ∈C1αu(f ) + (1 − α)u(h) dπ1 ⊗ π2∫u(f )dπ1 + (1 − α) minu(h)dπ2
S1 ×S2π2 ∈C2S1S2= αV (f ) + (1 − α)V (h)and similarly V (αg ⊕ (1 − α)h) = αV (g) + (1 − α)V (h), (2) holds.15
To be speciﬁc, again, consider a Ellsberg-type experiment; Let Sk = {Rk , Bk }, k = 1, 2.
Consider choice situations D = {D1 , D2 }, where D1 = {R1 , H} and D2 = {B2 , T }, andRi (resp. Bi ) is the bet that wins if a red (resp. black) ball is drawn from the ith urn.
Furthermore, H (resp. T ) is the bet that wins when a ﬂipped fair coin lands with on heads(resp. tails). Then, D is a list of independent choice situations. Therefore, RIS elicits thetrue preferences of a participant who admits representation (1).
As an example of choice situations that violate the independence condition, consider
D′ = {D1′ , D2′ }, where D1′ = {R1 , H} and D2′ = {B1 , T }. The diﬀerence between D′ and D isthat, in the second choice situation, the ambiguous bet wins if a black ball is drawn from theﬁrst urn, not from the second. If the RIS assigns a probability of 1/2 to each choice situation,
the participant has no reluctance to choose ambiguous bets in both situations.

15

In the second line, we identify f and h as their obvious counterparts deﬁned on S1 and S2 , respectively.

13

4

Experiment

Our experiment is intended to test the validity of RIS in indirect and direct manners. First,
we measure participants’ attitudes toward ambiguity, randomization, and reversal of uncertainty resolution order. Second, we compare the performance of the standard RIS and I-RIS.
Speciﬁcally, we ask the following:• Are more randomization seeking participants more likely to misreport their preferences
under the standard RIS?
• Does the use of I-RIS improve incentive compatibility compared with the standard RIS?
• Does the improvement by I-RIS, if any, depend on randomization attitude?
We recruited a total of 195 participants from the subject pool of ISER, Osaka University,
managed by ORSEE (Greiner, 2015). Among them, 192 (95 in IND, and 97 in COR) completed both parts of the experiment, which were run two weeks apart. We used Qualtrics(www.qualtrics.com) to conduct our online experiment. Registered participants were asked
to individually complete the task by clicking the link they received via e-mail within the sameday. The link for the experiment site was sent to all the registered participants around 10:00
am on the date of the experiment. The median of times spent by participants in completingthe ﬁrst and second parts of the experiment are 20 and three minutes, respectively.16 They
earned, on average, 979 JPY and 533 JPY in the ﬁrst and second parts of the experiment
including 500 and 300 JPY as participation fees, respectively. Participants received theirreward in the form of an Amazon gift card, e-mail version.

16

In the text, we report the median times in the experiment because there are outliers. The mean times are 29
and 7 minutes for the ﬁrst and second parts, respectively.

14

4.1

Part one – preference elicitation

4.1.1

Task and incentive

In this part, participants face a series of choice situations where they are asked to report their
WTP for a lottery, which pays an uncertain amount of money. The lotteries are classiﬁed
into the following four classes in terms of the sources of uncertainty.
• Risky lotteries that pay monetary rewards contingent on a roll of a four-sided fair die.
• Ambiguous lotteries that are obtained from risky lotteries by changing the source of
uncertainty, from a die to a four-color Ellsberg urn, while keeping possible monetary
outcomes the same.
• Coin–ball lotteries that resolve in two steps: ﬁrst a fair coin is tossed and then a ball isdrawn from the Ellsberg urn. From an ambiguous lottery, we constructed a corresponding coin–ball lottery in the following way. Suppose the ambiguous lottery f that gives
f1 , f2 , f3 , and f4 JPY when the drawn ball is red, blue, green, and yellow, respectively.
Now, construct a corresponding symmetric lottery f ′ that gives f4 , f3 , f2 , and f1 JPY
when the ball is red, blue, green, and yellow, respectively. The coin–ball lottery corresponding to f reduces to f when the coin lands with heads up, but it reduces to f ′when the coin lands with tails up.
• ball–coin lotteries that are obtained from coin–ball lotteries by reversing the order of acoin toss and a ball draw.
An example from each class is shown in Figure 1. The list of lotteries can be found in
Table 2 in the Appendix. An English translation of the instruction is provided in the online
supplementary material.
Each class consists of 10 lotteries. For each lottery, we employ the BDM method to elicit
their WTPs. Here, we employ I-RIS to enhance the credibility of the elicited preferenceindices.

15

(a) Risky lottery(b) Ambiguous lottery(c) Coin–ball lottery(d) Ball–coin lottery

Figure 1: Example of each class of lottery

4.1.2

Preference indices

From WTPs that we have elicited, we construct three indices, the ambiguity attitude index(IA ), the preference for randomization index (IP R ), and the reversal-of-order index (IRO ), tosummarize the preference characteristics of each participant. Suppose the participant’s reR ) and V A = (v A , . . . , v A ),ported WTPs of risky and ambiguous lotteries are V R = (v1R , . . . , v10
110

where viR is the WTP of the ith risky lottery and similarly for ambiguous lotteries. Then,the IA of the participant is deﬁned by
IA = |{i|viR > viA }| − |{i|viR < viA }|as a measure of his/her degree of ambiguity aversion. IA ranges from −10 to 10. In theanalyses below, we label participants as ambiguity averse or seeking when IA ≥ 4 or IA ≤ −4,
respectively, and as ambiguity neutral when neither is the case.
Similarly, suppose the participant’s WTP in coin–ball lotteries and ball–coin lotteries are16

V CB and V BC . Then, we deﬁne IP R and IRO as
IP R = |{i|viCB > viA }| − |{i|viCB < viA }|,IRO = |{i|viBC > viCB }| − |{i|viBC < viCB }|.

To interpret IP R , observe that the ith coin–ball lottery is a fair lottery over two ambiguous
lotteries such that one of them is obtained from the other by interchanging ball colors. Moreover, one of such lotteries is the ith ambiguous lottery. Thus, assuming symmetric beliefs overthe likelihood of ball colors, comparing corresponding ambiguous/coin–ball lotteries gives adegree of preference for randomization.
IRO is calculated by comparing the valuations of ball–coin lotteries and coin–ball lotteries.
We are interested in this index because it reﬂects participants’ conformity to the reversal-oforder axiom.
As discussed in Section 2.2, if a participant is not ambiguity neutral, the reversal-oforder axiom implies his/her nonneutrality to randomization. Thus, we are interested in theirattitude toward reversal-of-order.
Hypothesis 1. The participants’ preferences satisfy the reversal-of-order axiom.
If a participant’s preferences satisfy the axiom, his/her IRO is zero because s/he is indiﬀerent
between the corresponding coin–ball lottery and ball–coin lottery, which are the same except
the timing of uncertainty resolution: viBC − viCB = 0, i = 1, . . . , 10. Thus Hypothesis 1 isrejected if these numbers are signiﬁcantly diﬀerent from zero.

4.1.3

Descriptive statistics

Before examining the joint analysis of the data from parts 1 and 2 of the experiment, let us
ﬁrst describe the elicited preferences.
Figure 2 shows the histograms of IAM (left), IP R (middle), and IRO (right) of the full17

(a) Histogram of IAM(b) Histogram of IP R(c) Histogram of IRO

Figure 2: Histograms of IAM , IP R , and IRO in the whole sample. Red vertical lines in each histogram
represent the mean of that index.

(a) Ambiguity averse participants(b) Ambiguity neutral participants (c) Ambiguity seeking participants

Figure 3: Histogram of IP R for each ambiguity attitude. The red vertical line in each histogram represents the mean of the subpopulation.
sample (192 observations each). Each of the three panels shows that whereas there is a
substantial fraction of participants with an index value of indices equal to zero, there exists
substantial heterogeneity in the value of each of the three indices. We classify participants
based on their IAM , and then examine the distribution of IP R and IRO .
Figure 3 shows the histograms of IP R for ambiguity averse (left), neutral (center), andseeking (right) participants. These three panels of Figure 3 demonstrate a heterogeneity ofpreferences for randomization among participants within each ambiguity class. Whereas themean IP R of ambiguity averse participants is signiﬁcantly diﬀerent from zero (p < 0.001,

18

(a) Ambiguity averse participants(b) Amguity neutral participants(c) Ambiguity seeking participants

Figure 4: IRO for each ambiguity attitude. The red vertical lines represent its mean of the subpopulation.
Wilcoxon signed-rank test), that of ambiguity seeking and neutral participants is not (p =0.068 and p = 0.619, respectively). It is noteworthy that a nonnegligible number of ambiguity
averse (seeking) participants have negative (positive) preference for randomization. This
result suggests that preferences for randomization are not determined solely by ambiguity
attitude. The IP R s are signiﬁcantly diﬀerent across the three ambiguity classes (p < 0.001,
Kruskal–Wallis test).
Figure 4 shows the histograms of IRO for ambiguity averse (left), seeking (center), andneural (left) participants. Whereas the mean of IRO for the ambiguity seeking participants issigniﬁcantly diﬀerent from zero (p = 0.042, Wilcoxon signed-rank test), that of the ambiguity
seeking and neutral participants is not (p = 0.333 and p = 0.681, respectively). The IRO s are
not signiﬁcantly diﬀerent across the three ambiguity classes (p = 0.116, Kruskal–Wallis test).
We now consider a test of the reversal-of-order axiom at the individual level. For eachparticipant, we tested the hypothesis that his/her mean of viBC − viCB (i = 1, . . . , 10) is
zero using a nonparametric bootstrap test. As a result, for 52 of the 195 participants, thehypothesis is rejected.17 Because a nonnegligible number of participants violated the reversalof-order axiom, Hypothesis 1 is rejected. However, this result does not guarantee the incentive
17

Among the participants whose null hypothesis is rejected, 20 tend to prefer coin–ball lotteries and 32 tend toprefer ball–coin lotteries.

19

Figure 5: Choice alternatives in Part 2.
compatibility of RIS. Indeed, the among ambiguity averse (resp. seeking) participants, 20 outof 29 (resp. 41 out of 66) are consistent with reversal-of-order. Thus, if such participants viewan experiment as a single decision choice problem, RIS would not be incentive compatible.
However, once we pooled all the participants and applied the same test, the hypothesis thatthe mean is zero is not rejected. This suggests that the aggregate behavior of the participants
does not contradict the reversal-of-order axiom, which is consistent with Oechssler et al.
(2019).
Result 1. A nonneglible number of participants violated the reversal-of-order axiom. The
aggregate behavior of participants is consistent with the reversal-of-order axiom.

4.2

Part two – Comparison of RIS and I-RIS

4.2.1

Task and incentive

In Part 2, participants are informed that they will face two identical two-alternative choices.
The two lotteries involved in these choices are ambiguous lotteries and are shown in Figure 5.
An English translation of the instructions is provided in the online supplementary material.
One of the lotteries pays 510 JPY if the drawn ball is red or blue and 10 JPY if thedrawn ball is green or yellow, whereas the other lottery pays 0 JPY if the drawn ball is red
or blue and 500 JPY if the drawn ball is green or yellow. Because participants are instructed20

about the repetition of the choice situation, if an incentive scheme employed here is incentive
compatible and a participant is not indiﬀerent between these two lotteries, s/he should choose
the same lottery twice.
There are two treatments, Independent (IND) and Correlated (COR), in which we useddiﬀerent incentive schemes. Whereas the standard RIS is used in COR, I-RIS is used in IND.
In both treatments, one of two choice situations is randomly picked to use for compensation
at the beginning of the experiment. The diﬀerence between the treatments is in the ﬁnal
stage of the experiment where the uncertainties of lotteries are resolved. Whereas only one
ball is drawn in COR, four balls are independently drawn with replacement in IND; they areused to evaluate four lotteries that were presented to the participants. Thus, the two choicesituations in COR do not satisfy the independence condition, but those of IND do.
The two lotteries in this part pay positive, and almost the same, amounts of money indiﬀerent states. Thus if a participant is ambiguity averse, s/he may prefer their state-wise
mixture of to themselves. In COR, if s/he believes ex ante randomization of RIS hedges
ambiguity likewise, she may intentionally choose diﬀerent alternatives in the two repetitions.
However, a participant has no incentive to choose diﬀerent alternatives in IND if s/he satisﬁes
Condition 1, which is what Proposition 1 states.
We now test our hypotheses using the data from Part 2. We say a participant is inconsistent if s/he chooses diﬀerent lotteries during the repetitions. If the use of I-RIS improves
incentive compatibility, we expect that the proportion of inconsistent participants is smaller
in IND than in COR.
Hypothesis 2. The proportion of participants making inconsistent choices is smaller in IND
than in COR.
Because the two lotteries in each choice situation pay more in diﬀerent states, behaving
inconsistently may hedge ambiguity. Therefore, it is expected that the more randomization
seeking (averse) a participant is, the more (less) s/he tends to be inconsistent. Thus, we

21

expect the following:
Hypothesis 3. In COR, randomization seeking participants are more likely to make aninconsistent choice than randomization averse participants.
Finally, we consider the treatment eﬀect conditional on randomization attitude. In Section
3, it is shown that I-RIS is incentive compatible under Condition 1. If I-RIS improves thestandard RIS, the proportion of randomization seeking participant who behave consistently
would increase in IND, compared with COR. However, randomization averse participants
would have a weaker incentive to behave inconsistently in IND than in COR.
Hypothesis 4. Randomization seeking (resp. averse) participants are more (resp. less) likely
to make inconsistent choices in COR than in IND.

4.2.2

Results

In IND, 13 out of 95 participants’ choices were inconsistent, whereas 17 out of 97 were inconsistent in COR. The fractions of participants making inconsistent choices are not signiﬁcantly
diﬀerent between the two treatments (p = 0.5523, Fisher’s exact test). Thus, we reject Hypothesis 2, which states that a smaller fraction of participants making inconsistent choices in
IND than in COR.
Result 2. There is not a signiﬁcant diﬀerence between IND and COR in the proportion ofparticipants behaving inconsistently.
To test Hypotheses 3 and 4, we focus on the participants with nonneutral randomization
attitudes and used logistic regressions to investigate the relationship between the elicited
preference indices and inconsistent choices.18 The dependent variable is a dummy variable
that equals 1 if a participant is making inconsistent choices, and 0 otherwise. The set ofindependent variables includes a dummy variable that equals 1 for COR, and 0 otherwise, a18

Including randomization neutral participants does not change the following results.

22

Dependent variable:inconsistent2.163∗
p = 0.054
2.115∗
p = 0.070−2.856∗p = 0.052−3.367∗∗∗p = 0.001

correlated
R seekingcorrelated:R seeking
Constant
Observations
Log Likelihood
Akaike Inf. Crit.
Note:

90
−33.99375.985∗p<0.1; ∗∗ p<0.05; ∗∗∗ p<0.01

Table 1: Logistic regressions with ambiguity-neutral participants omitted.
dummy variable that equals 1 if a participant is randomization seeking, and 0 otherwise, as
well as interactions between the randomization dummies and COR. Table 1 shows the resultsof the speciﬁcation we considered.
We used the Wald test to understand the relation of randomization attitudes and inconsistency of choices in COR. We found no signiﬁcant eﬀect of randomization attitudes toinconsistency (p = 0.4). Thus Hypothesis 3 is rejected.
Result 3. In COR, there is no signiﬁcant diﬀerence in the proportion of participants behaving
inconsistently between randomization seeking and randomization averse participants.
We, again, used the Wald test to understand the treatment eﬀect among the subpopulationof randomization seeking participants, and found that the treatment eﬀect is insigniﬁcant(p = 0.46). The null hypothesis that the coeﬃcient of the COR dummy is zero is not rejected(p = 0.054), which shows that the treatment eﬀect for randomization averse participants is
also insigniﬁcant. Thus, Hypothesis 4 is rejected.19
19

We also estimated a similar model in which the dummy for randomization attitudes is replaced with dummies

23

Result 4. There is no signiﬁcant diﬀerence between COR and IND in the proportion ofinconsistent participants among randomization seeking or averse participants.

5

Discussion

In this section, we discuss the results of our experiment. As explained in Section 2, sometheoretical studies ﬁnd that RIS is not incentive compatible when an experiment involves
ambiguity. They conjecture that participants may hedge ambiguity by exploiting the randomness of RIS, assuming they are (at least partially) indiﬀerent about the order of theuncertainties. The rejection of Hypothesis 1 means that their concern do not apply to allparticipants. However, this result does not guarantee the incentive compatibility of RIS because the majority of participants with nonneutral ambiguity attitudes is consistent withreversal-of-order.
Using data collected in Part 2, we tested three hypotheses. Hypothesis 3 states that in
COR, randomization seeking participants are more likely to make an inconsistent choice thanrandomization averse participants. Its rejection suggests that inconsistent choices when RIS
is used cannot be explained by preferences for randomization that have roots in ambiguity
attitudes.
Hypothesis 2 and Hypothesis 4 were about the treatment eﬀect. Hypothesis 2 states thatthe proportion of participants making inconsistent choices are smaller in IND than in COR.
Hypothesis 4 states that randomization seeking (resp. averse) participants are more (resp.
less) likely to make inconsistent choices in COR than in IND. We rejected that both of thesehypotheses. That is, no signiﬁcant improvement is obtained by the use of I-RIS. This may
be because the observed inconsistent behavior in Part 2 might not be caused by nonneutral
randomization attitudes. This is the case if participants viewed the choice situations infor ambiguity attitudes. Conducting similar analyses, we obtained results that are consistent with those presented
in this and the previous paragraph.

24

isolation and inconsistent choices are just errors without deliberation. Overall, the results
above suggest that misreporting under RIS, if any, is not motivated by the desire to hedge
ambiguity.

6

Conclusion

We proposed a modiﬁcation of RIS, named I-RIS, in which participants face ambiguity from
diﬀerent sources in diﬀerent choice situations. Then, we posed a preference condition thatguarantees the incentive compatibility of I-RIS.
We conducted an experiment to test the performance of RIS directly and indirectly andobtained the following results. First, whereas nonneglibile fraction of participants violate
the reversal-of-order axiom, the majority of participants with nonneutral ambiguity attitudes
may be consistent with it. This implies that RIS is not incentive compatible if they regard an
experiment as a single decision choice problem. Second, we found no signiﬁcant diﬀerence inthe performances of RIS and I-RIS. Third, randomization attitudes do not explain inconsistentbehaviors under RIS. Fourth, there is no signiﬁcant diﬀerence in performance between RIS
and I-RIS among randomization seeking or averse participants. These results suggest that
preferences for randomization, which are motivated by the hedging of ambiguity, do not
reduce the reliability of RIS.

References
Anscombe, F. J. and R. J. Aumann (1963) “A Deﬁnition of Subjective Probability,” The
Annals of Mathematical Statistics, Vol. 34, pp. 199–205.
Azrieli, Yaron, Christopher P. Chambers, and Paul J. Healy (2018) “Incentives in Experiments: A Theoretical Analysis,” Journal of Political Economy, Vol. 126, pp. 1472–1503.

25

(2020) “Incentives in Experiments with Objective Lotteries,” Experimental Economics, Vol. 23, pp. 1–29.
Bade, Sophie (2015) “Randomization Devices and the Elicitation of Ambiguity-Averse Preferences,” Journal of Economic Theory, Vol. 159, pp. 221–235.
Baillon, Aurelien, Yoram Halevy, and Chen Li (2019) “Experimental Elicitation of Ambiguity
Attitude,” Working Paper.
Becker, Gordon M., Morris H. Degroot, and Jacob Marschak (1964) “Measuring Utility by aSingle-Response Sequential Method,” Behavioral Science, Vol. 9, pp. 226–232.
Brown, Alexander L. and Paul J. Healy (2018) “Separated Decisions,” European Economic
Review, Vol. 101, pp. 20–34.
Cox, James C., Vjollca Sadiraj, and Ulrich Schmidt (2014) “Asymmetrically Dominated
Choice Problems, the Isolation Hypothesis and Random Incentive Mechanisms,” PLoS
ONE, Vol. 9, p. e90742.
(2015) “Paradoxes and Mechanisms for Choice under Risk,” Experimental Economics, Vol. 18, pp. 215–250.
Cubitt, Robin P., Chris Starmer, and Robert Sugden (1998) “On the Validity of the Random
Lottery Incentive System,” Experimental Economics, Vol. 1, pp. 115–131.
Dominiak, Adam and Wendelin Schnedler (2011) “Attitudes toward Uncertainty and Randomization: An Experimental Study,” Economic Theory, Vol. 48, pp. 289–312.
Eichberger, Jürgen, Simon Grant, and David Kelsey (2016) “Randomization and Dynamic
Consistency,” Economic Theory, Vol. 62, pp. 547–566.
Freeman, David J., Yoram Halevy, and Terri Kneeland (2019) “Eliciting Risk Preferences
Using Choice Lists,” Quantitative Economics, Vol. 10, pp. 217–237.

26

Gilboa, Itzhak and David Schmeidler (1989) “Maxmin Expected Utility with Non-Unique
Prior,” Journal of Mathematical Economics, Vol. 18.
Greiner, Ben (2015) “Subject Pool Recruitment Procedures: Organizing Experiments with
ORSEE,” Journal of the Economic Science Association, Vol. 1, pp. 114–125.
Harrison, Glenn W. and J. Todd Swarthout (2014) “Experimental Payment Protocols andthe Bipolar Behaviorist,” Theory and Decision, Vol. 77, pp. 423–438.
Holt, Charles A (1986) “Preference Reversals and the Independence Axiom,” American Economic Review, Vol. 76, pp. 508–515.
Karni, Edi and Zvi Safra (1987) “”Preference Reversal” and the Observability of Preferences
by Experimental Methods,” Econometrica, Vol. 55, p. 675.
Ke, Shaowei and Qi Zhang (2020) “Randomization and Ambiguity Aversion,” Econometrica,
Vol. 88, pp. 1159–1195.
Kuzmics, Christoph (2017) “Abraham Wald’s Complete Class Theorem and Knightian Uncertainty,” Games and Economic Behavior, Vol. 104, pp. 666–673.
Lu, Jay (2019) “Random Ambiguity,” Working Paper.
Oechssler, Jörg, Hannes Rau, and Alex Roomets (2019) “Hedging, Ambiguity, and the Reversal of Order Axiom,” Games and Economic Behavior, Vol. 117, pp. 380–387.
Oechssler, Jörg and Alex Roomets (2014) “Unintended Hedging in Ambiguity Experiments,”
Economics Letters, Vol. 122, pp. 243–246.
Saito, Kota (2015) “Preferences for Flexibility and Randomization under Uncertainty,” American Economic Review, Vol. 105, pp. 1246–1271.

27

Segal, Uzi (1990) “Two-Stage Lotteries without the Reduction Axiom,” Econometrica, Vol.
58, p. 349.
Seo, Kyoungwon (2009) “Ambiguity and Second-Order Belief,” Econometrica, Vol. 77, pp.
1575–1605.

28

A

List of lotteries
#
1
2
3
4
5
6
7
8
9
10

1
2
3
600 6000
600 400 200
600 300 300
800 8000
800 600 200
800 400 400
1000 1000
01000 800 2001000 500 500
1200 1200
0

4
0
0
0
0
0
0
0
0
0
0

#
red blue green yellow
1 600 600
002 600 400
200
03 600 300
300
04 800 800
005 800 600
200
06 800 400
400
0
7 1000 1000
008 1000 800200
09 1000 500
500
010 1200 1200
00(a) Risky lotteries(b) Ambiguous lotteries#

Head
red blue green yellow red
1 600 600
0002 600 400
200
0003 600 300
300
04 800 800
0005 800 600
200
006 800 400
400
00
7 1000 1000
00008 1000 800200
09 1000 500
500
0
0
0
10 1200 1200
0
0

blue
0
200
300
0
200
400
0
200
500
0

Tail
green
600
400
300
800
600
400
1000
800
500
1200

yellow
600
600
600
800
800
800
1000
1000
1000
1200

(c) Coin–ball lotteries

#
1
2
3
4
5
6
7
8
9
10

red
blue
green
head tail head tail head
tail
600
0
600
0
0 600
600
0
400 200
200 400600
0300 300
300 300
800
0
800
0
0 800
600 200
200 600800
0800
0400 400
400 40000 10001000
0 10001000
0800 200
200 8001000
0500 500
500 5001200
0 12000
0 1200yellow
head
tail
0 600
0 600
0 600
0 800
0 800
0 800
0 10000 10000 10000 1200(d) Ball–coin lotteries

Table 2: List of lotteries used in Part one.

29

B

Instructions

B.1

Instruction for Part 1

In Part 1, we randomized the order of the presentation of its subparts 3 and 4 in which
coin–ball lotteries and ball–coin lotteries are evaluated, respectively. The following are theinstructions for the case where subpart 3 is presented before subpart 4. The only diﬀerence
between this and the other case is the order of explanations of these subparts.

Contents of the Experiment
In this experiment, you will be asked to answer questions about lotteries that allow you toreceive an uncertain amount of money. At the end of the experiment, you will receive areward. The reward is a sum of money that depends on the outcome of the experiment andthe participation fee of 500 yen.
Decisions are made by you alone, and the actions of other participants do not aﬀect yourdecisions or rewards. The experiment consists of four parts. Please answer 10 questions ineach part.
The lotteries are designed to provide monetary rewards for uncertain events. One example
is the lottery illustrated below, where a four-sided die is thrown and you receive 600 yen if
the die rolls a 1 or 2, and 0 yen if the die rolls a 3 or 4.

30

In this experiment, you will be asked to evaluate the value of various lotteries, where thevalue is a certain amount of money that is worth as much to you as receiving the lottery
ticket. There is no right answer.
The value of a lottery ticket should range from the lowest possible value to the highest
possible value that you could receive from that lottery ticket. For the lottery shown above,
the range is from 0 yen to 600 yen.
The ﬁgure below is an example of the input screen. Below the lottery diagram, you can
see a text description of the lottery. Use the slider to enter the value of the lottery. Whenyou have ﬁnished, click the ”→” button to go to the next page.

Answer the same questions for a total of 40 diﬀerent lotteries. Starting from the next page,
we will explain the types of lotteries that are presented in each part.

Description of Part 1
The realizations of the lotteries in Part 1 are determined by throwing a four-sided die. Eachside has a probability of 1/4 of appearing. The ﬁgure below shows an example of this typeof lottery.

31

This is a lottery where you get 600 yen if the die roll is 1 or 2, and 0 yen if the die roll is 3
or 4. Here are some questions about the 10 diﬀerent types of lotteries.

Explanation of Part 2 to Part 4
The other day, a student at Osaka University ﬁlled an urn with 100 balls of four diﬀerent
colors: red, blue, green, and yellow. It is not necessary to have the same number of balls ofeach color. A computer simulation of this urn is used in parts 2, 3, and 4. The realized valueof the lottery in these parts depends on the color of the balls randomly taken from the urn.

Description of Part 2
The realizations of the lotteries in Part 2 are determined solely by the color of the ball thatis drawn from the urn. The ﬁgure below shows an example of this type of lottery.

This is a lottery where a ball is drawn and if it is red or blue you get 1000 yen, and if it32

is green or yellow you get 0 yen. There are some questions about the 10 diﬀerent types oflotteries.

Description of Part 3
The realizations of the lottery presented in Part 3 are determined by ﬁrst tossing a coin andthen drawing a ball from the urn. The probability of tossing a coin and having it lands onheads or tails is 1/2. The ﬁgure below presents an example of this type of lottery.

In this lottery, a coin is tossed and if it lands heads up and the drawn ball is red or blue, you
get 1000 yen. Furthermore, if the coin lands tails up and the drawn ball is green or yellow,
you get 1000 yen. Otherwise, you get nothing. The questions are about 10 diﬀerent lotteries

Description of Part 4
The realizations of lotteries that are presented in Part 4 are determined by ﬁrst drawing aball from the urn and then tossing a coin. The probability of the coin landing on heads ortails is 1/2. The ﬁgure below shows an example of this type of lottery.

33

In this lottery, if the ball is red or blue and the coin lands heads up, you get 1000 yen. If theball is green or yellow and the coin lands tails up, you get 1000 yen. Otherwise, you get 0
yen. We ask you about these 10 diﬀerent kinds of lotteries.

Reward Description
Compensation for the experiment is determined as follows.
1. Immediately after the explanation of the experiment, the computer• selects one lottery ticket from among all the lotteries, and• randomly chooses a value p in the range from the lowest to the highest amountthat you might receive from the lottery.
2. After you have evaluated all the lotteries, you will get whichever you prefer between thelottery that was chosen in step 1 and p yen with certainty.
• If the valuation of the chosen lottery is greater than or equal to p yen, then thelottery must be better, so we simulate the lottery and pay the realized value.
• If the value of the lottery is less than p yen, then p yen should be preferred, so wepay p yen.
In step 1 of the above procedure, every lottery ticket is chosen with the same probability,and every possible value of p is chosen with the same probability. However, the lottery andthe value of p are not be known until the stage where the reward is paid.

34

Under this procedure, it is optimal for you to provide your true valuation. We will explain
why it is optimal for you to answer the true valuation. For example, if your true value of alottery ticket is 200 yen, but you answer that it is worth 300 yen, you will lose money whenthe value of p lies between 200 and 300. As an example, consider the case where the value ofp is 250. In this case, you would have been rewarded with your preferred option out of theguaranteed 250 yen and the lottery. If you had answered 200 yen, you would have preferredthe 250 yen with certainty to the lottery, so you would have been paid 250 yen. However, if
you had answered 300 yen, it would be determined that you prefer the lottery to the 250 yen
with certainty, and you would be paid the lottery. Because your true valuation is 200 yen, you
would prefer 250 yen to the lottery. Therefore, if you had provided your true valuation, you
would have obtained a better result. This example illustrates the case where the valuation isoverreported, but underreporting will still result in a loss.
Each lottery realization is determined independently of each other by the following procedure, after you have answered all the questions.
Lotteries for Part 1 For each lottery on the computer, toss a coin to determine the realized
value.
Lotteries for Part 2 For each lottery, a ball is randomly drawn from the urn to determine
the realized value. Each time, the ball is returned to the urn.
Lotteries for Part 3 For each lottery, a coin is tossed and a ball is randomly drawn todetermine the realization. Each time, the ball is returned to the urn.
Lotteries for Part 4 For each lottery, a ball is drawn at random and then a coin is tossed
to determine the realization. Each time, the ball is returned to the urn.
In summary, the experiment will proceed along the following lines.
1. A lottery and a random number p will be chosen as rewards.
2. All lotteries will be evaluated by you.

35

3. According to the lottery value and the value of p chosen in step 1, you receive the lottery
ticket or p yen with certainty.
4. The realized values of all lotteries are determined independently.
5. If a lottery is received, the realized value is taken as the reward.
This concludes the explanation of the experiment.

B.2

Instruction for COR treatment in Part 2

Contents of the Experiment
In this experiment, you will be asked to answer questions about lotteries that pay an uncertain
amount of money. At the end of the experiment, you will receive a reward. Decisions are
made by you alone, and the actions of other participants do not aﬀect your decisions or
rewards.
The other day, a student at Osaka University ﬁlled an urn with 100 red, blue, green, andyellow balls. It is possible that not all balls have the same number In this experiment, we
will use a computer simulation of this urn.
This experiment uses with the two lotteries shown below.

On the left is a lottery where you receive 510 yen if the ball is red or blue, and 10 yen if it
is green or yellow. On the right is a lottery where you receive 0 yen if the ball is red or blue,and 500 yen if it is green or yellow.

36

In this experiment, the participants were asked twice which of the two lotteries they would
choose. The ﬁgure below shows the response screen.

Check one of the options on the left or right side of the screen, and click the ”→” button.
Then, the lottery shown on the next page will be the same pair of lotteries again. When you
answer the second question, the experiment is over.

Reward Description
At the beginning of the experiment, the computer chooses one question to be rewarded, either
the ﬁrst or the second. Both questions have the same probability of being chosen. After you
have answered both questions, the question chosen by the computer will be revealed. The
lottery you had chosen for that question will be the reward.
At the end of the experiment, a ball is drawn from the urn on the computer. The color of
the ball determines the realized value of the rewarded lottery. Your reward for this experimentwill be the realized value of the lottery plus 300 yen.
In summary, the experiment proceeds in the following way.
1. The computer randomly decides which question will be the target of the reward.

37

2. You answer the same question twice, which relates to which of the two lotteries you
would prefer.
3. You receive the lottery for the reward question the computer chose in step 1.
4. A ball is drawn at random from the urn to determine the realized value of the lottery,and you receive the realized value of your lottery plus the participation fee.

B.3

Instruction for IND treatment in Part 2

Contents of the Experiment
In this experiment, you will be asked to answer questions about lotteries that allow you toreceive an uncertain amount of money. At the end of the experiment, you will receive areward. Decisions are made by you alone, and the actions of other participants do not aﬀect
your decisions or rewards.
The other day, a student at Osaka University ﬁlled an urn with 100 red, blue, green, andyellow balls. It is possible that not all balls have the same number. In this experiment, we
will use a computer simulation of this urn.
This experiment uses with the two lotteries shown below.

On the left is a lottery where you receive 510 yen if the ball is red or blue, and 10 yen if it
is green or yellow. On the right is a lottery where you receive 0 yen if the ball is red or blue,and 500 yen if it is green or yellow.

38

In this experiment, the participants were asked twice which of the two lotteries they would
choose. The ﬁgure below shows the response screen.

Check one of the options on the left or right side of the screen, and click the ”→” button.
Then, the lottery shown on the next page will be the same pair of lotteries again. When you
answer the second question, the experiment is over.

Reward Description
At the beginning of the experiment, the computer chooses one question to be rewarded,
either the ﬁrst or the second question. Both questions have the same probability of being
chosen. After you have answered both questions, the question chosen by the computer will
be revealed. The lottery you had chosen for that question will be the reward.
At the end of the experiment, the realizations of all the lotteries are determined. Twolotteries are shown for the ﬁrst and second questions, so there are four lotteries in total. The
realized value of each lottery is determined by drawing a ball from the urn independently. In
other words, the procedure of drawing one ball from the urn and putting it back is repeatedfour times. Your reward for this experiment is the realized value of the lottery plus the39participation fee of 300 yen.
In summary, the experiment proceeds in the following way.
1. The computer randomly decides which question will be the target of the reward.
2. You answer the same question twice, which relates to which of the two lotteries you
would prefer.
3. You receive the lottery for the reward question the computer chose in step 1.
4. Four balls are drawn at random and independently to determine the realized values ofthe lotteries, and you receive the realized value of your lottery plus the participationfee.

40