An Optimizing Neuroeconomic Model ofDiscrete Choice∗
Michael Woodford
Columbia University
February 2, 2014

Abstract
A model is proposed in which stochastic choice results from noise in cognitive processing rather than random variation in preferences. The mental processused to make a choice is nonetheless optimal, subject to a constraint on available
information-processing capacity that is partially motivated by neurophysiological evidence. The optimal information-constrained model is found to offer abetter fit to experimental data on choice frequencies and reaction times thaneither a purely mechanical process model of choice (the drift-diffusion model)
or an optimizing model with fewer constraints on feasible choice processes (therational inattention model).

∗I would like to thank Ian Krajbich for sharing the data from Krajbich et al. (2010), Ian Krajbich,
Stephen Morris, Antonio Rangel, and Michael Shadlen for helpful discussions, Stéphane Dupraz andKyle Jurado for research assistance, and the Institute for New Economic Thinking and the Kumho
Visiting Professorship, Yale University, for supporting this research.

1

Explaining the Stochasticity of Choice

The standard approach to the explanation and prediction of choice behavior in economics posits that a decisionmaker (DM) will on any occasion reliably choose theoption, among the set presented as alternatives, that has the highest value according to some utility function (or more generally, some preference ordering) that isassumed to remain constant across choice situations. The idea that the individual’s
preferences are well-defined and constant across situations is relied upon, not only
in seeking to predict behavioral responses to contemplated interventions (say, theimposition of a tax on a particular consumer good), but when economists propose to
use the preferences implicit in past observed behavior to make judgments about thedegree to which individual welfare should be improved or reduced by a contemplated
intervention.
A basic implication of this approach is that the choice made by an individual
should be a completely deterministic function of the characteristics of the options
available on a given occasion (outside the very special case of an exact tie between
the valuations of two options), at least for a given individual and if all of the relevantcharacteristics are taken into account. Yet since at least the work of Mosteller andNogee (1951) it has been noted in experimental studies of choice behavior that choices
have an apparently random element — not only in the sense that less than 100
percent of the variation in choices across trials can be accounted for by a deterministic
function of a particular list of measured characteristics, but that a given subject does
not always make the same choice when presented again with exactly the same setof alternatives, sometimes only a few minutes later (Hey, 1995, 2001; Ballinger andWilcox, 1997; Cheremukhin et al., 2011). More than 50 years ago, Luce (1959)
already noted that the “presupposition ... that choice behavior is best described asa probabilistic, not an algebraic [i.e., deterministic] phenomenon, ... is by now acommonplace in much of psychology,” though “a comparatively new and unprovenpoint of view in utility theory.”1
In economics, the most orthodox interpretation of stochastic choice has been one
according to which DMs do choose their preferred options on all occasions, but theirutility functions contain a random element — so-called “random utility models” (e.g.,
McFadden, 1974; Manski, 1977). Another possible interpretation, however, is that1

He also suggests that “economists when pressed will admit that the psychologist’s assumption isprobably the more accurate, but they have argued that the resulting simplicity warrants an algebraic
idealization” (p.2), a view that he seeks to rebut.

1the randomness in observed choice is due to random errors in the DM’s assessmentof the choice situation. Indeed, this is one interpretation sometimes given to themathematical formalism of random-utility models: the random utility assigned to aparticular choice option on a particular occasion represents a random error in themeasurement of the true utility.2
In our view, there are important reasons to prefer the interpretation of stochastic
choice as representing random errors in cognitive processing, akin to random errors inperception. The first is that even when random preferences can be hypothesized thatwould account for observed data as trial-by-trial optimal choices, the kind of variationin preferences required is not especially plausible. For example, if the choice madebetween a pair of options fluctuates after the passage of only a few minutes, thenone might hypothesize that preferences vary randomly and that the persistence of apreference state is shorter than the interval between trials, but such high-frequency
variation seems implausible in something deserving to be called a genuine preference;on the other hand, under the hypothesis of random errors in perception, one would
expect the correlation between responses to be equally low, regardless of the time
elapsed between the occasions on which independent assessments are made, as longas the time is sufficient for the subject not to recognize the question as an exact
repetition and to recall the previous judgment rather than assessing the alternatives
afresh.
Second, a longstanding literature in experimental psychology and neurosciencehas documented the randomness of the responses that subjects give when asked tomake perceptual judgments, in a variety of sensory domains. The probability of the2

Lu (2013) discusses the formal equivalence of a particular model of stochastic choice, in which
the randomness of choice (relative to the information available to an econometrician) is attributed
to dependence on an imperfect private signal about the value of a state variable that affects theDM’s rewards from action, and a type of random-utility model. Lu’s model differs, however, fromthe kind of interpretation proposed here. In his framework, choice is random from the standpoint of
the econometrician only because the rewards from action depend on random states not observableby the econometrician prior to the decision, and about which the DM has superior information. The
model proposed here is instead one in which choice will appear random even to an econometrician
with knowledge of the state x, which is fully sufficient to determine the reward from each of thepossible actions; for here the DM’s information set is not assumed to include everything observableby the econometrician. Thus the present model is consistent with the existence of randomness evenin cases where all payoff-relevant aspects of the choice options are under the complete control of theexperimenter.

2

subject’s giving one response instead of the other (when required to choose between
two possible responses) is found to vary continuously with continuous changes in thecharacteristics of the stimuli, rather than jumping discontinuously as the correct answer (say, to the question which of two tones is louder) would; and the “psychometricfunctions” that are plotted showing the experimentally determined relationship3 aresimilar in form to plots showing how probabilities of choice among lotteries, say, varywith variation in the promised payoffs.4 In the case of perceptual discrimination
tasks, variation in preferences is clearly not a candidate explanation; but one hasthen a more parsimonious theory if randomness of the relation between objective
characteristics and their mental representation is also the explanation for stochastic
choice.
And finally, in economic choices just as in the case of perceptual discrimination
tasks, there is observed to be a systematic relation between the time required forexperimental subjects to make a decision and the characteristics of the alternatives;
specifically, the average response time is shorter in the case of “easier” choices, which
are also the ones for which repeated trials yield the same response a greater fractionof the time.5 Clithero and Rangel (2013) and Krajbich et al. (2013) both findthat the relationship between response time and the frequencies with which different
choices are made is sufficiently systematic that data on a given subject’s response
times when choosing between particular pairs on goods on past information improves
one’s ability to predict the subject’s future choices (even if response times themselves
are not of intrinsic interest). This association between response time and the degreeof predictability of the subject’s eventual choice is easily understood in terms of amodel in which the randomness of choice results from random noise in the mental
representation of the available options (which interferes more with the subject’s ability
to make a clear choice when the true values of the options are not too different), as we
discuss further below; but there is no reason to expect such an association under theview that the values of the options are observed with perfect precision, but preferences
are different at different points in time.
The hypothesis that choices are based on errors, however, rather than on DMs’
true preferences, has some potentially unappealing features, in the absence of a more
3

See, e.g., Green and Swets (1966) or Gabbiani and Cox (2010), chap. 25.
See, e.g., Figure 2 from Mosteller and Nogee (1951).
5
See Figure 1 below, for an example of choice between alternative consumption goods.
43specific theory about the nature of the errors. First, there is the question whether
such a theory places any testable restrictions at all on the behavior that may beobserved.6 More specific predictions can be made under some strong assumption
about the probability distribution of valuation errors, but if this is chosen arbitrarily(perhaps simply for analytical or computational convenience, as is often the case),
one must worry about the extent to which one’s conclusions may be artifacts of a
possibly incorrect specification. And second, there is a question whether observed
behavior provides any basis for welfare evaluation of contemplated interventions, if
one cannot infer from the observation that something is chosen that it provides moreof what the DM would like to have.
The present study provides an account of stochastic choice between discrete alternatives that seeks to address these problems, by representing the mental process used
to make a choice as an optimal choice algorithm, from the standpoint of maximizing
expected utility, subject to a constraint on the information-processing capacity of theneural pathways that supply the information about the current choice situation on
the basis of which the choice is made. The kind of choice algorithm that is optimalin this constrained sense involves random errors, not because they are assumed to be
inevitable but because such an algorithm reduces the amount of information about
the variation in choice situations that must be obtained; and it makes specific (andempirically testable) predictions about the way in which the rate at which random
errors are made should very with the choice situation. And because the choice algorithm is hypothesized to be optimized to serve a particular objective, it allows one
to infer “true” preferences of the DM from observed stochastic choice behavior (andreaction times), even if these are not simply identified with the outcomes that arechosen. These true preferences can then reasonably provide a basis for individualistic welfare analysis of contemplated interventions, in addition to positive predictions
about the behavior that one expects to see.
In proposing that that the accuracy of feasible choice algorithms is limited byan information-theoretic constraint, the model here is an application of Sims’ (2003,
2010) theory of “rational inattention.”7 However, unlike Sims’ theory, the model
6

Haile et al. (2008) raise this question about the concept of “quantal response equilibrium” as asolution concept for strategic games that allows random errors in players’ actions.
7
Previous discussions of rational inattention as a source of randomness in discrete choices include
Woodford (2008, 2009), Pinkovskiy (2009), Cheremukhin et al. (2011), Matějka and MacKay (2013),

4

proposed here imposes additional constraints on the class of choice algorithms that are
assumed to be feasible, motivated by both behavioral and neurophysiological data.8
We turn first to a brief review of that evidence, before presenting the proposed model.

1.1

Process Models of Stochastic Choice

As noted above, a long literature in psychophysics — the study by experimental
psychologists of the relation between subjective perceptions and the objective physical characteristics of sensory stimuli — has documented the way in the probabilitiesof particular judgments (from among a discrete set of possible responses) vary continuously with variation in the stimuli. At least since Thurstone (1927), a standard
interpretation of such “psychometric functions” has been to assume that the subject’s
responses are based on subjective perceptions that are sampled from a probability
distribution of possible subjective perceptions corresponding to each objective stimulus.9 A further development of the approach, that seeks to account for measured
response times as well as the frequencies with which different responses are given,
replaces the hypothesis of a single sample from a distribution of possible perceptions
by a model in which the subjective perception is the realization of stochastic process, unfolding over time as the subject observes the stimulus and contemplates theappropriate response.
The best-known of such stochastic-process models of binary perceptual discrimination tasks is the “drift-diffusion model” (DDM), a member of the broader familyof “diffusion-to-bound” models.10 In this model, a continuous-valued subjective statevariable, that one may think of as an evolving perception of the weight of sensory
evidence in favor of one response relative to the other, follows a random walk withdrift on a bounded interval; a decision is made in favor of one or the other of thetwo possible responses when the bound corresponding to that decision is reached.
The instantaneous variance of the diffusion process is assumed to be independent ofand Caplin and Dean (2013). See further discussion of this theory in section 3.
8
Webb (2013) also seeks to discipline the specification of a model of stochastic choice on the basisof behavioral and neurophysiological evidence, but does not seek to provide optimizing foundations
for the proposed specification.
9
See Green and Swets (1966) for a canonical exposition of this approach.
10
Early proponents of models of this type include Stone (1960), Laming (1968), Link and Heath(1975), and Ratcliff (1978). Reviews of the mathematical properties of models in this family can befound in Smith (2000), Bogacz et al. (2006), and Shadlen et al. (2007).

5

the stimulus presented, while the drift of the process is assumed to depend on thestimulus (specifically, the degree to which it possesses the property that implies that
one response rather than the other is correct). Under a particular (typically linear)
specification of how the drift of the process depends on stimulus characteristics, themodel has been found to give a fairly accurate quantitative account of the effects ofvariation in the stimulus on both response frequencies and reaction times.
Further evidence for the realism of this account of the mental processing involvedin perceptual discriminations comes from measurement of electrical activity in partsof the brain involved in recognizing the direction of motion of visual stimuli whilemonkeys perform a motion-discrimination task. In a fascinating series of experiments
by William Newsome, Michael Shadlen, and their collaborators, beginning in the late1980s, monkeys were trained to indicate the perceived dominant direction of motion
in “moving-dot” stimuli. In one of these experiments, the monkey is trained to focus
its attention on a screen where several hundred dots are moving; at any point in
time, a certain fraction of these are moving in a particular direction with a particular
speed, while each of the other dots is moving in a randomly selected direction with arandomly selected speed. The fraction of dots moving coherently define the “motion
strength” in the particular direction. The monkey is rewarded (by juice) if it correctly
indicates (typically, by one of two possible eye movements) whether the dominantdirection of motion of the dots is in one direction (say, to the right) or the opposite
direction (i.e., to the left).
The DDM fits well the observed variation both in the frequency with which themonkey indicates motion to the right and in the average time that the monkey takes
to respond, as functions of the degree of actual (net) “motion strength” to the right.11
Moreover, the neurophysiological measurements uncover signatures of processes similar to those postulated by the model.12 For example, Britten et al. (1993) found thatthe rate of firing of neurons in the middle-temporal area (MT) of the visual cortex,
an area already known to be involved in recognizing visual motion, was affected by
the motion of the dots; specifically, the average firing rate of “right-preferring” MTneurons increased by more the greater the degree of motion strength to the right.
They further measured the random variation in the number of times that these neurons would fire over a given time interval in the case of each type of stimulus, and11
12

See Figure 10.1 in Shadlen et al. (2007).
See section 10.4 of Shadlen et al. (2007) for a more detailed discussion.

6

showed that the randomness of the monkey’s responses in the case of varying degrees
of motion strength (i.e., the “psychometric function”) could be accounted for by the
degree of random variation in the difference between the number of times that rightpreferring and left-preferring neurons would fire (Britten et al., 1992), suggesting thatthe difference between the firing rates of the two groups of neurons behaves like therandom innovations in the diffusion process postulated by the DDM.
Roitman and Shadlen (2002) further found that the rate of firing of directionsensitive neurons in the lateral intraparietal area (LIP) appeared to integrate thecumulative evidence in favor of one response or the other, as is postulated to occur
in the DDM. Unlike the MT neurons, the firing rate of which fairly quickly reaches asteady rate (around which there are only white-noise variations) that depends on the
degree of motion strength in the stimulus, the firing rates of the LIP neurons follow
path more like random walks with drift, until the decision is made; and as postulatedby the DDM, the drifts vary systematically with the motion strength of the stimulus.
Random variation in the degree to which the activation level of particular LIP neurons
drifts up on a particular trial correlates with whether a particular decision is made onthat trial, as shown by the difference in average sample paths for the cases in which themonkey eventually chooses one response or the other. And finally, as also postulatedby the DDM, the timing of the monkey’s decision appears to be determined by thetime when the firing rate of certain LIP neurons reaches a particular threshold, that is
independent of the stimulus characteristics and the time taken to reach it. All of theseobservations suggest that some form of diffusion-to-bound model is implemented by
the brain in making perceptual decisions of this kind.
Authors such as Milosavljevic et al. (2010) and Krajbich et al. (2010) haveproposed the DDM as a model of the way in which economic choices are made as
well.13 These authors show that also in this context, the DDM can successfully
account for the way in which variation in the relative value of the options presented
affects both the frequency with which one is option is chosen over the other, and theaverage time required to make a choice.14 And while the neural implementation of
13

See Fehr and Rangel (2011) for a review of this literature. Busemeyer and Townsend (1993)
had earlier proposed a variant form of accumulation-to-bound model to account for experimentallyobserved choice behavior.
14
See Figure 1 below, plotting data from Krajbich et al. (2010) for choice between pairs of fooditems that the subject may eat. The figure is presented in the same format as Figure 10.1 of Shadlen
et al. (2010), showing the similarity of the phenomena to be explained in the two domains.

7such a mechanism would have to be different in the case of choices among economic
options — the particular areas of the cortex specialized in the perception of visualmotion will obviously not be the ones involved in judgments of the relative value ofconsumption goods — one might well suppose that the underlying principles that
result in this form of neural computation being used by the brain in one area should
result in similar computations being used to make comparative judgments of othertypes as well.
Here we suppose that choices in economic contexts are indeed made as a resultof computations similar to those depicted by the DDM. However, rather than simply
positing a mechanical process for the decision dynamics without interpretation, we
propose that the particular stochastic dynamics that are observed are those thatsatisfy a particular constrained optimality property. In this way, we seek to provide afunctional interpretation for these dynamics, that should be useful in suggesting theway that one should expect the model to generalize to more complex types of decision
problems. The resulting dynamics, for the kind of binary choice problem considered
here, are similar to those implied by the DDM, but not quite identical. Moreover,
the way in which they differ from those of the DDM seems to allow an improvementin the empirical fit of the model.

2

A Information-Constrained Dynamic Model ofDiscrete Choice

We now present a simple example of a process model of choice that can be derived
from an optimization problem, subject to a constraint on the information-processing
capacity of the structures through which the DM becomes aware of the nature of the
choices faced at a given point in time. We then compare the properties of the optimal
choice mechanism according to this theory with the predictions of the DDM, with
empirical data, and with some other possible normative theories of stochastic choice.

8

2.1

Choice Algorithms

Suppose that choice among a discrete set of alternatives occurs using a mechanismwith two elements.15 First, there is a sensor that produces a signal about the natureof the current choice situation each time it is employed; and second, there is a decoderthat receives the stream of signals from the sensor, and indicates the decision to betaken, based only on the signals from the sensor. Suppose furthermore that the signal
each time the sensor is activated is a letter s in some finite alphabet S, and let H be
the set of all possible finite sequences of letters of this alphabet (the set of possible
finite signal histories), including an element ∅ corresponding to a situation in which
no signals have been received. The operation of the sensor can then be specified by afunction σ : X × H → ∆(S) indicating the probability with which a given signal will
be sent by the sensor, as a function of the state of the world x ∈ X at the time thatthe sensor is operated, and the history h ∈ H of signals previously sent.16
The decoder decides, after each additional signal from the sensor is received,
whether to announce a decision (an action a chosen from among a finite set of possible
actions A) or to request another signal from the sensor. The algorithm begins withprior signal history ∅, and terminates as soon as a signal history h is reached forwhich the decoder announces a decision; once it terminates, the DM receives a reward
specified by a function U : A × X → R, that depends on both the action chosen andthe state of the world at the time of decision. The operation of the decoder can thenbe specified by a set C ⊂ H of “codewords” for which the decoder will announce a
decision, and a decision rule δ : C → A indicating the decision made when each ofthese codewords is encountered. Since we assume that the algorithm terminates assoon as a signal history c ∈ C is encountered, the set C must constitute a “prefix
15

Natenzon (2013) is another example of an optimization-based theory of discrete choice that
postulates a mechanism belonging to this class. However, Natenzon takes as exogenously given theoperation of what we call the “sensor,” while proposing an optimal “decoder” (based on Bayesian
decision theory), whereas the present analysis takes as exogenously given the operation of the “decoder” and instead considers the optimal design of the “sensor” (an information-constrained optimal
control problem).
16
The assumption that the operation of the sensor may depend on the history of previous signals,
but not the history of previous states of the world reflects an assumption that the sensor itself ismemoryless, but that it may be influenced by the current state of the decoder unit. The decoder, inturn, has no direct access to the state of the world, and instead observes only the history of signals
generated by the sensor.

9

code.”17
The decision problem, for which we wish to design a choice algorithm, can thenbe defined by a prior over possible histories of states of the world (assumed to evolve
independently of both observations and actions of the DM), the action set A and thereward function U . A specification of the sets S and C and the functions σ and δ
completely describes a choice algorithm (that may or may not reach a decision in finitetime) for this decision problem. For any specification of a sequence of states of theworld associated with a given experimental trial, the algorithm yields a probability
distribution of possible outcomes, where each outcome (in which a decision is reached)
is a specification both of an action a ∈ A that is chosen and a number of time steps
T that are required for the decision.18
The DDM model of binary choice is an example of an algorithm of this kind.
In this kind of decision problem, it is assumed that the action set A consists of
two possible choices, which we may call “left” (L) and “right” (R), and in typicalapplications the state of the world x (i.e., the specification of the values of the two
possible actions, UL and UR ) is assumed to remain constant over a given trial, until
a decision is reached. The prior over possible histories of states of the world thenreduces to a simple specification of a prior π ∈ ∆(X) over possible constant states on
a given trial. A discrete version of the DDM (which is more commonly specified as acontinuous diffusion,19 as in section 2.4) assumes that at each step of the algorithm
prior to termination (i.e., a decision), the sensor produces one of two possible signals,
that we may also denote L and R — the idea being that the signal L is additional
evidence in support of the desirability of action L, while the signal R is additional
evidence in support of action R.
In the neural implementation of the DDM proposed by Shadlen et al. (2007),the momentary evidence that causes the decision variable to drift in one direction or
17

That is, it has the property that no codeword c ∈ C is also the prefix (the initial sequence of
signals) of some longer codeword in C (Cover and Thomas, 2006, chap. 5).
18
Below, the framework is extended to allow as well for termination of the algorithm due toan exogenous event, “disappearance of the choice opportunity.” In this extension of the model, thefunctions σ and δ continue to describe the functioning of the algorithm, conditional on an exogenous
termination not having occurred.
19
See, however, Shadlen et al. (2007) for a discrete-time presentation of the model, that also
allows for the possibility that the momentary evidence is a discrete random variable, as assumed
here.

10

the other corresponds to the difference in firing rates between “right-preferring” and“left-preferring” neurons in region MT, where the firing rate of each type of neuron isa function of the state x (strength of motion to the right, in the experiments discussed
by these authors). The neurons produce discrete spikes with random timing (often
approximated as a Poisson process), rather than being continuously active at somelevel, and it is this randomness of the production of spikes that results in the randomvariation in the momentary evidence associated with a given state in the neural model.
If we think of the successive steps in our algorithm as the instants in time at which MTneurons spike, then at each step the signal received will be of one of two types: either
another spike from a “right-preferring” neuron (which we shall call the signal R) or
another spike from a “left-preferring” neuron (the signal L). Thus our restriction toa binary signal space is not necessarily unrealistic. (In addition, as discussed further
below, the Brownian motion with drift assumed in classic expositions of the DDM
can be viewed as a continuous-time limit of this binary-signal model.)At each step, the signal R is produced with some probability 0 < λ(x) < 1 that
depends on the (constant) state x, while L is produced with probability 1 − λ(x).
The specific version of the DDM advocated by Fehr and Rangel (2011)20 as a modelof value-based choice assumes not only that these probabilities are independent of
the history of previous signals, but that the log odds of the two signals are a linear
function of the difference in value between the two options,
ln

α
λ(x)
=
[UR (x) − UL (x)],1 − λ(x)N(2.1)

where α > 0 is a parameter indicating the sensitivity of the momentary evidence tothe degree of difference between the two available choices, and N ≥ 1 is a parameterof the decoder discussed below.21 This specifies a function σ for the sensor, in theterminology proposed here.
20

Fehr and Rangel actually discuss the continuous-time version of this process, presented in section
2.4 below, rather than the discrete version reflected in equation (2.1). They also discuss an extensionof the model, proposed by Krajbich et al. (2010), in which the rate of drift (corresponding to thelog odds here) depends not only on the relative value of the options, but also on the current visual
fixation of the subject. We abstract from the latter complication in the analysis here.
21
Writing α/N rather than a simple positive coefficient in (2.1) has the advantage that the model’s
predicted choice frequencies, given by equation (3.4) below, are then a function of α, and independentof the value assumed for N . This form also has the advantage that in this case α corresponds directly
to a parameter of the continuous-time version of the model.

11

The classic DDM assumes that a decision is made as soon as the net evidence infavor of one alternative over the other reaches a certain threshold. For any possible
signal history h ∈ H, let n(h) denote the number of occurrences of the signal R minus
the number of occurrences of the signal L. The algorithm is assumed not to decide,and so to continue collecting evidence, in the case of any finite signal history for which−N < n(h) < N, where N ≥ 1 defines the required evidentiary threshold; it halts,
and chooses the action R, if n(h) = N ; and it halts, but chooses the action L, if
n(h) = −N. This specifies a decision function δ for the decoder, where the codebook
C consists of all finite sequences h such that n(h) is equal to either N or −N , andsuch that −N < n < N for all prefixes (truncations) of h.
Alternatively, the operation of the decoder can be described by a finite-state
automaton, with states {−N, −N +1, . . . , N −1, N }. The automation begins in state 0(corresponding to n(∅)). Whenever the automaton is in state n, for any −N < n < N,
it collects another signal from the sensor, and if signal s is received the state moves
to n0 = Ts (n), where the transition function is given by
TR (n) = n + 1,

TL (n) = n − 1.

(2.2)

If the automaton reaches either of the states {−N, N }, it halts, and announces thedecisiona(−N ) = L,a(N ) = R.
(2.3)
It can be shown that such an algorithm produces a decision in finite time with probability 1, for any values UL , UR for the choices.

2.2

The Optimal Sensor Problem

We are interested in evaluating choice algorithms of this kind according to theirimplications for the DM’s expected reward E[U ], where U (a, x) is evaluated at thetime of choice22 and the expectation is over all possible sequences of states of theworld (using the prior) and, given a sequence of states of the world, over all possible
22

We may restrict attention to algorithms that imply that a decision is reached with probability
1. In fact, below we mainly emphasize a case in which the opportunity to choose disappears in finite
time with probability 1 if the decision is delayed too long, and in this case there is no difficulty
defining the DM’s reward in the event of an algorithm that never reaches a decision: eventually theopportunity disappears, resulting in a zero reward.

12

signal histories (using the conditional probabilities defined by σ). For simplicity, we
here consider only binary choice problems in which in each possible state of the world,
UR = V (x), UL = −V (x), for some real quantity V (x) (that may be either positiveor negative) that depends on the state.23 The prior used to evaluate the expected
reward implied by a given choice algorithm is then a prior over possible sequences ofvalues for V .
This prior need not be identified with the rule used by an experimenter to select thedecision situations presented to an experimental subject on successive trials; rather,it should reflect the probability with which the DM can expect to encounter different
situations, given the environment to which its choice procedure has been adapted;
the particular situations created by the experimenter may be quite atypical, relative
to this prior. This is true even in the case where the experimenter tells the subject
the probability distribution from which the experimental situations will be drawn.
We may suppose that the DM’s deliberations make use of two mental systems, as
in Cunningham’s (2013) refinement of the dual-systems theory of Kahneman (2011):
a “system 1” that produces a quick recommendation a about the action to take, on
the basis of fine-grained “low-level” information x about the characteristics of theoptions presented on the current occasion, and a more reflective “system 2” that
can also draw inferences from “high-level” information about the way in which theexperimental situation has been designed, and that observes the recommendation aof system 1 without having access to all of the information x used to produce it.
The model presented here is intended as a theoretical account of the automatic
processing of the choice situation by system 1; the prior for which the algorithm usedby system 1 has been optimized derives from some considerable body of previous
experience by the organism, but may not be able to take into account “high-level”
information about the experimental design, even when the DM’s system 2 is aware ofit. As long as the high-level information about the experimental design does not givesystem 2 a reason to change its expectation that the action recommended by system 1
is more likely to be the correct one than the reverse, the two-system DM will continue
to take the action a recommended by system 1, even if the information processing bysystem 1 has been optimized for a prior that does not correspond precisely to system
2’s probability beliefs. Note that in this respect the theory proposed here differs from
23

This is without loss of generality, since the optimal algorithm remains the same if the constant(UL + UR )/2 is subtracted from all rewards.

13

Sims’ (2003, 2010) theory of rational inattention. In order to minimize the numberof free parameters used to fit the experimental data discussed below, we shall make afairly simple assumption about the prior, and one under which the situations actuallypresented in the experiment would not have been judged extremely unlikely ex ante.
However, we shall not assume that whatever state x is observed at a given point
in time is expected to persist indefinitely, regardless of the time that the DM may
take to decide between the two currently available options; it is reasonable to assume
that the more typical situation is one in which opportunities for action appear, butwill disappear again if a choice is not made sufficiently quickly, and that the choicealgorithm is optimized for an environment with this characteristic.
Specifically, we shall assume that some measure π over the real line describes theprior probability for the value of V for the options that are initially available whena choice situation is presented and the algorithm is initiated. Then, each time thatthe algorithm fails to make a choice between the two options, there is a probability
0 < ρ < 1 that the choice opportunity will continue to be available, in which casethe sensor can be used to produce another signal about the relative value of theoptions, and another opportunity to choose or to defer choice will be presented. But
with probability 1 − ρ, an action is forced and the algorithm necessarily terminates;
there is an equal probability of either action being selected (with no input from thedecoder) in this case, so that the expected reward is zero.24
A given choice algorithm (S, C, σ, δ) and anticipated persistence of opportunities
ρ then define probabilities {p(h|V )} of reaching each possible finite history h andengaging the sensor, if the initially presented options are such that UR = V, UL = −V.
We can similarly let p∗ (h|V ) be the probability of reaching any history (of length
greater than or equal to 1), whether the algorithm terminates at this history or not.
Let D ⊂ H be the set of histories such that neither h nor any prefix of h belongs to
the codebook C. (That is, D is the set of histories for which the algorithm will not
yet have halted.) Then these probabilities are defined recursively by the relationsp∗ (hs|V ) = p(h|V )σ(s|V, h)for any h ∈ D and s ∈ S, where hs denotes the signal history obtained by adjoining s
to the previous signal history h, and σ(s|h, V ) is the probability of observing signal s
if the sensor is activated when the value of option R is V and the prior signal history
24

More generally, the expected reward would be (UL + UR )/2 in this case.

14

is h; andp(∅|V ) = 1
for all V , whilep(h|V ) = ρp∗ (h|V )for any h ∈ D̄, the set of histories in D of length greater than or equal to 1. We
define p(h|V ) = 0 for any h ∈/ D and correspondingly p∗ (hs|V ) = 0 for all of thesuccessors of any h ∈/ D. Given a prior π over the possible initial values of V , theexpected reward from using the algorithm will then be25
E[U ] =

X
V

π(V )

X

p∗ (c|V )U (δ(c), V ),

(2.4)

c∈C

where we now write U (a, V ) for the reward from choosing a if V takes this value anda decision is made before an action is forced.
In the present analysis, we shall furthermore take as given the rule used by the
decoder, and consider only the optimal design of the sensor for a given class of potential decision problems. Specifically, we shall assume that the rule used by the decoder
is given by (2.2)–(2.3), for some value of N , as in the case of the DDM, and consider
the optimal choice of the signalling function σ. Note that while the assumption thatthe decoder uses the rule (2.2)–(2.3) implies something about the consequences ofsending a signal R as opposed to L at any stage of the algorithm, we make no apriori assumptions about which signal must be sent under any particular conditions,so that the dynamics of the decision process remain relatively unconstrained. One
advantage of focusing on the optimal sensor design problem is that, as we shall see,
it is possible to characterize the optimal signalling function σ under only a relatively
weak assumption about the prior π over possible relative values V ; the optimal designof the decoder will instead generally be sensitive to further details of the prior, sincea single choice of the decoder must be made to apply regardless of the value of V on
a particular trial.
If we simply consider the problem of choosing a sensor function to maximize
E[U ], under no constraints beyond those already mentioned, the solution is simple:
the sensor should always indicate the value R in a state in which V > 0, and the25

Here we assume that the set X is countable. This allows us to write sums rather than integrals,
though the formalism is easily extended to deal with the case of a continuum of possible values V .

15value L in a state in which V < 0. Such a rule implies that the correct (rewardmaximizing) choice will always be made, assuming that an action is not forced before
a choice can be made (i.e., before N observations of the signal have been completed);
and it will maximize the probability of getting to choose before an action is forced (byalways generating the shortest possible codeword among those leading to the correct
choice). Such an analysis cannot, however, explain the experimental data reported
in studies such as Krajbich et al. (2010); it implies that, in an experiment where
the choice opportunity is maintained long enough for the subject to reach a decision,
there should be no random variation in the choice made between any two options(except in the case of exactly identical values), and it implies that the time taken to
make the decision should be the same for “hard” choices as for “easy” ones.
According to the hypothesis proposed here, this does not occur because it would
require the output of the sensor to be too precisely coordinated with the current stateof the world. We shall suppose that there is a limit to the information-processing
capacity of the sensor, and model this (as in Sims, 2003, 2010) by a bound on themutual information between the output of the sensor and the state of the world.26
Following Shannon, we can define the rate of information transmission by the sensor
by the average amount by which receipt of a signal s reduces an observer’s uncertainty(measured by entropy) about the conditions z that have influenced the signal (Coverand Thomas, 2006, chap. 2). In our case, the operation of the sensor is definedby conditional probabilities λ(z) of producing the signal R when the sensor receives
inputs z ≡ (V, h). The prior over states of the world together with the specificationof the algorithm imply a set of unconditional probabilities π(z) of being in a given
state z when the sensor is engaged, given byπ(V )p(h|V )π(V )p(h|V )π(V, h) = P P
=
,
E[T
]
π(
Ṽ
)p(
h̃|
Ṽ
)
Ṽ
h̃∈H
where
E[T ] ≡XX
V

π(V )p(h|V )h∈His the expected value of T, the (random) number of times the sensor is employed (andhence the number of steps in the algorithm) before a decision is reached.
The ex ante uncertainty about the value of z, for someone who observes only that26

See also Wolpert and Leslie (2012) for use of a similar constraint.

16

the sensor has been engaged, is then given by the (unconditional) entropy27
XH≡−
π(z) ln π(z).
zObservation of a signal s implies a set of posterior probabilities π(z|s) given by
π(z|R) = π(z)λ(z)/λ̄,π(z|L) = π(z)(1 − λ(z))/(1 − λ̄),

where
λ̄ =

X

π(z)λ(z)(2.5)

zis the ex ante probability of receiving signal R when the sensor is engaged. The
entropy of the posterior distribution is thereforeX
H(s) ≡ −π(z|s) ln π(z|s),z

and the mutual information I is then defined as the average entropy reduction peruse of the sensor,
I ≡ H − [λ̄H(R) + (1 − λ̄)H(L)]¯ where the upper
We assume that the only feasible sensors are ones for which I ≤ I,
bound I¯ is assumed to be considerably less than ln 2 nats (i.e., one binary digit) per
signal.28
Substitution of the above definitions allows us to alternatively write
XXX
I = λ̄
π(z|R) ln π(z|R) + (1 − λ̄)π(z|L) ln π(z|L) −π(z) ln π(z)
z

=

X

z

z

π(z)λ(z) [ln π(z) + ln λ(z) − ln λ̄]

z

+

X

π(z)(1 − λ(z)) [ln π(z) + ln(1 − λ(z)) − ln(1 − λ̄)] −

z

=

X

X

π(z) ln π(z)zπ(z) D(λ(z)||λ̄)z27

The use of the natural logarithm rather than a logarithm of base 2, as in classic expositions of
information theory, is simply a change in the units in which the rate of information flow is measured:from “bits” (binary digits) per signal to “nats” per signal.
28
The bound I ≤ ln 2 is satisfied by any binary signalling mechanism, including the (deterministic)
optimal reporting rule characterized above.

17

for the average rate of information transmission per use of the sensor, whereD(λ||λ̄) ≡ λ ln(λ/λ̄) + (1 − λ) ln(1 − λ/1 − λ̄)is the relative entropy of a distribution that assigns probability λ to the receipt ofsignal R relative to one that assigns probability λ̄ to that outcome.29 We assume thatthe set of feasible choice algorithms is constrained by an upper bound on the average
cumulative information flow through the sensor per trial, summing the informationtransmitted each time the sensor is used. Thus the proposed information constraint
requires that the sensor be operated in a way that satisfies the bound I · E[T ] ≤ K,
for some finite value of K, or alternatively,
X
X
π(V )
p(h|V ) D(λ(V, h)||λ̄) ≤ K,
(2.6)
V

h∈D

where λ̄ is given by (2.5).
Note furthermore that if λ̄ is treated as a separate design parameter (rather thanbeing defined by (2.5), then for any choice algorithm (implying values for the π(z)and λ(z)), the value of λ̄ that minimizes the left-hand side of (2.6) is given by (2.5).
We can accordingly restate our problem as the choice of signal probabilities {λ(z)} foreach of the states that can possibly be reached, and a constant λ̄, so as to maximize
E[U ] subject to constraint (2.6). Alternatively, we can state the problem as the choiceof {λ(z)} and λ̄ to maximize a Lagrangian of the form L ≡ E[U ] − θI · E[T ], or
(
)
X
X
X
L=
π(V )
p∗ (c|V )U (δ(c), V ) − θp(h|V ) D(λ(V, h)||λ̄) ,
(2.7)
V

c∈C

h∈D

where θ is a non-negative Lagrange multiplier indicating the value of relaxing theinformation constraint (2.6).

2.3

Optimal Evidence Accumulation Dynamics

We now characterize the signaling function that solves problem (2.7), for a given
value of the shadow cost of processing capacity θ > 0. We note that for any V , the29

For the properties of relative entropy as a measure of the distance of one probability distribution
from another, see, e.g., Cover and Thomas (2006), chaps. 2 and 11. This formulation makes it clearthat a limit on information flow makes it difficult to arrange for the sensor to produce L and R
signals with frequencies that differ systematically with the conditions under which the sensor isoperating.

18

final term in (2.7) can be written in the form
X
XXp(h|V ) D(λ(V, h)||λ̄) =p∗ (hs|V ) ln(p∗ (hs|V )/p(h|V )p̄(s)h∈D s∈S

h∈D

=

X

∗

varφ(p (c|V )) +

c∈C



1−ρ
ρ

X
h∈D̄

ϕ(p∗ (h|V ) −Xp∗ (hs|V ) ln p̄(s),h∈D,s∈S

where we define p̄(R) ≡ λ̄, p̄(L) ≡ 1 − λ̄, and ϕ(p) ≡ p ln p for any 0 < p < 1, with
ϕ(0) ≡ ϕ(1) ≡ 0. Thus this term is a strictly convex function of the probabilities{p∗ (h|V )} and a strictly convex function of the probabilities {p̄(s)}. It follows thatfor any θ > 0, (2.7) is a strictly concave function of the probabilities {p∗ (h|V )}(implied by the probabilities {λ(z)}) and a strictly concave function of λ̄.
If for any probabilities {p∗ (h|V )} defined over signal histories that are immediate
successors of histories h ∈ D, we let {p∗† (h|V )} be the probabilities that reverse theroles of L and R (that is, p∗† (h|V ) ≡ p∗ (h† |V ), where if h = LLRL, h† = RRLR, andso on), then (2.7) can be written in the form
X
L=
π(V )L(V )V

where for each value of V , L(V ) is a function of the probabilities p∗ (·|V ) and of λ̄such that
L(V, p∗ (·|V ), λ̄) = L(−V, p∗† (·| − V ), 1 − λ̄).
Let us now assume further that the prior π over possible values of V is symmetric:
that is, the probability of a decision situation in which UR − UL = V is the same as
the probability of one in which UR − UL = −V, for any V . (This simply means thatthe assignment of which of the options will be the “right” option on a given trial isindependent of the options to be compared.) It then follows that
L(p∗ , λ̄) = L(p∗† , 1 − λ̄),if the Lagrangian is evaluated for an arbitrary signalling function σ (defined for all
possible values of V ) and for an arbitrary value of λ̄.
Then for any signalling function giving rise to probabilities {p∗ (h|V )} (defined for
all V and for all h ∈ D), the reversed probabilities {p∗† (h| − V )} define an alternative
feasible signalling function, and the convex combination11p̄∗ (h|V ) ≡ p∗ (h|V ) + p∗† (h| − V )2219defines yet another feasible signalling function. The strict concavity of L then impliesthat1
1
L(p̄∗ , 1/2) ≥ L(p∗ , λ̄) + L(p∗† , 1 − λ̄) = L(p∗ , λ̄),22where the inequality is strict unless p∗ = p∗† and λ̄ = 1/2. From this we can conclude
that the optimal value of λ̄ must be 1/2, and consider the problem of maximizing(2.7) over choices of the {λ(z)}, taking the value λ̄ = 1/2 as given.
It is now possible to consider the problem of choosing the {λ(V, h)} for a given
value of V so as to maximize the term L(V ) in the Lagrangian, independently of
the signalling function that is to be used in any other states. In particular, theprior π over possible values of V plays no role in this problem. Thus we can derivepredictions about the probability of choosing one action over the other and about theprobability distribution of reaction times for alternative values of V , without requiring
any assumption about the prior distribution over possible values of V , apart from thesymmetry assumption just invoked.
Let us consider this problem for a particular value of V . Suppressing the index
V , our goal is to choose transition probabilities {λ(h)} for the finite histories h ∈ D
so as to maximizeXX
p(h) D(λ(h)),(2.8)
p∗ (c)U (δ(c)) − θ
c∈C

h∈D

where we now adopt the shorthand D(λ) ≡ D(λ||1/2). This can be solved using
the method of dynamic programming. For any history h ∈ D, let W (h) denote themaximum achievable value of the continuation objectiveXXp∗ (c|h)U (δ(c)) − θp(h̃|h) D(λ(h̃)),
c∈Ch

h̃∈Dh

where Ch ⊂ C is the set of codewords for which h is a prefix, and Dh ⊂ D is theset consisting of h and the other non-terminating histories for which h is a prefix;p∗ (c|h) is the probability of reaching codeword c, conditional on having received theinitial signal history h; and p(h̃|h) is the probability of reaching h̃ and activating thesensor, conditional on the initial history h. In this definition, the maximization isover possible choices of {λ(h̃)} for the h̃ ∈ Dh . This value function must satisfy a
Bellman equation of the form
W (h) = max {ρ[λ(h)W (hR) + (1 − λ(h))W (hL)] − θD(λ(h))} ,
λ(h)

20

where for histories c ∈ C we define W (c) ≡ U (c).
One can furthermore observe that the set of continuation plans and the payoffs
associated with them are exactly the same in the case of all histories h ∈ D for whichn(h), the number of occurrences of the signal R minus the number of occurrences ofthe signal L, has the same value n. Thus we can simply write W (n) for the common
value of W (h) in the case of all such histories, and write the Bellman equation in theform
W (n) = max {ρ[λ(n)W (n + 1) + (1 − λ(n))W (n − 1)] − θD(λ(n))}

(2.9)

λ(n)

for any −N < n < N. The value function (for any value V of the action R) is thengiven by a sequence of values {W (n)} for −N ≤ n ≤ N that satisfy (2.9) for each−N < n < N, together with the boundary conditions
W (−N ) = −V,

W (N ) = V.

(2.10)The solution to the optimization problem in (2.9) is easily seen to be given by the
λ(n) corresponding to the log odds
ln

λ(n)
ρ
= [W (n + 1) − W (n − 1)].
1 − λ(n)θ(2.11)

One observes that for any finite values {W (n)}, the solution to (2.11) will imply that0 < λ(n) < 1 for all −N < n < N ; thus regardless of the value of V , and regardless ofthe previous signal history, there will at each point in time be positive probabilities
of receiving either L or R as the next signal, so that the value of n will fluctuate
randomly, drifting up over some time intervals and down over others, as in the DDM.
Note that this is not an assumption, but rather a conclusion about the kind of processthat economizes on information: for our model assumes that it is feasible, at finite
cost, to arrange for the transition probability λ(h) to equal either 0 or 1, though itturns out never to be efficient for the decision process to evolve so predictably.
Substituting the solution (2.11) into (2.9) we obtain

ρ
 1
ρ

1
W (n) = θ ln
exp W (n − 1) + exp W (n + 1)
(2.12)2θ2
θ
for each −N < n < N. The value function can then be characterized as the solution
to this nonlinear difference equation consistent with boundary conditions (2.10).
21

As with the DDM, the present model implies that the bias of this process toward
one of the absorbing barriers or the other depends on the sign and magnitude of V
(i.e., of the difference in value of the two options). Condition (2.11) implies thatthe log odds are an increasing function of the gradient of the value function, andthe boundary conditions (2.10) imply that the average value of this gradient must
be proportional to V . However, this model implies that the log odds of obtaining asignal R as opposed to L will also depend on the current balance of the accumulated
evidence (i.e., the value of n at a given point in the process), and not only on thevalue of V . As with the DDM, the decision dynamics can be described by a Markov
chain on the set of subjective states {−N, −N + 1, . . . , N − 1, N }, but the transition
law is no longer precisely a random walk with drift.
In the special case ρ = 1 (so that opportunities are expected to persist as longas may be necessary for a choice to be made), the difference equation (2.12) has aclosed-form solution,
W (n) = θ[a + ln(n + b)],eV /θ − e−V /θ
,2NbeV /θ + e−V /θ
= V /θ> 1,
Ne − e−V /θ

e−V /θ − eV /θ
,2Nb
eV /θ + e−V /θ
= V /θ
< −1,
Ne − e−V /θ

where a = ln

if V > 0, orW (n) = θ[a+ln(−b−n)],where a = ln

if V < 0. (In the case that V = 0, the solution is simply W (n) = 0 for all n.) It thenfollows from (2.11) that11λ(n) = +.
(2.13)
2 2(n + b)
It follows that for any value of n, λ(n) is a monotonically increasing function of
V /θ (approaching 0 as V /θ → −∞ and approaching 1 as V /θ → +∞). However,the probability of receiving a signal R (and hence the average rate of drift of theaccumulated evidence) depends on the current value of n: for any V 6= 0, λ(n) is amonotonically decreasing function of n.
Under the more realistic assumption that ρ < 1, a closed-form solution is unavailable, but the difference equation (2.12) can be solved numerically for each possible
specification of the boundary conditions. One finds in general that λ is a function of
the values of V /θ and n.

22

2.4

The Continuous-Time Limit

Suppose that we let the value of N become large, but shrink the length of time ∆required for each additional use of the sensor in proportion to N −2 , so that N 2 ∆is held constant as N is increased. Then if we let τ ≡ t∆ be the amount of clock
time that has passed after t uses of the sensor, the model approximates one in which
the rescaled state variable z ≡ ∆1/2 n evolves with a continuous sample path as τ
increases, The range over which this variable varies will be the interval [−B, B],
where the decision threshold B ≡ (N 2 ∆)1/2 is independent of N .
Furthermore, over any time interval that is sufficiently small for z (and hence
the probability λ(z) of receiving a signal R on each use of the sensor) to change
little over the interval, the cumulative change in the value of z over the interval isapproximately the sum of a large number of independent draws of a bounded randomvariable, so that the distribution approaches a Gaussian as N is made large. If forany −B < z < B, the log odds ln(λ(z)/1 − λ(z)) become small at the rate N −1 as N
is made large (as specified for example in (2.1)), thenµ(z) ≡

λ(m)N
ln
2B 1 − λ(m)(2.14)

has a well-defined limit as N is made large, and the mean increment in z over a small
time interval will equal µ(z) times the length of the time interval, neglecting an errorof order N −1 in the estimate of the drift. Under the same assumption, the varianceof the increment in z will equal the length of the time interval,30 where we again
neglect an error of order N −1 in this estimate of the variance per unit time. Hencein the limit of large enough N , the process approximates the trajectories of a Wiener
processdzτ = µ(zτ )dτ + dWτ ,(2.15)
where Wτ is a standard Brownian motion (with zero drift and a unit instantaneous
variance).
The continuous-time version of the DDM is then given by equation (2.15), in
which µ is a constant (i.e., a number independent of z) given byµ=α(UR − UL ),2B

30

The normalization used in the definition of z has been chosen to imply this unit instantaneous
variance.

23

as a consequence of (2.1) and (2.14). This is the version of the DDM presented, forexample, in Fehr and Rangel, 2011. Note that the testable predictions of the model
about a given choice situation are entirely functions of two parameters, the value ofB (that is the same for all choice situations) and the value of α(UR − UL ) for theparticular options available on this occasion.
In the case of the optimal information-constrained model (OICM), instead, thedrift will depend on the current value of z. Let us suppose that the arrival rate of
the event that forces an action is fixed per unit of time (rather than per use of thesensor) as the time required for each use of the sensor shrinks, so thatδ ≡ (− ln ρ)/∆ > 0
is held constant as N increases. Then (2.11) implies thatµ(z) =

w0 (z)
θ(2.16)

for all −B < z < B, where w(z) ≡ W (∆−1/2 z) expresses the value function in termsof the rescaled state variable.
In this continuous-time limit, the Bellman equation (2.9) becomes


θ 2
1 00
0
δw(z) = max w (z)µ + w (z) − µµ22
for all −B < z < B. The first-order condition for the inner problem is easily seen tobe (2.16), and substitution of this for µ(z) yields the differential equation
δw(z) =

1 0 2 1 00
w (z) + w (z)2θ2as the continuous analog of (2.12) above. If we alternatively define the normalized
value function v(z) ≡ w(z)/θ, we can characterize it by the differential equation
2δv(z) = v 0 (z)2 + v 00 (z)(2.17)

for all −B < z < B, together with the boundary conditionsv(−B) = −ν,v(B) = ν,where ν ≡ V /θ. We observe from (2.16) thatµ(z) = v 0 (z),24

(2.18)

so that computation of the normalized value function suffices to determine the driftof the process (2.15).
We observe from the form of the equations (2.17) and (2.18) that the solution
for the normalized value function (and hence for the optimal drift µ(z) at all points
in the interval −B < z < B) for any given decision problem depends only on thevalues of the parameters B, δ, and ν. Accordingly, the predictions of the OICM withregard to both the probability of choosing each of the two options and the probability
distribution of response times depend only on these three parameters. It follows
that in the large-N limit, it is only these parameters that can be identified fromobservations of choice and reaction time; we should not expect to be able to identify
numerical values for N , ρ or ∆. Alternatively (as in the discussion of parameter
estimates below), we can arbitrarily fix a large value of N , and use the data toidentify the implied values of ρ, ∆, and θ; but it should be understood that thenumerical values of ρ and ∆ obtained in this way are only meaningful in terms of thevalues that they imply for δ and B. It should also be noted that arbitrarily fixing
an assumed value for N does not imply that one is making an assumption about the
distance between the decision thresholds, rather than determining this empirically;
for the value of B (which determines the distance between the thresholds, in unitsof the instantaneous standard deviation of the Brownian motion) is not implied by agiven choice of N .

3

Comparison with Experimental Evidence

We turn now to a discussion of the degree to which the OICM succeeds as an explanation of observed behavior.

3.1

Explaining Logistic Choice

The model can be used to predict the probability of choosing each of the two options,as functions of the relative value of the two options to the DM. In using the model
to predict the outcomes that should be observed in experimental settings where theexperimenter allows the subject to take as much time as desired to decide between theoptions, we must now consider the probability distribution of outcomes predicted by
the choice algorithm derived above, for a given value of V and under the assumption
25

that the opportunity to choose persists indefinitely, rather than a decision being
forced with probability 1 − ρ after each additional use of the sensor.31
Suppose that the value of the opportunity presented on a particular occasion isV (that is, that UR = V, UL = −V ), and that the opportunity persists until a choice
is made. Under this assumption, the probability that the algorithm will eventuallyterminate in a choice of R, conditional upon the signal history received thus far, will
depend only on the quantity n by which the number of R signals exceeds the numberof L signals. Furthermore, this probability Λ(n) must satisfy
Λ(n) = λ(n)Λ(n + 1) + (1 − λ(n))Λ(n − 1)

(3.1)

for all −N < n < N, together with the boundary conditions
Λ(−N ) = 0,

Λ(N ) = 1,

(3.2)

where {λ(n)} is the sequence determined in the way discussed above. (Here thedependence of both λ(n) and Λ(n) on the value of V /θ is not made explicit.)
In the special case ρ = 1, the sequence {λ(n)} is given by (2.13), and (3.1) has aclosed-form solution,b+N n+NΛ(n) =,
2N n + b
so that the initial probability of an eventual choice of R is equal toΛ(0) =

e2V /θ
b+N=.
2b
1 + e2V /θThus in this case the model predicts a logistic relation between the difference in valueof the two options and the frequency with which they will each be chosen,
Prob(R) =e(UR −UL )/θ
.
1 + e(UR −UL )/θ

(3.3)

This kind of logistic relationship is very commonly fit to data on binary choices(both from laboratory experiments and the field); if the difference in the values of
31This does not require that we solve the equations of the previous section under the assumption
ρ = 1. The value of ρ indicates the degree of persistence that has been typical of the environment towhich the subject’s cognitive system has adapted, but this may differ from the degree of persistence
for the particular set of experiments for which we now wish to derive a probability distribution ofpredicted outcomes.

26the two alternatives is assumed to be a linear function of some vector of measured
characteristics, one obtains the familiar logistic regression model.32 The standard interpretation given to this statistical specification is in terms of a random-utility model(RUM), in which R is chosen over L if and only if UR + R > UL + L , where L , R
are two independent draws from an extreme value distribution of type I (McFadden, 1974). But while the RUM provides a possible justification for the econometric
practice, there is no obvious reason to expect that the additive random terms inpeople’s valuations (even supposing that they vary randomly from each occasion of
choice to the next) should be drawn from this particular type of distribution, so that
there is little reason to expect logistic regressions to be correctly specified, under thisinterpretation.
The DDM provides an alternative interpretation of logistic choice. Substitutionof (2.1) into (3.1) yields a difference equation with the solution
Λ(n) =eα(UR −UL ) − e−α(UR −UL )n/N,eα(UR −UL ) − e−α(UR −UL )so thateα(UR −UL ),(3.4)
1 + eα(UR −UL )again a logistic function of UR − UL .33 This makes the logistic outcome depend on
assumptions that seem less arbitrary; but it still depends on the particular functional
form (2.1) for the relation between the relative value of the two options and the driftof the diffusion process, which is not motivated by any considerations deeper thananalytical convenience.
The OICM instead provides an explanation for the empirical fit of a logistic relationship (or something close to it) that does not depend on the a priori assumption ofeither a special probability distribution or any special functional forms (apart fromthe information-theoretic measure of the information required by different sensors).
The exact logistic form (3.3) is predicted only in the case that ρ = 1; however, even in
the case of a value of ρ slightly less than 1 (argued below to be the empirically realistic
Λ(0) =

32

For a variety of applications, see Cramer (2003).
Thus we observe that the parameter θ of the OICM has the same consequences for the predicted
relationship between relative value and choice probabilities as the parameter α−1 in the DDM.
Note however that the decision dynamics implied by the two models are not equivalent under thisidentification: for example, the predicted values of Λ(n) are not the same, except when n = 0.
3327

case), the predicted relationship is quite similar to a logistic curve, as illustrated by
Figure 1(a) below.
The explanation given by the OICM for the stochasticity of discrete choice, andeven for logistic choice (or at least a nearly-logistic functional relationship) could also
be obtained more simply from the theory of “rational inattention” (RI) as formulated
by Sims (2003, 2010), without any need to discuss the specific class of dynamic
decision processes proposed here.34 Sims postulates that decisions must be based, as
in the model proposed here, on a signal s that is a stochastic function of the true
state x (that determines the reward U (a, x) from alternative possible actions a); but
he supposes that the set S of possible signals, the function σ : X → ∆(S) describing
the conditional probability of different signals being received in any given state, andthe decision rule δ : S → A are all chosen so as to maximize E[U ], subject only to an
upper bound on the mutual information between x and s.35
One can easily show that the maximum achievable value of E[U ] is attained in thecase of a signal that simply indicates which action to take. In this case, the information structure is specified by a function λ : X → [0, 1] indicating the probability ofchoosing R in any state, and the Lagrangian for the optimal information structure
problem isL=Xπ(x) (1 − λ(x))UL (x) + λ(x)UR (x) − θD(λ(x)||λ̄) ,xPwhere λ̄ ≡x π(x)λ(x) is the average frequency of occurrence of the signal that
results in the choice of R. This in turn is maximized by a function λ(x) given by theright-hand side of (3.3).
This might seem a simpler explanation for the empirical success of the logistic
specification. However, it fails to provide an explanation of the data on response
times, to which we turn next.
34

See Woodford (2008), Cheremukhin et al. (2011), and Matějka and McKay (2013) for previous
discussions of this implication of RI. Ortega and Braun (2013) also derive logistic choice from atheory that is essentially equivalent to RI.
35
Here we use the notation s to refer to the complete set of signals on the basis of which thedecision must be made, rather than to the signal obtained from a single use of the “sensor,” as in
the exposition above.

28

3.2

Explaining Variation in Response Times

We now compare the predictions of the OICM to those of the DDM with regard to themeasured time required for subjects to announce their decision. As these predictions
are considered purely numerically, we focus on the degree to which the model can fit aparticular set of experimental data, from one of the leading studies finding support for
the DDM as an account of the mental processing underlying observed choice behavior,namely that of Krajbich et al. (2010).
Krajbich et al. (2010) ask subjects to rate how much they would like to eat eachof 70 possible food items at the end of the experiment, on an integer scale between-10 and 10. (These rankings provide a measure of the value of each of the possible
outcomes to the individual subject, independent of any comparison between the goods
and any particular alternative.) In a second stage of the experiment, a subject is thenasked to choose between pairs of food items that have previously been ranked between
0 and 10; the experimenters record both the subject’s choice in the case of each pairand the time taken to decide. After making 100 such decisions, the subject is allowed
to actually eat the item chosen in one of the binary choices, selected at random.
Figure 1 plots certain summary statistics of the choices of 39 subjects, each ofwhom made 100 binary choices. In each panel, trials are grouped according to theextent to which the right option was ranked higher than the left option by thatparticular subject; thus a “relative value of R” equal to 3 might mean that the right
option was ranked 8 by that subject while the left option was ranked 5, or that theright option was ranked 3 while the left option was ranked 0. Panel (a) shows thefraction of trials on which the option R was chosen, as a function of the relative rankof option R; panel (b) shows the average time taken to decide (in milliseconds), again
as a function of the relative value. (In each panel, the height of the bar indicates thevalue of the statistic for a given bin, while the width of the bar is proportional to thenumber of trials assigned to that bin.36 )In the same figure, the open circles indicate the predictions of the DDM for these
two statistics, if the relative reward UR − UL is assumed to be proportional to thesubject’s reported relative ranking of the two options, with the same constant of
36

There are also a small number of trials for which the relative rank has an absolute value greaterthan five. But because the number of trials in these bins are small, and the estimated statistics arecorrespondingly inaccurate, these bins have not been shown in the figures, though all data are usedin the model evaluation exercise below. The truncation of the figures follows Krajbich et al. (2010).

29

proportionality for all subjects, and if the other parameters of the model are assumed
to be identical for all subjects as well. The assumption that UR −UL is proportional to
the relative rank amounts to an assumption that each subject’s utility from consuming
the various food items is a roughly linear function of the rank assigned to them on
the scale from -10 to 10. Let the slope of this relationship be called the subject’s“marginal utility of rank,” mur. Since the model predictions depend only on theratio (UR − UL )/θ for each pair of goods (or α(UR − UL ) in the notation used above
for the DDM), a crucial additional assumption of this test of the model is that theratio mur/θ (or α · mur) is the same (or sufficiently similar) for all subjects.37
In the case of the DDM, it follows from our discussion above of the large-N limitthat there are two parameters that should be identifiable from data on choice andresponse times, the parameter α introduced in equation (2.1) — or more precisely,
the value of αmur38 — and the parameter B ≡ ∆1/2 N introduced in section 2.4.
Alternatively, since N is not identified (any sufficiently large value should lead toan equivalent value), we may arbitrarily fix a value for N (set equal to 100 in the
numerical work reported here39 ) and then identify the values of α and ∆ that best fit
the data. In the fit shown in Figure 1(b), we also introduce a third free parameter, aconstant intercept A in the prediction
Response Time = A + T ∆for measured response time, where T is the random number of times that the sensor is employed in making the decision. The allowance for an intercept reflects anassumption that the observed response time may include an additional time interval(required to engage the choice mechanism and/or to communicate the choice once it
has been made) that is independent of the choice options presented on a given trial,
37

In the case of the OICM, it is a prediction of the model (assuming that each subject’s classification system has been adapted to the same frequency distribution of choice situations, and that eachsubject’s system is subject to the same information constraint I¯ in nats per millisecond) that theLagrange multiplier θ for each subject will be proportional to that subject’s mur. In the case of the
DDM, there is no underlying theory of the determinants of the parameter α, so that it would be acoincidence for this to be true, unless we assume that both mur and α are the same across subjects.
38
We call this parameter α, measuring utility in units of the numerical rank supplied by the
subjects, so that mur is assumed to equal 1. This is without loss of generality, since only α · mur
is identified.
39
We verify numerically that choosing N = 50 or 200 instead makes only a negligible difference
for the reported results.

30

in addition to the time required to execute the choice algorithm modeled above.40 ) In
the figure, these parameters have been chosen to minimize a weighted sum of squared
prediction errors.41
The solid triangles instead indicate the predictions of the OICM, if the parametersof the model are chosen to minimize the same weighted-least-squares criterion. In
this case, there are again three parameters that are adjusted to fit the data. It follows
from our discussion above that in the large-N limit, the only identified parameters of
this model are θ (actually, θ/mur), δ and B. Alternatively, we can fix an arbitrarylarge value for N (again, N = 100 in the results reported here), and then determine
the best-fitting values for θ, ρ and ∆.42 One observes that with the same (fairly small)number of free parameters, the OICM does a reasonably similar job of accounting forthese statistics; in panel (a), the weighted sum of squared errors is slightly smaller for
the OICM than for the DDM, though the errors are somewhat larger for the OICM
than for the DDM in panel (b).
The predictions of the two models are not identical, and one of the more noteworthy differences is shown in panel (a) of Figure 2. Here the mean response time isplotted not as a function of the relative value assigned to the R option, but rather asa function of the relative value of whichever option was actually chosen by the subject
in that particular trial. The predictions of the DDM are the same in Figure 2(a) as
in Figure 1(b): the model predicts that in the case of two options that differ in value
by a given amount, the mean response time is the same in those cases in which thelower-valued option is chosen as in those in which the higher-valued option is chosen. The data, instead, show that “correct” choices (cases in which the higher-valued
option is chosen) are on average made more quickly than “incorrect” choices, for agiven degree of difficulty of the choice (based on the absolute difference in rankings).
40

This is standard in empirical tests of the DDM; see, for example, the curve-fitting exercise shownin Figure 1 of Shadlen et al. (2007), in which a 3-parameter version of the DDM is fit to two curvesof the kind shown in panels (a) and (b) of Figure 1.
41
See the appendix for a precise statement of the criterion that is minimized, and the best-fitting
parameter values.
42
No intercept A is allowed in the linear relationship between T and the measured response time inthis case. In fact, allowing for a positive intercept would not reduce the sum of squared prediction
errors in this case; allowing for a negative intercept would, but would have no interpretation interms of the theoretical model. The best-fitting values of these parameters are again indicated in
the appendix.

31

This is in fact a common feature of experimental data on response times in binary
perceptual classification tasks as well, and is a famous empirical failing of the DDM(Luce, 1986).43 The OICM instead correctly predicts that correct choices should bemade more quickly, and by roughly the amount by which the mean response times aredifferent in the data.44 This prediction results from the way in which the probabilityof receiving an R signal varies with the net accumulated evidence in favor of the R
choice in the OICM; a departure of this sort from the assumption of a constant-drift
diffusion process thus has some empirical support.
The OICM’s predictions regarding response times are also more accurate thanthose of the basic (unconstrained) RI model, which, as noted above, is as successful
as the DDM in explaining the data on choice frequencies alone. One might think that
RI simply makes no prediction about response times, and so is neither confirmed nor
disconfirmed by such evidence, but this is not true. To the extent that response timeis informative about the decision problem faced on a given occasion, the algorithm
that produces the decision in that period of time must have access to (at least) that
information.
If we treat the pair (a, T ) as the output of the choice algorithm, and ask what
stochastic algorithm will maximize E[U ] subject to an upper bound on the mutual
information between the vector (a, T ) and the state x, under the assumption thatthe reward depends only on the initial state x and the action chosen, we obtain asimple prediction: (i) the probability of choosing option R in any state x should begiven by (3.3), for some Lagrange multiplier θ ≥ 0 on the information constraint; and(ii) conditional on the action a that is chosen, the response time T should convey noadditional information about the state x. This requires that the probability distribu43

According to Shadlen and Kiani (2013, Box 2), this “apparent refutation” led many mathematical psychologists to abandon the DDM as an explanation of stochastic choice and reaction times,
until the more recent discovery of neurophysiological evidence for a mechanism of a similar form. A
number of variations on the basic DDM presented here have been proposed in the literature that canallow for shorter average response time in the case of correct responses (see, e.g., Link and Heath,
1975; Ratcliff and Rouder, 1998; Ditterich, 2006; or Drugowitsch et al., 2012).
44
The model prediction differs substantially from the same mean response time in the data onlyin the case when the relative value of the chosen option equals -5. There are, however, only a few
trials in this bin (as shown by the narrowness of the bar), since subjects seldom chose the less-valued
option in the case of such an extreme difference, and the response times were quite variable acrossthe trials in this bin (as shown in panel (b)), so that the mean response time for this case should be
assigned a wide confidence interval.

32

tion of response times on those trials in which R is chosen should be independent ofx, and similarly for the distribution of response times when L is chosen. Hence theaverage response time would have to be an affine transformation of λ(x),E[T ](x) = (1 − λ(x))TL + λ(x)TR ,where Ta is the mean response time conditional on action a.45 But this is clearly
not consistent with a comparison between the two panels of Figure 1. If we add the
additional postulate (roughly consistent with the data shown in Figure 1(b)) that the
average response time for a given choice is independent of which option is presented
as the left option as opposed to the right option, then TL would have to equal TR ,and RI would require average response time to be completely independent of x.
Even without this last assumption, if the frequency distribution of value differences
UR − UL is symmetric around zero, the bins in Figure 2 should each contain as manycases in which R is chosen as cases in which L is chosen, and so the distribution of
response times for trials in each bin should be the same (an equally weighted mixtureof the distribution for trials in which R is chosen and the distribution for trials inwhich L is chosen). It follows that in Figure 2(a), the mean response time should be
the same for each relative value. But again, this is clearly not the case.
These predictions of RI are obtained under the assumption that response time
has no consequences for reward. If instead we assume, as in our derivation of theOICM, that the choice algorithm is optimized for a prior under which choice may beforced with a certain probability if the decision is delayed, delay influences expected
reward. In this case, however, in the absence of any constraints other than theinformation constraint, it would be optimal to always decide immediately (or in theminimum feasible time). Then average response time should be independent of thevalues of both of the options presented; but this is again inconsistent with bothFigures 1(b) and 2(a). Hence an additional constraint on possible algorithms, beyond
the information constraint alone, is necessary in order to account simultaneously forchoice behavior and response times. The OICM represents a relatively simple example
of such an additional constraint.
This does not mean, however, that our (still very simple) model successfully explains all aspects of observed response times with only three parameters. For example,
45

The RI hypothesis would provide no reason for TL to differ from TR , but it would not preclude
such an asymmetry, either.

33

as shown in panel (b) of Figure 2, when the parameters of the model are chosen to
fit the data on choice frequencies and mean response times, it generally over-predicts
the variability of response times. On the other hand, this is also a problem for theclassic DDM as well.46 Quite possibly it results from the fact that the version of
the OICM analyzed here, like the DDM, assumes that the choice algorithm takes noaccount of the passage of time, but only the net accumulation of evidence in favor of
one option over the other; some authors have argued instead that the threshold for adecision decreases with the passage of time (e.g., Drugowitsch et al., 2012), making
very long response times less likely than in the models analyzed here. This feature of
the OICM results from our adoption of the simple form of “decoder” assumed by the
DDM, rather than any intrinsic consequence of the hypothesis of optimal informationconstrained classification. It will be interesting to consider, in future work, whether
a more sophisticated form of decoder might be both more in conformity with the
hypothesis of economizing on information-processing capacity and more consistent
with empirical evidence.

46

Figure 2(b) is even more problematic for the RI theory. As explained above, RI would imply
that the probability distribution of response times should be the same for each of the bins in Figure
2, so that along with the mean response time (discussed above), the variance of the response timeshould be the same in each bin; but this is not at all what Figure 2(b) shows.

34References[1] Ballinger, T. Parker, and Nathaniel T. Wilcox, “Decisions, Error and Heterogeneity,” Economic Journal 107: 1090-1105 (1997).
[2] Bogacz, Rafal, Eric Brown, Jeff Moehlis, Philip Holmes, and Jonathan D. Cohen,“The Physics of Optimal Decision Making: A Formal Analysis of Models of
Performance in Two-Alternative Forced-Choice Tasks,” Psychological Review
113: 700-765 (2006).
[3] Britten, Kenneth H., Michael N. Shadlen, William T. Newsome, and J. AnthonyMovshon, “The Analysis of Visual Motion: A Comparison of Neuronal and Psychophysical Performance,” Journal of Neuroscience 12: 4745-4765 (1992).
[4] Britten, Kenneth H., Michael N. Shadlen, William T. Newsome, and J. AnthonyMovshon, “Responses of Neurons in Macaque MT to Stochastic Motion Signals,”
Visual Neuroscience 10: 1157-1169 (1993).
[5] Busemeyer, Jerome R., and J.T. Townsend, “Decision Field Theory: A DynamicCognitive Approach to Decision Making in an Uncertain Environment,” Psychological Review 100: 432-459 (1993).
[6] Caplin, Andrew, and Mark Dean, “Rational Inattention and State Dependent
Stochastic Choice,” unpublished, New York University, June 2013.
[7] Cheremukhin, Anton, Anna Popova, and Antonella Tutino, “Experimental Evidence on Rational Inattention,” Federal Reserve Bank of Dallas Research Department Working Paper no. 1112, December 2011.
[8] Clithero, John A., and Antonio Rangel, “Combining Response Times and Choice
Data Using a Neuroeconomic Model of the Decision Process Improves Out-ofSample Predictions,” unpublished, California Institute of Technology, November
2013.
[9] Cover, Thomas M., and Joy A. Thomas, Elements of Information Theory, New
York: Wiley-Interscience, 2d ed., 2006.
[10] Cramer, J.S., Logit Models from Economics and Other Fields, Cambridge: Cambridge University Press, 2003.
35

[11] Cunningham, Tom, “Biases and Implicit Knowledge,” unpublished, Institute forInternational Economic Studies, Stockholm University, September 2013.
[12] Ditterich, Jochen, “Stochastic Models of Decisions about Motion Direction: Behavior and Physiology,” Neural Networks 19: 981-1012 (2006).
[13] Drugowitsch, Jan, Rubén Moreno-Bote, Anne K. Churchland, Michael N.
Shadlen, and Alexandre Pouget, “The Cost of Accumulating Evidence in Perceptual Decision Making,” Journal of Neuroscience 32: 3612-3628 (2012).
[14] Fehr, Ernst, and Antonio Rangel, “Neuroeconomic Foundations of Economic
Choice — Recent Advances,” Journal of Economic Perspectives 25(4): 3-30
(2011).
[15] Gabbiani, Fabrizio, and Steven J. Cox, Mathematics for Neuroscientists, Amsterdam: Academic Press, 2010.
[16] Gold, Joshua I., and Michael N. Shadlen, “Banburismus and the Brain: Decoding
the Relationship between Sensory Stimuli, Decisions and Reward,” Neuron 36:
299-308 (2002).
[17] Green, David M., and John A. Swets, Signal Detection Theory and Psychophysics, New York: Wiley, 1966.
[18] Haile, Philip A., Ali Hortaçsu, and Grigory Kosenok, “On the Empirical Content
of Quantal Response Equilibrium,” American Economic Review 98: 180-200(2008).
[19] Hey, John D., “Experimental Investigations of Errors in Decision Making underRisk,” European Economic Review 39: 633-640 (1995).
[20] Hey, John D., “Does Repetition Improve Consistency?” Experimental Economics 4: 5-54 (2001).
[21] Kahneman, Daniel, Thinking, Fast and Slow, New York: Farrar, Straus andGiroux, 2011.
[22] Krajbich, Ian, Carrie Armel, and Antonio Rangel, “Visual Fixations and theComputation and Comparison of Value in Goal-Directed Choice,” Nature Neuroscience 13: 1292-1298 (2010).
36

[23] Krajbich, Ian, Bastiaan Oud, and Ernst Fehr, “Benefits of Neuroeconomics Modeling: New Policy Interventions and Predictors of Preference,” unpublished, Ohio
State University, September 2013.
[24] Laming, D.R.J., Information Theory of Choice-Reaction Times, London: Academic Press, 1968.
[25] Link, S.W., and R.A. Heath, “A Sequential Theory of Psychological Discrimination,” Psychometrika 40: 77-105 (1975).
[26] Lu, Jay, “Random Choice and Private Information,” unpublished, Princeton
University, October 2013.
[27] Luce, R. Duncan, Individual Choice Behavior: A Theoretical Analysis, New
York: Wiley, 1959.
[28] Luce, R. Duncan, Response Times, New York: Oxford University Press, 1986.
[29] Manski, Charles F., “The Structure of Random Utility Models,” Theory andDecision 8: 229-254 (1977).
[30] Matějka, Filip, and Alisdair McKay, “Rational Inattention to Discrete Choices:
A New Foundation for the Multinomial Logit Model,” unpublished, Boston University, January 2013.
[31] McFadden, Daniel, “Conditional Logit Analysis of Qualitative Choice Behavior,”
in P. Zarembka, ed., Frontiers in Econometrics, New York: Academic Press, 1974.
[32] Milosavljevic, Milica, Jonathan Malmaud, Alex Huth, Christof Koch, and Antonio Rangel, “The Drift Diffusion Model Can Account for the Accuracy andReaction Times of Value-Based Choice under High and Low Time Pressure,”
Judgment and Decision Making 5: 437-449 (2010).
[33] Mosteller, Frederick, and Philip Nogee, “An Experimental Measurement of Utility,” Journal of Political Economy 59: 371-404 (1951).
[34] Natenzon, Paulo, “Random Choice and Learning,” unpublished, Washington
University, September 2013.

37

[35] Ortega, Pedro A., and Daniel A. Braun, “Thermodynamics as a Theory ofDecision-Making with Information-Processing Costs,” Proceedings of the Royal
Society A 469: 20120683 (2013).
[36] Pinkovskiy, Maxim, “Rational Inattention and Choice under Risk: Explaining
Violations of Expected Utility through a Shannon Entropy Formulation of theCosts of Rationality,” Atlantic Economic Journal 37: 99-112 (2009).
[37] Ratcliff, Roger, “A Theory of Memory Retrieval,” Psychological Review 85: 59108 (1978).
[38] Ratcliff, Roger, and Jeffrey N. Rouder, “Modeling Response Times for TwoChoice Decisions,” Psychological Science 9: 347-356 (1998).
[39] Roitman, Jamie D., and Michael N. Shadlen, “Response of Neurons in the Lateral
Intraparietal Area During a Combined Visual Discrimination Reaction Time
Task,” Journal of Neuroscience 22: 9475-9489 (2002).
[40] Shadlen, Michael N., Timothy D. Hanks, Anne K. Churchland, Roozbeh Kiani,
and Tianming Yang, “The Speed and Accuracy of a Simple Perceptual Decision:
A Mathematical Primer,” in K. Doya, S. Ishii, A. Pouget, and R.P.N. Rao, eds.,
Bayesian Brain: Probabilistic Approaches to Neural Coding, Cambridge, MA:
MIT Press, 2007.
[41] Shadlen, Michael N., and Roozbeh Kiani, “Decision Making as a Window on
Cognition,” Neuron 80: 791-806 (2013).
[42] Sims, Christopher A., “Implications of Rational Inattention,” Journal of Monetary Economics 50: 665-690 (2003).
[43] Sims, Christopher A., “Rational Inattention and Monetary Economics,” in B.M.
Friedman and M. Woodford, eds., Handbook of Monetary Economics, vol. 3A,
Amsterdam: Elsevier, 2010.
[44] Smith, Philip L., “Stochastic Dynamic Models of Response Time and Accuracy: A Foundational Primer,” Journal of Mathematical Psychology 44: 408-463(2000).

38[45] Stone, Mervyn, “Models for Choice-Reaction Time,” Psychometrika 25: 251-260(1960).
[46] Thurstone, Louis L., “A Law of Comparative Judgment,” Psychological Review
34: 273-286 (1927).
[47] Webb, Ryan, “Dynamic Constraints on the Distribution of Stochastic Choice:
Drift Diffusion Implies Random Utility,” unpublished, New York University, August 2013.
[48] Wolpert, David, and David S. Leslie, “Information Theory and Observational
Limitations in Decision Making,” The BE Journal of Theoretical Economics,
volume 12, issue 1, article 5, 2012.
[49] Woodford, Michael, “Inattentiveness as a Source of Randomized Discrete Adjustment,” unpublished, Columbia University, April 2008.
[50] Woodford, Michael, “Information-Constrained State-Dependent Pricing,” Journal of Monetary Economics 56(S): 100-124 (2009).

39

(a)

(b)

1

3000
DDM
OICM

0.9
2500
0.8

Mean Response Time (msec)

0.7

Prob(R)

0.6

0.5

0.4

2000

1500

1000

0.3

0.2
500
0.1

0

−5 −4 −3 −2 −1 0 1 2
Rel Value(R)

3

4

0

5

−5 −4 −3 −2 −1 0 1 2
Rel Value(R)

3

4

5

Figure 1: Experimental data and fitted theoretical predictions for (a) probability ofchoosing R and (b) mean time required to make a decision, each plotted as a functionof the amount by which the subject’s reported valuation of option R exceeds thatof option L. Bars plot the experimental data of Krajbich et al. (2010), circles thepredictions of the DDM, and triangles the predictions of the present model.

40

(a)

(b)

3000

3000

2500

2500

2000

2000

SD Response Time (msec)

Mean Response Time (msec)

DDM
model

1500

1000

500

0

1500

1000

500

−5 −4 −3 −2 −1 0 1 2 3
Rel Value(Chosen)

4

0

5

−5 −4 −3 −2 −1 0 1 2 3
Rel Value(Chosen)

4

5

Figure 2: Empirical data and fitted theoretical predictions for (a) mean responsetime and (b) standard deviation of response times across trials, each now plotted as afunction of the amount by which the subject’s reported valuation of the option chosen
exceeds that of the option not chosen. Experimental data are again from Krajbich etal. (2010).

41

A

Appendix: Numerical Parameter Values

Here we provide additional details of the parameters that are used in the numericalpredictions shown in Figures 1 and 2. In the case of both the DDM and the OICM,
parameters are chosen to minimize a weighted sum of prediction errors,
−1
Ntot

X
j

1 −1 X
1 −1 X
Nj (aj − âj )2 + Ntot
Nj (bj − b̂j )2 NtotÑj (cj − ĉj )2 ,
2
2
j
j

(A.1)

taking into account the model predictions for the quantities plotted in Figures 1(a),1(b), and 2(a). (In focusing on the model predictions for choice frequencies and mean
reaction times, we follow much of the literature on the empirical support for the
DDM; see for example the plots in Shadlen et al. (2007) and Krajbich et al. (2010),
focusing on the same statistics as in Figures 1(a) and 1(b) here.)In this expression, for each integer 10 ≤ j ≤ 10, Nj is the number of trials on
which the subject’s ranking of the R option exceeds the ranking of the L option byj, and Ñj is the number of trials on which the ranking of the option that is chosen
exceedsthePranking of the other option by j. Ntot is the total number of trialsP( j Nj = j Ñj = Ntot ). Note that in computing (A.1), we sum over all of the bins
−10 ≤ j ≤ 10 for each of the two ways of classifying the data, and not just the bins
−5 ≤ j ≤ 5 shown in Figures 1 and 2. (Most of the data fall in the bins −5 ≤ j ≤ 5,
however, and given the weights in the goodness-of-fit criterion, these bins are largely
responsible for the conclusions about the best-fitting parameter values. This is why
only the central bins are shown in the figures.)
The quantities aj , bj and cj are the quantities plotted in Figures 1(a), 1(b) and2(a), respectively, while the corresponding hatted variables are the predictions of themodel for these same quantities. Thus the model parameters are chosen to minimize
a weighted sum of squared prediction errors, where the weights applied to the squared
prediction errors for each bin are proportional to the number of trials in that bin. The
factors (1, 1/2, 1/2) that pre-multiply the three terms are chosen so as to put equal
weight on fitting the average frequency of R choices and on fitting average response
times; and then, with regard to the goal of fitting the evidence on average response
times, putting equal weight on fitting the evidence in Figure 1(b) and fitting theevidence in Figure 2(a).
The quantity aj is the fraction of the trials on which the relative rank is j in which
the subject chose R; it varies over a theoretical range from 0 to 1. The quantities bj
and cj are instead mean response times in milliseconds, divided by 1600, so that therange of variation in these variables is of a similar order of magnitude to the range ofvariation in the variable aj .
In the case of both models, the value N = 100 is fixed arbitrarily, as discussed
in the text. In the case of the DDM, the parameters α, ∆, and A are then chosen to42

Table 1: Parameter values used to fit the experimental data, for each of the twomodels.

DDM
OICM

θ (α−1 )
1.53
2.08

ρ
—
0.9996∆ (msec) A (msec)
0.19
578
0.25
0

Table 2: Goodness-of-fit statistics, for each of the two models.
h(a − â)2 i
DDM 0.0019369
OICM 0.0015782

1h(b − b̂)2 i
2

1h(c − ĉ)2 i
20.0033134
0.0048181

0.0092264
0.0059117

Loss
0.014477
0.012308

minimize the criterion (A.1); in the case of the OICM, the parameters θ, ρ, and ∆are chosen to minimize the same criterion, fixing A = 0 in this case. The best-fitting
parameter values for each model are reported in Table 1.
The minimized values of the criterion (A.1) in each case are reported in the final
column of Table 2. Table 2 also reports the values of each of the three terms in (A.1)
separately, in the first three columns of the table, evaluated at the parameter values
reported in Table 1. (Thus the final column in Table 2 is the sum of the other threecolumns.) On this criterion, the OICM fits somewhat better overall; but admittedly,such a conclusion is sensitive to the relative weights chosen for the different parts ofthe goodness-of-fit criterion.
The best-fitting parameterizations of the two models imply fairly similar dynamics. Apart from the similarity of the predictions for the the statistics plotted in thetwo panels of Figure 1, one observes from Table 1 that the parameter indicating the
average sensitivity of the sensor log odds to variations in the value gradient (θ inthe case of the OICM, or α−1 in the case of the DDM) is of similar magnitude in
the two cases. The best-fitting value of ∆ is similar in magnitude in both cases as
well; this means that the implied value of B (the distance to one of the decisionbarriers, in units of the instantaneous standard deviation of the diffusion process inthe continuous-time limiting model) is similar for the two models.
The modest differences in the best-fitting numerical parameter values do point tosome differences in the dynamics implied by the two models, however. The fact thatthe best-fitting θ for the OICM is larger than α−1 for the DDM accounts for the factthat the predicted curve in Figure 1(a) is slightly steeper for the DDM. In fact (asshown by the first column of Table 2), the fit of the OICM to Figure 1(a) is slightly
43better than that of the DDM. This indicates that in the case of the DDM, there ismore tension between the value of the sensitivity parameter α needed to fit the choicefrequency data and the value that allows the model to better fit the data on average
response times.
The best-fitting value of ∆ is also about 25 percent shorter in the case of the DDM
than that used to fit the OICM. This means that the DDM is only able to accountas well as the OICM for the average response time by assuming a substantial fixed
time requirement A (a value that accounts for more than a quarter of the average
response time), in addition to the time required for the stochastic algorithm to reacha decision threshold. Thus in the case of the DDM, there is a tension between thevalue of ∆ needed to account for the average response time and the value neededto account for the difference in response times between “easy” and “hard” choices,that is minimized through the introduction of an additional free parameter, A, while
this is not needed in the case of the OICM. (On the other hand, the OICM has afree parameter, ρ, with no analog in the case of the DDM, and this parameter is also
crucial for allowing the model to match the observed difference in average response
times between “easy” and “hard” choices.)44