Strategies in Stochastic
Continuous-Time Games∗
Yuichiro Kamada†

Neel Rao‡

December 22, 2023

Abstract
We develop a new set of restrictions on strategy spaces for continuous-time
games in stochastic environments. These conditions imply that there exists
a unique path of play with probability one given a strategy profile, and also
ensure that agents’ behavior is measurable. Various economic examples areprovided to illustrate the applicability of the new restrictions. We discuss how
our methodology relates to an alternative approach in which certain payoffs are
assigned to nonmeasurable behavior.

JEL Classification: C73, C78, D25
Keywords: dynamic games, continuous time, stochastic processes, timing∗

We thank Ken-Ichi Akao, Eduardo Faingold, Drew Fudenberg, Oliver Hart, Michihiro Kandori,
Akitada Kasahara, Jay Lu, Bentley MacLeod, Eilon Solan, Phillip Strack, Takuo Sugaya, Satoru
Takahashi, Jean Tirole, Juuso Toikka, and Yasutora Watanabe for helpful suggestions. We also
appreciate the comments of seminar participants at Chinese University of Hong Kong, Harvard
University/Massachusetts Institute of Technology, Hitotsubashi University, University of California
at Davis, University of Tokyo, and the Summer Workshop on Economic Theory 2018 at Otaru
University of Commerce. Some of the results reported in this paper were included in an earlier
version of another working paper “Sequential Exchange with Stochastic Transaction Costs” by thesame authors.
†
Haas School of Business, University of California Berkeley, 2220 Piedmont Avenue, Berkeley, CA
94720-1900, USA, and University of Tokyo, Faculty of Economics, 7-3-1 Hongo, Bunkyoku, Tokyo,
113-0033, Japan, e-mail: y.cam.24@gmail.com‡
University at Buffalo, SUNY, Department of Economics, 415 Fronczak Hall, Buffalo, NY 142601520, USA, e-mail: neelrao@buffalo.edu

11Introduction

Continuous-time models have proven useful for the analysis of dynamic strategicinteractions because closed-form solutions can be obtained while such results may
be difficult to derive in discrete-time models. However, the specification of a gamein continuous time entails some technical complexities. As discussed by Simon andStinchcombe (1989) and Bergin and MacLeod (1993), many of these difficulties stemfrom the ability of agents to instantaneously react to the actions of other agents.
In order to resolve such problems, the strategy spaces need to be suitably defined,
and those authors propose techniques for doing so in a deterministic environment.
Although those methods are useful in that context, continuous-time modeling has
been successful in the analysis of stochastic environments as well, and hence it isdesirable to develop suitable restrictions for strategy spaces in such models. This
paper is devoted to the discussion of restrictions on strategy spaces for continuoustime games in stochastic settings.
We develop a new set of conditions to restrict strategy spaces in continuous time.
We begin by defining the concept of consistency between a strategy and a history,
which means that a given strategy can generate a given history. One problem incontinuous time is that there are cases where zero or multiple histories are consistent
with a specific profile of strategies. In order to rule out these two possibilities, wedevelop two novel concepts. Traceability helps prevent the former situation, requiringthat when the other agents do not move in the future, there exists a history consistent
with an agent’s strategy. Frictionality eliminates the latter possibility, requiring thatin any history that is consistent with an agent’s strategy, the agent can move only
finitely many times during any finite time interval. By applying these criteria, weshow that each strategy profile induces a unique path of play.
A further issue regards the measurability of the action process. For expected
payoffs to be well defined, the stochastic process describing the moves of agents should
be progressively measurable. In addition, specifying the strategy space so as to ensure
the measurability of actions is complicated in our model due to perfect monitoring.
Simply requiring the action process of an agent to be progressively measurable (asis standard in the literature) is problematic because the strategy of one agent affects
whether the strategy of another agent induces a measurable action process under2

perfect monitoring.1 Thus, simply restricting the action process of an agent to be
progressively measurable even when the behavior of another agent is nonmeasurable(which would correspond to the standard approach) would result in an extremely
limited set of strategies as explained in section 4.1, ruling out a variety of situations
in which the actions of one agent are influenced by the behavior of another agent.
Therefore, we invent a two-step technique for delineating the strategy space. A
strategy is called quantitative if the behavior of an agent is measurable regardlessof the strategy of its opponents. A strategy is called calculable if the behavior ofan agent is measurable when its opponents play quantitative strategies. We showthat each agent’s behavior is measurable if every agent uses a calculable strategy.
Moreover, the set of calculable strategies is the largest strategy space for each agent
that includes the set of all quantitative strategies and implies measurable actions.
A few methods have previously been proposed for defining strategy spaces incontinuous time when the environment is deterministic, but they are not directly
applicable to the stochastic case. In particular, the “inertia” assumption from Berginand MacLeod (1993) would ensure that each strategy profile induces a unique pathof play. This condition essentially requires that at each history up to a given time t,
there exists a time interval of positive length during which agents do not change theiractions.2 There are at least two ways to extend this definition to a stochastic setting.
In one approach, the aforementioned time interval can depend on the history up to
time t but not on the realization of uncertainty after time t. The other approachallows this time interval to also depend on the realization of uncertainty after time t.
The former version of inertia may be too strong in stochastic environments, where
given a history up to time t and any  > 0, the state variable may change quickly
within the time interval (t, t + ), so that the analyst may want to consider the
possibility of an equilibrium in which agents change their actions during the interval(t, t + ). We present various examples to illustrate that such a situation naturally
arises in stochastic settings.3 The latter version is weaker but does not guarantee
1This type of complexity does not arise in models of continuous-time games with imperfectly
observable actions such as Sannikov (2007).
2
Bergin and MacLeod (1993) also consider a weaker condition involving the completion of theset of inertia strategies, and we discuss it in footnote 43.
3
When comparing our approach with existing techniques, we consider many examples that involve rapid transitions between states, but as the applications in section 5 illustrate, such a property
is not necessary for our methodology to apply in situations that are not covered by existing techniques in the literature.

3

measurable behavior. In a deterministic environment, the inertia condition would
essentially suffice to ensure that the resulting path of play is measurable, but more is
involved in a stochastic setting because the actions of the agents can be contingent on
moves of nature.4 Accordingly, we introduce the notion of calculability. Furthermore,neither version of inertia implies nor is implied by traceability and frictionality, as
shown in section 3.3. We present an example to demonstrate that even the weaker
formulation of inertia rules out some applications of interest covered by traceability
and frictionality in which the timing of moves by one agent depends on the actionsof another agent.5
Relatedly, the “admissibility” criterion in Simon and Stinchcombe (1989) when
extended to a stochastic environment would imply existence and uniqueness of thepath of play. However, this condition requires the number of moves by an agent tobe bounded.6 As will be seen from our examples in section 5, imposing the bounduniformly so as not to depend on the realization of uncertainty can sometimes be too
restrictive. Specifically, the equilibria that we analyze entail no upper bound on thenumber of moves during any finite time interval. Agents may make infinitely many
moves over an infinite horizon with probability one.
Allowing the bound to depend on the realization of uncertainty does not suffice
to cover some relevant applications, particularly those in which the number of moves
by one agent depends on the behavior of the other agent. Furthermore, since admissibility applies off as well as on the path of play, it eliminates some simple strategies
that require an agent to move multiple times following a deviation. By contrast,
traceability and frictionality are weaker requirements and do not preclude strategies
in which the number of moves in a given finite time interval may be arbitrarily large.7
4
For instance, the strategies in example 8 satisfy inertia as well as traceability and frictionality
but do not induce measurable behavior.
5
Although the inertia condition may be insufficient to model optimal behavior in some of theapplications we consider, it may be enough to accommodate -optimal strategies. In item 3 of remark
3, we explain that restricting attention to -optimal strategies without defining strategy spaces moregenerally creates a problem of circularity.
6
Simon and Stinchcombe (1989) also restrict how agents may condition their behavior on thepast by requiring strategies not to distinguish between two histories in which the same actions occur
at different times that are sufficiently close to each other. Some of the equilibria we consider in our
examples violate this assumption.
7
More precisely, fix an interval of the nonnegative real line with positive but finite length aswell as any realization of uncertainty. For each agent, our model allows for the possibility that
there exists a strategy satisfying the restrictions of the model such that the following holds. For
any nonnegative integer n, one can find a history that is consistent with this strategy such that the4As shown in section 3.3, admissibility when applied to our model implies traceability
and frictionality, but not vice versa. Moreover, even if agents use admissible strategies, the behavior that arises may not be measurable, necessitating an additional
condition like calculability.8
Our model is not intended to cover all possible stochastic environments, and our
focus is on games in which agents strategically choose the timing of discrete moves.
This, in particular, precludes games in which agents move at every moment in continuous time9 (which are not allowed by Simon and Stinchcombe (1989) or Bergin and
MacLeod (1993), either). As the reader will hopefully see, even though our framework
is not totally comprehensive, defining strategy spaces in a stochastic environment is
a nontrivial task, and our examples show that our model encompasses a number of
economically relevant situations. We emphasize that our objective is not to undermine the usefulness of existing restrictions in the literature like those of Bergin and
MacLeod (1993) and Simon and Stinchcombe (1989). We discuss those restrictions
simply to underscore the additional complications that arise when the environmentis stochastic.
Continuous-time modeling has been widely employed in stochastic settings, and its
use is growing. Section 5 of our paper considers various applications. An importantquestion is the timing of investment under uncertainty, which has been studied by
McDonald and Siegel (1986) in continuous time with a single agent. In section 5.1,
we consider a model of forest management in which multiple agents decide when to
harvest trees whose volumes evolve stochastically over time. A major class of problems
involving the timing of moves is search models, in which agents must choose between
trading at the present time and waiting for a better opportunity to trade. Section 5.2
examines a supply chain in the petroleum industry, where the price of oil varies overtime, and a well is deciding when to extract each unit oil and transfer it to a refinery,
which then processes the input and delivers the output to a customer. The oil wellnumber of moves by the agent during the given time interval is equal to n.
8
See example 8 for a particular instance of admissible strategies that do not generate a measurable
path of play.
9
Such models have been applied to study, for example, repeated games (Sannikov, 2007), contracting (Sannikov, 2008), reputation (Faingold and Sannikov, 2011), signaling (Dilme, 2017), andoligopoly (Bonatti, Cisternas, and Toikka, 2017). These papers assume the imperfect observability of opponents’ actions and an inability to condition on one’s own past actions, which simplify
the description of strategies, thereby preventing issues related to the existence, uniqueness, andmeasurability of behavior that are discussed in our paper.

5

faces a search problem, and its optimal strategy is characterized by a reservation
price like in the classic model of McCall (1970). A topic related to investment under
uncertainty is entry and exit by a firm, which is studied by Dixit (1989) in continuous
time where the price follows a diffusion process. In section 5.3, we analyze a situationin which two firms contemplate the timing of entry into a market, where the cost ofentry varies with time and the benefit depends on entry by the competitor.
Some existing models limit agents to moving at random points in time. For
instance, there is a literature on bargaining games in continuous time, including themodel in Ambrus and Lu (2015), where agents can make offers at random times andmust reach an agreement by a specified deadline. Another class of models is revision
games as formalized by Kamada and Kandori (2020), where agents have opportunities
to alter their actions before interacting at a designated time. In section 5.4, we apply
our restrictions to a finite-horizon game that has a deadline by which a buyer must
place an order with a seller. The buyer’s taste for a good is stochastically changing
over time, and the seller receives opportunities at random times to fulfill the order.
Further applications of our framework are considered in the online appendix. Ortner (2019) specifies a bargaining model in continuous time, where the bargaining
power of each agent is governed by a Brownian motion. As mentioned in the online appendix, Kamada and Rao (2018) develop a model of bargaining and trade inwhich there is a transaction cost that may evolve according to a geometric Brownian
motion. We also present an example in which a pair of criminals play a prisoner’s
dilemma at times that arrive according to a Poisson process, as well as continuously
deciding whether to remain partners in crime. Another issue related to the timingof investment is technology adoption, which has been studied in continuous time by
Fudenberg and Tirole (1985). The online appendix contains an example where two
agents repeatedly decide on when to adopt a technology that has positive externalities and a stochastically evolving cost. A continuous-time formulation has also beenused for models of adjustment costs, including the analysis of pricing by Caplin andSpulber (1987). In the online appendix, we examine interactions between a retailer
and a distributor in a model of inventory adjustment with a randomly changing stock
as well as a model of exchange where prices evolve stochastically.
Given the difficulty of restricting the strategy space to prevent nonmeasurable
behavior, a pertinent question is whether the issue can simply be avoided by allowing
nonmeasurable behavior and exogenously assigning it a (for example, low) expected6

payoff. In section 6, we examine this question, and relate the equilibria under thisalternative approach to those under our original methodology. A problem is thatthe assignment of expected payoffs to nonmeasurable behavior is inherently arbitrary
as the conditional expectation may be undefined and the set of equilibria depends
on the particular assignment of expected payoffs to nonmeasurable behavior. Under
some conditions, a type of “folk theorem” can even be obtained, whereby arbitrary
behavior can be sustained in equilibrium by punishing deviations with nonmeasurable
behavior that is assigned an extremely low payoff. In addition, we show that any
equilibrium under the calculability restriction can be supported as an equilibrium
under the payoff assignment approach for some assignment of expected payoffs tononmeasurable behavior. A partial converse of this result is also obtained.
The rest of the paper proceeds as follows. In section 2, we specify the model.
Section 3 shows that the concepts of traceability and frictionality together imply theexistence of a unique path of play given a strategy profile. Section 4 deals withmeasurability issues and shows that the calculability assumption implies that agents’behavior is measurable. Section 5 contains the aforementioned applications of our
methodology. In section 6, we compare our approach to an alternative approach inwhich the strategy space is not restricted but a certain payoff is assigned to nonmeasurable behavior. Proofs of the major results are in the appendix. The online
appendix provides further applications, extensions, derivations, and discussions.
2

Model

There are a finite number of agents, and the set of agents is denoted by I. Time runs
continuously in [0, T ) where T ∈ (0, ∞]. Each agent i ∈ I has a measurable action
space Ai . Let (Ω, F, P ) be a probability space, and let {Ft }t∈[0,T ) be a filtration of thesigma-algebra F. The shock level at time t, which is denoted by st , evolves according
to a stochastic process whose state space is S endowed with the sigma-algebra B(S).10
The state space S can be any set but in applications is typically the product of the setof variables representing the environment (e.g., the amount of resources, the price, thecost) and R+ representing calendar time.11 The shock process {st }t∈[0,T ) is assumed
10

The notation B(X) represents the Borel sigma-algebra on a topological space X.
The former set could also be a binary set representing whether there is a Poisson hit at a given
moment.
117

to be progressively measurable.12
In every instant of time, each agent i observes the current realization of the shocklevel and chooses an action from a subset of Ai that can depend on the history.
Formally, for each i ∈ I and t ∈ [0, T ), let ait represent the action that agent i chooses
at time t. The collection {ait }t∈[0,T ) of agent i’s actions indexed by time t ∈ [0, T )is called an action path. Letting u ∈ [0, T ), the collection {ait }t∈[0,u) is said to be
an action path up to time u. Neither the probability space nor the shock processdepends in any way on the actions of the agents.13
A history of the game is represented as h = {st , (ait )i∈I }t∈[0,T ) . That is, a history
consists of the realization of the shock process along with the actions chosen by theagents at each time. Given a history h, a history hu up to time u is defined as{st }t∈[0,u] , {(ait )i∈I }t∈[0,u) . Note that hu includes information about the shock attime u but does not contain information about the actions at time u. By convention,
h0 is used to denote the null history (s0 , {}) at the start of the game. Letting Ht be
S
the set of all histories up to time t, define H = t∈[0,T ) Ht .
For each i ∈ I, define the feasible set of actions at every history by the function
Āi : H → 2Ai . We assume that there exists an action z ∈ Ai such that z ∈ Āi (ht ) for
any ht ∈ H. The action z can be interpreted as “not moving,” whereas an action otherthan z is regarded as a “move.” A strategy for agent i ∈ I is a map πi : H → Ai .14 Let
Πi with generic element πi represent the set of all strategies for agent i.15 In addition,
12

A stochastic process {xt }t∈[0,T ) can be treated for any υ ∈ [0, T ) as a function x(t, ω) on the
product space [0, υ] × Ω. It is said that {xt }t∈[0,T ) is progressively measurable if for any υ ∈ [0, T )the function x(t, ω) is measurable with respect to the product sigma-algebra B([0, υ]) × Fυ .
13
The assumption that the shock process does not depend on the actions of the agents is not particularly restrictive. For example, in order to model a situation where uncertainty evolves according
to an Itô process that depends on the actions of the agents, the shock process could be specified as a
Wiener process, and the value of the Itô process at each time could be encoded in the action space,which may be history-dependent. A related example is the application in section 5.1, where thevolume of a forest, which is affected by the actions of agents to harvest wood and evolves according
to an arithmetic Brownian motion between harvests, is specified in the set of feasible actions, which
we define shortly.
14
Simon and Stinchcombe (1989) allow for sequential moves at a single moment of time. The
definition of the strategy space here rules out such behavior. We can thereby express histories in anatural way as specifying a profile of actions at each time and define strategies in the standard way
as a mapping from histories to actions. In addition, it may not be realistic to allow players to movesequentially without any elapse of time. This restriction is innocuous except in the partnership andcooperation game between criminals in the online appendix, where we argue that restricting moves
to happen at Poisson opportunities may cause an inefficient delay. If agents could immediately
respond to a deviation at the current Poisson jump instead of waiting for the next Poisson jump to
respond, such an inefficiency would not arise.
15
The definition of strategies thus far does not eliminate the possibility of flow moves, whereby8

a strategy πi for agent i ∈ I is said to be feasible if it satisfies the restriction thatπi (ht ) ∈ Ā(ht ) for any ht ∈ H. Let Π̄i denote the set of all feasible strategies foragent i. For any profile of sets (Dj )j∈I and any i ∈ I, let D−i = ×j∈I\{i} Dj .
3
3.1

Existence and Uniqueness of the Action Path
ExamplesSince the model is formulated in continuous time, the definition of the strategy space
is not trivial as noted by Simon and Stinchcombe (1989) and Bergin and MacLeod(1993). As explained below, we need to develop a novel approach for restricting thestrategy space in our context. The following two examples illustrate the problems
that our restrictions help eliminate.16
Example 1. (No action path consistent with a given strategy profile) Suppose that Āi (ht ) = {x, z} for each i ∈ I and every ht ∈ H. We argue that there isno action path consistent with the following strategy profile. If there is a positive
integer m such that the current time t is equal to 1/m and no agent has chosen action
x before time t, then each agent chooses x at time t. Otherwise, all agents choose
action z. To see that there is no action path consistent with these strategies, noticethat on any path of play of the given strategy profile, there must exist exactly one
time t > 0 at which action x is taken. However, if the agents were to choose x at thistime t, then they would be deviating from the given strategy profile because there
exists m < ∞ large enough that 1/m ∈ (0, t).
In the preceding example, an action path consistent with the specified strategy
profile fails to exist because a given set of times may not have a least element incontinuous time. Similarly, restricting attention to Markov strategies is not enoughto ensure that strategy spaces are well defined because in continuous time there may
not exist a least time at which a state variable satisfies a given condition.17
actions other than z are taken for a positive measure of times. For example, when harvesting trees,
agents may cut a tree continuously over time so that the amount cut at each instant of time is zero,but the total amount cut over an interval of time may be positive. The subsequent sections restrict
the strategy space so as to avoid flow moves, thereby preventing such behavior in the tree harvesting
problem (section 5.1).
16
These problems arise because the strategy of an agent is a contingent plan in which the actionof an agent at a given time may be conditional on past actions. Similar examples can be constructedwith just one agent, but in that case such difficulties could be avoided by defining a strategy as a
mapping from each time to an action at that time without reference to previous actions.
17
A strategy is said to be Markov if the action that it specifies at any history up to a given time9

Example 2. (Multiple action paths consistent with a given strategy profile)
Suppose again that Āi (ht ) = {x, z} for each i ∈ I and every ht ∈ H. We argue that
there is more than one action path consistent with the following strategy profile. If
the current time t is equal to 1/m for some positive integer m and another agent haschosen action x at time 1/n for every positive integer n greater than m, then eachagent chooses x at time t. Otherwise, each agent chooses z.
The following two action paths can be outcomes of this strategy profile. First, allagents choose action z at all t ∈ [0, T ). Second, each agent chooses x if the current
time t is equal to 1/m for some positive integer m, and they choose z otherwise. It
is straightforward to check by inspection that all the agents are following the given
strategy profile in each case.
To overcome the difficulties with specifying strategies in continuous-time games
as in the above examples, we use a framework in which strategies are defined as
mappings that assign to each history an action at that history. However, a questionis whether there is an alternative formulation that bypasses these difficulties. A
seemingly simple possibility may be to define strategies using a sequence of stopping
times. Nonetheless, this approach involves a number of complications, as explained
below.
Remark 1. (Formulating strategies in terms of stopping times) Suppose thateach agent chooses a positive stopping time at the null history as well as a move to
make once that time is reached. Thereafter, whenever some agent moves, each agent
chooses another stopping time and move to make at that time.
An issue with such a formulation of strategies is that the resulting path of play
may not be determined for all times during the game. In particular, the sequence of
stopping times may be infinite and have an upper bound that is less than the time atwhich the game ends. Deriving the path of play after this upper bound would involve
taking the limit of this sequence and specifying another sequence in a well-definedmanner.
This formulation also leaves open the question of how to define behavior at histories where the current time is already greater than the applicable stopping time.
One solution to this problem may be to specify the stopping time as a function that
depends only on the current value of a state variable that summarizes the history of the game up tothat time.

10

depends on the current time as well, but that would be no simpler than our approach
where a strategy is just a mapping from histories to actions. Moreover, such a specification in which the stopping time depends on the current time would necessitate an
additional condition to guarantee the consistency of stopping times across different
histories.
In addition, note that under this formulation, a strategy is not simply a sequenceof stopping times but also entails specifying a stochastic process encoding the moveat each stopping time. As in section 4, the action process would need to be suitably
restricted so as to be progressively measurable, thereby ensuring that expected payoffs
are well defined.
3.2

Traceability and Frictionality

We introduce a series of concepts so as to eliminate strategies like those in the examples above. We define three conditions, which we call consistency, traceability,and frictionality. Consistency is a property of histories, whereas traceability andfrictionality are properties of strategies.
Definition 1. Given i ∈ I, the history h = st , (ajt )j∈I t∈[0,T ) is said to be consistent
with strategy πi at time t if πi (ht ) = ait .
Roughly speaking, a history is said to be consistent with a strategy for a given
agent if the history is a possible outcome when that agent plays the strategy.
For any action path {bit }t∈[0,u) of agent i ∈ I up to an arbitrary time u, let
Γi ({bit }t∈[0,u) ) be the set consisting of any action path {ait }t∈[0,T ) such that {ait }t∈[0,u) ={bit }t∈[0,u) . Given any history ku = {gt }t∈[0,u] , {(bjt )j∈I }t∈[0,u) up to time u, let−i
WiT R (ku ; πi ) be the set of all ω ∈ Ω such that for any {a−it }t∈[0,T ) ∈ Γ−i ({bt }t∈[0,u) )with ajt = z for all t > u and j 6= i, there exists {ait }t∈[0,T ) ∈ Γi ({bit }t∈[0,u) ) for which
the history h = st (ω), (ajt )j∈I t∈[0,T ) is consistent with πi at each t ∈ [u, T ). In otherwords, WiT R (ku ; πi ) represents the set of all sample paths of the shock process suchthat given the action paths of the agents up to time u, there exists a history consistent
with agent i’s strategy πi from time u onwards, assuming that agent i’s opponents do
not move after time u.
Definition 2. Given i ∈ I, the strategy πi ∈ Πi is traceable if for any historyku = {gt }t∈[0,u] , {(bjt )j∈I }t∈[0,u) up to time u, the set WiT R (ku ; πi ) is measurable withrespect to F and has conditional probability one given that {st }t∈[0,u] = {gt }t∈[0,u] .
11Intuitively, a strategy for a given agent is said to be traceable if there exists a
history that is consistent with the strategy when the other agents do not move in the
future. Traceability excludes the strategy in example 1.
Let Ξi (t) denote the set consisting of every action path {aiτ }τ ∈[0,T ) of agent i ∈ I
for which there exists no u > t such that the set {τ ∈ [t, u] : aiτ 6= z} contains

infinitely many elements. Given any history ku = {gt }t∈[0,u] , {(bjt )j∈I }t∈[0,u) up to
time u, let WiF R (ku ; πi ) be the set of all ω ∈ Ω such that {ait }t∈[0,T ) ∈ Ξi (u) for all−i{ait }t∈[0,T ) ∈ Γi ({bit }t∈[0,u) ) such that there exists {a−it }t∈[0,T ) ∈ Γ−i ({bt }t∈[0,u) ) for
which the history h = {st (ω), (ajt )j∈I }t∈[0,T ) is consistent with πi at each t ∈ [u, T ). In
other words, WiF R (ku ; πi ) represents the set of all sample paths of the shock process
such that given the action paths of the agents up to time u, agent i’s strategy πi
induces only finitely many moves in any finite interval of time in the future, regardlessof the behavior of agent i’s opponents from time u onwards.
Definition 3. Given i ∈ I, the strategy πi ∈ Πi is frictional if for any historyku = {gt }t∈[0,u] , {(bjt )j∈I }t∈[0,u) up to time u, the set WiF R (ku ; πi ) is measurable withrespect to F and has conditional probability one given that {st }t∈[0,u] = {gt }t∈[0,u] .
Intuitively, a strategy for a given agent is said to be frictional if any history thatis consistent with that strategy has the property that the agent moves (i.e., takes anaction other than z) only a finite number of times in any finite time interval, where
the number of moves can depend on the sample path of the shock process as well as
the action paths of the other agents. Frictionality excludes the strategy in example
2. It also rules out flow moves.
Traceability and frictionality restrict the strategy space for each agent. Note thatthese conditions are defined in terms of the strategy of an individual agent as opposedto the strategy profile of all agents. In addition, observe that these conditions impose
requirements on a strategy not only after the null history but after every history up
to any time. We can now state a main result.18
Theorem 1. Choose any profile (πj )j∈I of strategies that satisfy traceability and fric
tionality. Choose any history ku = {gt }t∈[0,u] , {(bjt )j∈I }t∈[0,u) up to time u.
18

The statement of theorem 1 is cumbersome, but it is inevitably so. For example, we cannot
simplify the statement of the theorem by making reference to the conditional probability of an event
given the history ku up to time u because the action path {(bjt )j∈I }t∈[0,u) of the agents up to timeu, which is part of ku , cannot be treated as a well defined random variable at this point. Hence, wefirst fix the history ku in the statement of the theorem and then consider the conditional expectation
given the shock process {st }t∈[0,u] up to time u.

12

1. Given that {st }t∈[0,u] = {gt }t∈[0,u] , there is conditional probability one that there
exists a unique profile ({ajt }t∈[0,T ) )j∈I of action paths with {ajt }t∈[0,T ) ∈ Γj ({bjt }t∈[0,u) )for each j ∈ I such that the history h = st , (ajt )j∈I t∈[0,T ) is consistent with πifor each i ∈ I at every t ∈ [u, T ).
2. Given that {st }t∈[0,u] = {gt }t∈[0,u] , there is conditional probability one that theaction paths in the first part of the theorem satisfy {ajt }t∈[0,T ) ∈ Ξj (u) for eachj ∈ I.
To paraphrase, suppose that each agent uses a strategy that satisfies the restrictions. For any realization of the shock process following a history up to a given time,
the game has a unique action path. Moreover, the action path is such that there is
only a finite number of non-z actions in any finite time interval. The proof entails thefollowing complication. The strategies of an agent’s opponents can specify actions
that depend on the agent’s behavior. However, the definitions of traceability andfrictionality refer not to the opponents’ strategies, but only to the opponents’ action
paths. Thus, the existence and uniqueness of action paths consistent with a given
strategy profile are not immediate consequences of these assumptions. Indeed, there
may exist a profile of traceable strategies such that no history is consistent with allagents’ strategies at all times.19 The proof of theorem 1 employs frictionality as wellto identify one by one each time a non-z action is taken, utilizing the two restrictions
at every step.
Hereafter, we restrict attention to strategies that satisfy traceability and frictionality. Henceforth, let ΠTi F denote the set consisting of every traceable and frictional
strategy for agent i. Below we comment on the consequences of relaxing the frictionality assumption.
Remark 2. (Weak frictionality and existence of a unique outcome) Frictionality requires that for any action path of the other agents, an agent moves only
finitely many times in a finite interval of time. A less restrictive way to specify frictionality is to impose this requirement only when the other agents move just finitely
often in a finite time interval. Formally, a strategy is weakly frictional if it meetsthe definition of frictionality when WiF R (ku ) is restricted to be the set of all ω ∈ Ωsuch that {ait }t∈[0,T ) ∈ Ξi (u) for all {ait }t∈[0,T ) ∈ Γi ({bit }t∈[0,u) ) such that there exists
19

See the online appendix for an example of such a strategy profile.

13

j
−i
{a−i
t }t∈[0,T ) ∈ Γ−i ({bt }t∈[0,u) )∩Ξ−i (u) for which the history h = {st (ω), (at )j∈I }t∈[0,T )is consistent with πi at each t ∈ [u, T ).
Suppose first that all the agents use traceable and weakly frictional strategies.
Then the resulting path of play may not be unique. For instance, the strategies inexample 2 satisfy traceability and weak frictionality, but there are multiple action
paths consistent with this strategy profile. Suppose next that all the agents use
traceable and weakly frictional strategies and no more than one agent uses a strategy
that is not frictional. Then both items in theorem 1 continue to hold.20 In particular,
the continuation path at any history up to a given time is unique, with each agent
moving only finitely many times in any finite time interval. This is a useful extension
because it enables modelling situations where one agent quickly responds to every
move by another agent (see section 5.2 for an example).

3.3

Comparison with Previous Literature

A question is how traceability and frictionality relate to restrictions on the strategy
space that were previously developed for a deterministic setting. Comparing our
methodology to existing approaches helps to assess how robust it is and clarify what
its limits may be. One condition is inertia from Bergin and MacLeod (1993), which
requires agents to wait for a certain interval of time after changing actions. A suitableapplication of their restriction to our model would eliminate pathologies related to
the nonexistence or nonuniqueness of the path of play. The inertia condition might
be extended to a stochastic environment in at least two ways.
One definition involves imposing inertia uniformly at each history. This versionof the condition requires that given any history ht up to an arbitrary time t, there
exists  > 0 such that an agent does not move in the time interval (t, t + ), where cannot depend on the realization of the shock process {sτ }τ ∈(t,∞) after time t.
Formally, a strategy πi ∈ Πi is said to be uniformly inertial if for every historyku = {gt }t∈[0,u] , {(bjt )j∈I }t∈[0,u) up to time u, there exists  > 0 such that πi (hτ ) = z

for all τ ∈ (u, u + ) and every history h = st , (ajt )j∈I t∈[0,T ) with {st }t∈[0,u] ={gt }t∈[0,u] and {ajt }t∈[0,u) = {bjt }t∈[0,u) for all j ∈ I. Nonetheless, this approach is too
restrictive in the current setting. For instance, in the tree harvesting problem (section5.1), the optimal behavior of the agents is such that for every  > 0, there is positive
conditional probability of the agents cutting trees in the time interval (u, u + ) given
20

The proof is in appendix A.1.

14

that the agents cut trees at time u. This violates the uniform inertia condition, which
requires that if the agents cut trees at time u, then there exists  > 0 such that they
do not cut trees during the time interval (u, u + ).
Another approach would be to apply inertia pathwise from each history. Formally, a strategy πi ∈ Πi is said to be pathwise inertial if for every historyku = {gt }t∈[0,u] , {(bjt )j∈I }t∈[0,u) up to time u and any realization {gt }t∈(u,T ) of theshock process after time u, there exists  > 0 such that πi (hτ ) = z for all τ ∈ (u, u + )and every history h = gt , (ajt )j∈I t∈[0,T ) with {ajt }t∈[0,u) = {bjt }t∈[0,u) for all j ∈ I. In
other words, for every history ht up to an arbitrary time t and any realization of theshock process {sτ }τ ∈(t,∞) after time t, there is required to exist  > 0 such that an
agent does not move in the time interval (t, t+).21 This formulation enables arbitrarily quick responses to changes in the shock level. However, it does not in itself ensure
measurability of the path of play in a stochastic setting. Some additional restriction
is needed, but the iterative procedure that we use to prove theorem 2 would not beapplicable if the traceability and frictionality restrictions were replaced with either
version of inertia.22 Example 8 in section 4.1 describes a strategy profile satisfying
both versions of inertia (as well as traceability and frictionality) while the resulting
behavior is not measurable.
Irrespective of whether the former or the latter version of inertia is used, the setof strategies satisfying inertia is neither a subset nor a superset of the set of traceable
and frictional strategies. As the result below states, even the pathwise formulationof inertia would imply traceability if applied to the current setting because inertia
guarantees the existence of a history consistent with a given strategy at each time.
Proposition 1. Any pathwise inertial strategy is traceable.
The next example shows that frictionality is not satisfied by even the uniform
specification of inertia because inertia allows for some strategies that induce infinitely
many moves in a finite time interval.
21

This condition could be modified so that zero probability events would never cause it to beviolated. In particular, we could require that for every history ht up to an arbitrary time t, there
is conditional probability one given {sτ }τ ∈[0,t] of {sτ }τ ∈(t,T ) being such that there exists  > 0 for
which an agent does not move in the time interval (t, t + ). However, such a redefinition would notaffect the analysis of the applications that we present, particularly given that diffusion processes aremodelled as a canonical Brownian motion as explained in footnote 38.
22
For this reason, it is unclear to us whether calculability in conjunction with either uniform or
pathwise inertia would ensure that the action processes of the agents are measurable.

15

Example 3. (Uniform inertia does not imply frictionality) Suppose thatĀi (ht ) = {x, z} for every ht ∈ H. Consider the strategy in which agent i chooses
x if and only if the current time t is such that t = 1 − 1/2m for some positive integer
m. This strategy satisfies uniform inertia because for any time t, agent i does not
move in the time interval (t, ∞) if t ≥ 1, and agent i does not move in the timeinterval (t, 1 − 1/2l ) if t < 1, where l is the least integer k such that 1 − 1/2k > t.
This strategy violates frictionality because any history consistent with it is such thatagent i moves at each of the infinitely many times t < 1 at which t = 1 − 1/2m forsome positive integer m.
There also exist traceable and frictional strategies that do not satisfy even thepathwise definition of inertia. In particular, inertia eliminates some traceable andfrictional strategies in which the response time of one agent varies with the actionsof another agent. An example of such a strategy is given below.
Example 4. (Traceability and frictionality do not imply pathwise inertia)
Suppose I = {1, 2} and that Āi (ht ) = {x, z} for each i ∈ I and every ht ∈ H.
Consider the strategy π̃i in which for any t > 0, agent i chooses x at time t if andonly if agent i has not chosen x before time t, agent −i has chosen x at time t/2, andagent −i has not chosen x before time t/2. The strategy π̃i does not satisfy pathwise
inertia because for any  > 0, there exists a history consistent with this strategy inwhich agent i chooses x at time /2 and agent −i chooses x at time /4.
We argue that π̃i satisfies traceability and frictionality. Choose any history up to
an arbitrary time u as well as any action path for agent −i. If there exists a least
time t̃−i such that agent −i chooses x, if t̃−i satisfies 2t̃−i ≥ u, and if x is not chosen
by agent i before time u, then a history h is consistent with π̃i from time u onwards if
and only if h is such that x is chosen by agent i at time 2t̃−i and z is chosen by agent
i at every other time t ≥ u. Otherwise, a history h is consistent with π̃i from time u
onwards if and only if h is such that z is chosen by agent i at every time t ≥ u. Since
there exists a history consistent with π̃i at and after time u and in any such history
x is chosen by agent i at most once during this interval, the strategy π̃i is traceable
and frictional.
Another condition specified for deterministic models is admissibility in Simon andStinchcombe (1989), which would ensure the existence of a unique path of play if
appropriately applied to our framework. This condition comprises three restrictions
16

that we explain in what follows. The first is F1, which bounds the number of moves
a strategy can induce. One approach to extending F1 to a stochastic setting is to
impose the bound uniformly. Formally, a strategy πi ∈ Πi satisfies uniform F1 if for
any time ũ ∈ [0, T ), there exists an integer m such that the following holds. Choose
any history of the form h = st , (ajt )j∈I t∈[0,T ) as well as any time û ∈ [0, ũ] such thath is consistent with πi for t ∈ [û, ũ]. Suppose that the set consisting of each timet < û such that ait 6= z is nonempty and has a maximum denoted by u. If πi (hu ) = aiu
and there are at least m distinct values of t < û such that ait 6= z, then ait = z fort ∈ [û, ũ].
An alternative is to specify the requirement pathwise. Formally, a strategy πi ∈ Πisatisfies pathwise F1 if for any realization {gt }t∈[0,T ) of the shock process as well as
any time ũ ∈ [0, T ), there exists an integer m such that the following holds. Choose
any history of the form h = gt , (ajt )j∈I t∈[0,T ) as well as any time û ∈ [0, ũ] such thath is consistent with πi for t ∈ [û, ũ]. Suppose that the set consisting of each timet < û such that ait 6= z is nonempty and has a maximum denoted by u. If πi (hu ) = aiu
and there are at least m distinct values of t < û such that ait 6= z, then ait = z fort ∈ [û, ũ].
Intuitively, a strategy for agent i satisfies uniform F1 if for any finite interval oftime, there exists an upper bound on the number of times agent i can move duringthat time interval. If, however, the bound has been reached or exceeded due to a
deviation by agent i, then agent i is permitted to move at most one more time. The
pathwise version of F1 is defined similarly except that the bound on the number ofmoves in each interval of time can also depend on the sample path of the shock process.
The uniform version of F1 rules out some applications of interest. For example, themaximal equilibrium of the tree harvesting problem (section 5.1) is such that given
any proper interval of time along with a nonnegative integer m, there is positive
probability that trees are cut more than m times during this interval. Pathwise F1
is less restrictive, but does not cover every situation. For example, the equilibriumof the supply chain example (section 5.2) is such that given any proper interval oftime and any price path along with a nonnegative integer m, there exists a history
consistent with the strategy of the oil refinery in which it processes more than mbatches of oil during this interval in response to oil being extracted more than mtimes.
The second criterion for admissibility is F2, which requires a strategy to in17

duce a well defined action path when the other agents do not move in the future. Formally, a strategy satisfies F2 if the following holds. Choose any historyku = {gt }t∈[0,u] , {(bjt )j∈I }t∈[0,u) up to time u. Given that {st }t∈[0,u] = {gt }t∈[0,u] ,−ithere is conditional probability one that for any {a−it }t∈[0,T ) ∈ Γ−i ({bt }t∈[0,u) ) suchthat ajt = z for all t > u and j 6= i, there exists a unique action path {ait }t∈[0,T ) ∈Γi ({bit }t∈[0,u) ) for which the history h = st , (ajt )j∈I t∈[0,T ) is consistent with πi at eacht ∈ [u, T ), and this action path satisfies {ait }t∈[0,T ) ∈ Ξi (u). Intuitively, a strategy foragent i satisfies F2 if there exists a unique history that is consistent with the strategy
when the other agents do not move in the future and this history is such that agent
i moves only finitely many times in any finite time interval in the future.
The third requirement is F3, which imposes a special form of continuity on strategies. In particular, if one history up to a given time specifies the same moves as
another history but at times that are just slightly greater, then an agent is required
to take the same action at these two histories. This condition is violated by thesolutions to all the applications in section 5 and the online appendix except for theones in sections 5.3 and 5.4.23 However, Simon and Stinchcombe (1989) note that
this condition is not needed to prove the existence and uniqueness of an outcome
consistent with a given strategy profile. It is used only to relate continuous-time
outcomes to the limits of outcomes in discrete-time games.24 For our purposes, wedefine uniformly admissible strategies as those that satisfy uniform F1 and F2 andpathwise admissible strategies as those that satisfy pathwise F1 and F2.
The result below shows that pathwise and hence also uniform F1 imply frictionality and that F2 implies traceability. Intuitively, suppose that agent i plays anadmissible strategy πi and that the other agents do not move in the future. ThenF2 guarantees the existence of an action path for agent i that is consistent with πi .
Hence, traceability is satisfied. Moreover, both uniform and pathwise F1 ensure that
πi induces only finitely many moves in any finite interval of time. Hence, frictionality
23

For instance, consider two histories up to a given time specifying the same moves but at slightly
different times, where one history but not the other is on the equilibrium path of play. If an
equilibrium in grim-trigger strategies is played as in section 5.1, then the behavior prescribed bythese strategies may differ between these two histories.
24
Since traceability and frictionality do not imply F3, the approach in Simon and Stinchcombe(1989) cannot easily be extended to relate the continuous-time equilibria that we consider to theequilibria of a discretized version of the game that is played on an increasingly fine grid. We believe
this should not be considered a deficiency as continuous-time and discrete-time methods involve
different models that do not necessarily have the same implications, and it is unclear a priori whichapproach is more plausible.

18

is satisfied.
Proposition 2. If πi satisfies pathwise F1, then πi is frictional. If πi satisfies F2,then πi is traceable.
Hence, the set of admissible strategies, regardless of whether defined pathwise or
uniformly, is a subset of the set of traceable and frictional strategies. The converseof this result does not hold for two reasons. First, not only do both versions ofproperty F1 apply on the path of play, but they also place important restrictions on
the behavior of the agents off the path of play. In particular, if an agent violates theupper bound on the number of moves in an interval of time because of a deviation,then the agent can move only once more in that interval. As the example below
illustrates, this rules out some simple strategies that require an agent always to moveat specified times.
Example 5. (Strategy violating pathwise F1 off the path of play) Supposethat Āi (ht ) = {x, z} for every ht ∈ H. Consider the following strategy π̃i . Let t0 andt00 be any two times with t00 > t0 > 0. Agent i chooses x if and only if the time iscurrently t0 or t00 . Note that π̃i is traceable and frictional. For any positive integer m,
let hm be a history in which agent i chooses x if and only if the time is equal to t00 or
to t0 · k/m for some positive integer k ≤ m. The history hm is consistent with π̃i attime t0 , but there is no value of m such that π̃i (hmt00 ) = z. Hence, strategy π̃i does notsatisfy pathwise F1.
Second, both versions of F1 preclude some traceable and frictional strategies inwhich the number of moves by one agent varies with the timing of moves by another agent. Below is an example of a traceable and frictional strategy that violates
pathwise F1, even when it is applied only on the path of play.
Example 6. (Traceability and frictionality do not imply pathwise F1) Suppose I = {1, 2} and that Āi (ht ) = {x, z} for each i ∈ I and every ht ∈ H. Consider
the following strategy π̃i . Agent i does not choose x at any time in the interval [0, 1].
If there exists a positive integer m such that agent −i chooses x at time 1/m andagent −i chooses z at every time t < 1 such that t 6= 1/m, then agent i chooses x attime t > 1 if and only if there exists a positive integer k ≤ m such that t = 1 + k/m.
Otherwise, agent i chooses z at each time t > 1. This strategy does not satisfy pathwise F1 because for every integer m > 0, there exists a history consistent with this19strategy in which agent −i chooses x at time 1/m and agent i chooses x at m distinct
times during the interval (1, 2], meaning that there is no upper bound on the numberof moves by agent i in this interval.
We argue that π̃i satisfies traceability and frictionality. Choose any history up to
an arbitrary time u as well as any action path for agent −i. If there exists an integerm > 0 such that agent −i chooses x at time 1/m and agent −i does not choose x at
any time t < 1 such that t 6= 1/m, then a history h is consistent with π̃i from time u
onwards if and only if h is such that x is chosen by agent i at every time t ≥ u forwhich there exists a positive integer k ≤ m such that t = 1 + k/m and z is chosen
by agent i at every other time t ≥ u. Otherwise, a history h is consistent with π̃ifrom time u onwards if and only if h is such that z is chosen by agent i at every timet ≥ u. Since there exists a history consistent with π̃i at and after time u and in any
such history x is chosen by agent i finitely many times during this interval of time,
the strategy π̃i is traceable and frictional.
Traceability alone does not imply F2. Although example 2 satisfies F2, this restriction is violated by the following single-agent variant of that example. There exist
traceable strategies that induce more than one path of play or an action path with
infinitely many moves in a finite interval of time.
Example 7. (Traceable strategy violating F2) Suppose that Āi (ht ) = {x, z} forevery ht ∈ H. Consider the traceable strategy π̃i defined as follows. If the current
time t is equal to 1/k for some positive integer k and agent i has chosen action x attime 1/m for every positive integer m greater than k, then agent i chooses x at timet. Otherwise, agent i chooses z. A history h is consistent with π̃i if agent i chooses
z at all times or if agent i chooses x if and only if the current time t is equal to 1/k
for some positive integer k. Noting that there exists more than one history consistent
with π̃i (including one in which agent i moves infinitely many times in a finite intervalof time), π̃i does not satisfy F2.
When taken together, traceability and frictionality imply F2, as the result below
shows. Intuitively, suppose that agent i plays a traceable and frictional strategy πiand that each of the other agents does not move in the future, which is also a traceable
and frictional strategy. By theorem 1, there exists a unique action path for agent ithat is consistent with πi and this action path specifies only finitely many moves in20

any finite time interval. Hence, F2 is satisfied. This result and its proof also hold if
the frictionality requirement is relaxed to weak frictionality as defined in remark 2.
Proposition 3. Any traceable and frictional strategy satisfies F2.
Finally, there are uniformly and pathwise admissible strategies, like π̃1 in example
8, that may not induce measurable behavior. A further restriction is necessary, as
discussed in the next section.
4
4.1

Measurability of the Action Process
An Example

Another matter concerning the specification of the model involves the measurabilityof the path of play. This issue causes little trouble in nonstochastic situations including Bergin and MacLeod (1993) and Simon and Stinchcombe (1989). Indeed, if
there were no uncertainty about the shock level, then constraints such as traceability and frictionality would be enough to define the payoff to each agent. However,the stochastic nature of the shock complicates matters since agents can condition
their behavior on moves by Nature. In order to compute expected payoffs, the action
process should be progressively measurable.
In discrete time, it is relatively uncomplicated to specify a game so that the pathof play is measurable. The analyst can simply require the strategies of the agents tobe measurable functions from the history up to each time to the actions at that time.
Under such an assumption, if the path of play is measurable up to and including
any given time, then the behavior of the agents will be measurable in the following
period as well. Hence, the measurability of the path of play up to every time can beshown by induction. In continuous time, however, such an iterative procedure is notapplicable because the next period after any given time is not well defined.
In continuous-time games with imperfect monitoring like Sannikov (2007), it isagain unproblematic to define the strategy space so as to ensure the measurabilityof the actions taken by agents. The strategy of each agent can be specified as astochastic process that is progressively measurable with respect to an exogenously
given filtration. Nonetheless, this task is less straightforward in the current context
where the actions induced by an agent’s strategy may or may not be measurable
depending on the other agents’ behavior. The example below demonstrates this sortof interdependence in terms of the measurability of actions.
21

Example 8. (Actions induced by a strategy may or may not be measurable)
Let {st }t∈[0,∞) be an arbitrary stochastic process with state space S ⊆ R++ . Assume
that there exists S̃ ⊆ S along with t̃ > 0 such that {ω ∈ Ω : st̃ (ω) ∈ S̃} is not ameasurable subset of the probability space (Ω, F, P ).25 Suppose I = {1, 2} and thatĀi (ht ) = {x, z} for each i ∈ I and every ht ∈ H.
Suppose that agent 1 plays the strategy π̃1 defined as follows. If agent 2 chooses
z at time t̃ > 0, then agent 1 is required to choose x at time 2t̃. Otherwise, agent 1 isrequired to choose x at time 2t̃ if the shock st̃ at time t̃ is in the set S̃ and to choose
x at time t̃ + st̃ if the shock st̃ at time t̃ is not in the set S̃. Suppose that agent 2
plays the strategy π̃2 defined as follows. Agent 2 is required to choose x at time t̃. If
agent 1 chooses x at time 2t̃, then agent 2 is required to choose x at time 3t̃. The
agents do not take action x except as specified above.
For i ∈ {1, 2} and t ∈ [0, T ), the action ait of agent i at time t can be treated as afunction from the probability space (Ω, F, P ) to {x, z}. Even if the action a2t of agent
2 is measurable at each time t ∈ [0, T ), the strategy π̃1 can induce a nonmeasurable
action a12t̃ by agent 1 at time 2t̃. By contrast, the strategy π̃2 induces a measurable
action a2t by agent 2 at every time t ∈ [0, T ), unless the action a12t̃ of agent 1 at time2t̃ is nonmeasurable.
In the ensuing analysis, we seek to eliminate strategies like π̃1 because such strategies can generate nonmeasurable behavior by one agent even if the actions of the otheragents are measurable. Nonetheless, we wish to retain strategies like π̃2 that ensure
the measurability of one agent’s actions given the measurability of the other agents’actions.
The following two simple ways of restricting the strategy space do not succeed inremoving π̃1 while retaining π̃2 , which is why we introduce a more complex procedure
in section 4.2. First, suppose that we delete strategies such that for some strategy of
one’s opponent, one’s behavior is not measurable. However, this would rule out bothπ̃1 and π̃2 . Second, suppose that we delete strategies such that for every strategy
of one’s opponent, one’s behavior is not measurable. However, this would rule outneither π̃1 nor π̃2 .
25

For instance, let the probability space be the interval (0, 1) with the Borel sigma-algebra andthe Lebesgue measure, and let the state space be the interval (0, 1). Suppose that st̃ is the uniform
random variable defined by st̃ (ω) = ω. Then a Vitali set can be used to provide an example of S̃.

22

4.2

Two-Step Procedure

In order to suitably phrase the definition, a two-step procedure is adopted. We first
specify a very restrictive set of strategies that always induce measurable behavior,and the first set is then used to construct a more inclusive second set. The elementsof the resulting strategy space are said to be calculable.
We begin by providing a formal definition of the action processes. Let π = (πj )j∈I
with πj ∈ ΠTj F for j ∈ I be a profile of traceable and frictional strategies. Choose
an arbitrary time u. Fix any path {(bjt )j∈I }t∈[0,u) of actions for the agents up to this
time. Define the history up to time u by ku = {gt }t∈[0,u] , {(bjt )j∈I }t∈[0,u) , where{gt }t∈[0,u] is any realization of shock levels from time 0 to u. Given that {st }t∈[0,u] ={gt }t∈[0,u] , let {φjt (ku , {sτ }τ ∈(u,T ) , π)}t∈[0,T ) j∈I with {φjt (ku , {sτ }τ ∈(u,T ) , π)}t∈[0,T ) ∈Γj ({bjt }t∈[0,u) ) for each j ∈ I be a profile of action paths for which the historyst , [φjt (ku , {sτ }τ ∈(u,T ) , π)]j∈I t∈[0,T ) is consistent with πi for each i ∈ I at everyt ∈ [u, T ). According to theorem 1, such a profile of action paths exists and is
unique with conditional probability one. Furthermore, these action paths satisfy{φjt (ku , {sτ }τ ∈(u,T ) , π)}t∈[0,T ) ∈ Ξj (u) for each j ∈ I with conditional probability one.
Denoting b = {(bjt )j∈I }t∈[0,u) and π = (πj )j∈I , the action process ξbi (π) for i ∈ I
represents the stochastic process defined as follows. At any time t ∈ [0, u), ξbi (π) = z
holds. Let g̃ = {g̃t }t∈[0,u] represent the realization of shock levels until time u, anddenote the resulting history up to time u by k̃u = (g̃, b). Given the realization of theshock {sτ }τ ∈(u,T ) after time u, ξbi (π1 , π2 ) = φit (k̃u , {sτ }τ ∈(u,T ) , π) holds at each timet ∈ [u, T ). The state space of ξbi (π) is Ai with sigma-algebra B(Ai ). To paraphrase, letb signify any path of actions for the agents up to an arbitrary time u. The stochastic
process ξbi (π) is simply equal to z up to this time. Thereafter, ξbi (π) records the actions
chosen by agent i when the strategy profile π is played.26
We now restrict the strategy space so as to ensure the progressive measurabilityof the action processes.
First step: quantitative strategies
The first step is to define a set of traceable and frictional strategies that induce
26

For simplicity in defining the process ξbi (π), we consider the continuation path of play for eachagent i under strategy profile π when the agents follow the fixed action path b up to time u. The
results in this section would not change under an alternative definition in which at a history ku up
to time u on the path of play of π, we instead consider the continuation path of play under π when
the agents follow strategy profile π up to time u.

23measurable behavior by one agent whenever the other agents play traceable andfrictional strategies. The elements of this set are said to be quantitative.
Definition 4. For i ∈ I, the strategy πi ∈ ΠTi F is quantitative if it belongs to
TF
the set ΠQsuch that the stochastic process ξbi (πi , π−i ) isi consisting of any πi ∈ Πi
progressively measurable for all b and every π−i ∈ ΠT−iF .
The composition of the set ΠQ
i can vary with the specification of the shock process.
It contains traceable and frictional strategies that depend neither on the realized
values of the shock process nor on the actions of one’s opponents. An example ofa quantitative strategy would be the strategy that requires agent i to choose action
x 6= z at time 1 regardless of the actions of agents −i and that specifies action z byagent i at other times. As shown in the online appendix, the set ΠQ
i may also containsome strategies that are contingent on the realization of the shock and the behaviorof one’s opponents. Nonetheless, the set ΠQ
i is extremely restrictive as a strategy
space. In particular, the strategies π̃1 and π̃2 in example 8 are both excluded.
Second step: calculable strategies
The second step is to define a set of traceable and frictional strategies that induce measurable behavior by one agent whenever the other agents play quantitativeCstrategies. That is, the set ΠQ
i is used to construct a larger strategy space Πi , theelements of which are said to be calculable.
Definition 5. For i ∈ I, the strategy πi ∈ ΠTi F is calculable if it belongs to the
TF
such that the stochastic process ξbi (πi , π−i ) isset ΠCi consisting of any πi ∈ Πi
progressively measurable for all b and every π−i ∈ ΠQ−i .
The strategy space ΠCi admits a relatively broad range of behavior. It allows forstrategies that depend on the actions of one’s opponents, including many grim-trigger
strategies. For example, consider the strategy that requires agent i to choose action
x at time 2 if and only if all the other agents choose action x at time 1 and thatspecifies no other non-z actions by agent i. This is a calculable strategy. If all the
opponents of i play quantitative strategies, then the action profile by those opponents
at time 1 is a measurable function from (Ω, F1 ) to A−i , and so the action by agent i
at time 2 is a measurable function from (Ω, F2 ) to Ai .
The result below shows that restricting the strategy space ensures measurable behavior. If each agent plays a calculable strategy, then the stochastic process encoding
24the actions of each agent is progressively measurable.
i
Theorem 2. If πi ∈ ΠCi for i ∈ I, then the stochastic process ξb (πi , π−i ) is progressively measurable for all b and each i ∈ I.

An iterative argument is used to establish the preceding result. Let (πi , π−i ) bea profile of calculable strategies, and let b denote the past actions of the agents. We
start by constructing a progressively measurable stochastic process that is the sameas ξbi (πi , π−i ) up to and including the time of the first non-z action by some agent.
We then do the same for the second non-z action, the third non-z action, and so on.27
Intuitively, if no agent has chosen a non-z action yet, then it is as if each agent’s
opponents are following the strategy of always choosing z, which belongs to the setCΠQ−i . Hence, if an agent is using a strategy in Πi , then its action process will satisfy
the requirements for progressive measurability up to and including the time when anon-z action is chosen.28
The final result justifies our restriction on strategies. In order to guarantee themeasurability of the action process, we seek to eliminate strategies that generatenonmeasurable behavior. There is no obvious reason to delete a quantitative strategy
because it induces measurable actions virtually regardless of the strategies played byC
the other agents. Given that any strategy in ΠQ
i is permitted, the set Πi of calculable
strategies is the largest strategy space for each agent that ensures the measurabilityof actions.
TF
Proposition 4. For each i ∈ I, let Ψi be any strategy space with ΠQi ⊆ Ψi ⊆ Πi .
Suppose that the stochastic process ξbi (π) is progressively measurable for all b, any
π ∈ ×j∈I Ψj , and each i ∈ I. Then Ψi ⊆ ΠCi for i ∈ I.

The logic is simple. Suppose that agent i uses some non-calculable strategy πi0 .
0
By definition, there exists a quantitative strategy π−isuch that the actions of agent
27

The second part of theorem 1 implies that, with probability one, the time of the k th non-z
action is well defined for every positive integer k.
28
The set of calculable strategies cannot be defined using wording similar to the traceability andfrictionality assumptions. In particular, suppose that agent i is simply required to play a strategy
that for any action path of the opponents, induces a progressively measurable action process foragent i. If the opponents play quantitative strategies, then agent i’s action process may not be
progressively measurable because, for example, the time of a non-z action by agent i may dependon the opponents choosing non-z actions at some time in a nonmeasurable set.

25

0
i may not be progressively measurable when i’s opponents play π−i. Hence, measurable behavior is not ensured if agent i can use some non-calculable strategy and theopponents can play any quantitative strategies.
The remainder of the paper considers for each i ∈ I primarily strategies in ΠC
i .

4.3

Expected Payoffs

This section defines expected payoffs and equilibrium concepts. For each i ∈ I,
let vi : (×j∈I Aj ) × S → R be a measurable utility function. Choose any historyh = {sτ , (ajτ )j∈I }τ ∈[0,T ) such that {ajτ }τ ∈[0,T ) ∈ Ξj (t) for each j ∈ I. The realized
payoff to agent i at time t is given by:
Vti (h) =Xvi [(ajτ )j∈I , sτ ],τ ∈Mt (h)where the set Mt (h) = {τ ∈ [t, T ) : ∃ j s.t. ajτ 6= z} represents the set of times fromt onwards at which some agent moves under the given history.29
That is, the payoff is the sum of discrete utilities from the times at which atleast one agent chooses an action other than z.30 It follows from the second part oftheorem 1 that, with probability one, the number of non-z actions chosen by eachagent is countable. Note also that discounting is not explicitly modeled. However, if
one wishes, it can be modelled by including the current time as part of the shock.31
For example, there may exist wi such that vi [(ajτ )j∈I , sτ ] = e−ρτ wi [(ajτ )j∈I ]. We also
define:Vti,p (h) =X

max{0, vi [(ajτ )j∈I , sτ ]} and Vti,n (h) =τ ∈Mt (h)Xmin{0, vi [(ajτ )j∈I , sτ ]},τ ∈Mt (h)29

The model can be extended to allow the utility function vi to depend on the time t at which
the payoff is calculated. Such an extension, which would allow for time-inconsistent behavior, would
leave the analysis in this section essentially unchanged.
30
The specification of the realized payoff reflects the normalization that for each element of S, thediscrete utility from the action profile in which all agents choose z is 0. The model could be extended
to allow the action profile in which all agents choose z to generate a flow utility for each agent that
depends arbitrarily on the shock process. Such an extension would not affect the equilibria of themodel in calculable strategies as long as the value of this flow utility does not depend on the pastactions of the agents.
31
In the applications in section 5 and the online appendix, the shock is formally defined to include
calendar time as one of its components.

26

whence the realized payoff can be expressed as Vti (h) = Vti,p (h) + Vti,n (h).
Under the preceding specification, no agent experiences a flow payoff. However,some of our examples in the online appendix illustrate how a game can be straightforwardly reformulated so that agents experience a stream of flow payoffs. We alsointerpret the different formulations, explaining how they emphasize different featuresof the economic setting.
Let Π̄Ti F = ΠTi F ∩ Π̄i denote the set of strategies for each agent i ∈ I that arefeasible as well as traceable and frictional. The space of sample paths of {st }t∈[0,u)is equipped with an arbitrary sigma-algebra A. We impose the following one-sided
boundedness condition.32 For any history ku = {gt }t∈[0,u] , {(bjt )j∈I }t∈[0,u) up to
time u and any strategy profile π ∈ ×j∈I Π̄Tj F such that the process ξbi (π) with b ={(bjt )j∈I }t∈[0,u) is progressively measurable for all i ∈ I, we have either Vip (ku , π) < ∞or Vin (ku , π) > −∞ for every i ∈ I, where we define for o ∈ {n, p}:
Vio (ku , π) =E{st }t∈(u,T ) Vui,o st , [φjt (ku , {sτ }τ ∈(u,T ) , π)]j∈I t∈[0,T ) |{st }t∈[0,u] = {gt }t∈[0,u] .
The conditional expectation is taken with respect to {st }t∈(u,T ) given that {st }t∈[0,u] ={gt }t∈[0,u] .33
The expected payoff to agent i at ku is specified as:
Vi (ku , π) = Vip (ku , π) + Vin (ku , π).

(1)CLet Π̄Ci = Πi ∩ Π̄i denote the set of strategies for each agent i ∈ I that are feasible aswell as calculable. For any π ∈ ×j∈I Π̄Cj , the expected payoff is well defined because
theorem 1 implies that the realized payoff can be uniquely computed with conditional
probability one, and theorem 2 along with the one-sided boundedness property ensuresthe existence of the conditional expectation.
A potential issue is the nonuniqueness of the conditional expectation, which is afunction from the sample space Ω to the set of sample paths of {st }t∈[0,u) . Math32

All of the applications considered in section 5 and the online appendix satisfy this property.
The following property is sufficient for the one-sided boundedness condition to hold. For
any u, letting H̄ denote the set consisting of every history h = {st , (ait )i∈I }t∈[0,T ) such that{ait }t∈[0,T ) ∈ Ξi (u) for each i ∈ I and such that ait ∈ Āi {sτ }τ ∈[0,t] , {(ajτ )j∈I }τ ∈[0,t) for allPt ≥ [u, T ) and each i ∈ I, we have either suph∈H̄ t∈Mu (h) max{0, vi [(ajt )j∈I , st ]} < ∞ orP
inf h∈H̄ t∈Mu (h) min{0, vi [(ajt )j∈I , st ]} > −∞ for every i ∈ I.
33

27

ematically, any two conditional expectation functions that differ on a set of probability measure zero may be regarded as the same random variable. This ambiguity
is resolved for each strategy profile π ∈ ×j∈I Π̄C
j and every time u by choosing any
conditional expectation function and holding this choice fixed. The expected payoff
Vi (ku , π) can thereby be specified uniquely for each π and every ku . This enables us
to use a sure version of optimality instead of an almost sure notion when defining anequilibrium.
Having specified the expected payoffs, we can now define subgame-perfect equilibrium (SPE). Formally, a strategy profile π with πj ∈ Π̄C
j for j ∈ I is a subgameperfect equilibrium if for any history ku = {gt }t∈[0,u] , {(bjt )j∈I }t∈[0,u) up to timeu, the expected payoff to agent i ∈ I at ku satisfies Vi (ku , π) ≥ Vi [ku , (πi0 , π−i )] for any
34
πi0 ∈ Π̄C
Although proving the existence of an SPE is difficult in general becausei .
of the complexity of the strategy space that we consider, we show that an SPE exists
in all of the applications that we analyze in section 5 and the online appendix.35
5Applications

The four examples in this section and the five additional ones in the online appendix
illustrate the applicability of our methodology to a broad range of settings. The existence of a unique solution to each model facilitates the derivation of comparative
statics, which demonstrates that our methodology enables substantive economic analysis. Although other methods could be used to analyze each specific application, ourpoint is that our method simultaneously covers all of the examples that we present,
hence providing a unified approach to a variety of stochastic continuous-time games.
Moreover, even though the equilibria we study often involve Markov strategies, it is
not sufficient, as explained in section 3.1, to restrict attention to Markov strategies
for strategy spaces to be well defined. These applications also illustrate how to adapt
our framework to accommodate particular situations that are seemingly beyond thescope of its assumptions.36
34

The online appendix provides an alternative but equivalent definition that simplifies checking
whether a strategy profile is an SPE given the restrictions on the strategy space. This definition isuseful in applications because it provides a simpler alternative to determining whether a potential
deviating strategy is traceable, frictional, and calculable.
35
This issue is not specific to our approach: for example, Simon and Stinchcombe (1989) andBergin and MacLeod (1993) do not demonstrate the existence of equilibria in their general model.
36
For example, the action A in section 5.3 may be interpreted as a way of patching the model toallow the agents to move sequentially at the same time. In the online appendix, we also consider an28More specifically, in each application, we compare our restrictions with existing
conditions that have been developed in the literature for a deterministic environment.
We explain this comparison in detail for the first two examples, in which existing
conditions are most problematic to apply. The uniform version of the inertia condition
from Bergin and MacLeod (1993) is not satisfied in any application, and pathwise
inertia is violated in section 5.2. As for the admissibility restriction of Simon andStinchcombe (1989), uniform F1 does not hold except in sections 5.3 and 5.4 andin one example in the online appendix. Moreover, pathwise F1 is violated by theexample in section 5.2 and also by one of the applications in the online appendix.
While any traceable and frictional strategies have property F2, property F3 fails inevery application except those in sections 5.3 and 5.4.
It is straightforward to confirm that the equilibrium strategies we study satisfy
the calculability restriction in each application (verifying that they are traceable andfrictional is simple as well). Doing so basically involves checking that each agent’s
behavior up to and including the time of the next move is progressively measurable
given that the behavior of all the agents up to the current time is progressively
measurable and the other agents are using quantitative strategies.37 In addition, it ispossible to demonstrate that a profile of calculable strategies is an SPE without having
to determine whether potential deviating strategies satisfy all of the requirements forcalculability (see also footnote 34).
While the applications we present have not been previously studied due to thelack of suitable techniques, the examples in sections 5.1 and 5.3 as well as those in
the online appendix where the shock follows a diffusion process can be regarded as
multiplayer extensions of the single-agent model of investment under uncertainty in
McDonald and Siegel (1986).
5.1

Tree Harvesting Problem

There are n ≥ 2 woodcutters harvesting trees in a common forest. The volume offorest resources evolves stochastically over time. It can increase due to natural growth,
example that involves flow payoffs and explain how it can be formulated within our framework.
37
For example, consider an equilibrium in grim-trigger strategies like in section 5.1 or two of theapplications in the online appendix. Suppose that agent i plays its equilibrium strategy and theother agents play arbitrary quantitative strategies. Then the behavior of agent i will be progressively measurable up to and including the first time that one of the other agents deviates from its
equilibrium strategy, after which the behavior of agent i simply involves not moving. Since thisbehavior is progressively measurable, it follows that the equilibrium strategy of agent i is calculable.

29which depends on the weather. It can also decrease because of damage to trees by windor fire. The forest is large, making it reasonable to assume that the volume follows
a continuous process. Specifically, we assume that it follows an arithmetic Brownian
motion with a lower bound of zero. Formally, let bt with b0 = 0 be a Brownian motion
having arbitrary drift µ and positive volatility σ: dbt = µdt + σdzt .38 The volume
qt of the forest at time t is given by the greater of bt − p and 0, where p is the total
amount cut in the past. The decision to harvest trees is described in what follows.
At every moment of time t ∈ [0, ∞), each woodcutter i decides whether to cut
trees and, if so, a total amount of wood from the forest to harvest eit ∈ (0, qt ] as well as
an amount of wood from the harvest to claim for itself fti > 0. The woodcutters needhelp from each other in order to cut trees, so that the forest is cut if and only if all thewoodcutters choose to harvest the same positive amount of wood. Suppose that attime t, every woodcutter i chooses to harvest the total amount of wood dt > 0. ThenP
the utility of each woodcutter i at that time is (fti / j ftj )(dt − κ), which represents
the difference between the benefit and cost of cutting trees. The amount of wood fromthe harvest dt and the total cost of harvesting trees κ > 0 are rationed proportionallyamong the woodcutters according to the amount each of them claims.39 If the forest
is not cut at time t (which happens when not all the woodcutters choose to cut thesame positive amount of trees), then each woodcutter receives the payoff 0 at thattime. The woodcutters discount the future at rate ρ > 0.
We consider traceable, frictional, calculable, and feasible strategies, denoted Π̄Ci40
for each woodcutter i. Call this game with such strategy spaces the tree harvesting
game. It is characterized by (n, µ, σ, κ, ρ). The analysis in sections 3 and 4 implies
that an SPE is well defined. A symmetric SPE is said to be maximal if there is nosymmetric SPE that yields a higher expected payoff to each agent.
Letx∗ = [1 + ακ + W (−e−1−ακ )]/α and x̄ = ln[n/(n − 1)]/α,38

In order to simplify the exposition throughout the paper, we consider the canonical versionof Brownian motion, in which every sample path is continuous, thereby reducing the need to referto zero probability events. For example, if it were possible for a sample path of the shock process
to be discontinuous in the application here, then a maximal equilibrium in which the path of playis always as specified in the statement of proposition 5 might violate pathwise inertia, unless thedefinition of this condition were extended as described in footnote 21.
39
The analysis would not change qualitatively if each woodcutter who harvests trees were instead
required to bear the full cost κ of cutting trees.
40
Formal definitions of histories and strategy spaces are provided in the online appendix.

30

pwhere α = (−µ + µ2 + 2σ 2 ρ)/σ 2 , and W is the principal branch of the Lambert W
function.41 Assume that κ < x̄, and define x̂ = min{x∗ , x̄}.
Proposition 5. The tree harvesting game has a maximal equilibrium for any (n, µ, σ, κ,ρ). Moreover, the path of play in any maximal equilibrium is such that, with probability one, the mth cutting of trees occurs at the mth time the volume reaches x̂ forevery positive integer m, where the trees are cut to volume 0 on each cutting.
In a maximal SPE, the continuation payoff from deviation is zero, which is theminmax payoff of each agent.42 To understand the intuition for why it is optimal
to cut the trees to zero, consider a strategy profile in which the trees are cut to thevolume y > 0 whenever they reach a volume x > y on the equilibrium path. This
policy would be dominated by one in which the trees are cut to zero when they firstreach a volume x and are thereafter cut to zero whenever they reach a volume ofx − y.
Remark 3.
1. (Relationship to inertia) In a maximal SPE, each agent’s strategy
violates uniform inertia but satisfies pathwise inertia. Consider any history up
to an arbitrary time t in which no agent has deviated in the past. For any  > 0,
there is positive conditional probability that qτ = x̂ for some τ ∈ (t, t + ), in
which case the agents are required to cut trees. Thus, there does not exist
 > 0 such that the agents do not move during the time interval (t, t + ),
meaning that uniform inertia is violated.43 However, pathwise inertia holds
given the continuity of the Brownian path. For any realization of the shockprocess {bτ }τ ∈(t,∞) after time t, there exists  > 0 such that qτ 6= x̂ for allτ ∈ (t, t + ), in which case the agents do not move during the time interval(t, t + ).
That is, W (−e−1−ακ ) is the larger of the two values of y satisfying −e−1−ακ = yey .
There are multiple possibilities for off-path strategies, as described in the proof of proposition
5 in the online appendix.
43
Bergin and MacLeod (1993) also expand the strategy space to the completion of the set ofinertia strategies, but this extension turns out not to apply for the same reason as why the uniform
version of inertia does not. As shown in the online appendix of Kamada and Rao (2018) in thecontext of their model, the outcomes of some strategy profiles of interest satisfying our restrictions
cannot be expressed as the limit of the outcomes generated by a Cauchy sequence of strategy profiles
satisfying the uniform version of inertia, even when the approach in Bergin and MacLeod (1993) is
extended to the stochastic case. We are currently working on a paper about the problem of generally
defining the completion of the set of inertia strategies.
41

42

31

2. (Relationship to admissibility) In any maximal SPE, uniform F1 is violated
because there is no upper bound on the number of times that trees may be
harvested in any proper time interval. However, pathwise F1 would hold because
for any realization of the shock process {bτ }τ ∈[0,t] up to time t, the agents are
required to harvest trees only finitely many times during the time interval [0, t].
According to proposition 3, any traceable and frictional strategy has property
F2. Nonetheless, property F3 in Simon and Stinchcombe (1989), which in somesense requires behavior to depend continuously on the history, is violated by amaximal equilibrium, in which agents use grim-trigger strategies where even asmall deviation incurs a large punishment.
3. (-optimal strategies) In the current example, the complexities of defining strategy spaces in continuous time could be mitigated by focusing on a class of optimal strategies that is relatively well behaved. For example, the strategiesof the agents might be discretized so that agents can move if and only if thecurrent time is a multiple of η > 0, where η can be made arbitrarily small so
as to approximate the expected payoff to each player in a maximal SPE. Such
strategies would satisfy both uniform and pathwise inertia. Nevertheless, thisapproach is subject to a problem of circularity. In order to determine whether astrategy is -optimal, the supremum expected payoff must be computed amonga more general class of strategies such as those satisfying traceability, frictionality, and calculability. However, doing so requires defining expected payoffs
for such a class of strategies in the first place. This circularity problem is not
specific to this example but may arise in general when trying to restrict attention to a set of -optimal strategies without first having been able to computethe expected payoffs given a larger space of strategies. We will not repeat thesame point in the subsequent examples where we solve for an optimal strategy
profile.
4. (Comparative statics) The cutoff x̂ is the lesser of x∗ , which represents thevolume at which it is socially efficient to harvest wood, and x̄, which represents
the highest volume at which it is incentive compatible to do so.44
The threshold x∗ is increasing in µ, σ, and κ and decreasing in ρ. Intuitively,a higher value of µ or σ reduces the time required for the forest to grow by a
44

Proofs of the comparative statics results that follow are provided in the online appendix.

32

given amount, and a lower value of ρ makes it less costly to postpone the gains
from harvesting wood, which makes it optimal to wait for the forest to grow to alarger volume before cutting it. A higher value of κ means that harvesting wood
is more costly, so the associated benefits must be larger for it to be optimal to do
so. In addition, the payoff maximizing behavior as represented by x∗ does notdepend on n because the expected payoff to each agent from any given profile
of symmetric strategies is inversely proportional to the number of agents.
The threshold x̄ is increasing in µ and σ and decreasing in ρ and n. Intuitively,a higher value of µ or σ or a lower value of ρ raises the present value of the
future benefits from cooperation by the agents in harvesting wood, and when n islower, each agent receives a bigger share of these benefits. This makes it possible
to enforce cooperation among the agents while harvesting a larger amount ofwood on each cutting. In addition, the incentive compatibility constraint as
represented by x̄ does not depend on κ because both the expected payoff toeach agent in a maximal equilibrium and the supremum of the expected payoffs
from a deviation are proportional to x̂ − κ.
5.2

Petroleum Supply Chain

Another application of our restrictions is search models, which involve the timing of
exchanges. Opportunities to transact occur randomly over time, and an agent may
delay trade in pursuit of a better opportunity. In the current example, one agent
reacts to the behavior of another agent that is engaged in a search problem. The
equilibrium strategy of the former agent in responding to the latter agent violates
both pathwise inertia and pathwise admissibility (and hence it also violates uniform
inertia and uniform admissibility).
There are two agents: an oil well W and an oil refinery R. Let pt denote the priceof oil at time t. The price of oil changes at the jump times of a Poisson process withparameter λ > 0. The resulting price is independently and identically distributed
according to the cumulative distribution function G on the positive real line and hasa finite expectation. Both agents discount the future at the rate ρ > 0.
The well contains a known quantity q of oil. Let xt denote the amount of oil
withdrawn before time t. At each moment in time t ∈ [0, ∞), the well decides whether
to extract oil and, if so, an amount et ∈ (0, q − xt ] to remove. Let c(xt ) represent

33the marginal cost of extracting oil when xt units have already been withdrawn, wherec : [0, q] → R+ is an increasing function. Correspondingly, the cost to the oil wellat time t of extracting the quantity et when xt is the amount that has already been
R x +eextracted is given by xtt t c(ξ)dξ.
Any oil extracted at time t is immediately transferred to the refinery, and therefinery pays the well the amount pt et at time t. Upon receiving the input at time t, therefinery must process it, which takes an amount of time d(et ), where the measurable
function d : (0, q] → R++ satisfies lime↓0 d(e) = 0. Let f (et ) ≥ 0 represent theassociated processing costs discounted to time t. Having finished processing the oil
received at time t, the refinery decides at each time τ ≥ t + d(et ) whether or not todeliver the output to a customer, if it has not done so already.
The refinery can be compensated in a variety of ways. In general, if the material
that it started processing at time t is delivered to the customer at time t0 ≥ t + d(et ),then the refinery is paid the amount y(et , pt ) > exp[ρd(et )][pt et +f (et )] by the customer
at time t0 , where y : R2++ → R++ is a measurable function. In the case where d isbounded, for example, the payment to the refinery may have the cost-plus form
¯ where d¯ = supỹ · [pt et + f (et )] with ỹ > exp(ρd),
e∈(0,q] d(e). If in addition G has abounded support and f (e)/e is bounded for e ∈ (0, q], then the refinery may be paid
¯ + f¯), so that it receives ŷ · et , where p̄ is thea constant unit price ŷ > exp(ρd)(p̄supremum of the support of G, and f¯ = supe∈(0,q] f (e)/e.
We consider traceable, frictional, calculable, and feasible strategies, denoted Π̄CW,for the oil well and traceable, weakly frictional, calculable, and feasible strategies,
45
Call this game with such strategy spaces thedenoted Π̂CR , for the oil refinery.
supply chain model. It is characterized by (G, c, d, y, q, λ, ρ). The analysis in sections
3 and 4 implies that an SPE is well defined.
The equilibrium of the model is presented in the result below. Let ς : R → R be
45

Formal definitions of histories and strategies are provided in the online appendix. In the formal
specification of the model, W ’s action at a given time is defined as a pair representing the respective
amounts of oil extracted at the current time and in the past, and R’s action at a given time is defined
as a function representing the payments it obtains from delivering the output produced from the oil
received at each time. This way of defining action spaces enables the instantaneous utility of eachagent to be expressed solely as a function of the actions of the agents at the current time, so thatthe expected payoffs in this application can be calculated using the general approach in section 4.3.

34

the function that solves the following reservation price equation for all κ ∈ R:
λς(κ) = κ +ρZ ∞p − ς(κ)dG(p).

(2)

ς(κ)

It is straightforward to show that there exists a unique value of ς(κ) satisfying theabove equation for each value of κ and that ς is continuous, increasing, and surjective.

Denoting the inverse of ς by ς −1 , let ζt denote the supremum of the set c−1 [0, ς −1 (pt )] ,
where c−1 [0, ς −1 (pt )] is the preimage of the interval [0, ς −1 (pt )] under c. Let {θt,k }lk=1
be a finite sequence of times at which oil was extracted from the well before time t,
and for each index k, let ξt,k be an indicator that is equal to 1 if the refinery has
by time t processed the oil received at time θt,k but has not before time t delivered
the output from the oil received at time θt,k and that is equal to 0 otherwise. If therefinery has received infinitely many batches of oil by time t, then assume that it isinfeasible for the refinery to continue operating from time t onwards.
Proposition 6. In the supply chain model with (G, c, d, y, q, λ, ρ), the following holdin any SPE:
1. The strategy of W is such that:(a) If xt < ζt , then W extracts the amount et = ζt − xt at time t.
(b) If xt ≥ ζt , then W does not extract oil at time t.
2. The strategy of R is such that for any k satisfying ξt,k = 1, R delivers the
product of the oil extracted at time θt,k to the customer at time t.
The well faces a search problem in deciding when to sell each unit of oil, and its
equilibrium strategy has the reservation price property. In particular, each unit ofoil is sold once the price reaches or exceeds a certain threshold, which is increasingin the marginal cost of extracting that unit of oil. The best response of the refinery
is to deliver each batch of output to the customer as soon as the relevant input has
been processed.
Remark 4.
1. (Relationship to inertia) R’s equilibrium strategy violates both theuniform and pathwise versions of inertia. In particular, consider any history up
to an arbitrary time t and any  > 0. Since lime↓0 d(e) = 0, there exists χ > 035

such that d(χ) < /2. If W extracts the amount χ at any time u ∈ (t, t + /2),
then R delivers the resulting output to the customer at time u + d(χ) < t + .
Thus, there is no  > 0 such that R does not move in the interval (t, t + ).
W ’s equilibrium strategy does not in general satisfy uniform inertia. For example, suppose that G has full support on R+ and that c(0) > 0. Consider
any history up to an arbitrary time t such that pt < ς[c(0)] and no oil has
been extracted yet. For any  > 0, there is positive conditional probability
that pτ > ς[c(q)] for some τ ∈ (t, t + ), in which case W extracts oil. Thus,
there does not exist  > 0 such that W does not move during the time interval(t, t+). However, W has an equilibrium strategy that satisfies pathwise inertia.
For any price path {pτ }τ ∈(t,∞) after time t, there exists  > 0 such that pτ = pt
for all τ ∈ (t, t + ), in which case W is not required to extract oil during thetime interval (t, t + ).
2. (Relationship to admissibility) R’s equilibrium strategy violates both uniform
and pathwise F1. Choose any integer m ≥ 0 and any t > 0. Since lime↓0 d(e) =0, there exists χ > 0 such that d(χ) < t/(m + 1). If W extracts the amountχ at any time τ for which there exists an integer k ≥ 0 such that τ = kd(χ),
then R makes more than m deliveries during the time interval [0, t]. Hence,
there is no upper bound on the number of moves by R up to a given positive
time. Moreover, W ’s equilibrium strategy does not generally satisfy uniform
F1. For example, assume that G has full support on R+ and that c is positive,
continuous, and increasing. Suppose that p0 < ς[c(0)], and choose any integerm ≥ 1 and any t > 0. There is positive probability that the price path isnondecreasing, has m discontinuities during the interval [0, t], and is such thatfor every positive integer k ≤ m, there exists τk ∈ [0, t] satisfying ς{c[(k −1)q/m]} < pτk < ς[c(kq/m)], in which case W extracts oil m times during thetime interval [0, t]. Hence, the number of moves by W in a finite time interval is
not bounded. However, W has an equilibrium strategy that satisfies pathwise
F1. The number of times that W needs to extract oil during any finite timeinterval [0, t] is bounded above by one plus the number of discontinuities up to
time t in the price path {pτ }τ ∈[0,∞) , and the number of discontinuities is finite.
F2 is satisfied by any traceable and weakly frictional strategy.
R’s equilibrium strategy violates the strong continuity property F3 because a36

small difference in when W extracts a given amount of oil results in some difference in when R delivers the resulting output. W has an equilibrium strategy
that satisfies F3. In particular, W ’s behavior depends only on the amount ofoil remaining and the current price of oil, so the specific times when oil waspreviously extracted or processed do not affect W ’s behavior.
3. (Role of weak frictionality) While W ’s equilibrium strategy is frictional, R’s isnot. For example, suppose that for every positive integer k, W extracts theamount q/2k at time 1/k. Then since lime↓0 d(e) = 0, R moves infinitely manytimes in the finite time interval [0, ) for any  > 0. However, R’s equilibrium
strategy is weakly frictional. This is because the number of times that R moves
up to a given time t is bounded above by the number of times that W moves up
to that time. Hence, if W moves only finitely many times in any finite intervalof time, then the same is true of R.
4. (Comparative statics) It is straightforward to show from equation (2) that thereservation price ς[c(xt )] of a unit of oil with marginal extraction cost c(xt ) is
nondecreasing in the rate parameter λ and nonincreasing in the discount rate
ρ. If ς[c(xt )] is less than the supremum of the support of G, then ς[c(xt )] is
strictly increasing in λ and strictly decreasing in ρ. Intuitively, a higher valueof λ makes the price more volatile, raising the option value of waiting for the
price to rise, and a higher value of ρ means that an agent is more impatient andless willing to delay sale until the price rises.
5.3

Entry Game between Competing Firms

Our restrictions on strategy spaces are useful for analyzing investment games, inwhich agents optimally time their decisions based on a shock process that may bemodeled by a diffusion process such as geometric Brownian motion. In the following
example, investment is modelled as entry into a market, which is a common setting
in industrial organization. Uniform inertia is violated, although this condition holds
when defined pathwise.
Two firms, 1 and 2, are deciding whether to enter a market. Assume that both
firms are initially out of the market. At each moment of time t ∈ [0, ∞) such thatno firm has entered, each firm can choose between two moves: I (“get in”) and A(“accommodate”). Choosing I means that the firm enters the market. When a firm
37

chooses A at time t, it enters the market if and only if the other firm does not choose
I at the same time t. Once a firm has entered, it cannot move any longer. At any
time such that one firm has entered but not the other, then the latter firm chooses F(“follow the other firm by entering”) or not. In any event, not moving (i.e., choosing
action z) means that the firm does not enter for now or has already entered.
Let ct be the discrete cost incurred by a firm when it enters the market at timet. The entry cost evolves according to a geometric Brownian motion: dct = µct dt +σct dzt , with initial condition c0 = c̃ for some c̃ ∈ R++ . Let b1 be the discrete benefit
to a firm from entering the market at time t if it is the only firm to enter the market
up to and including time t. Let b2 be the discrete benefit from entering at time t if
both firms enter the market at or before time t. Assume that b1 > b2 > 0 and c0 > b1 .
Agents discount the future at rate ρ > 0.
We consider traceable, frictional, calculable, and feasible strategies, denoted Π̄Cifor each agent i = 1, 2.46 Call the game with such strategy spaces the entry game. It
is characterized by (b1 , b2 , µ, σ, ρ). The analysis in sections 3 and 4 implies that an
SPE is well defined.
Below we characterize Markov perfect equilibria, which we define as SPE in
Markov strategies.47 A strategy is said to be Markov if the action prescribed at
any history up to a given time depends only on the current value of the cost andwhether or not each firm has already entered.
Proposition 7. In the entry game with (b1 , b2 , µ, σ, ρ), there exist κ1 and κ2 with
0 < κ2 < κ1 < ∞ such that in any Markov perfect equilibrium, the following hold at
any time t.
1. Suppose that no firm has entered yet.
(a) If κ1 < ct , then both firms choose z.
(b) If κ2 < ct ≤ κ1 , then one firm chooses I, and the other firm chooses A.
(c) If ct ≤ κ2 , then both firms choose I, or both firms choose A.
2. Suppose that firm i has already entered but −i has not.
46

Formal definitions of histories and strategy spaces are provided in the online appendix.
There exist non-Markov SPE in which the identity of the first entrant depends on the pathof the cost process in an arbitrary manner. Here we rule out such complications by assuming theMarkov property to focus on key issues.
47

38

(a) If κ2 < ct , then both firms choose z.
(b) If ct ≤ κ2 , then i chooses z while −i chooses F .
That is, on the path of play of any Markov perfect equilibrium, one firm enterswhen the cost reaches κ1 for the first time, and the other firm enters when the costreaches κ2 for the first time. The cutoffs κ1 and κ2 are such that each firm is indifferent
between being the first and second entrant (i.e., taking action I and A when the cost
ct reaches κ1 for the first time). This is because if being the first entrant is better,then the second entrant would deviate by choosing I or A just before ct reaches κ1 forthe first time, and if being the second entrant is better, then the first entrant would
deviate by choosing z whenever ct > κ2 and choosing F at the first time ct reaches
κ2 .
Remark 5.
1. (Relationship to inertia and admissibility) Uniform inertia is violated in any Markov perfect equilibrium, but pathwise inertia is satisfied. The
example satisfies all of the requirements for admissibility, regardless of whether
F1 is defined pathwise or uniformly. The details are discussed in the online
appendix.
2. (Comparative statics) For each j = 1, 2, κj is increasing in the benefit bj atentry because the j th entry is more attractive if bj is higher. The cutoff κ1 isdecreasing in b2 because the expected payoff of the second entrant is greater
when the value of b2 is higher, so that the pressure to be the first entrant islower and the first entrant can wait for the cost to fall. The cutoff κ2 clearly
does not depend on b1 . Both cutoffs are increasing in µ and ρ because the future
gains from waiting for the entry cost to fall are lower when these parameters arehigher. The cutoffs are decreasing in σ since the effect of σ on the probabilityof the cost falling is opposite to that of µ.48
5.4Model of Ordering with a Deadline

Our methodology applies to environments like, for example, standard discrete-time
repeated games and revision games as formalized by Kamada and Kandori (2020), in
which each agent is restricted to change actions at discrete times or Poisson opportunities and the payoff to each agent is determined by the actions and shock levels
48

Proofs of these comparative statics results are provided in the online appendix.

39

at these moving times. The following is an example in which one agent can take a
non-z action (i.e., has an opportunity to move) only at the arrival times of a Poissonprocess. Requiring each agent to move only at discrete state changes (i.e., Poissonhits) may perhaps suffice for the purpose of restricting strategy spaces so as to ensure
a well defined outcome. Nonetheless, in the example below, there is another agent
who can move at any time in an interval of the real line, and restricting that agent tomove only at Poisson arrival times is problematic due to the nonstationarity of themodel.
Consider a buyer B who is contemplating the timing of an order to a seller S,
who faces a predetermined deadline for providing a good. The game is played incontinuous time, where time is denoted by t ∈ [0, T ) with T > 0. The buyer B’s taste
xt evolves according to a Brownian motion with zero drift and positive volatility σ:
dxt = σdzt , with initial condition x0 = x̃ for some x̃ ∈ R. The deadline may, forexample, correspond to Christmas day, and the good may be a Christmas present.
B’s taste may vary over time because, for example, the gift is intended for child who
has fickle preferences.
At each moment in time, B observes her taste, and if she has not yet ordered,chooses whether or not to place an order indicating her preferred specification of
the good. Once an order is placed, B is unable to revise it. After learning thespecification, S has stochastic chances arriving according to a Poisson process withparameter λ > 0 to produce and supply the ordered good to B. Specifically, at each
Poisson arrival time, S chooses whether to supply the good or not.
If B places an order and the order is fulfilled, then her utility is v − (s − xT )2 − p,where v > 0 is her valuation for a good whose specification perfectly matches her
taste, s is the specification of the good actually purchased, and p ∈ (0, v) is the fixedprice that B pays. Otherwise, her utility is 0. The seller S’s payoff is p if he fulfills
an order and 0 otherwise.49
We consider traceable, frictional, calculable, and feasible strategies, denoted Π̄Cifor each agent i = B, S.50 Call the game with such strategy spaces the finite-horizonordering game. It is characterized by (λ, T, σ, p, v). The analysis in sections 3 and 449
It is assumed without loss of generality that S’s cost of producing and supplying the good iszero. The equilibrium strategies would not change if S were to face a constant cost strictly less thanthe price.
50
Formal definitions of histories and strategy spaces are provided in the online appendix.

40

implies that an SPE is well defined.51
Proposition 8. In the finite-horizon ordering game with (λ, T, σ, p, v), there exists a
unique SPE. In this SPE, there exists t∗ ∈ [0, T ) such that B places an order at time
t if and only if she has not done so yet and t ≥ t∗ , and S sells the good at time t
if and only if he has not done so yet, an order has already been placed, and there is
Poisson hit at time t.
The result implies that on the equilibrium path of the unique SPE, B places anorder at time t∗ and S sells the good at the first Poisson hit after time t∗ .52
Remark 6.
1. (Relationship to inertia and admissibility) B’s equilibrium strategy
satisfies both uniform and pathwise inertia, and S’s equilibrium strategy ispathwise inertial but not uniformly inertial. The example satisfies all of therequirements for admissibility, regardless of whether F1 is defined pathwise or
uniformly. The details are discussed in the online appendix.
2. (Non-z action at a time without a Poisson hit) B’s unique equilibrium strategy
would not satisfy a condition requiring that a non-z action be taken only at
the times of discrete changes in the shock. There is probability zero that theshock discretely changes at time t∗ , where the set of times at which the shock
discretely changes is defined as the set of Poisson arrival times.
3. (Comparative statics) The existence of a unique equilibrium facilitates thederivation of comparative statics.53 We focus on the case where t∗ > 0. First,t∗ is increasing in σ. Intuitively, as her taste becomes more volatile, B wants to
wait longer to better match the specification to her taste. Second, t∗ is decreasing in v. The reason is that as her valuation for the product becomes higher,
B increasingly wants S to have an opportunity to fulfill her order. Third, t∗ isincreasing in p since the effect of p is opposite to that of v. Finally, t∗ is increasing in λ. As opportunities to provide the product arrive more frequently to S,
51

In our general framework, the shock level, action profile, and instantaneous utility are notdefined at the terminal time T . However, our description of the finite-horizon ordering game suggeststhat B receives a payoff at the deadline that depends on action profiles before the deadline as wellas the shock level at the deadline. As detailed in the online appendix, the game here can bereinterpreted so as to conform with our general framework.
52
Note that the traceability restriction would be violated if B were to follow a strategy of placing
an order if and only if she has not done so yet and t > t∗ .
53
Proofs of these comparative statics results are provided in the online appendix.

41

B wants to wait longer so as to set the specification closer to the realization ofher taste at time T .
6

Payoff Assignment with Nonmeasurable Behavior

In section 4, we observed that the expected payoffs are not well defined when astrategy profile generates nonmeasurable behavior. There are at least two approaches
to resolving this problem. The first method is to restrict the strategy space. The
calculability assumption considered in section 4 is an example of the first approachand can be justified as being the most inclusive restriction in a certain sense. The
second method is to assign an expected payoff to nonmeasurable behavior. Thissection is devoted to an exploration of the second approach.
We begin by specifying how to assign expected payoffs to nonmeasurable behavior(section 6.1). Then we point out a few properties of this methodology that may beproblematic (section 6.2). Ultimately, the disadvantage of the second approach is
the lack of clarity about what expected payoff to assign to nonmeasurable behavior.
The choice of an expected payoff is necessarily arbitrary because the conditional
expectation may be undefined when the behavior of the agents is nonmeasurable,
and such arbitrariness may be problematic because the set of equilibria depends onthe choice of an expected payoff. Hence, there is not an obvious way to argue that
one choice of expected payoff is better than another.54 With some restrictions, weshow that any behavior on the path of play can be sustained in an SPE under aparticular assignment of expected payoffs (section 6.3). Given the absence of an ideal
assignment, the last parts of this section examine when the second approach would
provide the same solution as the first approach. We first show that any SPE under
the calculability assumption is an SPE for a certain way of assigning payoffs (section
6.4). Then we provide conditions under which the converse is also true (section 6.5).
54

For a particular class of games, some payoff assignments might be more reasonable than others.
For example, an agent may be able to secure a certain payoff by choosing a specific action. In sucha case, this payoff could be assigned to nonmeasurable behavior. More generally, nonmeasurable
behavior might be assigned the minmax payoff. However, this approach suffers from circularity
because the minmax payoff may not be defined until the expected payoffs have already been defined.

42

6.1

Formulation of Expected Payoffs

We consider the assignment of expected payoffs to strategy profiles that induce nonmeasurable behavior.55 For each agent i ∈ I, define a function χi : H × Π̄Ti F →R ∪ {−∞, ∞}.

Choose any strategy profile π ∈ ×i∈I Π̄Ti F . Let ku = {gt }t∈[0,u] , {(bit )i∈I }t∈[0,u)be any history up to time u, and denote b = {(bit )i∈I }t∈[0,u) . If the process ξbi (π) is
progressively measurable for each i ∈ I, then the expected payoff to agent i at ku isgiven by Ui (ku , π) = Vi (ku , π), where Vi (ku , π) is as specified in equation (1) in section
4.3. Otherwise, the expected payoff to agent i at ku is given by Ui (ku , π) = χi (ku , π).56
Given a strategy space ×i∈I Π̂i ⊆ ×i∈I Π̄Ti F , we say that π ∈ ×i∈I Π̂i is a subgameperfect equilibrium if for any history ku up to time u, the expected payoff to agenti ∈ I at ku satisfies Ui (ku , π) ≥ Ui [ku , (πi0 , π−i )] for any πi0 ∈ Π̂i . Since the assignmentof expected payoffs to nonmeasurable behavior is not based on an extensive form
(see footnote 56), the standard one-shot deviation principle does not hold in general.
Thus, it is crucial for the definition of SPE to consider deviations to a strategy in theentire subgame. The following example illustrates.
Example 9. (Deviation to Measurable Behavior) Let {st }t∈[0,T ) be an arbitrary
stochastic process with state space S. Assume that there exists S̃ ⊆ S along witht̃ > 0 such that {ω ∈ Ω : st̃ (ω) ∈ S̃} is not a measurable subset of the probability
space (Ω, F, P ).57 Suppose I = {1} and that Ā1 (ht ) = {x, z} for every ht ∈ H. The
utility function satisfies v1 (x, s) = 0 for all s ∈ S. Let χ1 (ht , π1 ) = −1 for all ht ∈ H
and any π1 ∈ Π̄T1 F . A strategy in which agent 1 chooses action x at time t̃ if and only
if st̃ is in S̃ is not optimal at the null history because agent 1 can deviate to a strategy
55

We assume traceability and frictionality, so that the behavior of the agents is well defined in
the sense that there exists a unique action path consistent with each strategy profile. We could
also consider the assignment of expected payoffs to nondefined behavior, but there is little difference
between assigning payoffs to nondefined and nonmeasurable behavior. In order to define equilibria,
every element in a given set of strategy profiles should be mapped to an expected payoff, andnondefined behavior like nonmeasurability would ordinarily preclude such a mapping. The results
here can be extended to allow for strategy profiles that induce zero or multiple action paths, butthese additional results are not stated so as to simplify the exposition.
56
The extensive form of the game, which assigns a payoff profile to each deterministic history,
is well defined. However, if a profile of traceable and frictional strategies induces nonmeasurable
behavior, then there is no standard method to compute the expected payoff, even though the strategy
profile is associated with a unique payoff under each realization of the shock process. In this sense,
the assignment of expected payoffs is not based on an extensive form.
57
For example, let {st }t∈[0,T ) be a standard Brownian motion, and suppose that the set consistingof every continuous function c : [0, T ) → R with c(0) = 0 and c(t̃) ∈ S̃ is not Wiener measurable.

43

that induces measurable behavior. However, there is no history up to a given time atwhich a one-shot deviation would increase the expected payoff of the agent.
In what follows, we will compare SPE under the calculability restriction with
SPE under a payoff assignment. Whenever there is ambiguity about the strategy

space or payoff assignment being considered, we identify the problem as Γ Π̂, (χi )i∈I ,where Π̂ ⊆ ×i∈I Π̄Ti F is the space of strategy profiles in consideration. For example, our analysis in the main sections corresponds to considering the game withC
Γ ×i∈I Π̄Ci , (χi )i∈I . Since Ui (ht , π) = Vi (ht , π) for all i ∈ I whenever π ∈ ×i∈I Π̄i , thespecification of (χi )i∈I is irrelevant in this case, so that we denote Γ ×i∈I Π̄Ci , (χi )i∈Iby Γ ×i∈I Π̄Ci .
6.2

Problems with Payoff Assignment

Assigning a payoff to nonmeasurable behavior may result in a model with objectionable properties. We first observe that assigning expected payoffs to nonmeasurable
behavior may lead to a non-monotonic relationship between expected and realized
payoffs.
Example 10. (Non-Monotonicity of Expected Payoffs in Realized Payoffs)
Suppose I = {1}. Let Ā1 (ht ) = {x, z} if t = 1, and let Ā1 (ht ) = {z} otherwise.
Let {st }t∈[0,T ) be an arbitrary stochastic process with state space S. Assume that
there exists S̃ ⊆ S such that {ω ∈ Ω : s1 (ω) ∈ S̃} is not a measurable subset of theprobability space (Ω, F, P ). The utility function satisfies v1 (x, s) = 1 for all s ∈ S.
Let χ1 (ht , π1 ) = −1 for all ht ∈ H and any π1 ∈ Π̄T1 F .
Consider a class of strategies, each of which is indexed by a set C ⊆ S, where π Cprescribes action z at any time t 6= 1 and action x at time t = 1 if and only if s1 is in
C. It may be natural for the expected payoffs to satisfy the following monotonicity
000
condition: U1 (h0 , π S ) ≤ U1 (h0 , π S ) if S 00 ⊆ S 0 . That is, the expected payoff ismonotonic in the realized payoffs in the sense of statewise dominance. However,U1 (h0 , π ∅ ) = 0 > −1 = U1 (h0 , π S̃ ) even though ∅ ⊆ S̃. Hence, the monotonicity
condition fails.
As shown by the example below, the specific assignment of expected payoffs tononmeasurable behavior affects the set of payoffs that can be supported in an SPE.
Example 11. (Dependence of Equilibrium Set on Payoff Assignment) Suppose I = {1, 2}. For each i ∈ {1, 2}, let Āi (ht ) = {x, z} if t = i − 1, and let
44

Āi (ht ) = {z} otherwise. Let {st }t∈[0,T ) be an arbitrary stochastic process with state
space S. Assume that there exists S̃ ⊆ S such that {ω ∈ Ω : s1 (ω) ∈ S̃} is not ameasurable subset of the probability space (Ω, F, P ). The utility functions satisfy
v1 [(x, z), s] = 1, v1 [(z, x), s] = 0, and v2 [(x, z), s] = v2 [(z, x), s] = 0 for all s ∈ S.
First, suppose that χ1 (ht , π) = −1 and χ2 (ht , π) = 0 for all ht ∈ H and any
π ∈ Π̄T1 F × Π̄T2 F . Then there exists an SPE in which agent 1 receives an expected
payoff of 0. For example, agent 2 may use a strategy of choosing action x at time 1
if and only if agent 1 chooses action x at time 0 and s1 is in S̃.
Second, suppose that χ1 (ht , π) = χ2 (ht , π) = −1 for all ht ∈ H and any π ∈TFΠ̄1 × Π̄T2 F . Then there does not exist an SPE in which agent 1 receives an expected
payoff of 0. The reason is that there is no history h1 up to time 1 for which agent
2 has an incentive to choose a strategy π2 such that U2 [h1 , (π1 , π2 )] = χ2 [h1 , (π1 , π2 )]for some strategy π1 of agent 1. Hence, it is always optimal for agent 1 to choose x
at time 0, so that agent 1 receives an expected payoff of 1.
The general problem is that when the agents’ behavior is nonmeasurable, there
is not a well defined probability distribution over future paths of play. Hence, theexpected payoff assigned by the function χi to a strategy profile involving nonmeasurable behavior does not have any natural relationship with the realized payoffs at
future times as determined by the function vi . Despite such a problem, the usual
concept of SPE is well defined.
6.3

Arbitrary Behavior in Equilibrium

Now we examine the implications for agents’ incentives of assigning payoffs to nonmeasurable behavior. Given any history ku up to time u, let H̄(ku ) denote the set con
sisting of every history h = {st , (ajt )j∈I }t∈[0,T ) such that {st }t∈[0,u] , {(ajt )j∈I }t∈[0,u) =ku , {ait }t∈[0,T ) ∈ Ξi (u) for each i ∈ I, and aiτ ∈ Āi {st }t∈[0,τ ] , {(ajt )j∈I }t∈[0,τ ) forall τ ≥ u and each i ∈ I. That is, H̄(ku ) includes every feasible history withfinitely many moves in a finite time interval for which ku is a subhistory. LetPζi (ku ) = inf h∈H̄(ku ) τ ∈Mu (h) vi [(ajτ )j∈I , sτ ] denote the greatest lower bound on thefeasible payoffs to agent i at ku .58 In this section, we consider the possibility of letting χi (ku , π) ≤ ζi (ku ) for all π ∈ ×j∈I Π̄Tj F and each i ∈ I. That is, nonmeasurable
58

Recall that Mu (h) denotes the set of times at and after u where some agent moves under historyh.

45

behavior is assigned an expected payoff no greater than the infimum of the set of
feasible payoffs.59
A motivation for this specification of payoffs is that assigning an extremely low
payoff to nonmeasurable behavior disincentivizes agents from pursuing such behavior,
thereby ensuring that the path of play is measureable, in which case the computationof expected payoffs is straightforward. In what follows, we demonstrate that such apayoff assignment may have a significant impact on the set of SPE. Intuitively, if an
extremely low payoff is supportable in an SPE, then it may be used to severely punish
deviations. In fact, we can prove a type of “folk theorem” under certain conditions. To
illustrate this point, we first describe how both agents using nonmeasurable behavior
can be supported as an SPE.
Example 12. (Mutual Nonmeasurability) Suppose I = {1, 2}. For each i ∈{1, 2}, let Āi (ht ) = {x, z} if t = 1, and let Āi (ht ) = {z} otherwise. Let {st }t∈[0,T ) be
an arbitrary stochastic process with state space S. Assume that there exists S̃ ⊆ Ssuch that {ω ∈ Ω : s1 (ω) ∈ S̃} is not a measurable subset of the probability space(Ω, F, P ). The utility function of each agent i ∈ {1, 2} satisfies vi [(a1 , a2 ), s] ≥ 0 for
all s ∈ S and any (a1 , a2 ) ∈ A1 × A2 such that (a1 , a2 ) 6= (z, z). For each i ∈ {1, 2},let χi (ht , π 0 ) = χi (ht , π 00 ) ≤ 0 for all ht ∈ H and any π 0 , π 00 ∈ Π̄T1 F × Π̄T2 F .
Let π̃ be a strategy profile in which each agent chooses x at time 1 if and only if s1
is in S̃. This strategy profile is an SPE because at any history ht up to a time t ≤ 1,
there is no unilateral deviation that would enable an agent to obtain an expected
payoff greater than χi (ht , π̃) ≤ 0.
We next show how this behavior may be used as a punishment to support otherpaths of play.
Example 13. (Folk Theorem) Suppose I = {1, 2}. For each i ∈ {1, 2}, let Āi (h0 ) =Ai , Āi (ht ) = {x, z} if t = 1, and Āi (ht ) = {z} if t ∈/ {0, 1}. Let {st }t∈[0,T ) be
an arbitrary stochastic process with state space S. Assume that there exists S̃ ⊆S such that {ω ∈ Ω : s1 (ω) ∈ S̃} is not a measurable subset of the probability
space (Ω, F, P ). The utility functions satisfy v1 [(a1 , a2 ), s] ≥ v1 [(z, a2 ), s] ≥ 0 and
59

Similar results hold under an alternative definition in which ζi (ku ) = inf π∈Π̄Tb F Vi (ku , π) at any
history ku = {gt }t∈[0,u] , {(bjt )j∈I }t∈[0,u) up to time u, where Π̄Tb F denotes the set consisting of any
strategy profile π ∈ ×j∈I Π̄Tj F such that the process ξbi (π) with b = {(bjt )j∈I }t∈[0,u) is progressively
measurable for all i ∈ I.

46

v2 [(a1 , a2 ), s] ≥ v2 [(a1 , z), s] ≥ 0 for all s ∈ S and any (a1 , a2 ) ∈ A1 × A2 such that
(a1 , a2 ) 6= (z, z). For each i ∈ {1, 2}, let χi (ht , π 0 ) = χi (ht , π 00 ) ≤ 0 for all ht ∈ H andany π 0 , π 00 ∈ Π̄T1 F × Π̄T2 F .
Choose any pair of actions (ã1 , ã2 ) ∈ A1 ×A2 . Let π̃ be the strategy profile defined
as follows. Each agent i ∈ {1, 2} chooses ãi at time 0. If every agent i takes action ãi
at time 0, then the agents choose x at time 1. If some agent i does not take action ãi
at time 0, then the agents choose x at time 1 if and only if s1 is in S̃. This strategy
profile is an SPE. A unilateral deviation at h0 would result in an expected payoff ofχi (h0 , π̃) ≤ 0 to agent i. If strategy profile π̃ is followed at time 0, then playing x attime 1 is a best response for each agent to the action of the other agent. If strategy
profile π̃ is not followed at time 0, then neither agent has an incentive to deviate
again for the same reason as in example 12.
The preceding example illustrates how any profile of actions at the null history
can be implemented in equilibrium.60 Now we identify general conditions under which
arbitrary behavior after the null history can also be supported in equilibrium bysuitably assigning payoffs to nonmeasurable behavior.
Proposition 9. Let |I| ≥ 2 and T = ∞. Consider the game Γ ×j∈I Π̄Tj F , (χj )j∈I
where χi (ht , π̂) = χi (ht , π̄) ≤ ζi (ht ) for all π̂, π̄ ∈ ×j∈I Π̄Tj F , any ht ∈ H, and every i ∈ I. Assume that for each i ∈ I and any ht ∈ H, there exists ã ∈ Āi (ht )such that ã 6= z. Suppose that there exists t̃ > 0 along with a collection of sets
{S̃t }t∈[0,t̃] such that {ω ∈ Ω : st (ω) ∈ S̃t , ∀ t ∈ [0, t̃]} is not a measurable subset ofthe probability space (Ω, F, P ). Choose any π ∈ ×i∈I Π̄Ti F such that for any profile
of action paths {bit }t∈[0,u) i∈I up to an arbitrary time u > 0, there exists with prob
ability one some t < u such that πi {sτ }τ ∈[0,t] , {(bjτ )j∈I }τ ∈[0,t) 6= bit for some i ∈ I.
Then there exists an SPE π 0 ∈ ×i∈I Π̄Ti F such that {φit (h0 , {sτ }τ ∈(0,T ) , π 0 )}t∈[0,T ) i∈I ={φit (h0 , {sτ }τ ∈(0,T ) , π)}t∈[0,T ) i∈I with probability one.
The result is proved by constructing a strategy profile π 0 with the following properties. At the null history h0 , agent i has no incentive to deviate because doing so
60

In our model, actions are assumed to be perfectly observable. Hence, given a strategy profile inwhich behavior on the path of play is measurable, assigning an extremely low payoff to nonmeasurable
behavior at information sets that can be reached only after a deviation does not affect expected
payoffs at the null history. Bonatti, Cisternas, and Toikka (2017) consider a related approachin which a payoff of negative infinity is assigned to strategy profiles with undesirable properties.
However, their model assumes imperfect monitoring, so that punishment using an infinitely negative
payoff results in an infinitely negative payoff at the null history.

47

would result in an expected payoff of χi (h0 , π), which is no more than the infimal
feasible payoff ζi (h0 ) based on the utility function. At any history ku up to a positive
time u, the action path b up to time u is such that the process ξbi (π 0 ) is not progressively measurable given the restriction on strategy profile π in the statement of the
proposition as well as the assumption that ξbi (π 0 ) records the continuation path ofplay when the action path up to time u is fixed at b. Hence, the expected payoff ofagent i at history ku is constant at χi (ku , π 0 ) when the other agents play π 0 , so thatagent i has no incentive to deviate.61
Note that simply requiring progressive measurability of the shock process does
not enable an arbitrary path of play to be implemented as an equilibrium. To see
this, suppose that the shock can take values only in a finite set and can change values
only at discrete times. Then the shock process would be progressively measurable,
but the finiteness of the state space and the discrete timing of state changes make it
impossible for a strategy profile to induce nonmeasurable behavior.
6.4

From Calculability Restriction to Payoff Assignment

We examine the relationship between the SPE under the calculability restriction and
the SPE when payoffs are assigned to nonmeasurable behavior. As in section 6.3, we
associate nonmeasurable behavior with an expected payoff no more than the greatest
lower bound on the feasible payoffs. The following result shows that the set of SPE
in this case is at least as large as the set of SPE under the calculability restriction.
Proposition 10. Let Āi (ht ) = Ai for every ht ∈ H and each i ∈ I. If π is an SPETFof Γ ×i∈I Π̄Ci , then π is an SPE of Γ ×i∈I Π̄i , (χi )i∈I with χi (ht , π̃) ≤ ζi (ht ) forall π̃ ∈ ×j∈I Π̄Tj F , any ht ∈ H, and every i ∈ I.
Therefore, if the model has an SPE under the calculability restriction, then there
exists an SPE under the approach where nonmeasurable behavior is assigned an
expected payoff no greater than the infimal feasible payoff, even when we do not
associate each instance of nonmeasurable behavior with the same expected payoff.
We assume only for simplicity when defining ξbi (π 0 ) that the action path up to time u is fixedirrespective of the shock realization up to time u. The reasoning in the proof does not entirely apply
under the alternative definition in footnote 26 where behavior up to time u may be determinedby π 0 . However, it can still be shown that at any history up to a given time that is reached withprobability one when playing π, π 0 specifies the same action profile as π, and πi0 designates a best0
response to π−i. As mentioned in footnote 26, the main results in section 4 are valid under bothdefinitions of the action process.
61

48

This result implies that insofar as the concept of SPE is concerned, the calculability restriction is not picking up strategy profiles that would be ruled out by every
assignment of expected payoffs to nonmeasurable behavior.
6.5

From Payoff Assignment to Calculability Restriction

Here we identify when an SPE under the method of assigning payoffs to nonmeasurable behavior is also an SPE under the calculability restriction.
We begin by defining a set of strategy profiles with certain measurability properties. Let the random variable θ : Ω → [0, T ] be a stopping time.62 Given any
π 0 , π 00 ∈ ×i∈I Πi , let ψ(π 0 , π 00 , θ) be the strategy profile satisfying the following two
properties for each i ∈ I:
1. ψi (π 0 , π 00 , θ) {st (ω)}t∈[0,u] , {(bjt )j∈I }t∈[0,u) = πi0 {st (ω)}t∈[0,u] , {(bjt )j∈I }t∈[0,u) for
each i ∈ I, all u ≥ 0, every {(bjt )j∈I }t∈[0,u) , and any ω ∈ Ω with u ≤ θ(ω);

2. ψi (π 0 , π 00 , θ) {st (ω)}t∈[0,u] , {(bjt )j∈I }t∈[0,u) = πi00 {st (ω)}t∈[0,u] , {(bjt )j∈I }t∈[0,u) for
each i ∈ I, all u ≥ 0, every {(bjt )j∈I }t∈[0,u) , and any ω ∈ Ω with u > θ(ω).
In other words, the strategy ψi (π 0 , π 00 , θ) plays πi0 until and including the stopping time,and plays πi00 thereafter. A strategy profile π ∈ ×i∈I ΠTi F is said to be measurablyattachable if for each action path b, every stopping time θ, and any π̃ ∈ ×i∈I ΠTi F suchthat ξbi (π̃) is progressively measurable for all i ∈ I, the strategy ψi (π̃, π, θ) is traceable
and frictional for all i ∈ I and the process ξbi [ψ(π̃, π, θ)] is progressively measurable
for all i ∈ I. That is, π is required to induce progressively measurable behavior after
any progressively measurable behavior up to and including an arbitrary random time.
Let ΠA ⊆ ×i∈I ΠTi F be the set of measurably attachable strategy profiles. In addition,a strategy profile π ∈ ×i∈I Πi is said to be synchronous if for any ht ∈ H, πj (ht ) = z
for all j ∈ I whenever πi (ht ) = z for some i ∈ I. That is, π requires the agents tomove at the same time as each other.63
62

That is, it satisfies {ω ∈ Ω : θ(ω) ≤ t} ∈ Ft for all t ∈ [0, T ).
The synchronicity assumption is satisfied by the maximal equilibrium of the tree harvesting
problem (section 5.1) and of the sequential exchange model and technology adoption game in the
online appendix as well as by a Markov perfect equilibrium of the inventory restocking application in the online appendix. In addition, any asynchronous strategy profile can be expressed as asynchronous strategy profile by adding a payoff irrelevant action to the action space of each agent
and requiring this action to be chosen by an agent that does not move when another agent moves.
All the equilibria studied in section 5 and the online appendix satisfy synchronicity after such areformulation.
63

49

According to the following result, any synchronous and measurably attachable
strategy profile that is an SPE when payoffs are assigned to nonmeasurable behavior
is also an SPE under the calculability restriction.
Theorem 3. If the synchronous strategy profile π ∈ ΠA is an SPE of Γ ×i∈I Π̄Ti F ,(χi )i∈I , then π is an SPE of Γ ×i∈I Π̄Ci .
The theorem implies that the restriction to calculable strategies does not exclude
from the set of SPE any synchronous and measurably attachable strategy profile thatis supported as an SPE under some assignment of expected payoffs to nonmeasurable
behavior.
To prove this result, we first show that any profile of calculable strategies that is an
SPE under an assignment of payoffs to nonmeasurable behavior is also an SPE under
the calculability restriction. Intuitively, when the other agents are playing calculable
strategies, a deviation by an agent from one calculable strategy to another calculable strategy produces the same change in expected payoffs under the calculability
restriction as under the payoff assignment method.
We then confirm that any synchronous and measurably attachable strategy profile
π is also a profile of calculable strategies. This part of the proof involves an iterative
procedure as in the proof of the main theorem stating that calculable strategies gen0
be a profile of quantitative
erate a measurable path of play. Specifically, we let π−i0strategies for the agents other than i. If the agents are playing (πi , π−i), then the0
behavior induced by (πi , π−i) is progressively measurable up to and including the first0
time that π or (πi , π−i ) prescribes a move. Because π induces progressively measurable behavior after any progressively measurable behavior up to and including an
0
arbitrary random time, the behavior induced by (πi , π−i) is progressively measurable0
up to and including the next time that π or (πi , π−i) prescribes a move. We can apply
0
this argument iteratively in order to show that the behavior induced by (πi , π−i) isprogressively measurable. This sort of reasoning establishes that πi is calculable.
The restriction to synchronous SPE ensures that the aforesaid iterative procedure0characterizes the path of play of (πi , π−i) over the entire course of the game. To see
this, choose any time t ∈ [0, T ). By the traceability and frictionality assumptions,0strategy profile (πi , π−i) with probability one induces only a finite number of moves
before time t. Moreover, the synchronicity assumption implies that the procedure is0such that π prescribes a move only if (πi , π−i) does so. Hence, the iterations withprobability one reach time t after only finitely many steps.
50Without the assumption that π is measurably attachable, the proposition fails.
This is illustrated by the following example, in which there are two agents, two times
at which the agents can move, and two possible actions at each of these times. We
specify a strategy profile π ∗ that is an SPE for a given assignment of payoffs tononmeasurable behavior but that is not measurably attachable because the action ofeach agent at time 2 is not measurable if one or both agents choose a non-z action
at time 1. We explain that πi∗ is not calculable because agent i’s behavior at time 2
under πi∗ is not measurable if agent −i follows the quantitative strategy π̃−i of always
choosing a non-z action at time 1 and always choosing z at time 2.
Example 14. (Role of Measurable Attachability) Suppose I = {1, 2}. For
each i ∈ {1, 2}, let Āi (ht ) = {w, z} if t = 1, Āi (ht ) = {x, z} if t = 2, and letĀi (ht ) = {z} otherwise. Let {st }t∈[0,T ) be an arbitrary stochastic process with state
space S. Assume that there exists S̃ ⊆ S such that {ω ∈ Ω : s2 (ω) ∈ S̃} is nota measurable subset of the probability space (Ω, F, P ). For all s ∈ S, the utility
function of each agent i ∈ {1, 2} satisfies vi [(x, x), s] = 1 and vi [(a1 , a2 ), s] = 0 for
any (a1 , a2 ) ∈ A1 × A2 such that (a1 , a2 ) 6= (x, x). Let χi (ht , π) = 0 for each i ∈ I, all
ht ∈ H, and any π ∈ Π̄T1 F × Π̄T2 F .
Let π ∗ be a strategy profile in which both agents choose z at time 1 and choose z
at time 2 if and only if some agent i ∈ {1, 2} chooses w at time 1 and s2 is in S̃. First,i[ψ(π̃, π ∗ , θ̃)] is not progressively
π ∗ is not measurably attachable since the process ξ{}
measurable, where π̃ is a strategy profile in which both agents always choose w attime 1 and choose z at time 2, and the stopping time θ̃ is equal to the constant 1.
Second, π ∗ is an SPE of Γ ×i∈I Π̄Ti F , (χi )i∈I because for all i ∈ I and πi0 ∈ Π̄Ti F , we∗have Ui (ht , π ∗ ) = Vi (ht , π ∗ ) = 1 ≥ Ui [ht , (πi0 , π−i)] for every history ht up to time 1 andevery history ht up to time 2 where z is chosen by both agents at time 1 and because∗
for all i ∈ I and πi0 ∈ Π̄Ti F , we have Ui (ht , π ∗ ) = χi (ht , π ∗ ) = 0 = χi [ht , (πi0 , π−i)] =0∗Ui [ht , (πi , π−i )] for every history ht up to time 2 where w is chosen by some agent at
itime 1. Third, πi∗ is not calculable since the process ξ{}(πi∗ , π̃−i ) is not progressively
measurable, where π̃−i is the quantitative strategy of always choosing w at time 1
and choosing z at time 2.
Note that measurable attachability is not a restriction on the strategy space of anindividual agent but on the space of strategy profiles. Hence, ΠA does not necessarily
have a product structure. Although we find it unsatisfactory, one could define a notion
51

of SPE under the restriction of measurable attachability, where the set of strategies
to which an agent can deviate depends on the strategy profile of its opponents. In
the online appendix, we provide a formal definition of this concept and demonstratethat any synchronous strategy profile satisfying this notion of equilibrium is an SPE
under the calculability restriction.
7

Conclusion

This paper considers the problem of defining strategy spaces for continuous-time
games in a stochastic environment. We introduced a new set of restrictions on individual strategy spaces that guarantee the existence of a unique action path as well as
the measurability of the induced behavior. Specifically, traceability and frictionality
ensure the former, and calculability ensures the latter. Existing techniques developed for a deterministic environment do not guarantee all of these properties in astochastic setting, and they are not sufficient to cover some applications of interest.
We also compared our method to an alternative approach in which specific payoffs
are assigned to strategies inducing nonmeasurable behavior, and found a certain relationship between our method and this alternative. A variety of economic examples
were presented to illustrate the applicability of our framework.
As we mentioned in the introduction, our methodology does not cover every possible situation. For example, although it applies to timing games in which agentschoose when to take actions, it is not applicable to settings where agents continuously change their actions. We hope that future research will address such settings
as well. In addition, we focused on the case in which past shocks and actions are perfectly observable. Allowing for imperfect information about them would enable theanalysis of a wider range of applications. One example would be strategic experimentation with multiple agents, in which the actions of the agents affect the acquisition of
information.64 Other examples may include models of repeated games as well as bargaining with incomplete information as in Daley and Green (2012) and Ortner (2017)
in which at least some actions are perfectly observed instantaneously. The entry gamein section 5.3 could also be extended to allow firms to have private information about
their own costs and benefits of entering the market. Finally, we restricted attention
to pure strategies. With mixed strategies, measurability problems may arise even64

For example, Keller and Rady (2015) consider a multi-agent bandit problem in continuous time
but restrict agents to Markov strategies.

52

in the absence of an exogenous shock because one agent’s behavior may conditionon the realization of another agent’s action in a nonmeasurable way. Despite thoselimitations, we showed the relevance of our methodology to a wide range of problems,
and we hope that it will be useful in future work on these applications and others forappropriately defining strategy spaces.
A

Appendix

A.1

Proofs for Sections 3 and 4

Proof of Theorem 1. We divide the proof of the theorem into three steps. The firststep shows uniqueness in part 1, the second shows part 2, and the third shows exis
tence in part 1. Fix an arbitrary history ku = {gt }t∈[0,u] , {(bjt )j∈I }t∈[0,u) up to timeu. Assume that the realization of the shock process {st }t∈[0,T ) is such that st = gt fort ∈ [0, u].
For t ∈ [0, u), let ajt = bjt for each j ∈ I. First, we show that there is probability
one that {st }t∈(u,T ) is such that there exists at most one profile {(ajt )j∈I }t∈[0,T ) of action
paths for which the history h = st , (ajt )j∈I t∈[0,T ) is consistent with πi for each i ∈ I
at every t ∈ [u, T ). Second, we show that there is probability one that {st }t∈(u,T )is such that {ajt }t∈[0,T ) ∈ Ξj (u) for each j ∈ I if the history h = st , (ajt )j∈I t∈[0,T )is consistent with πi for each i ∈ I at every t ∈ [u, T ). Third, we show that there
is probability one that {st }t∈(u,T ) is such that there exists a profile {(ajt )j∈I }t∈[0,T ) of
action paths for which the history h = st , (ajt )j∈I t∈[0,T ) is consistent with πi for eachi ∈ I at every t ∈ [u, T ).
For the first step, we use the definition of frictionality. For t ∈ [0, u) and j ∈ I,
let pjt = bjt and qtj = bjt . Suppose to the contrary that there is positive probability that {st }t∈(u,T ) is such that there exist two distinct profiles {(pjt )j∈I }t∈[0,T ) and
{(qtj )j∈I }t∈[0,T ) of action paths for which the histories hp = st , (pjt )j∈I t∈[0,T ) and
hq = st , (qtj )j∈I t∈[0,T ) are both consistent with πi for each i ∈ I at every t ∈ [u, T ).
The frictionality assumption implies that there is zero probability that {st }t∈(u,T ) issuch that {pjt }t∈[u,T ) or {qtj }t∈[u,T ) has infinitely many non-z actions in any finite interval of time for some j ∈ I. It follows that one can find a first time v ≥ u suchthat there exists j ∈ I such that {pjt }t∈[u,T ) is different from {qtj }t∈[u,T ) .65 Let hpv be
To see this, let X be the set of times at which {pjt }t∈[u,T ) is different from {qtj }t∈[u,T ) for somej ∈ I. If inf(X) = ∞, it means there is no such time, which is a contradiction. Hence, inf(X) < ∞.
If the time inf(X) does not belong to the set X, then there exists  > 0 such that there are infinitely
65

53

the history up to time v when hp is the history between times 0 and T , and let hqv
be the history up to time v when hq is the history between times 0 and T . By thedefinition of v, it must be that hpv = hqv . It follows that πi (hpv ) = πi (hqv ) for eachi ∈ I. Moreover, since both hp and hq are consistent with πi for each i ∈ I at everyt ∈ [u, T ), it must be that πi (hpv ) = piv and πi (hqv ) = qvi for each i ∈ I. Hence, we have
piv = qvi for each i ∈ I, which contradicts the fact that v is the first time no less thanu such that {pjt }t∈[u,T ) is different from {qtj }t∈[u,T ) for some j ∈ I.
The second step is straightforward. We again use the definition of frictionality.
Suppose to the contrary that there is positive probability that {st }t∈(u,T ) is such that
there exists {(ajt )j∈I }t∈[0,T ) for which {ajt }t∈[0,T ) is not in Ξj (u) for some j ∈ I and forwhich the history h = st , (ajt )j∈I t∈[0,T ) is consistent with πi for each i ∈ I at everyt ∈ [u, T ). In this case, πj would clearly violate the frictionality assumption for somej ∈ I.
For the third step, we use the following construction that relies on an iterativeargument. We apply the definitions of both traceability and frictionality. With
probability one, {st }t∈(u,T ) is such that the following algorithm can be applied. For
j,0
j
j,0
j ∈ I, define the action path {aj,0t }t∈[0,T ) so that at = bt for all t ∈ [0, u) and at = z
for all t ∈ [u, T ). Let k = 0.
1. If the history {st , (aj,kt )j∈I }t∈[0,T ) is consistent with πi for each i ∈ I at everyt ∈ [u, T ), then we are finished. Otherwise, continue to Stage 2.
2. From traceability, for each j ∈ I, one can find an action path {dj,kt }t∈[0,T )
j,k
j
j,k
i,k
with dt = bt for t ∈ [0, u) such that the history {st , (dt , (at )i6=j )}t∈[0,T ) is
consistent with πj at each t ∈ [u, T ), where dj,k= aj,kfor t ∈ [u, vk ] if k ≥ 1.
ttj,kFrom frictionality, {dt }t∈[0,T ) can be treated as having only finitely many nonz actions in any finite interval of time that is a subset of [u, T ). It follows
that one can find a first time vk ≥ u such that {aj,kt }t∈[u,T ) is different fromj,k{dt }t∈[u,T ) for some j ∈ I. For j ∈ I, define the action path {aj,k+1}t∈[0,T ) sotj,k+1
that aj,k+1= aj,k= dj,k+1
for t = vk . The historytt for t 6= vk and so that attj,k+1{st , (at)j∈I }t∈[0,T ) is consistent with πi for each i ∈ I at every t ∈ [u, vk ].
Redefine k as k + 1. Return to Stage 1.
many times in X greater than inf(X) but less than inf(X) + . This implies that at least one of
the 2|I| paths {(pjt )j∈I }t∈[u,T ) , {(qtj )j∈I }t∈[u,T ) differs from z at infinitely many points in the timeinterval between inf(X) and inf(X) + . In this case, there exists j ∈ I such that πj violates thefrictionality assumption.

54

Consider the case where the preceding algorithm does not terminate after a finite
number of iterations. By construction, vk is increasing in k. Note also that forj ∈ I along with any fixed value of t, there exists l such that aj,kis constant in ktj,k
for k > l. Thus, limk→∞ at is well defined for all t. Consider the history hT =limk→∞ {st , (aj,kt )j∈I }t∈[0,T ) . The history hT is consistent with πi for each i ∈ I fort ∈ [u, limk→∞ vk ).
Suppose that limk→∞ vk < T . With probability one, {st }t∈(u,T ) is such that thefollowing argument can be applied. From traceability, one can find action pathsj,∞{(dj,∞= bjt for every j ∈ I and all t ∈ [0, u) such that fort )j∈I }t∈[0,T ) with dt
i,keach j ∈ I, the history {st , (dj,∞t , limk→∞ (at )i6=j )}t∈[0,T ) is consistent with πj at eacht ∈ [u, T ), where dj,∞= limk→∞ aj,kfor j ∈ I and t ∈ [u, limk→∞ vk ). Since thettalgorithm does not terminate after a finite number of iterations, there exists j ∈ Isuch that {dj,∞t }[0,T ) 6∈ Ξj (u), which implies πj violates the frictionality assumption
for some j ∈ I. Thus, limk→∞ vk = T .
jHence, the history hT is such that (i) st = gt for all t ∈ [0, u], (ii) limk→∞ aj,kt = btfor j ∈ I and t ∈ [0, u), and (iii) hT is consistent with πi for each i ∈ I at everyt ∈ [u, T ).
Proof of Remark 2. Suppose that the first sentence of the statement of theorem 1 ismodified so that (πj )j∈I is a profile of traceable and weakly frictional strategies forwhich there exists l such that πj is frictional for all j 6= l. We show that the remainderof the statement of theorem 1 continues to hold. The proof of theorem 1 still applies
with the changes below.
The fourth sentence of the third paragraph should be replaced with the following.
The frictionality assumption implies that there is zero probability that {st }t∈(u,T )is such that {pjt }t∈[u,T ) or {qtj }t∈[u,T ) has infinitely many non-z actions in any finite
interval of time for some j 6= l. Therefore, from weak frictionality, there is zeroprobability that {plt }t∈[u,T ) or {qtl }t∈[u,T ) has infinitely many non-z actions in any
finite time interval.
The second pair of sentences in the fourth paragraph should be replaced with thefollowing. Suppose to the contrary that there is positive probability that {st }t∈(u,T )is such that there exists {(ajt )j∈I }t∈[0,T ) for which {ajt }t∈[0,T ) 6∈ Ξj (u) for some j 6= l
and for which the history h = st , (ajt )j∈I t∈[0,T ) is consistent with πi for each i ∈ I at
every t ∈ [u, T ). In this case, πj would clearly violate the frictionality assumption forsome j 6= l. Since no such {(ajt )j∈I }t∈[0,T ) can exist with positive probability, it follows
55

from the weak frictionality assumption that there is zero probability that {st }t∈(u,T )is such that there exists {(ajt )j∈I }t∈[0,T ) for which {alt }t∈[0,T ) 6∈ Ξl (u) and for which thehistory h = st , (ajt )j∈I t∈[0,T ) is consistent with πi for each i ∈ I at every t ∈ [u, T ).
In the second sentence of item 2 of the iterative procedure, frictionality should
be replaced with weak frictionality. In the second to last paragraph, the second
to last sentence should be replaced with the following. Since the algorithm does not
terminate after a finite number of iterations, there exists j ∈ I such that dj,∞6∈ Ξj (u).
tj,∞
If dt 6∈ Ξj (u) for some j 6= l, then πj violates the frictionality assumption forsome j 6= l. Thus, it must be that {dj,∞t }t∈[0,T ) ∈ Ξj (u) for all j 6= l, so thatj,k{limk→∞ at }t∈[0,T ) ∈ Ξj (u) for all j 6= l. Hence, if dl,∞
6∈ Ξl (u), then πl violates thetweak frictionality assumption.
Proof of Proposition 1. Assume that the strategy πi is pathwise inertial. For eachj 6= i, let π̃j ∈ be the pathwise inertial strategy that prescribes z at every history
up to an arbitrary time. Choose any history ku = {gt }t∈[0,u] , {(bjt )j∈I }t∈[0,u) up to
time u as well as any realization {gt }t∈(u,T ) of the shock process after time u. From
theorem 1 in Bergin and MacLeod (1993), there exists a unique profile ({ãjt }t∈[0,T ) )j∈Iof action paths with {ãjt }t∈[0,T ) ∈ Γj ({bjt }t∈[0,u) ) for each j ∈ I such that the historyh = gt , (ãjt )j∈I t∈[0,T ) is consistent with πi and π̃−i at every t ∈ [u, T ). For eachj 6= i, from the definition of π̃j , the action path {ãjt }t∈[0,T ) satisfies ãjt = z for allt ∈ [u, ∞). Combining the two preceding statements, πi is traceable.

Proof of Proposition 2. Fix an arbitrary history ku = {gt }t∈[0,u] , {(bjt )j∈I }t∈[0,u) up
to time u. Assume that πi ∈ Πi satisfies F2. Then given that {st }t∈[0,u] = {gt }t∈[0,u] ,−ithere is conditional probability one that for any {a−it }t∈[0,T ) ∈ Γ−i ({bt }t∈[0,u) ) suchthat ajt = z for all t > u and j 6= i, there exists an action path {ait }t∈[0,T ) ∈Γi ({bit }t∈[0,u) ) for which the history h = st , (ajt )j∈I t∈[0,T ) is consistent with πi ateach t ∈ [u, T ). Hence, πi is traceable.
Choose any realization {gt }t∈(u,T ) of the shock process after time u and any time0u > u. Assume that πi ∈ Πi satisfies pathwise F1. Then given any {a−it }t∈[0,T ) ∈ii
Γ−i ({b−it }t∈[0,u) ) for which there exists {at }t∈[0,T ) ∈ Γi ({bt }t∈[0,u) ) such that the historyh = {gt , (ajt )j∈I }t∈[0,T ) is consistent with πi at each t ∈ [u, T ), there exists an integerm such that there are at most m distinct values of t ∈ [u, u0 ] at which ait 6= z. Hence,
πi is frictional.
Proof of Proposition 3. Assume that πi ∈ ΠTi F . For each j 6= i, let π̃j ∈ ΠTj F be56

a strategy that prescribes z at every history up to an arbitrary time. Choose anyhistory ku = {gt }t∈[0,u] , {(bjt )j∈I }t∈[0,u) up to time u. The following hold with conditional probability one given that {st }t∈[0,u] = {gt }t∈[0,u] . From the first item of theorem 1, there exists a unique profile ({ãjt }t∈[0,T ) )j∈I of action paths with {ãjt }t∈[0,T ) ∈Γj ({bjt }t∈[0,u) ) for each j ∈ I such that the history h = st , (ãjt )j∈I t∈[0,T ) is consistent
with πi and π̃−i at every t ∈ [u, T ). For each j 6= i, from the definition of π̃j , theaction path {ãjt }t∈[0,T ) satisfies ãjt = z for all t ∈ [u, ∞). From the second item oftheorem 1, {ãit }t∈[0,T ) ∈ Ξi (u). Combining the three preceding statements, πi has
property F2.
jProof of Theorem 2. For i ∈ I, choose any πi ∈ ΠC
i . Let {(bτ )j∈I }τ ∈[0,u) be any pathof actions by the agents up to an arbitrary time u. Given the realization of theshock {sτ }τ ∈[0,u] until time u, denote the history up to time u by k̃u ({sτ }τ ∈[0,u] ) ={sτ }τ ∈[0,u] , {(bjτ )j∈I }τ ∈[0,u) .
It is helpful to define a set of strategies that depend only on the realizationof the shock and not on the behavior of the agents. For i ∈ I, a strategy π̃i ∈ΠTi F is said to be individualistic if π̃i (hpt ) = π̃i (hqt ) for any two histories hpt ={gτp }τ ∈[0,t] , {(pjτ )j∈I }τ ∈[0,t) and hqt = {gτq }τ ∈[0,t] , {(qτj )j∈I }τ ∈[0,t) up to an arbitrary
time t such that {gτp }τ ∈[0,t] = {gτq }τ ∈[0,t] .
Next some additional terminology regarding the action process is introduced. For
any realization of the shock process {sτ }τ ∈[0,T ) , let Λ({sτ }τ ∈[0,T ) ) be an arbitrary
0
0
00
00
0
00
subset of the interval [u, T ). Choose any π = (πj )j∈I and π = (πj )j∈I with πj , πj ∈000
ΠTj F for j ∈ I. The strategy profiles π and π are said to almost surely induce
the same path of play by agent i ∈ I for all t ∈ Λ({sτ }τ ∈[0,T ) ) if the following
holds. There is probability one of the realization of the shock process {sτ }τ ∈[0,T ) being
000
such that φit [k̃u ({sτ }τ ∈[0,u] ), {sτ }τ ∈(u,T ) , π ] = φit [k̃u ({sτ }τ ∈[0,u] ), {sτ }τ ∈(u,T ) , π ] for allt ∈ Λ({sτ }τ ∈[0,T ) ).
Now observe that the actions of each agent depend only on the realization of theshock if the agents play a fixed profile of traceable and frictional strategies. In particular, theorem 1 implies that there exists a profile (πj∗ )j∈I of individualistic strategies
such that π = (πj )j∈I and π ∗ = (πj∗ )j∈I almost surely induce the same path of playby every agent for all t ∈ [u, T ). We prove below that ξbi (πi , π−i ) is progressively
measurable for i ∈ I.
For i ∈ I, let πi0 be the strategy that requires agent i to choose action z ati0every history, and note that πi0 ∈ ΠQ
i . Moreover, the stochastic process ξb (πi , π−i )57

is progressively measurable for i ∈ I. Given any realization of the shock process{sτ }τ ∈[0,T ) , let Λ1 ({sτ }τ ∈[0,T ) ) denote the set consisting of every time t ∈ [u, T ) for
which there does not exist a time t̃ ∈ [u, t) such that φit̃ [k̃u ({sτ }τ ∈[0,u] ), {sτ }τ ∈(u,T ) , π] 6=z for some i ∈ I. That is, Λ1 ({sτ }τ ∈[0,T ) ) represents the interval consisting of alltimes no less than u and no greater than the time that the first non-z action after
reaching k̃u ({sτ }τ ∈[0,u] ) would be chosen when playing π. Note that the strategy0) almost surely induce the same path of play by agent
profiles (πi , π−i ) and (πi , π−i1i ∈ I for all t ∈ Λ ({sτ }τ ∈[0,T ) ).
For i ∈ I, let πi1 be the strategy that requires agent i to follow πi∗ at any timet ∈ Λ1 ({sτ }τ ∈[0,T ) ) and to choose action z at any time t ∈/ Λ1 ({sτ }τ ∈[0,T ) ). That is,
the strategy πi1 requires agent i to choose action z at each time less than u, to follow
the strategy πi∗ at any time no less than u and no greater than the time that thefirst non-z action after reaching k̃u ({sτ }τ ∈[0,u] ) would be chosen when playing π, and0∈ ΠT−iF , the strategy profiles
to choose action z thereafter. Note that for any π−i00(πi1 , π−i) and (πi , π−i) almost surely induce the same path of play by agent i for all
10t ∈ Λ ({sτ }τ ∈[0,T ) ). Moreover, it was noted above that ξbi (πi , π−i) is progressivelyQmeasurable. Hence, πi1 ∈ Πi for i ∈ I.
1
) is progressively measurable.
For i ∈ I, the strategy πi is such that ξbi (πi , π−i
Given any realization of the shock process {sτ }τ ∈[0,T ) , let Λ2 ({sτ }τ ∈[0,T ) ) denote theset consisting of every time t ∈ [u, T ) for which there exists at most one time t̃ ∈ [u, t)such that φit̃ [k̃u ({sτ }τ ∈[0,u] ), {sτ }τ ∈(u,T ) , π] 6= z for some i ∈ I. That is, Λ2 ({sτ }τ ∈[0,T ) )represents the interval consisting of all times no less than u and no greater than thetime that the second non-z action after reaching k̃u ({sτ }τ ∈[0,u] ) would be chosen when1
playing π. Note that the strategy profiles (πi , π−i ) and (πi , π−i) almost surely induce2
the same path of play by agent i ∈ I for all t ∈ Λ ({sτ }τ ∈[0,T ) ).
For i ∈ I, let πi2 be the strategy that requires agent i to follow πi∗ at any timet ∈ Λ2 ({sτ }τ ∈[0,T ) ) and to choose action z at any time t ∈/ Λ2 ({sτ }τ ∈[0,T ) ). That is,
the strategy πi2 requires agent i to choose action z at each time less than u, to follow
the strategy πi∗ at any time no less than u and no greater than the time that thesecond non-z action after reaching k̃u ({sτ }τ ∈[0,u] ) would be chosen when playing π,
0and to choose action z thereafter. Note that for any π−i∈ ΠT−iF , the strategy profiles
01(πi2 , π−i) and (πi , π−i) almost surely induce the same path of play by agent i for all
2
1t ∈ Λ ({sτ }τ ∈[0,T ) ). Moreover, it was noted above that ξbi (πi , π−i) is progressivelyQmeasurable. Hence, πi2 ∈ Πi for i ∈ I.
58

Let k be an arbitrary positive integer. Given any realization of the shock process{sτ }τ ∈[0,T ) , let Λk ({sτ }τ ∈[0,T ) ) denote the set consisting of every time t ∈ [u, T ) for
which there exist at most k−1 values of t̃ ∈ [u, t) such that φit̃ [k̃u ({sτ }τ ∈[0,u] ), {sτ }τ ∈(u,T ) ,π] 6= z for some i ∈ I. That is, Λk ({sτ }τ ∈[0,T ) ) represents the interval consisting of
all times no less than u and no greater than the time that the k th non-z action after
reaching k̃u ({sτ }τ ∈[0,u] ) would be chosen when playing π. For i ∈ I, let πik be thestrategy that requires agent i to follow πi∗ at any time t ∈ Λk ({sτ }τ ∈[0,T ) ) and tochoose action z at any time t ∈/ Λk ({sτ }τ ∈[0,T ) ). That is, the strategy πik requires
agent i to choose action z at each time less than u, to follow the strategy πi∗ at any
time no less than u and no greater than the time that the k th non-z action after reaching k̃u ({sτ }τ ∈[0,u] ) would be chosen when playing π, and to choose action z thereafter.
Proceeding as above, it follows that πik ∈ ΠQ
i for i ∈ I.
For i ∈ I, let ψi be the strategy that requires agent i to behave as follows. Agenti chooses action z at each time before u. Agent i plays strategy πi1 at any timet ∈ Λ1 ({sτ }τ ∈[0,T ) ). That is, agent i follows πi1 at any time no less than u and nogreater than the time that the first non-z action after reaching k̃u ({sτ }τ ∈[0,u] ) would bechosen when playing π. For every integer k ≥ 2, agent i plays strategy πik at any time
t satisfying t ∈/ Λk−1 ({sτ }τ ∈[0,T ) ) and t ∈ Λk ({sτ }τ ∈[0,T ) ). That is, agent i follows πik
between the times that the (k−1)th and k th non-z actions after reaching k̃u ({sτ }τ ∈[0,u] )would be chosen when playing π. Note that ψ = (ψj )j∈I and π = (πj )j∈I almost surely
induce the same path of play by every agent for all t ∈ [u, T ). We prove below thatξbi (ψi , ψ−i ) is progressively measurable for i ∈ I. It will then follow that ξbi (πi , π−i ) is
progressively measurable for i ∈ I.
For any positive integer k, let Θk denote the set that consists of each pair (t, ω) ∈[u, T ) × Ω for which there exist exactly k − 1 values of t̃ ∈ [u, t) such that we haveφit̃ [k̃u ({sτ (ω)}τ ∈[0,u] ), {sτ (ω)}τ ∈(u,T ) , ψ] 6= z for some i ∈ I. For k = 1, this conditionmeans that time t is no less than u and no greater than the time that the first non-z
action after reaching k̃u ({sτ }τ ∈[0,u] ) would be chosen when the sample point is ω andstrategy profile ψ is played by the agents. For k > 1, this condition means that time
t is greater than the time of the (k − 1)th but no greater than the time of the k th
non-z action after reaching k̃u ({sτ }τ ∈[0,u] ) when the sample point is ω and strategy
profile ψ is played by the agents.
Recall that πik ∈ ΠQ
i for i ∈ I and every positive integer k, which implies thatkξbi (πik , π−i) is progressively measurable. It follows that for each positive integer k,59the set Θk is progressively measurable.66 For i ∈ I and any positive integer k, letξ˜bi [(πjk )j∈I ] denote the stochastic process that is equal to ξbi [(πjk )j∈I ] on the set Θkand is equal to zero elsewhere. It also follows that for each positive integer k, thestochastic process ξ˜bi [(πjk )j∈I ] is progressively measurable for i ∈ I.
For any t ≥ u and positive integer k, let Θtk denote the set consisting of everySttpair (τ, ω) ∈ Θk such that τ ≤ t. Define Θt = ∞k=1 Θk . For any t ≥ u, let Υ
denote the set consisting of every pair (τ, ω) ∈ [u, t] × Ω. Recall that πi ∈ ΠC
i isa traceable and frictional strategy for i ∈ I. Hence, theorem 1 implies that given
any realization of the shock process {sτ }τ ∈[0,u] up to time u, there is conditional
probability one that there exists only finitely many values of t̃ ∈ [u, T ) such thatφit̃ [k̃u ({sτ }τ ∈[0,u] ), {sτ }τ ∈(u,T ) , π] 6= z for some i ∈ I. It follows that for any t ≥ u, theset consisting of each pair (τ, ω) such that (τ, ω) ∈ Υt and (τ, ω) ∈/ Θt has measurezero with respect to the product measure on B([0, t]) × Ft .
Note that for any positive integer k and i ∈ I, the stochastic process ξbi [(ψj )j∈I ]is equal to the stochastic process ξ˜bi [(πjk )j∈I ] on the set Θk . Recall that each set Θkalong with every stochastic process ξ˜bi [(πjk )j∈I ] is progressively measurable. Hence,
ξbi (ψi , ψ−i ) is progressively measurable for i ∈ I.
Proof of Proposition 4. Assume that for some j ∈ I, there exists ψj ∈ Ψj such that
Q
ψj ∈
/ ΠC
j . By definition, there exists ψ−j ∈ Π−j along with v such that the stochastic
process ξvj (ψj , ψ−j ) is not progressively measurable. It follows from ΠQ−j ⊆ Ψ−j thatψ−j ∈ Ψ−j . Hence, there exists i ∈ I such that the stochastic process ξbi (πi , π−i ) is
not progressively measurable for some πi ∈ Ψi , π−i ∈ Ψ−i , and b.
A.2

Proofs for Section 6

Proof of Proposition 9. Choose any strategy profile π ∈ ×i∈I Π̄Ti F . Define the strategy

profile π 0 ∈ ×i∈I Π̄Ti F as follows. Let ku = {st }t∈[0,u] , {(ait )i∈I }t∈[0,u) be any history
up to an arbitrary time u. If ait = πi {sτ }τ ∈[0,t] , {(ajτ )j∈I }τ ∈[0,t) for each i ∈ I andall t ∈ [0, u), then let πi0 (ku ) = πi (ku ) for all i ∈ I. If there exists t ∈ [0, u) and i ∈ Isuch that ait 6= πi {sτ }τ ∈[0,t] , {(ajτ )j∈I }τ ∈[0,t) , then for each i ∈ I, let πi0 (ku ) 6= z if st
is in S̃t for all t ∈ [0, t̃] and u = nt̃ for some positive integer n, and let πi0 (ku ) = z
otherwise.
66

Given any Θ ⊆ [0, T ) × Ω, let χΘ denote the indicator function of Θ. The set Θ is said to be
progressively measurable if for any υ ≥ 0 the function χΘ is measurable with respect to the productsigma-algebra B([0, υ]) × Fυ .

60

The strategy profile π 0 is an SPE because if agent i ∈ I deviates at h0 , then the
expected payoff to agent i is χi (h0 , π 0 ) ≤ ζi (h0 ). Moreover, consider any history ku =
{gt }t∈[0,u] , {(bjt )j∈I }t∈[0,u) up to an arbitrary time u > 0. By assumption, there exists
with probability one some t < u such that πi {sτ }τ ∈[0,t] , {(bjτ )j∈I }τ ∈[0,t) 6= bit for somei ∈ I. It follows that for each i ∈ I, the function φint̃ {st (ω)}t∈[0,u] , {(bjt )j∈I }t∈[0,u) ,{st (ω)}t∈(u,∞) , π 0 from Ω to Ai is not measurable, where n is any integer such thatnt̃ ≥ u. Hence, the process ξbi (π 0 ) with b = {(bjt )j∈I }t∈[0,u) is not progressively measurable, so that the expected payoff to agent i is χi (ku , π 0 ) ≤ ζi (ku ) when the otheragents follow strategy profile π 0 .
Proof of Proposition 10. Let π be an SPE of Γ ×i∈I Π̄Ci , and consider the game
Γ ×i∈I Π̄Ti F , (χi )i∈I with χi (ht , π̃) ≤ ζi (ht ) for all π̃ ∈ ×j∈I Π̄Tj F , ht ∈ H, and i ∈ I.
Let ku = {gt }t∈[0,u] , {(bjt )j∈I }t∈[0,u) be any history up to an arbitrary time u, anddenote b = {(bjt )j∈I }t∈[0,u) .
For any i ∈ I, choose any πi0 ∈ Π̄Ti F . If there is some j ∈ I such that ξbj (πi0 , π−i )is not progressively measurable, then Ui [ku , (πi0 , π−i )] = χi [ku , (πi0 , π−i )] ≤ ζi (ku ),whereas Ui (ku , π) = Vi (ku , π) ≥ ζi (ku ). Suppose now that ξbj (πi0 , π−i ) is progressively measurable for all j ∈ I. Let πi00 with πi00 (ht ) = z for t < u be defined such that





πi00 {sτ }τ ∈[0,t] , {(djτ )j∈I }τ ∈[0,t) = φit {sτ }τ ∈[0,u] , {(bjτ )j∈I }τ ∈[0,u) , {sτ }τ ∈(u,T ) , (πi0 , π−i )for each realization of the shock process {sτ }τ ∈[0,T ) and any action path {(djτ )j∈I }τ ∈[0,t)up to an arbitrary time t ≥ u. Note that πi00 ∈ Π̄i given the assumption thatĀi (ht ) = Ai for all ht ∈ H and i ∈ I. By the definition of πi00 , πi00 ∈ ΠTi F , andthe stochastic process ξbj (πi00 , π−i ) is the same as ξbj (πi0 , π−i ) for all j ∈ I, which implies
that Ui [ku , (πi00 , π−i )] = Vi [ku , (πi00 , π−i )] = Vi [ku , (πi0 , π−i )] = Ui [ku , (πi0 , π−i )]. Moreover, π 00 is quantitative and hence calculable. Since π is an SPE of Γ ×i∈I Π̄Ci , itmust be that Ui (ku , π) ≥ Ui [ku , (πi00 , π−i )]. It follows that Ui (ku , π) ≥ Ui [ku , (πi0 , π−i )].
Hence, no agent i has an incentive to deviate from πi to πi0 at ku , which proves that
π is an SPE of Γ ×i∈I Π̄Ti F , (χi )i∈I .
Proof of Theorem 3. We first show that π is an SPE of Γ ×i∈I Π̄Ci , assuming that πis an SPE of Γ ×i∈I Π̄Ti F , (χi )i∈I and πi is calculable for each i ∈ I. Then we confirm
that πi is calculable for each i ∈ I given that π is synchronous and measurablyattachable.

To show that π is an SPE of Γ ×i∈I Π̄Ci , consider any ht ∈ H. Given anyTFi ∈ I, choose any πi0 ∈ Π̄C
.
Since
π
is
an
SPE
of
Γ
×
Π̄
,
(χ
)
, it must
i∈I
i
i∈I
i
i
61

be that Ui (ht , π) ≥ Ui [ht , (πi0 , π−i )], where Ui (ht , π) = Vi (ht , π) and Ui [ht , (πi0 , π−i )] =CVi [ht , (πi0 , π−i )] because π ∈ ×j∈I ΠC
j . It follows that π is an SPE of Γ ×i∈I Π̄i .
zGiven any i ∈ I, we now confirm that πi ∈ ΠC
i . Define πj (ht ) = z for each j ∈ I0∈ ΠQand every ht ∈ H. Choose any b = {(bjt )j∈I }t∈[0,u) as well as any π−i−i . Define
the stopping time θ0 as follows. For any ω ∈ Ω, let Υ̂0 (ω) denote the set consistingof all t ∈ [u, T ) such that
φet {sτ (ω)}τ ∈[0,u] , ({bjτ }τ ∈[0,u) )j∈I , {sτ (ω)}τ ∈(u,T ) , π 6= z
for some e ∈ I, and let Ῡ0 (ω) denote the set consisting of all t ∈ [u, T ) such that
0φet {sτ (ω)}τ ∈[0,u] , ({bjτ }τ ∈[0,u) )j∈I , {sτ (ω)}τ ∈(u,T ) , (πi , π−i) 6= zfor some e ∈ I. Let θ0 (ω) be the lesser of the infimum of Υ̂0 (ω) and the infimum of0Ῡ0 (ω). Note that ξbi [ψ(π, π z , θ0 )] and ξbi {ψ[(πi , π−i), π z , θ0 ]} are the same progressively
measurable stochastic process.
Apply the following procedure iteratively for every positive integer k. Define thestopping time θk as follows. For any ω ∈ Ω, let Υ̂k (ω) denote the set consisting of allt ∈ (θk−1 (ω), T ) such that
0φet {sτ (ω)}τ ∈[0,u] , ({bjτ }τ ∈[0,u) )j∈I , {sτ (ω)}τ ∈(u,T ) , ψ[(πi , π−i), π, θk−1 ] 6= z
for some e ∈ I, and let Ῡk (ω) denote the set consisting of all t ∈ (θk−1 (ω), T ) such
that



0
φet {sτ (ω)}τ ∈[0,u] , ({bjτ }τ ∈[0,u) )j∈I , {sτ (ω)}τ ∈(u,T ) , (πi , π−i) 6= zfor some e ∈ I. Let θk (ω) be the lesser of the infimum of Υ̂k (ω) and the infimum ofῩk (ω), where P ({ω ∈ Ω : θk (ω) > θk−1 (ω)} ∪ {ω ∈ Ω : θk−1 (ω) = ∞}) = 1 because000ψ[(πi , π−i), π, θk−1 ], (πi , π−i) ∈ ×j∈I ΠTj F . Note that ξbi ψ{ψ[(πi , π−i), π, θk−1 ], π z , θk }0and ξbi {ψ[(πi , π−i), π z , θk ]} are the same progressively measurable stochastic process.
Suppose that the sequence {θk }∞k=1 does not converge almost surely to ∞. ThenP [{ω ∈ Ω : limk→∞ θk (ω) < ∞}] 6= 0. For each ω ∈ Ω, let Ξ(ω) denote the set
consisting of all t ∈ [u, T ) such that
0φet {sτ (ω)}τ ∈[0,u] , ({bjτ }τ ∈[0,u) )j∈I , {sτ (ω)}τ ∈(u,T ) , (πi , π−i) 6= z62

for some e ∈ I. Letting E denote the set consisting of all ω ∈ Ω such that {t ∈ Ξ(ω) :t ≤ c} contains only finitely many elements for any c ∈ [u, ∞), we have P ({ω ∈ Ω :0) ∈ ×j∈I ΠTj F . The definition of E also implies that forω ∈ E}) = 1 because (πi , π−i
all ω ∈ E such that limk→∞ θk (ω) < ∞, there exists t̃(ω) ∈ [u, limk→∞ θk (ω)) such
that



0
φet {sτ (ω)}τ ∈[0,u] , ({bjτ }τ ∈[0,u) )j∈I , {sτ (ω)}τ ∈(u,T ) , (πi , π−i) =z
for each e ∈ I and all t ∈ [t̃(ω), limk→∞ θk (ω)).
For any ω ∈ E such that limk→∞ θk (ω) < ∞, choose any k̃(ω) ≥ 1 such thatθk̃(ω)−1 (ω) ≥ t̃(ω). Assuming now that π is synchronous,0φet {sτ (ω)}τ ∈[0,u] ,({bjτ }τ ∈[0,u) )j∈I , {sτ (ω)}τ ∈(u,T ) , ψ[(πi , π−i), π, θk̃(ω)−1 ]0= φet {sτ (ω)}τ ∈[0,u] , ({bjτ }τ ∈[0,u) )j∈I , {sτ (ω)}τ ∈(u,T ) , (πi , π−i) =z
for each e ∈ I, any t ∈ (θk̃(ω)−1 (ω), limk→∞ θk (ω)), and all ω ∈ E such that limk→∞ θk (ω)< ∞. This implies that θk̃(ω) (ω) ≥ limk→∞ θk (ω) for all ω ∈ E such that limk→∞ θk (ω) <∞, from which it follows that there is a set of nonzero measure consisting of ω ∈ Ewith limk→∞ θk (ω) < ∞ for which there exists l ≥ 1 such that θl (ω) > limk→∞ θk (ω).
However, θk (ω) is nondecreasing in k by construction, so this is a contradiction.
Thus, the sequence {θk }∞k=1 must converge almost surely to ∞. Since the stochasi0
tic process ξb {ψ[(πi , π−i ), π z , θk ]} is progressively measurable for all k ≥ 1 and thei0sequence {θk }∞k=1 converges almost surely to ∞, the stochastic process ξb (πi , π−i ) is
progressively measurable. It follows that πi ∈ ΠCi .
References
Ambrus, A., and S. E. Lu (2015): “A Continuous-Time Model of Multilateral
Bargaining,” American Economic Journal: Microeconomics, 7(1), 208–249.
Bergin, J., and W. B. MacLeod (1993): “Continuous Time Repeated Games,”
International Economic Review, 34(1), 1993.
Bonatti, A., G. Cisternas, and J. Toikka (2017): “Dynamic Oligopoly with
Incomplete Information,” Review of Economic Studies, 84(2), 503–546.
Caplin, A. S., and D. F. Spulber (1987): “Menu Costs and the Neutrality of
Money,” Quarterly Journal of Economics, 102(4), 703–726.
63

Daley, B., and B. Green (2012): “Waiting for News in the Market for Lemons,”
Econometrica, 80(4), 1433–1504.
Dilme, F. (2017): “Dynamic Quality Signaling with Hidden Actions,” Mimeo, University of Bonn.
Dixit, A. (1989): “Entry and Exit Decisions under Uncertainty,” Journal of Political
Economy, 97(3), 620–638.
Faingold, E., and Y. Sannikov (2011):
Games,” Econometrica, 79(3), 773–876.

“Reputation in Continuous-Time

Fudenberg, D., and J. Tirole (1985): “Preemption and Rent Equalization in the
Adoption of New Technology,” Review of Economic Studies, 52(3), 383–401.
Kamada, Y., and M. Kandori (2020): “Revision Games,” Econometrica, 88(4),1599–1630.
Kamada, Y., and N. Rao (2018): “Sequential Exchange with Stochastic Transaction Costs,” Mimeo, University of California, Berkeley.
Keller, G., and S. Rady (2015): “Breakdowns,” Theoretical Economics, 10(1),175–202.
McCall, J. J. (1970): “Economics of Information and Job Search,” Quarterly Journal of Economics, 84(1), 113–126.
McDonald, R., and D. Siegel (1986): “The Value of Waiting to Invest,” The
Quarterly Journal of Economics, 101(4), 707–727.
Ortner, J. (2017): “Durable Goods Monopoly with Stochastic Costs,” Theoretical
Economics, 12(2), 817–861.
(2019): “A Continuous-Time Model of Bilateral Bargaining,” Games andEconomic Behavior, 113(C), 720–733.
Sannikov, Y. (2007): “Games with Imperfectly Observable Actions in Continuous
Time,” Econometrica, 75(5), 1285–1329.
(2008): “A Continuous-Time Version of the Principal-Agent Problem,” Review of Economic Studies, 75(3), 957–984.
64

Simon, L. K., and M. B. Stinchcombe (1989): “Extensive Form Games in Continuous Time: Pure Strategies,” Econometrica, 57(5), 1171–1214.

65