Global Dynamics in Inﬁnitely Repeated Games
with Additively Separable Continuous Payoﬀs∗
Takashi Kamihigashi† and Taiji Furusawa‡
November 13, 2007
Abstract
This paper studies a class of inﬁnitely repeated games with two players
in which the action space of each player is an interval, and the oneshot payoﬀ of each player is additively separable in their actions. We
deﬁne an immediately reactive equilibrium (IRE) as a pure-strategy
subgame perfect equilibrium such that each player’s action in eachperiod is a stationary function of the other player’s last action. We
completely characterize IREs and their dynamics in terms of certain
indiﬀerence curves. In a special case we establish a folk-type theorem using only IREs that are continuous and punish deviations in
a minimal way. Our results are used to show that in a prisoners’
dilemma game with observable mixed strategies, gradual cooperation
occurs when the players are suﬃciently patient, and that in a certain
duopoly game, kinked demand curves emerge naturally.
Keywords: Immediately reactive equilibria; additively separable payoﬀs; kinked demand; gradual cooperation; prisoners’ dilemma∗This is a revised version of the paper entitled “Immediately Reactive Equilibria in
Inﬁnitely Repeated Games with Additively Separable Continuous Payoﬀs.” Earlier versions
were presented in seminars at Kyoto University, University of Venice, University of Paris
1, and GREQAM. We would like to thank Atsushi Kajii, Tomoyuki Nakajima, Tadashi
Sekiguchi, Olivier Tercieux, Julio Davila, Sergio Currarini, Piero Gottardi, and anonymousreferees for helpful comments and suggestions.
†
Corresponding author. RIEB, Kobe University, Rokkodai, Nada, Kobe 657-8501
JAPAN. Email: tkamihig@rieb.kobe-u.ac.jp. Tel/Fax: +81-78-803-7015.
‡
Graduate School of Economics, Hitotsubashi University, 2-1 Naka, Kunitachi, Tokyo
186-8601 JAPAN. Email: furusawa@econ.hit-u.ac.jp. Tel/Fax: +81-42-580-8866.

1

Introduction

In inﬁnitely repeated games with a prisoners’ dilemma-like stage game, Nashreversion trigger strategies (Friedman, 1971) are often used to show thatcooperation (or collusion) can be sustained by the threat to revert to a noncooperative Nash equilibrium. In such equilibria, each player continues tocooperate as long as all the other players cooperate, but will choose to behave
selﬁshly once anyone defects.
While Nash reversion equilibria are simple and intuitive, they seem tohave two disadvantages. First, small deviations are punished as harshly
as large deviations.1 Second, there are no nontrivial transition dynamics
between the cooperative state and the noncooperative state. The ﬁrst disadvantage can be avoided by considering continuous strategies, which areinsensitive to small deviations. This approach has been adopted by Samuelson (1987), Friedman and Samuelson (1990, 1994a, 1994b), and Langlois andSachs (1993), who established the existence of nontrivial continuous equilibria. As for nontrivial transition dynamics, however, very few general results
seem to be available in the literature.
In this paper we study a class of inﬁnitely repeated two-person games
in which the action space of each player is an interval. We follow Friedman (1968, 1973, 1976) in focusing on strategies such that the action chosen in each period is a stationary function of the other player’s last action.
We call such strategies immediately reactive, and say that a subgame perfect equilibrium is an immediately reactive equilibrium (IRE) if each playerchooses an immediately reactive strategy.2 IREs are the natural choice inalternating move games (e.g., Maskin and Tirole, 1988a; Bhaskar and VegaRedondo, 2002). Since they can be continuous or discontinuous, IREs extend
Nash reversion equilibria, avoiding their ﬁrst disadvantage mentioned above.
Furthermore, the global dynamics of IREs are typically nontrivial and cancompletely be characterized graphically.
In our framework, interesting dynamics arise naturally. For example,
1

This also applies to more complex trigger strategy equilibria (Green and Porter 1984;
Abreu, 1986, 1988).
2Friedman (1968, 1973, 1976) called IREs “reaction function equilibria.” We avoid
his terminology since it has been used in a broader sense in subsequent literature. The
concept of IRE is related to a few other ones (Friedman and Samuelson, 1994a; Kalai,
Samet, and Stanford, 1988; Maskin and Tirole, 1988a, 1988b). Detailed discussions are
given in Subsections 2.4 and 2.5.

1

cooperation is achieved gradually in a repeated prisoners’ dilemma with observable mixed strategies if the players are suﬃciently patient.3 In a duopoly
game, kinked demand curves emerge naturally.4 In a collusive steady state,each ﬁrm cuts its price if the other ﬁrm does so, but neither ﬁrm responds if
the other ﬁrm raises its price. In “ineﬃcient” IREs, the collusive steady stateis unstable: after a small price cut by either ﬁrm, price war starts gradually,
accelerates, and leads to noncollusive prices in the long run. In “eﬃcient”
IREs, however, the collusive steady state is stable: a price cut by either ﬁrm
is matched by a smaller price cut, and the steady state is restored in thelong run. All of these dynamic phenomena are properties of IREs in regularform, which we deﬁne as IREs that are continuous and punish deviations in
a minimal way.
Unfortunately, these results come at a cost. In particular, we assume
that the one-shot payoﬀ of each player is additively separable in their actions. This assumption is necessary since, as shown by Stanford (1986) andRobson (1986), the only possible IRE is a trivial one in certain duopoly gameswith additively non-separable payoﬀs. On the other hand, the assumption of
additive separability holds in various games, including a prisoner’s dilemma
with observable mixed strategies and a duopoly game in which each ﬁrm’s
one-shot payoﬀ is given by the logarithm of its proﬁt.5 Another example
is a two-country model of international trade in which each country sets its
tariﬀ rate to maximize the sum of tariﬀ revenue, consumer surplus, and producer surplus.6 These and other examples are discussed in Subsection 2.2.
In addition to additive separability, we assume that each player’s one-shot
payoﬀ is continuous, monotone in the other player’s action, and monotone
or unimodal in his own action.
3
Gradual cooperation is known to arise in certain partnership games (e.g., Kranton,
1996; Watson, 1999, 2002; Furusawa and Kawakami, 2006). Our example shows a simplest
mechanism of gradual cooperation.
4
Although there are game-theoretic models of kinked demand in the literature (e.g.,
Maskin and Tiroel, 1988b; Bhaskar, Machin, and Reid, 1991; Radner, 2003; Sen, 2004),
they typically require rather speciﬁc assumptions. Though our example also requires
speciﬁc assumptions, it allows one to derive and visualize kinked demand curves in an
extremely simple manner.
5This can be justiﬁed by assuming that the owner of each ﬁrm is “risk averse” or, more
precisely, prefers stable proﬁt streams to unstable ones.
6
Furusawa and Kamihigashi (2006) study such a model, focusing on issues speciﬁc tointernational trade. A preliminary version of Furusawa and Kamihigashi (2006) containedsome of the arguments in this paper, which now appear exclusively in this paper.

2

Given a stage game satisfying the assumptions mentioned above, we showthat the set of IREs in the simultaneous move game is identical to that in thealternating move game.7 In both games, we completely characterize IREs interms of indiﬀerence curves associated with what we call eﬀective payoﬀs.
The eﬀective payoﬀ of a player is the part of his discounted sum of payoﬀsthat is directly aﬀected by his current action. By additive separability, theeﬀective payoﬀ consists of only two functions. This structure substantiallysimpliﬁes each player’s dynamic maximization problem.8
We show that in any IRE, any equilibrium path stays on the associatedindiﬀerence curves except for the initial period. By this result, equilibrium
dynamics are always characterized by two indiﬀerence curves. Our main
result characterizes what pairs of indiﬀerence curves can be supported as
IREs: given a pair of indiﬀerence curves, there is an associated IRE if andonly if the following two graphical conditions are met. First, the intersectionof the areas on or above the indiﬀerence curves must be nonempty. Second,
the lowest point of each indiﬀerence curve must not be too low relative tothe other indiﬀerence curve.
In a special case in which each player’s payoﬀ depends monotonically on
his own action, we provide a necessary and suﬃcient condition for an IRE
to be eﬀective eﬃcient, i.e., Pareto optimal among IREs in terms of eﬀective
payoﬀs. The necessary and suﬃcient condition is that the intersection of theareas strictly above the indiﬀerence curves be empty. Eﬀective eﬃciency hasinteresting dynamic implications, such as gradual cooperation in a prisoners’
dilemma with observable mixed strategies, as well as globally stable collusive
prices in a duopoly game, as mentioned above. In the same special case, we
also obtain the following folk-type theorem: if both players are suﬃciently
patient, any “strictly individually rational” action proﬁle—or any pair ofactions in which each player’s payoﬀ is strictly greater than his minimax
payoﬀ—can be supported as a steady state of an IRE in regular form.
Our folk-type theorem is similar in spirit to those shown by Friedman
and Samuelson (1994a, 1994b). Their results show that the main idea of
7

Section 4 discusses the relationship of this result to Lagunoﬀ and Matsui’s (1997) antifolk theorem for alternating move games of pure coordination. See Haller and Lagunoﬀ(2000) and Yoon (2001) for further results on alternating move games.
8Eﬀective payoﬀs are similar to what Kamihigashi and Roy (2006) call partial gains
in an optimal growth model with linear utility. Equations (2.19)–(2.21) in this paper
are similar to (3.7)–(3.9) in Kamihigashi and Roy (2006), but essentially this is the only
similarity in analysis between the two papers.

3the standard folk theorem (Fudenberg and Maskin, 1986) is valid even if one
conﬁnes oneself to continuous equilibria. In our case the equilibria that we
consider are stationary, continuous, and immediately reactive. In addition,
the IREs used in our folk-type theorem punish deviations in a minimal way.
The rest of the paper is organized as follows. Section 2 describes the oneshot game and our assumptions, discusses several examples, and introduces
the simultaneous and the alternating move games. Section 3 characterizes
the best responses of a player given the other player’s strategy, developing
and utilizing various graphical tools. The main result of Section 3 has some
immediate implications on IREs, which are shown in Section 4. Section 5
discusses the dynamics induced by IREs. Section 6 gives a complete characterization of IREs. Section 7 characterizes eﬀectively eﬃcient IREs in a
special case and shows a folk-type theorem. Section 8 applies our results toa prisoner’s dilemma game and a duopoly game. Section 9 concludes thepaper. The appendix contains the proof of our characterization result.

2

The Games

2.1

The One-Shot Game

Before introducing repeated games, let us describe the one-shot game. There
are two players, 1 and 2. Deﬁne
Q = {(1, 2), (2, 1)}.

(2.1)

For (i, j) ∈ Q, let Si denote player i’s action space, πi : Si × Sj → R player
i’s payoﬀ. The following assumptions are maintained throughout.
Assumption 2.1. For i = 1, 2, Si ⊂ R is an interval with nonempty interior.
Assumption 2.2. For (i, j) ∈ Q, there exist ui : Si → [−∞, ∞) and vi :Sj → R such that∀(si , sj ) ∈ Si × Sj ,πi (si , sj ) = ui (si ) + vi (sj ).

(2.2)

Assumption 2.3. v1 and v2 are continuous. Either both are strictly increasing or both are strictly decreasing.

4

Assumption 2.4. For i = 1, 2, ui is continuous, and there exists ŝi ∈ Si suchthat ui is strictly increasing on Si ∩ (−∞, ŝi ) provided Si ∩ (−∞, ŝi ) = ∅, andstrictly decreasing on Si ∩ (ŝi , ∞) provided Si ∩ (ŝi , ∞) = ∅.9Assumption 2.5. For i = 1, 2, ui is bounded above, and vi is bounded.
Assumption 2.2 is our key assumption. Assumptions 2.4 and 2.5 imply
that ŝi is player i’s strictly dominant strategy and that (ŝ1 , ŝ2 ) is the uniquestatic Nash equilibrium. Assumption 2.2 allows ui to be unbounded below
because such cases are common in economic models.

2.2

ExamplesThough Assumption 2.2 may appear rather strong as a restriction on generalgames with two players, it is satisﬁed in various games. We provide speciﬁc
examples below. Our intention here is not to claim that our assumptions
are general, but to suggest that our framework is useful in analyzing certain
types of games as well as special cases of more general games.
2.2.1

Tariﬀ War

Consider a two-country world in which the payoﬀ of each country is given by
the sum of its tariﬀ revenue, consumer surplus, and producer surplus. Eachcountry is better oﬀ if the other country reduces its tariﬀ rate, while eachcountry has an incentive to choose the tariﬀ rate that maximizes the sum of
its tariﬀ revenue and consumer surplus. To be more speciﬁc, let ŝi be thismaximizing tariﬀ rate, and si be country i’s tariﬀ rate imposed on imports
from country j. Under standard assumptions, country i’s producer surplus
is strictly decreasing in sj , while the sum of its tariﬀ revenue and consumer
surplus is strictly increasing in si for si ≤ ŝi and strictly decreasing forsi ≥ ŝi . This game satisﬁes our assumptions, and is analyzed in detail in
Furusawa and Kamihigashi (2006).
2.2.2

Aggregative Games

Consider a game in which the payoﬀ of player i can be written as a functionof si and si + sj , i.e., πi (si , sj ) = π̃i (si , si + sj ) for some π̃i . This type of
9

We follow the convention that if ui (r) = −∞ for some r ∈ Si , then ui is continuous
at r if limsi →r ui (si ) = −∞. Such r can only be min Si or max Si by Assumption 2.4.

5

game is called an aggregative game (Corchon, 1994). For example, si canbe player i’s contribution to a public good, or his pollution emission. If π̃ isadditively separable and depends linearly on si + sj , then there are various
cases in which our assumptions are satisﬁed.
2.2.3

Bertrand Competition

Consider a game played by two ﬁrms, each producing a diﬀerentiated productwith a constant marginal cost ci and with no ﬁxed cost. Firm i faces a demand
function Di (pi , pj ) that depends on the prices pi and pj chosen by the two
ﬁrms. Firm i’s proﬁt is Di (pi , pj )(pi − ci ). Suppose Di is multiplicatively
separable: Di (pi , pj ) = dii (pi )dji (pj ) for some functions dii and dji . Then the
proﬁt maximization problem of ﬁrm i is equivalent to maximizing ui (pi ) +vi (pj ), whereui (pi ) = ln dii (pi ) + ln(pi − ci ),vi (pj ) = ln dji (pj ).

(2.3)

This transformation is innocuous in the one-shot game, and our assumptions
are satisﬁed under reasonable assumptions on dii and dji . In repeated games
the above transformation may be justiﬁed by assuming that the owners of
the ﬁrms are “risk averse” or, more precisely, prefer stable proﬁt streams tounstable ones.
2.2.4

Prisoner’s Dilemma

Though the action spaces are assumed to be intervals in this paper, ourframework applies to 2×2 games with mixed strategies. A case in point is theprisoner’s dilemma game in Figure 1 (with a, c > 0), which is a parametrizedversion of the game discussed by Fudenberg and Tirole (1991, p. 10, p. 111).
For i = 1, 2, let si be player i’s probability of choosing action C. Let πi (si , sj )be player i’s expected payoﬀ:
πi (si , sj ) = si sj c + si (1 − sj )(−a) + (1 − si )sj (c + a)
= −asi + (c + a)sj .

(2.4)
(2.5)

Let S1 = S2 = [0, 1]. Then all our assumptions are clearly satisﬁed withŝ1 = ŝ2 = 0.10
10

Furusawa and Kawakami (2006) use a payoﬀ function similar to (2.4) to analyze perfect
Bayesian equilibria in a model with stochastic outside options.

6

Player 2
C

D

C

c, c

−a, c + aD

c + a, −a0, 0Player 1

Figure 1: Prisoner’s dilemma
Player 2
1

2

1

11
p11
1 , p212p12
1 , p22

21
p21
1 , p222
p22
1 , p2

Player 1

Figure 2: General 2 × 2 game
2.2.5

General 2 × 2 GamesThe preceding example suggests that our framework applies to more general2 × 2 games. To see this, consider the 2 × 2 game in Figure 2. For i = 1, 2,
let si be player i’s probability of choosing action 1. Let πi (si , sj ) be player
i’s expected payoﬀ:
πi (si , sj )12
21
22
= si sj p11
i + si (1 − sj )pi + (1 − si )sj pi + (1 − si )(1 − sj )pi
22
21
22
11
12
21
22
22
= (p12i − pi )si + (pi − pi )sj + (pi − pi − pi + pi )si sj + pi .

(2.6)
(2.7)
(2.8)

21
It is easy to see that all our assumptions hold if and only if p12
 p22
i =
i , (p1 −
22
21
11
12
21
22
22
p1 )(p2 − p2 ) > 0, and pi − pi − pi + pi = 0. The last condition suggests some form of additive separability. For example, it can be written as
22
21
12
p11
i − pi = pi − pi , i.e., player i’s choice has the same eﬀect on his payoﬀ independently of player j’s action. Alternatively, it can be written as
22
12
21
p11
i − pi = pi − pi , i.e., player j’s choice has the same eﬀect on player i’s
payoﬀ independently of player i’s action.

7

2.3

Normalizing Assumptions

To simplify the exposition, we introduce some assumptions that can be madewithout loss of generality.
Assumption 2.6. For i = 1, 2, inf Si = 0 and sup Si = 1.
This can be assumed without loss of generality since none of our assumptions is aﬀected by strictly increasing, continuous transformations of Si . If
0 ∈ Si and/or 1 ∈ Si , we extend ui and vi to 0 and/or 1 as follows:ui (0) = lim ui (s),ui (1) = lim ui (s),(2.9)

vi (0) = lim vi (s),vi (1) = lim vi (s).

(2.10)

s↓0

s↑1

s↓0

s↑1

By the above and Assumption 2.5, the following can be assumed without lossof generality.
Assumption 2.7. For i = 1, 2, ui : [0, 1] → [−∞, ∞) and vi : [0, 1] → R arecontinuous, and ui ((0, 1]), vi ([0, 1]) ⊂ R.
Strictly speaking, the next assumption is not a normalization, but it isinnocuous and is made merely for notational simplicity.11
Assumption 2.8. S1 = S2 = [0, 1].
The following is our last normalizing assumption.
Assumption 2.9. For i = 1, 2, vi is strictly increasing.
To see that this is a normalization, suppose v1 and v2 are both strictly
decreasing (recall Assumption 2.3). For (i, j) ∈ Q, deﬁne S̃i = [0, 1], s̃i =1 − si , ũi (s̃j ) = ui (1 − s̃j ), and ṽi (s̃i ) = vi (1 − s̃i ). Then ṽ1 and ṽ2 are strictlyincreasing, and ũ1 , ṽ1 , S̃1 , ũ2 , ṽ2 , and S̃2 satisfy all the other assumptions.

2.4

The Repeated Game with Simultaneous Moves

Consider the inﬁnitely repeated game in which the stage game is given by
the one-shot game described above. For i = 1, 2, let δi ∈ (0, 1) be player
i’s discount factor. We restrict ourselves to pure-strategy subgame perfect11

It is innocuous because removing 0 and/or 1 from Si does not aﬀect our analysis.

8equilibria in which player i’s action in period t, si,t , is a stationary function fiof player j’s action in period t − 1, sj,t−1 . We call such strategies immediately
reactive.
Friedman (1968) called such strategies reaction functions. Immediately
reactive strategies are a special case of single-period-recall strategies (Friedman and Samuelson, 1994a) and reactive strategies (Kalai et al., 1988).
Single-period-recall strategies depend only on both players’ last actions, andreactive strategies depend only on the other player’s past actions. We focus
on stationary strategies that depend only on the other player’s last action.
This feature is shared by well-known strategies such as grim trigger andtit-for-tat.
Let F be the set of all functions from [0, 1] to [0, 1]. Let (i, j) ∈ Q. Taking
player j’s strategy fj ∈ F as given, player i faces the following problem:max{si,t }∞t=1∞


δit−1 [ui (si,t ) + vi (sj,t )](2.11)

t=1

s.t. ∀t ∈ N,∀t ∈ N,sj,t = fj (si,t−1 ),si,t ∈ [0, 1].

(2.12)(2.13)

We say that fi ∈ F is a best response to fj if for any (si,0 , sj,0 ) ∈ [0, 1]2 , theabove maximization problem has a solution {si,t }∞t=1 such that si,t = fi (sj,t−1 )for all t ∈ N. We call a strategy proﬁle (f1 , f2 ) ∈ F 2 an immediately reactive
equilibrium (IRE) if f1 is a best response to f2 and vice versa. Note that
f1 and f2 are not required to be continuous or even measurable, but the
maximization problem (2.11)–(2.13) is required to be well deﬁned given fj .12

2.5

The Repeated Game with Alternating Moves

Now consider the case of alternating moves. Player 1 updates his action in
odd periods, while player 2 updates his action in even periods.13 Deﬁne
T1 = {1, 3, 5, · · · },T2 = {2, 4, 6, · · · }.

12

(2.14)

Our results are unaﬀected even if f1 and f2 are required to be continuous or uppersemi-continuous. The same remark applies to the alternating move game.
13
In alternating move games, it is often assumed that the players play simultaneously
in the initial period and take turns afterwards. Such an assumption does not aﬀect ouranalysis, which is concerned only with stationary subgame perfect equilibria.

9As in the simultaneous move case, we restrict ourselves to subgame perfectequilibria in which each player chooses an immediately reactive strategy, i.e,
in each period t ∈ Ti , player i chooses an action si,t according to a stationary
function fi of player j’s last (or equivalently current) action sj,t−1 .
Let (i, j) ∈ Q. Given player j’s strategy fj ∈ F , player i faces thefollowing problem:max{si,t }∞t=1∞


δit−i [ui (si,t ) + vi (sj,t )](2.15)

t=is.t. ∀t ∈ Tj ,∀t ∈ Ti ,sj,t = fj (si,t−1 ), si,t = si,t−1 ,si,t ∈ [0, 1], sj,t = sj,t−1 .

(2.16)(2.17)

We say that fi ∈ F is a best response to fj if for any sj,i−1 ∈ [0, 1],14 the above
maximization problem has a solution {si,t }∞t=1 such that si,t = fi (sj,t−1 ) for
all t ∈ Ti . We call a strategy proﬁle (f1 , f2 ) ∈ F 2 an immediately reactive
equilibrium (IRE) if f1 is a best response to f2 and vice versa. This equilibrium concept is consistent with one deﬁnition of Markov perfect equilibrium(Maskin and Tirole, 1988b, Section 2), but distinct from another (Maskin
and Tirole, 2001) due to additive separability of payoﬀs.

2.6

Eﬀective Payoﬀs

We now introduce a function that plays a central role in our analysis. For(i, j) ∈ Q, deﬁne wi : [0, 1]2 → R+ by
wi (si , sj ) = ui (si ) + δi vi (sj ).

(2.18)

We call this function player i’s eﬀective payoﬀ since in both repeated games,
player i in eﬀect seeks to maximize the discounted sum of eﬀective payoﬀs.
Indeed, in both games, player i’s discounted sum of payoﬀs from period 1
14

Notice that for i = 1, 2, the ﬁrst period in which player i plays is period i.

10onward is written as∞δit−1 [vi (sj,t ) + ui (si,t )]t=1= vi (sj,1 ) += vi (sj,1 ) +

∞

t=1
∞


(2.19)

δit−1 [ui (si,t ) + δi vi (sj,t+1 )](2.20)

δit−1 wi (si,t , sj,t+1 ).

(2.21)

t=1

In both games, player i has no inﬂuence on sj,1 , so that player i’s problem is
equivalent to maximizing the discounted sum of eﬀective payoﬀs.

3Characterizing Best Responses

Let (i, j) ∈ Q. This section takes player j’s strategy fj ∈ F as given, andstudies player i’s best responses. We show ﬁrst a simple result that characterizes them. The purpose of this section is to reexpress the result in terms ofindiﬀerence curves associated with eﬀective payoﬀs so as to obtain a graphicalunderstanding of player i’s problem. The following result characterizes player
i’s best responses in both the simultaneous and alternating move games.
Proposition 3.1. In both the simultaneous and the alternating move games,fi ∈ F is a best response to fj if and only if∀sj ∈ [0, 1],fi (sj ) ∈ argmax wi (si , fj (si )) ≡ Mi (fj ).

(3.1)

si ∈[0,1]

Proof. Consider the simultaneous move game. From (2.19)–(2.21) and (2.12),
player i’s discounted sum of payoﬀs is written as∞δit−1 [ui (si,t ) + vi (sj,t )] = vi (sj,1 ) +

t=1

∞


δit−1 wi (si,t , fj (si,t )).

(3.2)

t=1

Thus the maximization problem (2.11)–(2.13) is equivalent to maximizing
the right-hand side of (3.2), which is maximized if and only if si,t ∈ Mi (fj )for all t ∈ N. Therefore, if fi ∈ F is a best response, then fi (sj,0 ) ∈ Mi (fj )for all sj,0 ∈ [0, 1]; thus (3.1) holds. Conversely, if fi ∈ F satisﬁes (3.1), thenit is a best response since si,t = fi (sj,t−1 ) ∈ Mi (fj ) for all t ∈ N.
11

Now consider the alternating move game. From (2.19)–(2.21), (2.16), and(2.17), player i’s discounted sum of payoﬀs from period i onward is writtenas∞δit−i [ui (si,t ) + vi (sj,t )]t=i= vi (sj,i ) +δit−i (1 + δi )wi (si,t , sj,t+1 )t∈Ti= vi (sj,i ) + (1 + δi )δit−1 wi (si,t , fj (si,t )).

(3.3)
(3.4)
(3.5)

t∈Ti

Thus the maximization problem (2.15)–(2.17) is equivalent to maximizing
the right-hand side of (3.5), which is maximized if and only if si,t ∈ Mi (fj )for all t ∈ Ti . Hence the proposition follows as in the simultaneous movecase.
To translate the above result into more usable forms, deﬁneR(fi ) = {fi (sj ) | sj ∈ [0, 1]}.

(3.6)

Note that R(fi ) is the range of fi . The following is a simple restatement of
Proposition 3.1.
Corollary 3.1. fi ∈ F is a best response to fj if and only if R(fi ) ⊂ Mi (fj ).
Let us now introduce indiﬀerence curves associated with eﬀective payoﬀs. Those indiﬀerence curves are closely connected with both players’ best
responses, and crucial in understanding IREs and their global dynamics.
Since vi is strictly increasing by Assumption 2.9, each indiﬀerence curvewi (si , sj ) = ω can be expressed as the graph of a function from si to sj . We
denote this function by gjω , i.e.,ω = wi (si , gjω (si )) = ui (si ) + δi vi (gjω (si )).

(3.7)

Depending on si and ω, however, gjω (si ) may or may not be deﬁned. We
specify the domain of gjω , denoted D(gjω ), as follows:D(gjω ) = {si ∈ [0, 1] | ∃sj ∈ [0, 1], ui (si ) + δi vi (sj ) = ω}= {si ∈ [0, 1] | ω − δi vi (1) ≤ ui (si ) ≤ ω − δi vi (0)}.
12

(3.8)
(3.9)

w (ŝi ,1)

gj i

(si )

D(gjω )1sj
1
gjω (si )w (ŝi ,0)

gj i

(si )

ŝi

0

si

Figure 3: Indiﬀerence curve gjω and D(gjω )See Figure 3. It follows from (3.7) that∀si ∈ D(gjω ),gjω (si ) = vi−1

ω − ui (si )δi.

(3.10)

The following lemma collects useful observations on gjω .
Lemma 3.1. Let Ω = [wi (ŝi , 0), wi (ŝi , 1)]. (i) For ω ∈ Ω, gjω (·) is continuouson D(gjω ), ŝi ∈ D(gjω ), and D(gjω ) is a nonempty closed interval. (ii) Ifω, ω  ∈ Ω with ω < ω  , then D(gjω ) ⊂ D(gjω ) and

∀si ∈ D(gjω ),gjω (si ) < gjω (si ).

(3.11)(iii) Let ω ∈ [wi (ŝi , 0), wi (ŝi , 1)). Then D(gjω ) is a closed interval with
nonempty interior. Furthermore, gjω (·) is strictly decreasing on D(gjω )∩[0, ŝi ]provided ŝi > 0, and strictly increasing on D(gjω ) ∩ [ŝi , 1] provided ŝi < 1.
Proof. Let ω ∈ Ω. The continuity of gjω is obvious. Both inequalities in (3.9)
hold with si = ŝi sinceω ≤ wi (ŝi , 1) = ui (ŝi ) + δi vi (1),ω ≥ wi (ŝi , 0) = ui (ŝi ) + δi vi (0).
13

(3.12)(3.13)

Hence ŝi ∈ D(gjω ). Note from Assumption 2.4 and (3.13) that∀si ∈ [0, 1],

ui (si ) ≤ ui (ŝi ) ≤ ω − δi vi (0).

(3.14)

D(gjω ) = {si ∈ [0, 1] | ω − δi vi (1) ≤ ui (si )}.

(3.15)

Thus by (3.9),

It follows by Assumption 2.4 and (3.12) that D(gjω ) is a nonempty closed
interval. We have veriﬁed (i).
To see (ii), note that the inequality in (3.11) is immediate from (3.10)
for si ∈ D(gjω ) ∩ D(gjω ). Thus it suﬃces to show D(gjω ) ⊂ D(gjω ). Letsi ∈ D(gjω ). Then the inequality in (3.15) holds with ω = ω  , so it holds forany ω ≤ ω  . It follows that D(gjω ) ⊂ D(gjω ).
To see (iii), let ω ∈ [wi (ŝi , 0), wi (ŝi , 1)). By (i), D(gjω ) is a nonempty
closed interval. Since ω < wi (ŝi , 1) = ui (ŝi )+δi vi (1), i.e., ω −δi vi (1) < ui (ŝi ),it follows by (3.15) that si ∈ D(gjω ) for si suﬃciently close to ŝi . Thus theﬁrst conclusion in (i) holds. The second conclusion is immediate from (3.10)
and Assumptions 2.9 and 2.4.
To understand Corollary 3.1 in terms of indiﬀerence curves gjω , we deﬁnewi∗ (fj ) = sup wi (si , fj (si )).

(3.16)si ∈[0,1]

Since vi is strictly increasing by Assumption 2.9,
wi (ŝi , 0) ≤ wi∗ (fj ) ≤ wi (ŝi , 1).

(3.17)

By Lemma 3.1(i), a higher indiﬀerence curve is associated with a higher
eﬀective payoﬀ. Thus by (3.17),w (ŝ ,0)w∗ (fj )gj i i (ŝi ) ≤ gj iw (ŝ ,1)(ŝi ) ≤ gj i i (ŝi ).

(3.18)

See Figure 3 (with ω = wi∗ (fj )).
Now consider the maximization problem associated with (3.1) (or (3.16)),
which can equivalently be expressed as
max wi (si , sj ) s.t. sj = fj (si ).

si ,sj ∈[0,1]14(3.19)

The graph sj = fj (si ) represents the set of feasible pairs (si , sj ) for playeri, who takes player j’s reaction as a constraint. Since the highest feasiblew∗ (f )
indiﬀerence curve is given by sj = gj i j (si ),w∗ (fj )∀si ∈ D(gj iw∗ (fj )fj (si ) ≤ gj i),(si ).

(3.20)

See Figure 4, which shows two ad hoc examples (recall that fj is arbitraryhere). It follows that the solution to (3.19) is to choose any pair (si , sj )w∗ (f )
satisfying sj = fj (si ) and sj = gj i j (si ). More precisely,w∗ (fj )Mi (fj ) = {si ∈ D(gj iw∗ (fj )) | fj (si ) = gj i(si )}.

(3.21)

See Figure 4 again. By Corollary 3.1 and (3.21),w∗ (fj )R(fi ) ⊂ Mi (fj ) ⊂ D(gj i),(3.22)

whenever fi is a best response to fj . We are ready to restate Corollary 3.1
in terms of indiﬀerence curves gjω .
Proposition 3.2. fi ∈ F is a best response to fj if and only if∀si ∈ R(fi ),w∗ (fj )si ∈ D(gj iw∗ (fj )), fj (si ) = gj i(si ).

(3.23)

Proof. This holds since (3.23) is equivalent to R(fi ) ⊂ Mi (fj ) by (3.21).
In Figure 4(a), Mi (fj ) is a singleton, so by Corollary 3.1, there is a uniquebest response, which is the constant function from sj to si corresponding to
the dotted vertical line. This function trivially satisﬁes (3.23). In Figure4(b), Mi (fj ) is an interval, so all functions from sj to si whose ranges are
conﬁned to that interval are best responses. Notice that all those functions
satisfy (3.23).

4

Immediate Implications on IREs

The following result is immediate from Corollary 3.1, Proposition 3.2, andthe deﬁnitions of IRE in Subsections 2.4 and 2.5.

15

sj
1

Ψωi i
w∗ (fj )

gj i

(si )

fj (si )Mi (fj )

0

1

si

(a)
sj
1

fj (si )0Ψωi i

Mi (fj )

1

si

(b)
Figure 4: Mi (fj ) and Ψωi i (deﬁned in (6.1)) with ωi = wi∗ (fj )16Theorem 4.1. In both the simultaneous and the alternating move games, astrategy proﬁle (f1 , f2 ) ∈ F 2 is an IRE if and only if∀(i, j) ∈ Q,

R(fi ) ⊂ Mi (fj ),(4.1)

or, equivalently,∀(i, j) ∈ Q, ∀si ∈ R(fi ),w∗ (fj )si ∈ D(gj iw∗ (fj )), fj (si ) = gj i(si ).

(4.2)

This result implies that the simultaneous and the alternating move games
are equivalent as far as IREs are concerned. This would appear in sharp contrast to the anti-folk theorem of Lagunoﬀ and Matsui (1997) for alternating
move games of pure coordination. They showed that there is a considerable diﬀerence between the simultaneous and the alternating move games in
the case of pure coordination. If ui (si ) = vj (si ) and vi (sj ) = uj (sj ) for all
si , sj ∈ [0, 1] and (i, j) ∈ Q, then the one-shot game described in Subsection
2.1 becomes a pure coordination game. Theorem 4.1 of course applies tothis case (which is consistent with our assumptions), but does not contradict Lagunoﬀ and Matsui’s result. This is because their result deals with all
subgame perfect equilibria, while Theorem 4.1 deals only with IREs.15
For the remainder of the paper, we do not distinguish between the twogames except when we explicitly consider dynamics. The diﬀerences in dynamics between the two games are discussed in Section 5.
To illustrate Theorem 4.1, let (f1 , f2 ) be given by fi (sj ) = ŝi for sj ∈ [0, 1]
and (i, j) ∈ Q. See Figure 5. This strategy proﬁle corresponds to the staticw∗ (f )Nash equilibrium. One can easily see from Figure 5 that f2 and g2 1 2w∗ (f )
coincide on R(f1 ) = {ŝ1 }, and f1 and g1 2 1 coincide on R(f2 ) = {ŝ2 }. Thus
(f1 , f2 ) satisﬁes (4.2), so it is an IRE by Theorem 4.1.
As another example, let (f1 , f2 ) be such that fi (sj ) = si if sj = sj , andfi (sj ) = ŝi otherwise, where s1 and s2 are as in Figure 6. In words, eachplayer “cooperates” as long as the other player does so, but reverts to thestatic Nash equilibrium if the other player deviates in any direction. One can
15
Theorem 4.1 suggests that the concept of IRE has some resemblance to that of conjectural variation equilibrium. The main diﬀerence between the two concepts is that whilea conjectural variation equilibrium is a static equilibrium that consists of an equilibrium
point and supporting conjectures that are typically required to satisfy certain local properties, an IRE is a fully dynamic equilibrium that consists of two functions that represent
the players’ actual reactions. See Sabourian (1992) and Tidball et al. (2000) for detailed
discussions on the relation between conjectural variation equilibria and repeated games.

17

s2
1
w∗ (f2 )

g2 1

ŝ2

(s1 )

f2 (s1 )f1 (s2 )w∗ (f1 )

g1 2

(s2 )

ŝ1

0

1

s1

Figure 5: Static Nash equilibriumw∗ (f )

see from Figure 6 that f2 and g2 1 2 coincide on R(f1 ) = {ŝ1 , s1 }, and f1w∗ (f )and g1 2 1 coincide on R(f2 ) = {ŝ2 , s2 }. Thus this strategy proﬁle satisﬁes(4.2), so it is an IRE.
The main purpose of this paper is to characterize the entire set of IREs,
many of which induce more interesting dynamics. To this end we deﬁne an
IRE associated with (ω1 , ω2 ) ∈ R2 as an IRE (f1 , f2 ) such that∀(i, j) ∈ Q,

ωi = wi∗ (fj ).

(4.3)

Notice that any IRE (f1 , f2 ) is associated with (w1∗ (f2 ), w2∗ (f1 )). The following result shows one way to construct an IRE with nontrivial dynamics.
Proposition 4.1. Let ω1 , ω2 ∈ R. Suppose∀(i, j) ∈ Q,

D(gjωi ) = [0, 1].

(4.4)

Then there exists an IRE associated with (ω1 , ω2 ). In particular, (g1ω2 , g2ω1 ) issuch an IRE.
Proof. Let fj = gjωi for (i, j) ∈ Q. For (i, j) ∈ Q and si ∈ [0, 1], we havewi (si , fj (si )) = wi (si , gjωi (si )) = ωi , so ωi = wi∗ (fj ). To verify (4.2), let (i, j) ∈w∗ (f )w∗ (fj )Q and si ∈ R(fj ). Then si ∈ [0, 1] = D(gj i j ) and fj (si ) = gj i(4.2) holds, so (f1 , f2 ) is an IRE by Theorem 4.1.
18(si ). Thus

s2
1

s1
w∗ (f2 )

g2 1

ŝ2

s2

(s1 )

f2 (s1 )f1 (s2 )w∗ (f1 )

g1 2

(s2 )

ŝ1

0

1

s1

Figure 6: Nash reversion
See Figure 7 for an example of an IRE satisfying (4.4) and (4.3). Sincew∗ (f )w∗ (f )
f1 = g1 2 1 and f2 = g2 1 2 , the example trivially satisﬁes (4.2).

5

Dynamics

Before we turn to a detailed characterization of IREs, it is useful to have abasic understanding of their dynamics. This section takes an IRE (f1 , f2 ) ∈F 2 as given and studies its dynamic properties.
Consider ﬁrst the alternating move game. Recall that in each periodt ∈ N, player i with t ∈ Ti updates his action as a function of player j’s last(or current) action. So the “state variable” in each period t ∈ Ti is player j’s
last action sj,t−1 ∈ [0, 1]. Given an initial condition s2,0 ∈ [0, 1], the entire
path {s1,t , s2,t }∞t=1 of the game is uniquely determined by∀(i, j) ∈ Q, ∀t ∈ Ti ,si,t+1 = si,t = fi (sj,t−1 ).

(5.1)

In step-by-step form,
s1,1 = f1 (s2,0 ), s2,2 = f2 (s1,1 ), s1,3 = f1 (s2,2 ), · · · .

19

(5.2)

s2
1
w∗ (f1 )f1 (s2 ) (= g1 2(s2 ))w∗ (f2 )ŝ2

f2 (s1 ) (= g2 1(s1 ))

s2,0

0

ŝ1

1

s1

Figure 7: Example of IRE satisfying (4.4) and (4.3).
For the alternating move game, we deﬁne an IRE path associated with (f1 , f2 )as a sequence {s1,t , s2,t }∞t=0 satisfying (5.1). See Figure 7 for an example an
IRE path.
Now consider the simultaneous move game. The state variable in eachperiod t ∈ N is the pair of both players’ last actions (s1,t−1 , s2,t−1 ) ∈ [0, 1]2 .
Given an initial condition (s1,0 , s2,0 ) ∈ [0, 1]2 , the entire path {s1,t , s2,t }∞t=1 ofthe game is uniquely determined by∀(i, j) ∈ Q, ∀t ∈ N,

si,t = fi (sj,t−1 ).

(5.3)

For the simultaneous move game, we deﬁne an IRE path associated with
(f1 , f2 ) as a sequence {s1,t , s2,t }∞t=0 satisfying (5.3). Any IRE path can be
decoupled into two sequences, one originating from s2,0 , the other from s1,0 :s1,1 = f1 (s2,0 ), s2,2 = f2 (s1,1 ), s1,3 = f1 (s2,2 ), · · · ,s2,1 = f2 (s1,0 ), s1,2 = f1 (s2,1 ), s2,3 = f2 (s1,2 ), · · · .

(5.4)
(5.5)

Obviously, given s2,0 ∈ [0, 1], the sequences given by (5.2) and (5.4) are
identical. The sequence given by (5.5) can be viewed as an IRE path for thealternating move game in which player 2 moves ﬁrst. Hence an IRE path forthe simultaneous move game is equivalent to a pair of IRE paths for the two20

alternating move games in one of which player 1 moves ﬁrst and in the otherof which player 2 moves ﬁrst.
The following result is a simple consequence of Theorem 4.1.
Theorem 5.1. Any IRE path {s1,t , s2,t }∞t=0 associated with (f1 , f2 ) for thesimultaneous move game satisﬁes∀t ≥ 2, ∀(i, j) ∈ Q,w∗ (fi )si,t = gi j(sj,t−1 ).

(5.6)

Furthermore, any IRE path {s1,t , s2,t }∞t=0 associated with (f1 , f2 ) for the alternating move game satisﬁes∀t ≥ 2, ∀(i, j) ∈ Q,w∗ (fi )t ∈ Ti ⇒ si,t = gi j(sj,t−1 ).

(5.7)

Proof. Consider the simultaneous move game. Let {s1,t , s2,t }∞t=0 be an IRE
path associated with (f1 , f2 ). Let (i, j) ∈ Q and t ≥ 2. Then sj,t−1 ∈ R(fj ).
w∗ (fi )Hence si,t = gi j (sj,t−1 ) by (5.3) and (4.2). Thus (5.6) follows. The proof
for the alternating move game is similar.
The above result shows that any IRE path is characterized by the corw∗ (f ) w∗ (f )responding pair of indiﬀerence curves (g1 2 1 , g2 1 2 ) except for the initialperiod. To better understand this result, consider the alternating move game.
The initial period must be excluded in (5.7) because s2,0 is an arbitrary initial condition that need not be optimal for player 2 given f1 , i.e., it need notw∗ (f )
satisfy s1,1 = g1 2 1 (s2,0 ). Since all subsequent actions must be individually
optimal, they must be on the optimal indiﬀerence curves. In Figure 7, any
IRE path satisﬁes the equality in (5.7) for all t ≥ 1. In Figure 5, by contrast,
an IRE path (not shown in the ﬁgure) violates the equality for t = 1 unlesss2,0 = ŝ2 , but trivially satisﬁes it for t ≥ 2.
Theorem 5.1 also shows that in both cases the dynamics of an IRE associated with (ω1 , ω2 ) ∈ R2 are essentially characterized by the same dynamicalsystem:(5.8)∀t ∈ T1 , s1,t+2 = g1ω2 (g2ω1 (s1,t )).
To be precise, the simultaneous move game has another equation, s2,t+2 =g2ω1 (g1ω2 (s2,t )) for t ∈ T2 , but this system is equivalent to (5.8) in terms ofdynamics. Hence one can obtain conditions for dynamic properties suchas monotonicity and chaos by applying numerous results available on onedimensional dynamical systems (e.g., Devaney, 1989).16
16

See Rand (1978) for an early example of complex dynamics in an “adaptive” dynamic

21

6

Characterizing IREs

Theorem 5.1 shows that the dynamics of an IRE are characterized by the
associated pair of indiﬀerence curves. The remaining question then is whatpairs of indiﬀerence curves are supported in IREs. To answer this question,
we need additional notation. For (i, j) ∈ Q and ωi , ωj ∈ R, deﬁneΨωi i = {(si , sj ) ∈ [0, 1]2 | wi (si , sj ) ≥ ωi }= {(si , sj ) ∈ [0, 1]2 | si ∈ D(gjωi ), sj ≥ gjωi (si )}.

(6.1)
(6.2)

The set Ψωi i is the collection of all pairs (si , sj ) with player i’s eﬀective payoﬀ
at least as large as ωi . In the (si , sj ) space, it is the area on or above thegraph sj = gjωi (si ); see Figure 4 in Section 3.
ωProvided Ψωi i ∩ Ψj j = ∅,17 deﬁne(ω ,ω )

ωsi i j = max{si ∈ [0, 1] | ∃sj ∈ [0, 1], (si , sj ) ∈ Ψωi i ∩ Ψj j },(ω ,ω )

(ω ,ωi )si i j = min{si ∈ D(gjωi ) | gjωi (si ) ≤ sj j
(ω ,ω )

(ω ,ω )

}.

(6.3)
(6.4)

(ω ,ω )By (6.2) and continuity, gjωi (si i j ) ≤ sj j i . Hence si i j exists as long asω(ω ,ω )
(ω ,ω )
Ψωi i ∩ Ψj j = ∅. See Figure 8. In the case of Figure 7, s1 1 2 = s2 2 1 = 1(ω ,ω )
(ω ,ω )and s1 1 2 = s2 2 1 = 0. It follows from Lemma 3.1(iii) that∀(i, j) ∈ Q,

(ω ,ω )

(ω ,ω )

si i j ≤ ŝi ≤ si i j .

(6.5)

See Figure 8 again.18 The following result characterizes all IREs in terms ofeﬀective payoﬀs.
model that has a structure similar to Figure 7. See Rosser (2002) for a recent survey ofadaptive duopoly/oligopoly models that generate complex dynamics. This paper does notconsider complex dynamics, which should be left to more specialized studies.
ω17iHere it is understood that the coordinates of Ψj j (or Ψωi ) are interchanged so thatωωij
Ψi and Ψj have the same order of the coordinates. Similar comments apply to similar
expressions below.
18
The ﬁrst inequality in (6.5) is immediate from (6.4) and Lemma 3.1(iii). To formallyverify the second inequality, let (i, j) ∈ Q and suppose si < ŝi . (We omit superscripts
here.) By (6.2) and (6.3), sj ≥ gj (si ) and si ≥ gi (sj ). Since gj is strictly decreasing at
si by Lemma 3.1(iii), both inequalities continue to hold even if si is slightly increased,
contradicting (6.3).

22

s2
1
(ω ,ω2 )s2 2

g2ω1 (s1 )Ψω1 1

f2 (s1 )

Ψω2 2

ŝ2

(ω ,ω1 )s2 2

f1 (s2 )0(ω ,ω2 )

s1 1

ŝ1

g1ω2 (s2 )(ω ,ω2 )

s1 1

1

s1

Figure 8: si , si , and IRE in regular form
Theorem 6.1. There exists an IRE associated with (ω1 , ω2 ) ∈ R2 if and only
if∀(i, j) ∈ Q,

Ψ1ω1 ∩ Ψω2 2 = ∅,(6.6)(ω ,ω )ωωŝj ∈ D(gi j ), si i j ≤ gi j (ŝj ).

(6.7)

In particular, under (6.6) and (6.7), (f1 , f2 ) ∈ F 2 is an IRE associated with
(ω1 , ω2 ) if for (i, j) ∈ Q,ω(ω ,ω )
ω
min{gi j (sj ), si i j } if sj ∈ D(gi j ),(6.8)
fi (sj ) =(ω ,ω )
si i jotherwise.
Proof. See Appendix A.
We say that an IRE satisfying (6.8) is in regular form. See Figure 8 foran example of an IRE in regular form. One can easily see that the example
satisﬁes (4.2) and thus is an IRE. We call (6.6) the nonemptiness condition,and (6.7) the no-sticking-out condition. The nonemptiness condition saysthat the intersection of the two sets Ψω1 1 and Ψω2 2 must be nonempty. Theω
no-sticking-out condition says that the graph of gi j must not “stick out” ofthe straight line si = si .
23

These conditions can be better understood by considering examples inwhich they are violated. In Figure 9(a), the nonemptiness condition (6.6) isviolated. In this case, if an IRE exists, any IRE path for the alternating movegame must behave like the path depicted in the ﬁgure (except for the initialperiod) by Theorem 5.1. But since such a path cannot stay on the indiﬀerence
curves forever, it cannot be an IRE path, a contradiction. In Figure 9(b), theno-sticking-out condition (6.7) is violated for (i, j) = (1, 2). In this case, if an(ω ,ω )
IRE exists, there is s2,0 such that f1 (s2,0 ) ≤ g1ω2 (s2,0 ) < s1 1 2 .19 As shownin the ﬁgure, the IRE path from such s2,0 cannot stay on the indiﬀerence
curves forever, contradicting Theorem 5.1.
We should mention that the IRE in regular form associated with (ω1 , ω2 ) ∈R2 is not the only IRE associated with (ω1 , ω2 ). In fact, fi (sj ) is arbitrary(ω ,ω ) (ω ,ω )for sj ∈ [sj j i , sj j i ] as long as it does not aﬀect R(fi ). However, any
IRE satisﬁes one restriction:
Proposition 6.1. Let (f1 , f2 ) be an IRE associated with (ω1 , ω2 ) ∈ R2 . Then∀(i, j) ∈ Q, ∀sj ∈ [0, 1],(ω ,ω )fi (sj ) ≤ si i j .

(6.9)

Proof. Immediate from (A.2), (A.12), and (6.3).
To see the idea of this result, suppose the inequality in (6.9) is violatedfor (i, j) = (1, 2). Consider the alternating move game. Then for some s2,0 ,(ω ,ω )
s1,1 = f1 (s2,0 ) > s1 1 2 . If this path is continued, it behaves like the one
depicted in Figure 8 by Theorem 5.1. But such a path cannot be an IRE
path since it cannot stay on the indiﬀerence curves forever.
Proposition 6.1 along with (6.8) implies that if (f1 , f2 ) is an IRE associated with (ω1 , ω2 ) ∈ R2 , and if (f 1 , f 2 ) is the IRE in regular form associatedwith (ω1 , ω2 ), then fi ≤ f i for i = 1, 2. In other words, in the IRE (f 1 , f 2 ),each player gives the other player the highest possible eﬀective payoﬀ consistent with an IRE associated with (ω1 , ω2 ) in response to any action by theother player. This becomes important when we discuss our folk-type theorem
in the next section.
In what follows, we say that an IRE (f1 , f2 ) is eﬀectively eﬃcient if there
is no IRE (f˜1 , f˜2 ) such that w1∗ (f2 ) ≤ w1∗ (f˜2 ) and w2∗ (f1 ) ≤ w2∗ (f˜1 ) with atleast one of them holding strictly. That is, (f1 , f2 ) is eﬀectively eﬃcient if it
19

The ﬁrst inequality holds by (3.20). In Figure 9(b), f1 (s2,0 ) = g1ω2 (s2,0 ).

24

s2
1

Ψω1 1

g2ω1 (s1 )

Ψω2 2

ŝ2

g1ω2 (s2 )

ŝ1

0

1

s1

(a)
s2
1
(ω ,ω )
s2 2 1

g2ω1 (s1 )

Ψω1 1

s2,0

Ψω2 2

ŝ2
g1ω2 (s2 )(ω ,ω1 )s2 2

0(ω ,ω2 )

s1 1

ŝ1

(ω ,ω2 )s1 1(b)Figure 9: Examples with no IRE

25

1

s1

s2
1
g2ω1 (s1 )Ψω1 1

f2 (s1 )
ŝ2

Ψω2 2

f1 (s2 )0

g1ω2 (s2 )

ŝ1

1

s1

Figure 10: Eﬀectively eﬃcient IRE
is not Pareto dominated by any other IRE in terms of eﬀective payoﬀs. As
shown in Section 8, eﬀective eﬃciency has important dynamic implications.
For (i, j) ∈ Q and ωi , ωj ∈ R, deﬁneΨ̃ωi i = {(si , sj ) ∈ [0, 1]2 | wi (si , sj ) > ωi }.

(6.10)

It is clear from Theorem 6.1 and Lemma 3.1(ii) that an IRE associated with
(ω1 , ω2 ) ∈ R2 is eﬀectively eﬃcient ifΨ̃1ω1 ∩ Ψ̃ω2 2 = ∅.

(6.11)

See Figures 10 and 9(a).
One might conjecture that (6.11) is also necessary for eﬀective eﬃciency.
Unfortunately it is not the case. This is because the no-sticking-out condition(6.7), a necessary condition for an IRE, is not stable under small perturbations to (ωi , ωj ). In other words, even when (6.11) does not hold, (6.7) canbe violated if either ωi or ωj is increased. For example, when (6.7) holds withequality for (i, j) = (1, 2), it can be violated after ω2 is slightly increased,
depending on how fast the two sides of the inequality in (6.7) vary with ω2 .
Even if (6.7) holds with strict inequality, (6.7) can be violated after small(ω ,ω )
perturbations to (ωi , ωj ), since si i j need not be continuous in (ωi , ωj ).
26

Figure 11 illustrates this point. There is an IRE in Figure 11(a), but there(ω ,ω )
is no IRE in Figure 11(b) due to violation of (6.7). Note that both s1 1 2(ω2 ,ω1 )are discontinuous in this example.20
and s2

7Eﬀective Eﬃciency and a Folk-Type Theorem: A Special Case

The anomaly in Figure 11 is largely due to the fact that the indiﬀerent curves
are unimodal there. The purpose of this section to characterize eﬀective
eﬃciency and to show a folk-type theorem under the assumption that bothindiﬀerence curves are “upward sloping.” More precisely, we focus on the casein which the following assumption holds.
Assumption 7.1. For i = 1, 2, ŝi = 0 or, equivalently, ui is strictly decreasing.21
This assumption holds, for example, in the prisonner’s dilemma gamein Subsection 2.2.4. More generally, it holds whenever an increase in si iscostly to player i but beneﬁcial to player j. Assumption 7.1 is maintained
throughout this section. The following result simpliﬁes Theorem 6.1 andfacilitates subsequent analysis.
Lemma 7.1. There exists an IRE (in regular form)22 associated with (ω1 , ω2 ) ∈R2 if and only if the nonemptiness condition (6.6) holds andω∀(i, j) ∈ Q,

0 ∈ D(gi j ).

(7.1)

Proof. By Theorem 6.1, it suﬃces to show that (7.1) is equivalent to the nosticking-out condition (6.7) under (6.6). By Assumption 7.1, (6.7) implies(7.1). Conversely, assume (6.6) and (7.1). Let (i, j) ∈ Q. By Assumption
20

Though in fact Figure 11 shows that the IRE in (a) is only “locally” eﬀectively eﬃcient,
it should be clear that one can easily construct a fully speciﬁed example of an eﬀectively
eﬃcient IRE that violates (6.11).
21
If one chooses to normalize vi in the opposite direction, i.e., if one chooses to assume
that vi is strictly decreasing for i = 1, 2 in Assumption 2.9, then the case considered here
corresponds to the case ŝi = 1 for i = 1, 2.
22
All the results stated in this section hold true with or without “in regular form.”
We include “(in regular form)” if the version with this qualiﬁcation is not an immediate
consequence of the version without it.

27

s2
1
(ω ,ω1 )s2 2

f1 (s2 )

Ψω1 1
Ψω2 2

ŝ2

(ω ,ω1 )s2 2

f2 (s1 )0ŝ1(ω ,ω2 )s1 1(ω ,ω2 )

s1 1

1

s1

(a)
s2
1

ŝ2

Ψω1 1

Ψω2 2

g2ω1 (s1 )g1ω2 (s2 )

0

ŝ1

1

s1

(b)
Figure 11: Eﬀectively eﬃcient IRE violating (6.11)28

(ω ,ω )ω(ω ,ω )

ω

7.1 and (6.5), si i j = 0. Since 0 ∈ D(gi j ), we have si i j = 0 ≤ gi j (0) =ω
gi j (ŝj ) by Assumption 7.1. Now (6.7) follows.
The above proof shows that under (6.6) and (7.1), the inequality in the
no-sticking-out condition (6.7) automatically holds. This implies that if an
IRE exists such that Ψ̃ω1 1 ∩ Ψ̃ω2 2 = ∅, then an IRE continues to exist when bothindiﬀerence curves are slightly shifted upward. Therefore an IRE cannot be
eﬀectively eﬃcient if Ψ̃ω1 1 ∩ Ψ̃ω2 2 = ∅. This is the idea of the following result.
Theorem 7.1. Suppose an IRE associated with (ω1 , ω2 ) ∈ R2 exists. Then
it is eﬀectively eﬃcient if and only if (6.11) holds, i.e., Ψ̃ω1 1 ∩ Ψ̃ω2 2 = ∅.
Proof. The “if” part is obvious, as mentioned earlier. To see the “only if”
part, suppose there is an IRE associated with (ω1 , ω2 ) ∈ R2 that is eﬀectively
eﬃcient. Suppose (6.11) does not hold, i.e.,Ψ̃1ω1 ∩ Ψ̃ω2 2 = ∅.

(7.2)

Since for (i, j) ∈ Q, gjωi (·) is strictly increasing by (3.10) and Assumption 7.1,
(7.1) and (7.2) imply 0 ≤ gjωi (0) < 1 for (i, j) ∈ Q. Since gjωi is continuous andstrictly increasing in ωi by (3.10), it follows that there is (ω̃1 , ω̃2 )  (ω1 , ω2 )such that 0 < gjω̃i (0) < 1 and Ψ̃1ω̃1 ∩ Ψ̃2ω̃2 = ∅ for (i, j) ∈ Q. Hence (6.6) and(7.1) hold with ω̃1 and ω̃2 replacing ω1 and ω2 . But this implies that thegiven IRE cannot be eﬀectively eﬃcient, a contradiction.
Let us now turn to the development of our folk-type theorem. The following result gives an alternative characterization of IREs that proves useful.
Lemma 7.2. There exists an IRE (in regular form) associated with (ω1 , ω2 ) ∈R2 if and only if there exists (s1 , s2 ) ∈ [0, 1]2 such that∀(i, j) ∈ Q,

ωi = wi (si , sj ),(7.3)

w (0,0)w (0,0)∩ Ψ2 2.
(s1 , s2 ) ∈ Ψ1 1

(7.4)

Proof. First we observe from (3.7), (6.1), and Lemma 3.1(ii) that
(7.3)
(7.3)

⇔

∀(i, j) ∈ Q, si ∈ D(gjωi ), sj = gjωi (si ),

(7.5)

⇒

(s1 , s2 ) ∈ Ψω1 1 ∩ Ψω2 2 .

(7.6)

If: Let (ω1 , ω2 ) ∈ R2 . Suppose there exists (s1 , s2 ) ∈ [0, 1]2 satisfying(7.3) and (7.4). Then by (7.6), the nonemptiness condition (6.6) holds. By
29

Lemma 7.1, it suﬃces to verify (7.1). Let (i, j) ∈ Q. By (7.4) and (6.1),
wi (si , sj ) ≥ wi (0, 0). Since ui is strictly decreasing by Assumption 7.1, ωi =wi (si , sj ) ≤ wi (0, 1). It follows thatui (0) + δi vi (0) ≤ ωi ≤ ui (0) + δi vi (1),(7.7)

which can be written as
ωi − δi vi (1) ≤ ui (0) ≤ ωi − δi vi (0).

(7.8)

Hence (7.1) holds by (3.9).
Only if: Let there be an IRE associated with (ω1 , ω2 ) ∈ R2 . By (7.1),
gjωi (0) ≥ 0 for (i, j) ∈ Q. Thus if the graphs of g2ω1 and g1ω2 have no intersection, then the nonemptiness condition (6.6) does not hold, a contradiction.
Hence the graphs of g2ω1 and g1ω2 have an intersection (s1 , s2 ) ∈ [0, 1]2 , i.e.,sj = gjωi (si ) for (i, j) ∈ Q. Thus (7.3) holds by (7.5). We have (7.4) by (7.3),(7.6), (4.3), (3.17), Assumption 7.1, and (6.1).
w (0,0)w (0,0)See Figure 12 for an illustration of Ψ1 1
∩ Ψ2 2. Note that both
w2 (0,0)w1 (0,0)
indiﬀerence curves g1
and g2emanate from the origin because theycorrespond to the eﬀective payoﬀs associated with the action proﬁle (0, 0).
Given an IRE (f1 , f2 ), we say that (s1 , s2 ) ∈ [0, 1]2 is a steady state if
s1 = f1 (s2 ) and s2 = f2 (s1 ). In other words, any intersection of f1 andf2 is a steady state. Needless to say, the IRE path starting from a steadystate remains there forever. Thus by Theorem 5.1, any steady state is anintersection of the associated indiﬀerence curves.
Proposition 7.1. There exists an IRE (in regular form) such that (s1 , s2 ) ∈[0, 1]2 is a steady state if and only if (7.4) holds.
Proof. Let there be an IRE such that (s1 , s2 ) ∈ [0, 1]2 is a steady state.
Deﬁne ω1 and ω2 by (7.3). Then this IRE is associated with (ω1 , ω2 ). Thus
(7.4) holds by Lemma 7.2. Conversely, let (s1 , s2 ) ∈ R2 satisfy (7.4). Deﬁneω1 and ω2 by (7.3). By Lemma 7.2, an IRE (f1 , f2 ) in regular form associatedwith (ω1 , ω2 ) exists. Thus (6.6) and (6.7) hold by Theorem 6.1. By (7.5), (a)(ω ,ω )
sj = gjωi (si ) for (i, j) ∈ Q. Since (s1 , s2 ) ∈ Ψω1 1 ∩ Ψω2 2 by (7.6), (b) si ≤ si i j(ω ,ω )for (i, j) ∈ Q. Hence by (a) and (b), for (i, j) ∈ Q, gjωi (si ) = sj ≤ sj j i ,so fj (si ) = gjωi (si ) by (6.8). This together with (a) shows that (s1 , s2 ) is a
steady state.
30

s2
1
g1w2 (s2 )|δ2 =1

g1w2 (s2 )g2w1 (s1 )w21Ψw
1 ∩ Ψ2

g2w1 (s1 )|δ1 =1

0

1
w (0,0)s1

w (0,0)Figure 12: Ψ1 1
∩ Ψ2 2and lower bounds of Ψ̃∗i(wi = wi (0, 0) in this ﬁgure)w (0,0)w (0,0)

By this result, the set Ψ1 1
∩ Ψ2 2can be viewed as the collectionof all steady states supported by IREs. For (i, j) ∈ Q, deﬁneΨ̃∗i = {(si , sj ) ∈ [0, 1]2 | ui (si ) + vi (sj ) > ui (0) + vi (0)}.

(7.9)

Since ui (0) + vi (0) is player i’s minimax payoﬀ in the one-shot game, Ψ̃∗1 ∩ Ψ̃∗2
may be called the set of “strictly individually rational” action proﬁles without
randomization.
Note that the deﬁnition of Ψ̃ωi i in (6.10) becomes identical to (7.9) if
δi = 1 and ωi = ui (0) + vi (0). It follows from (3.10) that for (i, j) ∈ Q,ui (0) − ui (si )wi (0,0)wi (0,0)−1), gj(si ) = vi∀si ∈ D(gj.
(7.10)
vi (0) +δiw (0,0)

shifts downward as δi increases; see Figure 12. The
Thus the graph of gj iw (0,0)w (0,0)idea of our folk-type theorem is that as δi ↑ 1, Ψ̃1 1∩ Ψ̃2 2“converges”∗∗∗∗
to Ψ̃1 ∩ Ψ̃2 , so by Proposition 7.1, any point in Ψ̃1 ∩ Ψ̃2 can be supported asa steady state of an IRE for (δ1 , δ2 ) suﬃciently close to (1, 1).
Theorem 7.2. Let(s1 , s2 ) ∈ Ψ̃∗1 ∩ Ψ̃∗2 .
31(7.11)

Then for (δ1 , δ2 )  (1, 1) suﬃciently close to (1, 1), there exists an IRE (inregular form) such that (s1 , s2 ) is a steady state.
Proof. Assume (7.11). By (7.9), for (δ1 , δ2 ) close enough to (1, 1),∀(i, j) ∈ Q,

ui (si ) + δi vi (sj ) > ui (0) + δi vi (0),(7.12)so (7.4) holds by (6.1). Now the theorem follows by Proposition 7.1.
Theorem 7.2 of course holds true without “in regular form,” but in thatcase, it can easily be shown by using an IRE based on Nash reversion;23
recall Figure 6. Part of the signiﬁcance of Theorem 7.2 lies in the fact that
IREs in regular form are continuous. In this regard the result is similar in
spirit to the folk-type theorems by Friedman and Samuelson (1994a, 1994b),
which show that the main idea of the standard folk theorem (Fudenberg and
Maskin, 1986) is valid even if one conﬁnes oneself to continuous equilibria.
Our result shows that any (s1 , s2 ) ∈ Ψ̃∗1 ∩ Ψ̃∗2 is supported as a steady stateof a continuous IRE for (δ1 , δ2 ) suﬃciently close to (1, 1).
In addition, it should be noted that a deviation is punished in a minimal
way in IREs in regular form. Indeed, after the initial period, player i isindiﬀerent between conforming to the current IRE path and choosing any(ω ,ω )
(ω ,ω )si ∈ [0, si i j ] (recall that si i j = 0 here by the proof of Lemma 7.1).
This suggests that sever punishment may not be necessary for maintaining
a subgame perfect equilibrium or even for establishing a folk-type theorem.

8

Applications

8.1

Prisoner’s Dilemma

Consider the alternating move game associated with the prisoner’s dilemma
game in Subsection 2.2.4.24 For simplicity, we assume directly that the oneshot payoﬀ of player i is given by (2.5),25 and that both players have the23

We owe this observation to an anonymous referee.
The simultaneous move game can be analyzed similarly; recall Proposition 6.1 andSection 5.
25
Alternatively one may assume that player i’s mixed action in period t ∈ Ti is observableto player j at the beginning of period t + 1. In this case, the expected one-shot payoﬀ ofplayer i in period t is −asi,t + (c + a)rj,t−1 , where si,t is player i’s probability of choosing
C, and rj,t−1 is player j’s realized action in period t − 1. Since rj,t−1 does not aﬀect player
24

32

same discount factor: δ1 = δ2 = δ ∈ (0, 1). The eﬀective payoﬀ of player i isgiven by
(8.1)
wi (si , sj ) = −asi + δesj ,where e = c + a. Replacing wi (si , sj ) with ωi and solving for sj , we see thatthe indiﬀerence curve associated with ω i ∈ R, or gjωi , is linear:aωi+ si .
(8.2)
δe δe
Since Assumption 7.1 holds here, all the results in Section 7 apply.
We consider three cases separately. First suppose δ < a/e, i.e., the slopeωof gjωi is strictly greater than one. By (7.1), gi j (0) ≥ 0 for (i, j) ∈ Q in any
ωIRE. Thus if gi j (0) > 0 for either (i, j) ∈ Q, the nonempiness condition (6.6)ω
will be violated; see Figure 13. Hence in any IRE, gi j (0) = 0 for (i, j) ∈ Q.
It follows that there is a unique IRE, which corresponds to the static Nashequilibrium, i.e., fi (sj ) = 0 for all sj ∈ [0, 1] and (i, j) ∈ Q. This is because(0,0)= 0 for all sj ∈ [0, 1] and (i, j) ∈ Q. See
by Proposition 6.1, fi (sj ) ≤ si
Figure 13 again. This IRE is eﬀectively eﬃcient by Theorem 7.1 (or simply
by uniqueness).
Now suppose δ = a/e, i.e., the slope of gjωi is equal to one. In this knifeedge case, the two indiﬀerence curves emanating from the origin coincide.
ωThe above argument still shows gi j (0) = 0 for (i, j) ∈ Q. Though, as in theprevious case, there is an IRE corresponding to the static Nash equilibrium,
there are many other IREs here. Figure 14 shows one example.
Finally, suppose δ > a/e, i.e., the slope of gjωi is strictly less than one.
In this case, there are many pairs of eﬀective payoﬀs supported by IREs. A“typical” case is depicted in Figure 15, where there is a unique and globallystable steady state. The existence of a unique and globally stable steadystate is a general property of this case by (5.8) and (8.2).
Figure 16 shows a symmetric IRE that is eﬀectively eﬃcient. In thiscase, gradual cooperation occurs, and full cooperation is achieved in the longrun.26 Figure 17 shows an eﬀectively eﬃcient IRE in which uneven gradualcooperation occurs: only player 2 fully cooperates in the long run, and player
1 enjoys the highest possible eﬀective payoﬀ.
gjωi (si ) =i’s preferences over his actions from period t onward, all our results hold in this case. This
argument is unnecessary for the simultaneous move game, where rj,t−1 must be replacedby sj,t .
26
Gradual cooperation is known to arise in certain partnership games; see Furusawa and
Kawakami (2006) and the references therein.

33

s2
1
w (0,0)

g1 2

(s2 )

f1 (s2 )w (0,0)

g2 1

(s1 )
f2 (s1 )0

1

s1

Figure 13: Prisoner’s dilemma with δ < a/es2
1f1 (s2 )f2 (s1 )0

1

s1

Figure 14: Prisoner’s dilemma with δ = a/e

34

s2
1
s2,0
f1 (s2 )f2 (s1 )

s2,0
0

1

s1

Figure 15: Prisoner’s dilemma with δ > a/e: “typical” case
By Theorem 7.2, any strictly individually rational action proﬁle (s1 , s2 ),which by deﬁnition satisﬁes −asi + esj > 0 for (i, j) ∈ Q, is supported asa steady state of an IRE for δ suﬃciently close to one. Notice that the setof payoﬀ proﬁles supported by strictly individually rational action proﬁles isconvex here, so that this set coincides with the set of “strictly individually
rational” payoﬀ proﬁles (Fudenberge and Maskin, 1986). Thus our folk-type
theorem coincides with the standard folk theorem in this example.27

8.2

Duopoly

Consider the alternating move game associated with the duopoly game of
Subsection 2.2.3.28 For simplicity we assume that the ﬁrms are symmetric.
Let c and δ denote their common marginal cost and discount factor. Recall
that ﬁrm i’s one-shot proﬁt is given by Di (pi , pj )(pi − c). We parametrize Di
as follows:(8.3)∀(i, j) ∈ Q, Di (pi , pj ) = (p − pi )pj ,27

See Stahl (1991) for a characterization of subgame perfect correlated equilibria of amore general repeated prisoners’ dilemma game.
28
Once again, the dynamics of the simultaneous move game can be analyzed similarly.

35

s2
1

f2 (s1 )f1 (s2 )

s2,0

0

1

s1

1

s1

Figure 16: Gradual cooperations2
1

f2 (s1 )s2,0
f1 (s2 )0

Figure 17: Uneven gradual cooperation36

where p > c. Let (i, j) ∈ Q. Recalling (2.3), we see that the eﬀective payoﬀ
of ﬁrm i is given bywi (pi , pj ) = ln(p − pi ) + ln(pi − c) + δ ln pj .

(8.4)

Replacing wi (pi , pj ) with ωi and solving for pj , we obtaingjωi (pi ) = exp[{ωi − ln(p − pi ) − ln(pi − c)}/δ].

(8.5)

Note that gjωi (c) = gjωi (p) = ∞. Direct calculation of the second derivative
shows that gjωi is strictly convex. It is easy to see that given pj , ﬁrm i’s oneshot proﬁt, as well as its eﬀective payoﬀ, is maximized at pi = p̂ ≡ (c + p)/2.
This is the price charged by both ﬁrms in the unique static Nash equilibrium.
Figure 18 illustrates a symmetric IRE in which both ﬁrms receive the effective payoﬀ corresponding to the static Nash equilibrium. The indiﬀerence
curves in this ﬁgure are similar to those in Figure 5, which shows the IRE
corresponding to the static Nash equilibrium. Figure 18 shows an alternative IRE (which is in regular form). In this IRE, there is a steady state inwhich both ﬁrms charge the static Nash price, as in Figure 5. In Figure 18,
however, there is another steady state with a higher symmetric price. At thissteady state, each ﬁrm faces a “kinked demand curve.” If one of the ﬁrms
raises its price, the other does not follow. Proposition 6.1 implies that thiskinked feature is a rather general property in the sense that in any IRE, theﬁrms never charge prices higher than those given by the highest intersectionof the two indiﬀerence curves. On the other hand, if one of the ﬁrms lowers
its price, this triggers price war, and the prices converge to the lower steadystate. Figure 18 shows an example of an IRE path after a small price cut byﬁrm 2 in period 0 (which is taken as the initial condition of the model).
Clearly the above properties of the two steady states continue to holdeven if the ﬁrms receive higher eﬀective payoﬀs, as long as there are twosteady states. It is easy to see that there can be at most two steady statesby strict concavity of gjωi , provided that the ﬁrms receive eﬀective payoﬀs
at least as large as the level associated with the static Nash equilibrium.
Note that eﬀective payoﬀs higher than the static Nash level correspond toindiﬀerence curves higher than those depicted in Figure 18.
If there is only one steady state, then the IRE is eﬀectively eﬃcient by(6.11). Figure 19 illustrates a symmetric, eﬀectively eﬃcient IRE in regularform. At the unique steady state, each ﬁrm faces a kinked demand curve once
again. This steady state, however, is globally stable. If one of the ﬁrms raises
37

p

p2

s2,0

f2 (s1 )p̂

f1 (s2 )
c

c

p̂

p

p1

Figure 18: Kinked demand curves with unstable collusion
its price, the other does not follow, as in Figure 18. If one of them lowers
its price, the other lowers its price too but by a smaller degree. Eventually
the prices return to the initial high levels. This process is shown in Figure
19 assuming that ﬁrm 2 cuts its price to the static Nash level in period 0. It
follows from Theorem 5.1 that the global stability of the unique steady stateis a general property of any eﬀectively eﬃcient IRE here.

9

Concluding Comments

This paper oﬀers a complete and graphical characterization of immediately
reactive equilibria (IREs) and their global dynamics for inﬁnitely repeatedgames with two players in which the action space of each player is an interval,
and the one-shot payoﬀ of each player consists of two continuous functions,
one unimodal in his own action, the other strictly monotone in the otherplayer’s action. IREs extend Nash reversion equilibria by allowing for continuous strategies and nontrivial dynamics. Characterized by two indiﬀerence
curves, the global dynamics of an IRE can be analyzed graphically. Though
IREs constitute only a small subset of the subgame perfect equilibria, thestructure of IREs is rich enough to allow us to show a folk-type theorem ina special case.
38

p

p2

f2 (s1 )s2,0

f1 (s2 )
c

c

p̂

p

p1

Figure 19: Kinked demand curves with stable collusion
Although additive separability, which is crucial to our analysis, is rather
restrictive, there are various interesting games that satisfy it. We have analyzed two such games and characterized their IREs by applying our general
results. We have shown among other things that gradual cooperation arises
in an eﬀectively eﬃcient IRE of a prisoners’ dilemma game, and that kinked
demand curves with stable collusion emerge in an eﬀectively eﬃcient IRE of
a duopoly game.
We believe that our results are useful not only in analyzing games thatsatisfy our assumptions, but also in constructing completely tractable special cases of more general games. Such special cases, whose dynamics canbe analyzed explicitly, would enhance understanding of various interesting
problems.

Appendix A

Proof of Theorem 6.1

Throughout the proof, we omit the superscripts ωi , ωj , (ωi , ωj ), and (ωj , ωi ).

A.1

Suﬃciency

The “if” part of the proposition follows from the following.
39

Lemma A.1. Let ω1 , ω2 ∈ R satisfy (6.6) and (6.7). Then the strategy
proﬁle (f1 , f2 ) given by (6.8) is an IRE associated with (ω1 , ω2 ).
Proof. It suﬃces to show R(fi ) ⊂ Mi (fj ) for (i, j) ∈ Q by Theorem 4.1. Let(i, j) ∈ Q. By (3.21) and (6.8),
Mi (fj ) = {si ∈ D(gj ) | gj (si ) ≤ sj }.

(A.1)

By (6.3) and (6.2), gj (si ) ≤ sj , so si ∈ Mi (fj ). By (6.4), gj (si ) ≤ sj , sosi ∈ Mi (fj ). Let si ∈ R(fi ). By (6.7), Lemma 3.1(iii), and (6.8), si ≤gi (ŝj ) ≤ si ≤ si . Thus si ∈ Mi (fj ) since si , si ∈ Mi (fj ) and Mi (fj ) is aninterval by (A.1) and Lemma 3.1(iii). It follows that R(fi ) ⊂ Mi (fj ).

A.2

Necessity

We show the “only if” part in a few steps. Throughout we take as given an
IRE (f1 , f2 ) associated with (ω1 , ω2 ) ∈ R2 . For (i, j) ∈ Q, deﬁne
ri = inf R(fi ), ri = sup R(fi ),r̃i = min{si ∈ D(gj ) | gj (si ) ≤ rj }.

(A.2)
(A.3)

Lemma A.2. For (i, j) ∈ Q,
ri , ri ∈ D(gj ),gj (ri ), gj (ri ) ∈ D(gi ),ri ≤ gi (gj (ri )), ri ≥ gi (gj (rj )).

(A.4)
(A.5)
(A.6)

Proof. Let (i, j) ∈ Q. Recall from (3.22) that
R(fi ) ⊂ Mi (fj ) ⊂ D(gj ).

(A.7)

Since D(gj ) is closed by (3.17) and Lemma 3.1(i), (A.4) follows from (A.7)
and (A.2). Note from (4.2) and (A.7) that∀sj ∈ R(fj ),fi (sj ) = gi (sj ).

(A.8)

ri ≥ sup gi (sj ).

(A.9)

Hence
ri ≤

inf

sj ∈R(fj )gi (sj ),sj ∈R(fj )40

By (A.8) and (A.7) (with i and j interchanged),∀si ∈ R(fi ),gj (si ) = fj (si ) ∈ R(fj ) ⊂ D(gi ).

(A.10)

Thus (A.5) follows by continuity of gj and closeness of D(gi ).
To see (A.6), let si ∈ R(fi ). By (A.10), gj (si ) ∈ R(fj ). Hence by (A.9),
ri ≤ gi (gj (si )) and ri ≥ gi (gj (si )). Thus (A.6) follows by continuity of gi andgj .
When (i, j) ∈ Q is given, we interchange the coordinates in Ψj so that Ψiand Ψj have the same coordinates, i.e., we redeﬁneΨj = {(si , sj ) ∈ [0, 1]2 | sj ∈ D(gi ), si ≥ gi (sj )}.

(A.11)

This is identical to (6.2) with i and j interchanged, except for the order ofthe coordinates; recall footnote 17.
Lemma A.3. For (i, j) ∈ Q,
(ri , gj (ri )) ∈ Ψi ∩ Ψj ,(a) ŝj ∈ D(gi ), (b) r̃i ≤ gi (ŝj ).

(A.12)(A.13)

Proof. Let (i, j) ∈ Q. Since ri ∈ D(gj ) by (A.4), we have (ri , gj (ri )) ∈ Ψi by(6.2) (with sj = gj (ri ) and si = ri ). By (A.5) and (A.6), gj (ri ) ∈ D(gi ) andri ≥ gi (gj (ri )). So by (A.11), (ri , gj (ri )) ∈ Ψj . Thus (A.12) follows.29
It remains to show (A.13). Note that (A.13)(a) is immediate from (3.17)
and Lemma 3.1(i). By (A.2), fj (si ) ≤ rj for si ∈ [0, 1]. Thus by (3.21),∀si ∈ Mi (fj ),gj (si ) = fj (si ) ≤ rj .

(A.14)

Hence Mi (fj ) ⊂ {si ∈ D(gj ) | gj (si ) ≤ rj } ≡ B. Thus by (3.20) and (4.1),
gi (ŝj ) ≥ fi (ŝj ) ∈ Mi (fj ) ⊂ B.

(A.15)

Since r̃i = min B by (A.3), (A.13)(b) follows.
Let us now complete the “only if” part of the proof. We have (6.6) by(A.12). Let (i, j) ∈ Q. By (A.12) and (6.3), rj ≤ sj . Thus the set in (6.4)
includes the set in (A.3), so sj ≤ r̃j . This together with (A.13) shows (6.7).
29

The corresponding result for ri does not hold since ri cannot be replaced by ri in(ri , gj (ri )) ∈ Ψj unless ri = gi (gj (ri )); recall (A.6).

41

References
Abreu, D., 1986, Extremal equilibria of oligopolistic supergames, Journal of
Economic Theory 39, 191–225.
Abreu, D., 1988, On the theory of inﬁnitely repeated games with discounting, Econometrica 56, 383–396.
Bhaskar, V., F., Vega-Redondo, 2002 , Asynchronous choice and Markov equilibria, Journal of Economic Theory 103, 334–350.
Bhaskar, V., S. Machin, G. Reid, 1991, Testing a model of the kinked demand curve, Journal of Industrial Economics 39, 241–154.
Corchon, L., 1994, Comparative statics for aggregative games: the strong
concavity case, Mathematical Social Sciences 28, 151–165.
Devaney, R.L., 1989, An Introduction to Chaotic Dynamical Systems, 2nd
ed. Redwood City: Addison-Wesley.
Friedman, J.W., 1968, Reaction functions and the theory of duopoly, Review
of Economic Studies 35, 257–272.
Friedman, J.W., 1971, A non-cooperative equilibrium for supergames, Review of Economic Studies 38, 1–12.
Friedman, J.W., 1973, On reaction function equilibria, International Economic Review 14, 721–734.
Friedman, J.W., 1976, Reaction functions as Nash equilibria, Review of Economic Studies 43, 83–90.
Friedman, J.W., L. Samuelson, 1990, Subgame perfect equilibria with continuous reaction functions, Games and Economic Behavior 2, 304–324.
Friedman, J.W., L. Samuelson, 1994a, Continuous reaction functions in duopolies,
Games and Economic Behavior 6, 55–82.
Friedman, J.W., L. Samuelson, 1994b, An extension of the “folk theorem”
with continuous reaction functions, Games and Economic Behavior 6,
83–96.
Fudenberg, D., E. Maskin, 1986, The folk theorem in repeated games withdiscounting or with incomplete information, Econometrica 54, 533–554.
Fudenberg, D., J. Tirole, 1991, Game Theory. Cambridge, MA: MIT Press.
Furusawa, T., T. Kamihigashi, 2006, Threats and concessions in tariﬀ setting, work in progress.

42Furusawa, T., T. Kawakami, 2006, Gradual cooperation in the existence ofoutside options, Journal of Economic Behavior and Organization, forthcoming.
Green, E.J., R.H. Porter, 1984, Noncooperative collusion under imperfectprice information, Econometrica 52, 87–100.
Haller, H., R. Lagunoﬀ, 2000, Markov perfect equilibria in repeated asynchronous choice games, working paper, Georgetown University.
Kalai, E., D. Samet, W. Standford, 1988, A note on reactive equilibria inthe discounted prisoner’s dilemma and associated games, International
Journal of Game Theory 17, 177–186.
Kamihigashi, K., S. Roy, 2006, Dynamic optimization with a nonsmooth,nonconvex technology: the case of a linear objective function, Economic
Theory 29, 325–340.
Kranton, R.E., 1996, The formation of cooperative relationships, Journal ofLaw, Economics, and Organization 12, 214–233.
Langlois, J.-P., J.A. Sachs, 1993, Existence and local stability of Pareto superior reaction function equilibria in discounted supergames, Journal of
Mathematical Economics 22, 199–221.
Lagunoﬀ, R., A. Matsui, 1997, Asynchronous choice in repeated coordination games, Econometrica 65, 1467–1477.
Maskin, E., J. Tirole, 1987, A theory of dynamic oligopoly, III: Cournot
competition, European Economic Review 31, 947–968.
Maskin, E., J. Tirole, 1988a, A theory of dynamic oligopoly, I: overview andquantity competition with large ﬁxed costs, Econometrica 56, 549–569.
Maskin, E., J. Tirole, 1988b, A theory of dynamic oligopoly, II: price competition, kinked demand curves, and edgeworth cycles, Econometrica 56,
571–599.
Maskin, E., J. Tirole, 2001, Markov perfect equilibrium I: observable actions,
Journal of Economic Theory 100, 191–291.
Radner, R., 2003, Viscous demand, Journal of Economic Theory 112, 189–
231.
Rand, D., 1978, Exotic phenomena in games and duopoly models, Journalof Mathematical Economics 5, 173–184.
Robson, A., 1986, The existence of Nash equilibria in reaction functions for
dynamic models of oligopoly, International Economic Review 27, 539–
544.
43

Rosser, J.B., Jr., 2002, The development of complex oligopoly dynamics theory. In: T. Puu, I. Sushko, (Eds.), Oligopoly Dynamics: Models andTools, Springer-Verlag, Berlin, pp. 15–29.
Sabourian, H., 1992, Rational conjectural equilibrium and repeated games.
In: P. Dasgupta, D. Gale, O. Hart, E. Maskin, (Eds.), Economic Analysis
of Markets and Games: Essays in Honor of Frank Hahn, pp. 228–257.
Samuelson, L., Non-trivial subgame perfect duopoly equilibria can be supported by continuous reaction functions, Economics Letters 24, 207–211.
Sen, D., 2004, The kinked demand curve revisited, Economics Letters 84,
99–105.
Stahl, D.O., II, 1991, The graph of prisoners’ dilemma supergame payoﬀs
as a function of the discount factor, Games and Economic Behavior 3,
368–384.
Stanford, W.G., 1986, Subgame perfect reaction function equilibria in discounted duopoly supergames are trivial, Journal of Economic Theory 39,
226–232.
Tidball, M., A. Jean-Marie, N. Querou, 2000, Conjectural variation equilibrium, reaction functions, and indiﬀerence curves: a survey, working paper
2002-7, LAMETA.
Yoon, K., 2001, A folk theorem for asynchronously repeated games, Econometrica 69, 191–200.
Watson, J., 1999, Starting small and renegotiation, Journal of Economic
Theory 85, 52–90.
Watson, J., 2002, Starting small and commitment, Games and Economic
Behavior 38, 176–199.

44