Gradualism and Rough Transition with Reference Dependence∗†

Takashi Onoda‡
Japan Bank for International Cooperation

August 9, 2024
Abstract
I study cooperation among reference-dependent, loss-averse players in a dynamic game with complete information. Each period, the players choose theircooperation levels, which determine their intrinsic payoffs, and update theirreference points in a backward-looking manner. I characterize the subgameperfect equilibrium that maximizes utility with Nash reversion, demonstratingthat the optimal cooperation development exhibits gradualism. After initiating
cooperation, the players experience higher payoffs than their initial reference
points. Consequently, the reference points rise, making the penalty for deviation more severe and enabling further cooperation. Additionally, I illustrate
how the developed cooperation responds to a structural shock. When a steadystate cooperation level shifts downward, transitioning to the new level entails aloss. This loss generates an additional deviation incentive, forcing the players
to undergo cooperation levels lower than the new steady-state level.
Keywords: Gradualism, Starting small, Reference dependence, Loss aversion, Long-term relationship, Trade liberalization JEL classification: C72, C73,
D91∗

I am deeply indebted to Michihiro Kandori and Ichiro Obara for their invaluable guidance andsupport. I also thank Rodrigo Adão, Jonathan Dingel, Taiji Furusawa, Gene Grossman, Robert Gulotty, Sota Ichiba, Fuhito Kojima, Elliot Lipnowski, Takeshi Murooka, Shunya Noda, Felix Tintelnot,
Yuichi Yamamoto, Kai Hao Yang, and seminar participants at the CIRJE and UTMD workshop at
the University of Tokyo, the DC conference at Osaka University, the Contract Theory Workshop,
and the International Trade Working Group at the University of Chicago.
†
An earlier version of this paper was circulated as “Gradual Development and Rough Transitionof Cooperation with Reference Dependence.”‡
Email: onoda.takashi@gmail.com

1

1

Introduction

Standard models explain cooperation among non-cooperative agents as an equilibriumoutcome in which they compare expected utilities from cooperation and deviation. In
the real world, agents are reference-dependent and loss-averse; they evaluate economic
returns against their reference points (Kahneman and Tversky, 1979) with a strong
aversion to losses—payoffs lower than the reference point. Can this property explain
phenomena that standard models cannot?
In this paper, I examine cooperation among reference-dependent agents and achieve
two aims. First, this paper shows that the optimal level of cooperation for referencedependent agents gradually develops over time, resembling gradualism or “starting
small,” which is prevalent in cooperative relationships. Second, it illustrates how thedeveloped cooperation responds to changes in the economic environment. Specifically, this paper demonstrates that when players must scale back their cooperation
to a new steady state, they temporarily experience cooperation levels lower than the
new steady state.
I investigate an infinite horizon dynamic game with complete information in which
players have reference points. The players are symmetric, choosing their cooperationlevels individually and simultaneously every period. Their flow utility consists ofan intrinsic payoff and a loss utility. The intrinsic payoff is a function of all theplayers’ levels of cooperation and exhibits prisoner’s-dilemma-type properties; thefirst-best cooperation is infeasible in a one-shot game because deviation is lucrative,
and a static Nash equilibrium yields low cooperation. The loss utility becomes nonzero and negative only if the intrinsic payoff is below the player’s reference point,reflecting loss aversion. The reference point is backward-looking, as in DellaVigna
et al. (2017) and Bowman et al. (1999). The reference point is updated every period
as the weighted average of the previous value and the latest intrinsic payoff, following
Bowman et al. (1999).

2

Throughout the paper, I consider subgame-perfect equilibria (SPEs) in which theplayers employ a Nash-reversion strategy; when a player deviates from a cooperative
level of cooperation, the others impose a static Nash equilibrium that generates low
intrinsic payoffs in the following periods as a penalty. An infinite number of SPEs
exist, but I focus on the one that maximizes the players’ present value of utility.
Reference dependence generates three forces that influence the development andtransitions of cooperation. The first is cooperation loss aversion, the only force at workwhen players start cooperation with low initial reference points. Once they receive
intrinsic payoffs higher than their reference points in return for their cooperation,
their reference points rise. Consequently, the penalty would cause a loss, inflicting
loss utilities in the deviation path. Thus, the experience of a high level of cooperation
makes the players averse to losing it—an additional deterrent to deviation.
Cooperation develops gradually as the cooperation loss aversion strengthens overtime. Experiencing high payoffs raises the players’ reference points, generating thecooperation loss aversion. This aversion disincentives the players to deviate, enablingthem to implement a higher level of cooperation in the next period. The resulting
higher cooperation produces higher payoffs, elevating the reference points further. In
turn, these elevated reference points widen the hypothetical losses after deviation,
strengthening the cooperation loss aversion. Consequently, an even higher level ofcooperation becomes possible in the following periods. Repeating this process results
in a gradual development of cooperation, eventually converging to a steady state.
When the economic environment changes unfavorably, the developed cooperation
faces two additional forces. When considering players whose cooperation has reacheda steady state and who have reference points at that level, a structural shock that
reduces intrinsic payoffs from a given level of cooperation implies a decline of thesteady-state payoff. Thus, the players would incur losses from the new steady-state
payoff. This situation generates two forces. First, the loss in the current period
incentivizes the players to avoid the loss, augmenting the incentive to deviate. Second,3

if loss utilities exist in future periods, they reduce the value of future cooperative
levels of cooperation, discouraging cooperation. These two effects counteract thecooperation loss aversion.
The net effect necessitates that players undergo cooperation levels lower than the
new steady-state level. Thus, a reference point higher than the new steady statehinders their cooperation. Only after experiencing payoffs lower than the new steadystate level and lowering their reference points can players reach the new steady state;in other words, those living in past glories are not trustworthy because they try
to cling to them by betraying others; it is not until they become used to the newand less pleasant reality that they can be trustworthy. This analysis highlights that
players’ reputations or trustworthiness in this game are not only shaped by the historyof actions, but also by the reference point, or equivalently the history of outcomes(intrinsic payoffs).
To the best of my knowledge, this paper is the first to demonstrate that reference
dependence and loss aversion generate gradualism, or “starting small,” in cooperation. Unlike my model, most existing works on gradualism in cooperation-building or
relationship-building feature incomplete information. Previous research has analyzed
situations in which players are uncertain about their partners’ types (Sobel 1985;
Watson 1999, 2002; Rauch and Watson 2003; Hua and Watson 2022; Furusawa and
Kawakami 2008). Additionally, in some cases, players can match with other partners
after relationship termination (Ghosh and Ray 1996; Kranton 1996).1 Research on
gradualism in games with complete information focuses on other factors, such as theirreversibility of cooperation (Lockwood and Thomas 2002).
The results of this work apply also to gradualism in other fields, such as trade
policy. The literature on trade policy analyzes trade liberalization with incomplete
tariff elimination as an equilibrium result of a tariff-setting game between strate1

Datta (1996) also studies gradualism in a model with random matching but with complete
information.

4

gic governments in an infinite time horizon. The welfare function that the governments maximize falls into a class of intrinsic payoff functions described in mymodel.2 While there have been various explanations of gradualism in trade liberalization (Staiger 1994; Devereux 1997; Furusawa and Lai 1999; Bond and Park 2002;
Maggi and Rodriguez-Clare 2007; Zissimos 2007; Chan 2019), research on reference
dependence as the driver in a strategic setting is lacking. Although reference dependence or loss aversion at an aggregate level is not as established a concept as at theindividual level, efforts have been made to theoretically and empirically investigate
behavioral effects in trade policy (Bernardes 2003; Freund and Özden 2008; Tovar2009).
The implication for transitional dynamics is a unique contribution. From collusion by firms to trade agreements to gang memberships, no cooperation is immune
to changes in their economic environments, such as new whistleblower regulations,
growing trade protectionism, or stricter law enforcement. However, analysis of thetransitional dynamics is limited in the literature. A mature relationship has resolved
the problem of incomplete information; therefore, models of cooperation developmentwith incomplete information typically imply an instant transition from a steady state
to a new one. My model provides new implications for transitional dynamics ofmature relationships and cooperation.
The rest of this paper is structured as follows. Section 2 introduces the model.
Section 3 examines Nash-reversion strategies in this model. Section 4 identifies thethree effects from reference dependence. Section 5 solves for the SPE that maximizes
utility with a Nash-reversion strategy, illustrating the gradual development of cooperation. Section 6 analyzes how the cooperation level transitions to a new steady state
when the economic environment changes. Section 7 concludes this paper.
2

The welfare for a government to maximize is the sum of consumer surplus, producer surplus,
and tariff revenues. When a government redistributes an increase in this welfare among its citizens,the welfare is the sum of the gains from tariff reductions for individual citizens. If citizens arereference-dependent and loss-averse, their individual loss utilities aggregate to a loss utility at the
government level.

5

2

Setup

I study a dynamic game with complete information by N symmetric players. I denotethe set of players by N. There are infinite periods, which I denote by t. In eachperiod, the players individually and simultaneously choose their cooperation levels.
I denote player i’s cooperation level in period t by αi,t ∈ A where A = [0, ᾱ] and thevector of cooperation levels in period t by αt = (α1t , α2t , . . . , αNt ).
Player i’s flow utility consists of an intrinsic payoff πi (αt ) and a loss utility ℓi (αt |rit ).
Given the reference point rit of player i in period t, it is given byui (αt |ri,t ) = πi (αt ) + ℓi (αt |rit ),
where






if π(αt ) ≥ ri,t ,0ℓi (αt |rit ) = −η [rit − πi (αt )] if π(αt ) < ri,t ,and 0 < η ≤ 1. The intrinsic payoff is symmetric among the players in the sensethat α0t = (α01t , α02t , . . . , α0Nt ) with αit = α0jt , α jt = α0it , and αkt = α0kt for else implies
πi (αt ) = π j (α0t ). The loss utility term ℓi (αt |rit ) captures the reference dependence andloss aversion. It measures losses in terms of intrinsic payoffs as in Kőszegi and Rabin(2006); when the intrinsic payoff πi (αt ) is below the reference point rit , they perceive
it as a loss, incurring the negative loss utility. The parameter η > 0 measures thedegree of loss aversion, and η = 0 would eliminate reference dependence, making themodel standard.
This functional form reflects simplifying assumptions. First, the flow utility ucomprises only the loss utility, not both loss and gain utilities.3 The lack of gain
utilities prevents us from weighining the importance of gain-loss utilities relative tointrinsic payoff separately from loss utility relative to gain utility. However, it retains
3

Other works that employ this formulation includes Freund and Özden (2008).

6

the property of loss aversion and simplifies the algebra. Moreover, in structural
estimations of a reference-dependent job search model, DellaVigna et al. (2017) reportthat a model only with loss utility fits the data similarly to a model with both gain
and loss utilities. Thus, using only the loss utility for tractability is a reasonablechoice. Second, it abstracts from the diminishing sensitivity of the prospect theory(Kahneman and Tversky 1979); the loss utility is piecewise linear in loss (rit − πi (αt )).
This functional form enables us to study the main idea of this paper in a relatively
simple model. The parameter η belongs to (0, 1], which is roughly consistent with theobservation that people evaluate an economic outcome to be twice as large when they
perceive it as a loss compared to when they perceive it as a gain in terms of absolutevalue in riskless decisions (Kahneman et al. 1990, Tversky and Kahneman 1991).
I assume several properties of the intrinsic payoff πi (αt ) that can be micro-foundedby either collusion in Cournot competition or trade liberalization as explained in theOnline Appendix. πi (αt ) is continuous in α j and twice differentiable with respect to α jfor all j ∈ N. In this paper, I focus on equilibria where players symmetrically choose
the same level of cooperation. To simplify the notations, I define π̃(α) as Definition 1
and assume π̃ is increasing as Assumption 1.
Definition 1 (Symmetric Cooperation). π̃(α) = πi (αt ) where αit = α for all i ∈ N.
Assumption 1. π̃(α) strictly increases in α.
I let π̄ = maxα∈A π̃(α) = π̃(ᾱ) and refer to ᾱ as the first-best cooperation. Assumption 1 ensures that the inverse function π̃−1 (π) exists. Using this inverse function, I
define the best-response intrinsic payoff b as a function of π.4
Definition 2 (Best-Response). b(π) = maxαi ∈A πi (αi , π̃−1 (π), . . . , π̃−1 (π)).
I assume two properties for the best-response intrinsic payoff as Assumption 2.
Assumption 2. b is convex, and there exists unique π ∈ [π̃(0), π̄) that solves b(π) = π.
4b(π) is the best response in terms of intrinsic payoff and in terms of flow utility.

7

Assumption 2 implies that π is the intrinsic payoff in a unique symmetric purestrategy Nash equilibrium in a one-shot game without a loss utility (η = 0). I denotethe cooperation level that produces π by αN , referring to αN as the Nash cooperation in
the rest of the paper, although whether π is the intrinsic payoff in a Nash equilibriumwith a loss utility (η > 0) is unclear at this point. Section 3 shows that π is indeed
the intrinsic payoff in a SPE even with a loss utility (η > 0).
The reference point is backward-looking and adaptive, following Bowman et al.
(1999) and DellaVigna et al. (2017). Specifically, it is given by






ρri,t−1 + (1 − ρ)πi (αt−1 ) if t ≥ 2,
ri,t = 




r1
if t = 1,(1)where 0 < ρ < 1. The initial reference point r1 ∈ R takes an exogenously given
common value among the players, which is public information. The parameter ρ
measures the persistence of a reference point.
This backward-looking reference point is a crucial assumption in my model. If
it were forward-looking, deviation from a given sequence of intrinsic payoffs would
always produce the same size of losses in subsequent periods. Thus, the optimal levelof cooperation would be invariant to the history of payoffs, becoming constant overperiods. 5 In the structural estimations by DellaVigna et al. (2017), the backwardlooking and adaptive reference point outperforms other types of reference points,
including the forward-looking one (Kőszegi and Rabin 2006) and the status quo,
which corresponds to ρ = 0 in my model, in terms of fitting job search patterns.67
5

I predict that completely eliminating the effect from expected utilities is unnecessary. As longas the payoff history affects the reference point, gradualism should appear for the same reasoning.
6
DellaVigna et al. (2017) use the average income over the N previous periods in their benchmark
model instead of the AR(1) formation like mine for computational reasons. They report that thefits of these two reference points are similar.
7
In addition to the empirical support, there is another reason why I do not allow ρ to be one.
ρ = 1 disconnects a reference point before deviation and the loss utilities by a penalty. If ρ = 1, thereference point in the first penalty period is the intrinsic utility by the best deviation, which I show
independent of the reference point in Section 3. Given this, the magnitude of the cooperation loss

8

Also, Post et al. (2008) report the persistence of reference points, although they use a
different functional form than mine. Thus, a backward-looking and adaptive reference
point is a reasonable assumption.
The players correctly understand how their own reference points evolve. Givena path of intrinsic payoffs, they derive their future reference points according to thelaw of motion (1), evaluating future intrinsic payoffs against them. They discountfuture flow utilities by a discount factor δ ∈ (0, 1) , maximizing the present value ofPttheir utility ∞t=0 δ u(αt |ri,t ). The Online Appendix presents a study of the game inwhich players incorrectly believe their reference points remain fixed at the current
level indefinitely. Gradualism and rough transitions still occur, demonstrating therobustness of the model’s results.

3

Nash-Reversion Strategy

The analysis of this paper focuses on the pure-strategy SPEs with Nash-reversion
strategies, which use a static Nash equilibrium as the penalty (Friedman 1971). Consequently, the range of cooperation levels feasible in the SPEs is generally limited
compared to the case with the optimal penalty (Abreu 1988). However, a relatively
simple form of penalty enables this study to obtain rich analytical results.
This section examines the Nash-reversion strategies and the best deviation when
the others employ a Nash-reversion strategy. We must first know where to revert (inother words, we must know the penalty). A natural candidate is a Nash strategy sN
in Definition 3.
Definition 3 (Nash Strategy). A Nash strategy sN is a sequence of mapping σtN thatN(t−1)and the initial reference point r1 ∈ R
maps any history of cooperations {αk }t−1
k=1 ∈ A

into the Nash cooperation αN . That is, ∀t ∈ N, σtN : AN(t−1) × R 7→ αN .
aversion would become independent of the reference point, and gradualism does not emerge.

9We need to examine if all the players employing sN , (sN , sN , . . . sN ), indeed forms a
SPE in this game. An SPE may not be formed for two reasons. First, the flow utilityu(αt |ri,t ) additionally contains the loss-aversion term ℓ(αt |ri,t ). Second, this model has
a dynamic effect. An action in a given period affects the reference points in thefollowing periods. Thus, the maximizer of the flow utility u possibly differs from thatP
k−tof the present value of utility ( ∞k=t δ ui,k ), even when the other players choose thesame actions every period. To see these effects clearly, I lay out the derivative of theutility with respect to πi (αt+k ) below:∞
X∂ri,t+k+m δt+k 1 + 1{πi (αt+k ) < ri,t+k } · η −.
δm 1{πi (αt+k+m ) < ri,t+k+m } · η ·
∂π
(α
)
i
t+k
m=1

This derivative shows the standard positive effect (δt+k · 1) and additional positive
and negative effects. First, the second term is the positive effect: a reduction of thecurrent loss utility when the intrinsic payoff πi (αt+k ) is in the loss region. Second, thethird term summarizes the indirect negative effect: a higher intrinsic payoff today
πi (αt+k ) raises future reference points (∂ri,t+k+m /∂πi (αt+k ) > 0), widening future losses,
if they exist. One can prove that the standard positive effect dominates the indirect
negative effect, making the net effect positive. Thus, Lemma 1 follows.
Lemma 1. (sN , sN , . . . sN ) is a SPE.
□

Proof. See Appendix A.

Given this result, I formally define a Nash-reversion strategy—a grim trigger strategy with the Nash penalty sN —in Definition 4.
Definition 4. A Nash-reversion strategy with a cooperation path {αct }∞t=1 for player i,c k∞s({αct }∞t=1 ), is a sequence of mapping {sk ({αt }t=1 )}k=1 such that





c
c


αk if ∀t ∈ {1, 2, . . . , k − 1}, ∀j∈ N, α jt = αt or if k = 1 andc kN(k−1)sk ({αt }t=1 ) : A
×R 7→ 
αN otherwise.
10

As the next step, I identify the best deviation. Given that the other players
employ sN from the next period, the best deviation maximizes the present value of
utility. Like the analysis of the Nash strategy, the marginal utility with respect tothe own cooperation level has, possibly, two additional effects: directly alleviating thecurrent loss utility and indirectly aggravating the future loss utilities by raising futurereference points. Nevertheless, the best deviation in my model becomes the same as
the best deviation in the model without the reference dependence (η = 0) because thestandard positive effect on the current intrinsic payoff dominates the indirect effectof aggravating the future loss utilities. Formally, Lemma 2 states this result.
Lemma 2. When the others play a Nash-reversion strategy s({αct }∞t=1 ), the best deviation in period t (αbt ) is unique and the same as that without the reference dependence(η = 0). That is, for any i ∈ N, ∀ri,t ∈ R,


∞
X
h
i

c
c
k
N
N
argmax πi (αit , α−i,t ) + ℓ(αit , α−i,t |ri,t ) +δ π̃(α ) + ℓ(α |ri,t+k )  = argmax πi (αit , αc−i,t ),αit ∈Aαit ∈A

k=1

where αc−i,t is the vector of cooperation levels by all the players but i where α jt = αct
for all j , i.
□

Proof. See Appendix B.

Having validated the penalty and identified the best deviation, I analyze the effects
of reference dependence in SPEs with Nash-reversion strategies in the next section.

4

Three Effects

This section identifies the three effects that reference dependence generates in SPEs
with Nash-reverting strategies. In the rest of the paper, I focus on cooperation paths
that take values between the Nash cooperation and the first-best cooperation, [αN , ᾱ].
8Given this interval, the focus on the symmetric equilibria with Nash-reversion
8

This focus does not forbid a best deviation from taking a value lower than αN .

11strategies, and Lemmas 1 and 2, I simplify notations using πt and b(πt ) where πt =π̃(αt ), as if the players directly choose πt instead of αt , and a player earns b(πt ) by
the best deviation. The notation for a Nash-reversion strategy changes accordinglyc ∞Nfrom s({αct }∞t=1 ) to s({πt }t=1 ). Since π̃ is increasing on [α , ᾱ], I sometimes refer to thelevel of πt as that of cooperation and, in particular, π̄ as the first-best cooperation.
A cooperation level greater than αN and an intrinsic payoff greater than π are called
cooperative. I assume the initial reference point falls into the same interval as intrinsic
payoffs as Assumption 3.
Assumption 3. π ≤ r1 ≤ π̄.
c ∞c ∞In this game with the initial reference point r1 , (s({πct }∞t=1 ), s({πt }t=1 ), . . . , s({πt }t=1 ))forms a SPE if and only if, for all t ∈ N, every player weakly prefers following s({πct }∞
t=1 )
c ∞
to the best deviation. I call a cooperation path {πct }∞t=1 feasible if and only if (s({πt }t=1 ),c ∞s({πct }∞t=1 ), . . . , s({πt }t=1 ) ) forms a SPE. Lemma 3 states the condition for feasibility.

Lemma 3 (Feasibility). {πct }∞t=1 is feasible if and only if,∀t ∈ N,b(πct ) − πct + 1{rt > πct } · η min{rt , b(πct )} − πct
∞

X


k c
≤
δ πt+k − 1{rt+k > πct+k } · η rt+k − πct+k
k=1

n
io 
h
d
d
− π − 1{rt+k> π} · η rt+k
−π ,

(2)

s.t.∀k ∈ N, rt+k = ρrt+k−1 + (1 − ρ)πct+k ,dd∀k ∈ N, rt+k
= ρrt+k−1+ (1 − ρ)π where rtd = rt .

□Proof. It follows from Lemmas 1 and 2.

This condition inherits, from a standard model, a tension between the short-term
gain and the long-term loss by deviation. The left-hand side of the inequality expresses
the short-term gain from deviation in period t. It consists of the improvement of thecurrent intrinsic payoff (b(πct ) − πct ) and the reduction of the loss utility, if any. The
12right-hand side is the difference in the present value of utilities from the following
periods between the equilibrium and deviation paths, which I refer to as a longterm loss. The value of future utilities from the cooperative levels of cooperation ismeasured with reference points in the equilibrium path {rt+k }∞k=1 , and that of deviationd ∞is measured with reference points in the deviation path {rt+k}k=1 .

To analyze the standard effects and the effects from reference dependence separately, I suppose that reference dependence does not exist (η = 0) and that thecooperation path is constant (πct = πc ). Inequality (2) becomesb(π ) − π ≤
c

c

∞
X



δk πc − π .

k=1

On the one hand, the short-term gain is convex and possibly becomes very large when
the cooperative intrinsic payoff πc is suﬀiciently high. On the other hand, the longterm loss is linear in πc . Thus, rising πc makes the short-term gain overwhelm thelong-term loss at some point, subject to parameter values, making high cooperationlevels infeasible.
Reference dependence adds three effects to this tension. To see them clearly, Idexpress the future reference points rt+k and rt+k
as functions of the current reference

point rt and future intrinsic payoffs. Manipulating inequality (2) yields the following
inequality, showing the three effects as:b(πct ) − πct + 1{rt > πct } · η min{rt , b(πct )} − πct|{z}Loss-evading incentive≤∞Xδk πct+k −

1





{rt+k > πct+k } · δk η ρk rt +

|
∞
X
k=1

δ π+
k

k−1
X

ρ

k−1−ℓ

ℓ=0





(1 − ρ)πct+ℓ − πct+k 

{z

k=1

k=1

−

∞
X

}

Future cooperation losses
∞
X

|

k=1

h
i
δk η ρk rt + ρk−1 (1 − ρ)b(πct ) − ρk−1 π .
{zCooperation loss aversion

13

}

(3)

First, the cooperation loss aversion adds to the long-term loss, and a higher reference point rt strengthens this effect. The last summation term on the right-hand side
is non-negative and increases with rt , capturing this effect. After deviation, a player
incurs a loss utility every period. A higher reference point in the current period has
a persistent effect on future reference points as ρ > 0, aggravating these loss utilities. Intuitively, the more players are used to high payoffs, the more they hesitate to
deviate due to future losses.
The second effect is the loss-evading incentive; loss from πct incentivizes players todeviate, captured by the third term on the left-hand side. Given a cooperation level
πct , the magnitude of this effect weakly increases with rt . The larger a loss is, themore a player can potentially gain by deviating. However, a reference point higher
than b(πct ) creates too big a loss to eliminate, making the magnitude of this effectunresponsive to a further increase.
Third, the equilibrium path can entail losses in the future (rt+k > πct+k ), labeled as
future cooperation losses, which discourage cooperation in the current period. The
second summation term on the right-hand side captures this effect, reducing the longterm loss by deviation. Given a cooperation path {πct }∞t=1 , the size of this effect increases
with rt ; a higher reference point implies more significant future losses because of its
persistence.
Whether a higher reference point rt relaxes or tightens this inequality condition
depends on whether the equilibrium path entails losses. Without a loss in any period,
only the cooperation loss aversion is operative; consequently, a higher rt relaxes thecondition. In contrast, if there is a loss every period (rt+k > πct+k for all k), all threeeffects exist, and each of them is strengthened with a higher rt . Eq. (3) shows that thecontributions of rt to the cooperation loss aversion and the future cooperation lossesPkk
are both ∞k=1 δ ηρ rt , canceling each other out. Thus, the only remaining effect is thestrengthening loss-evading incentive, tightening the condition. This case-dependent
net effect becomes crucial in the optimized path, which I study in Section 5.

14

I exclude an uninteresting case in which the first-best cooperation π̄ is feasible fromperiod 1 regardless of the initial reference point. Imposing Assumption 4 ensures it.
Assumption 4. The first-best cooperation every period {π̄}∞t=1 is not feasible when thereference point is at the Nash cooperation level (r1 = αN ) asb(π̄) − π̄ >



δ 
δη 
ρπ + (1 − ρ)b(π̄) − π .
π̄ − π +1−δ
1 − δρ

The less patient (low δ) and the less loss-averse (low η) the players are, and the lesspersistent a reference point is (low ρ), the more likely Assumption 4 holds. A smaller
discount factor lowers the present value of the long-term loss. Less persistence of areference point and less loss-aversion weaken the cooperation loss aversion. Anotherfactor that can support Assumption 4 is a lucrative best deviation. The higher b(π̄) isrelative to π̄, the more likely the inequality holds. Given Assumption 4, the first-bestcooperation π̄ is not feasible for some r1 , allowing us to study the development pathof cooperation in the next section.

5

Gradual Development

In this section, I investigate the optimal SPE, defined as one that maximizes thepresent value of utility among those feasible by a Nash-reversion strategy, showingthat cooperation optimally develops gradually. The path of πt in that equilibrium isthe solution to the following problem:v∗ (r1 ) = max∞{πt }t=1∞Xδt−1 πt − 1{rt > πt } · η(rt − πt ) ,

t=1

(4)

s.t.∀t ∈ N, πt ∈ Γ(rt , v∗ ),∀t ∈ N, rt+1 = ρrt + (1 − ρ)πt ,15

where the feasible set Γ(rt , v∗ ) is defined as"#δη(1 − ρ)Γ(rt , v ) = πt ∈ [π, π̄] : 1 −b(πt ) − 1{rt > b(πt )} · η [rt − b(πt )]1 − δρ
"
#

δ
δη
δηρ
+
+
π−
rt ≤ v∗ (rt ) .
1 − δ 1 − δρ
1 − δρ
∗The first constraint ensures the feasibility following Lemma 3; the inequality compares
the value of the deviation path on the left-hand side and that of the equilibrium pathon the right-hand side. To characterize the solution, I set up the corresponding
functional equation to apply dynamic programming. It takes rt as the state variableand πt as the control variable. I obtain Lemma 4.
Lemma 4. The function v∗ satisfies the following functional equation.
v∗ (r) = max∗ y − 1{r > y} · η(r − y) + δv∗ (ρr + [1 − ρ]y) .
y∈Γ(r,v )(5)□

Proof. See Appendix C.

The functional equation (5) differs from a standard dynamic programming problem in that the correspondence Γ(r, v∗ ) has the function v∗ as an argument. Nevertheless, Lemma 4 states that the function v∗ satisfies eq. (5), enabling us to characterize the value function v∗ and the policy function g∗ (r) = argmaxy∈Γ(r,v∗ ) y − 1{r >y} · η(r − y) + δv∗ (ρr + [1 − ρ]y).
The properties of the value function v∗ and the policy function g∗ depend onwhether the first-best cooperation π̄ is sustainable. I define sustainability in Definition
5.
Definition 5 (Sustainability). A cooperation (or reference-point) level π is sustainablec
if the constant cooperation path at that level ({πct }∞t=1 where πt = π) is feasible with aNash-reversion strategy when r1 = π. Otherwise, they are unsustainable.
The constant cooperation path at the initial reference point level does not change
16the reference point over periods, generating no loss. Thus, the condition for π to besustainable follows from Lemma 3 asb(π) − π ≤



δ 
δη 
π−π +
ρπ + (1 − ρ)b(π) − π .
1−δ
1 − δρ

(6)

Given this definition, I partition all the cases into two.
Case 1.

The first-best cooperation is sustainable.

Case 2.

The first-best cooperation is unsustainable.

Similar to the condition of Assumption 4, given π = π̄, the right-hand side of theinequality (6) increases in δ, η, and ρ; the more patient (higher δ) and the more
loss-averse (higher η) the players are, the more likely it falls into Case 1. Also, themore persistent a reference point is (higher ρ), the more likely it is Case 1. These
increase the magnitude of the long-term loss. On the other hand, a more lucrative
best deviation (higher b(π̄)) makes Case 2 more likely by raising the short-term gain.
In the rest of this section, I first provide analytical results for Case 1. Subsequently,
I provide analytical results for Case 2. Finally, I provide numerical examples of Case2, obtaining additional insights.

5.1 Case 1
For Case 1, Lemma 5 summarizes the main properties of the policy function g∗ .
Lemma 5 (Case 1). The policy function g∗ is continuous on [π, π̄] and strictly increasing and concave on [π, γ]. Additionally, r < g∗ (r) < π̄ on [π, γ), and g∗ (r) = π̄ on[γ, π̄]. Consequently, π̄ is the unique steady state.
□

Proof. See Appendix D.

17

Figure 1
Gradual Development Path in Case 1yg ∗ (r)

π4
π3

π2

π1

r1 = π

r2

r3

γ

r4

π̄

r

Notes: The solid blue line depicts the policy function g∗ according to the properties in Proposition 5 for Case 1. The blue dots
locate {π1 , π2 , π3 , π4 } and the red dots locate {r2 , r3 , r4 } of the optimized cooperation path with r1 = π.
Figure 1 illustrates the optimized cooperation path for players with the initialreference point at π according to these properties. The policy function g∗ (rt ) locatesan intrinsic payoffs πt (blue dots). Subsequently, πt determines the reference pointin the next period rt+1 (red dots) as the weighted average of πt and rt . From the
beginning, the optimized cooperation g∗ (rt ) is higher than rt , raising the reference
point. Consequently, the next optimized cooperation becomes even higher. This way,
the cooperation level gradually and monotonically rises and reaches the first-best levelπ̄ when the reference point reaches or exceeds γ. Thus, I obtain Proposition 1.
Proposition 1 (Cooperation Path in Case 1). (i) In a game of players with initialreference points lower than γ, they monotonically increase the level of cooperation18

over time, with the level converging to the first-best level. (ii) In a game of playerswith initial reference points not lower than γ, they keep implementing the first-bestcooperation level from period 1.
□

Proof. It follows from Lemma 5.

The driver of this monotonically increasing cooperation is the monotonically strengthening cooperation loss aversion (δηρrt /(1 − δρ) in Γ(rt , v∗ )). Experiencing a higher utility raises the reference point and aggravates the loss utilities in the deviation path.
Consequently, the aversion strengthens, enabling the players to shift up the cooperation level. In turn, this even higher cooperation level elevates the reference point,
reinforcing the aversion further. The repetition of this process continues until thecooperation level reaches the first-best level. This cooperation path never generates
a loss, ensuring the other two effects remain absent.

5.2 Case 2
In Case 2, reference point r can be unsustainable, allowing the other two effects
to operate. Lemma 6 states the properties of g∗ , describing how the path differs
depending on the sustainability of r.
Lemma 6 (Case 2). The policy function g∗ is continuous, strictly increasing, andstrictly concave on [π, z̄] where z̄ is the highest sustainable cooperation level that is
implicitly defined byb(z̄) − z̄ =



δ 
δη 
z̄ − π +ρz̄ + (1 − ρ)b(z̄) − π .
1−δ
1 − δρ

(7)

Additionally, r < g∗ (r) < z̄ for r < z̄, g∗ (z̄) = z̄, and g∗ (r) < z̄ for r > z̄. Consequently,z̄ is the unique steady state.
□

Proof. See Appendix E.

19

Lemma 6 immediately implies Proposition 2.
Proposition 2 (Cooperation Path in Case 2). (i) In a game of players with initial reference points lower than z̄, the players monotonically increase the level of cooperation
over time, with the level converging to z̄. (ii) In a game of players with initial reference points higher than z̄ , they start cooperation below z̄ , with the level eventuallyreaching z̄.
□Proof. It follows from Lemma 6.

When players’ initial reference point r1 is not as high as z̄—the case we expect
for a new relationship—the development of cooperation exhibits gradualism. The
properties of the policy function on [π, z̄] are the same as those on [π, γ] in Case 1.
Heuristically, this result is due to the non-existence of a loss from any reference point
r1 ∈ [π, z̄]. Thus, only the cooperation loss aversion is operative, driving graduallydeveloping cooperation similarly to Case 1.
An excessively high reference point hinders cooperation. In Case 2, a high b(r)prevents the players from implementing a cooperation level above z̄, making a reference point higher than z̄ unsustainable. Given such a reference point, the players
must follow an equilibrium path with losses. The future cooperation losses cancelout the cooperation loss aversion in incentivizing cooperation, as discussed in Section 4. Additionally, the loss-evading incentive—the only remaining effect—operates,
preventing the players from maintaining the cooperation level even at z̄.
Whether a higher reference point in the region above z̄ lowers the cooperationlevel is generally unclear because the magnitude of the loss-evading incentive can beunresponsive to r, as discussed in Section 4. Nevertheless, Assumption 5 enables us
to obtain the monotonicity as Proposition 3.
Assumption 5. The highest sustainable z̄ is suﬀiciently close to π̄.
Proposition 3 (Case 2 with Additional Assumption). Given Assumption 5, g∗ isstrictly decreasing on [z̄, π̄]. Thus, the higher the players’ initial reference points are,20

the lower the cooperation level is in the first period, as long as their initial reference
points are above z̄.
□Proof. See Appendix E.

The decreasing g∗ reflects the strengthening loss-evading incentive. Assumption 5
ensures that the best deviation payoff always exceeds a reference point (b(g(r)) > r for
all r), guaranteeing the strictly increasing loss-evading incentive. The next subsection
provides numerical examples, consistent with this result.

5.3 Numerical Examples of Case 2This subsection provides numeric examples of Case 2 to obtain more sense of the
optimized paths. Figure 2 shows Example 1: an example with high ρ and not verylow δ. The policy function g∗ (r) (solid blue) is upward-sloping until it intersects the
45-degree (dashed green) line and then becomes downward-sloping. The intersection
corresponds to the highest sustainable cooperation z̄, and the properties of g∗ areexactly as stated in Lemma 6 and Proposition 3. The other solid line (orange) locatesthe reference point in the next period (ρr + (1 − ρ)g∗ (r)), which increases even after theintersection. In this example, when the initial reference point is unsustainable (r > z̄)
along the equilibrium path, the reference point gradually declines, the cooperation
gradually rises, and both converge to the intersection from the right of the graph.
Throughout this converging path, the players incur loss utilities.

21

Figure 2
Case 2 - Example 1: Policy Function and Reference Point in Next Period

Notes: The intrinsic payoff π(α) is derived from Cournot competition by N = 15 firms with common marginal cost c = 1 withP
the inverse demand function 10 − xi where xi is the quantity
supplied by firm i.
An unsustainable reference point inhibits cooperation more severely when theplayers are less patient and when a reference point is less persistent. Figure 3 shows
Example 2 of Case 2 in which δ and ρ are smaller than in Figure 2. There are four
differences from Figure 2. First, the optimized cooperation g∗ slope in the sustainableregion (π ≤ r ≤ z̄) flattens, demonstrating that the cooperation loss aversion becomes
weaker with less persistent losses in the deviation path and more discounting of future
losses. Second, the highest sustainable cooperation z̄ becomes smaller, caused by thesmaller standard long-term loss with more discounting and the weakening cooperation
loss aversion. Third, the downward slope in the unsustainable region (z̄ < r) close to z̄
becomes steeper. This steeper slope indicates that the magnitude of the loss-evading
incentive becomes relatively greater than in Example 1. A smaller discount factor
and a less persistent reference point both reduce the long-term losses, whereas thestandard short-term gain or the loss-evading incentive is unaffected. Consequently, agiven increase in the loss-evading incentive tightens the constraint in Γ(r, v∗ ) relatively
22more, generating a greater drop in cooperation. Finally, there are kinks in g∗ (r) and,
subsequently, ρr + (1 − ρ)g∗ (r) around r = 0.95. Where the reference point is higher
than this point, the best deviation cannot eliminate a loss in the current period(r > b(g∗ (r)). A higher reference point in this region does not generate an additional
incentive for deviation. This example works as a base for the analysis of transitional
dynamics in the next section.
Figure 3
Case 2 - Example 2: Policy Function and Reference Point in Next Period

Notes: The intrinsic payoff π(α) is derived from Cournot competition by N = 15 firms with common marginal cost c = 1 withP
the inverse demand function 10 − xi where xi is the quantity
supplied by firm i.

6

Rough Transition

In this section, I study the model’s implications for transitions of cooperation levels
when the economic environment changes. To this end, I consider Case 2 for richer
implications and assume that a reference point is at the steady state z̄, corresponding
to a mature relationship.
Reference dependence generates asymmetric transitional dynamics when z̄ shifts.

23

The steady state z̄ moves when any of function b and parameters (π, π̄, δ, η, ρ)changes, which occurs for various reasons; for example, an entry or exit in a Cournot
competition changes b, π, and π̄.9 In the following analysis, I consider a Cournotcompetition example in which the incumbents are tacitly colluding and achieving thehighest sustainable profit z̄ below the monopolist level (π̄).
Cooperation monotonically transitions to the new steady state when the environment favorably changes. Suppose an incumbent exits the market at the end of periodt. Then, it increases the profits of the remaining firms from period t + 1, working as
a positive shock for them. Consequently, z̄ rises, becoming higher than the reference
point rt+1 , which is unaffected.10 The result of g(r) > r for r < z̄ implies higher profits
for the remaining firms in period t + 1. Subsequently, the profits monotonically rise toz̄. Figure 4 shows this path by a solid blue line. It uses the same parameters except
for the number of firms as in Example 2 in Section 5, which uses a functional formof Cournot competition among 15 firms. In Figure 4, the number of firms decreases
from 15 to 14 in period 2. This figure illustrates a smooth transition for firms, who
do not suffer losses after the shock.
9

A more straightforward example of the shift of z̄ is a permanent intrinsic payoff shock. ThatoldNis, πnewt+k (α) = πt+k (α) + ϵ for all α ∈ A and all k ∈ N for a shock at the beginning of period t + 1.
This shock shifts everything but the reference point to the same extent. The insights in this section
applies also to this simple case.
10
Additionally, π̄ and π rise, and b(π) changes.

24Figure 4
Transitional Dynamics from z̄ with N = 15 in Cournot competition

Notes: These lines trace the level of intrinsic payoffs with η =0.8, δ = 0.3, and ρ = 0.3 and a shock to the number of firms
at the end of period 1 in collusion under Cournot competition.
Firms have a common marginal cost c = 1 with the inversePdemand function 10 − xi where xi is the quantity supplied byfirm i. In period 1, r1 = π1 = z̄ for N = 15. In period 2 and later,
πt moves according to the policy functions for N = 14 (blue) andN = 16 (orange) with r2 = π1 .
In contrast, a negative shock causes an excessive decline in profits in the shortterm, traced by the solid orange line in Figure 4 where the number of firms increases
to 16. When a new firm enters the market and joins the tacit collusion, individual
firms’ profits fall. Thus, the entry has a negative effect, pushing down the steadystate z̄.11 The result of g(r) < z̄ for r > z̄ implies a profit in period t + 1 (period 2
in Figure 4) lower than the new steady state level z̄, let alone the previous steadystate level. This overshoot reflects the loss-evading incentive as discussed in Section
5. Firms are accustomed to a high level of profit. Even after this high profit level
becomes infeasible in a new environment, they try to cling to it by deviating from the11

I assume the entrant has the same reference point as the incumbents when it enters.

25

collusion. This incentive makes their cooperation diﬀicult, lowering the profit. Afterthe drop in period t + 1, the profit converges to the new steady state level after firms
eventually adapt to a lower profit level.
These dynamics are similar to those of Green and Porter (1984) in that a sharp
drop in the payoff level does not imply a collapse of collusion or cooperative relationships. Green and Porter (1984) study a model with unobserved demand shocks ina stationary environment, showing that prices and profits can sharply decline fromand return to the monopolistic levels even when collusion operates. In contrast to
their work, the return to a higher price—the new steady state—is gradual in mymodel; thus, reference-dependent firms “restart small.” As a similar exercise, I analyze how cooperation levels respond to unexpected one-off payoff shocks in the Online
Appendix. The shock does not affect the steady state, making the analysis more comparable to Green and Porter (1984). Following the one-off shock, reference points shift
away from the steady state, preventing players from implementing the steady-statelevel in the next period; thus, even a positive shock can reduce the cooperation level.
The cooperation returns to the steady-state level only gradually, unlike in Green andPorter (1984).

7

Conclusion

This paper provides a new explanation of the gradual development of cooperation.
In existing models of gradualism with incomplete information, the existence of myopic players forces players to start cooperation at a low level. In contrast, reference
dependence raises the initial level of cooperation and facilitates it further over time.
In contrast, reference points provide an additional incentive to deviate—the lossevading incentive—dragging cooperation levels down when players must scale backtheir cooperation levels. A limitation of the analysis is that I analyze SPEs by Nashreversion strategies to obtain the implications of reference dependence in a tractable26

way. The investigation of the optimal penalty in the presence of reference dependence
potentially provides additional results.

References
Abreu, D. (1988). On the theory of infinitely repeated games with discounting. Econometrica: Journal of the Econometric Society, 383–396.
Bernardes, L. G. (2003). Reference-dependent preferences and the speed of economic
liberalization. The Journal of Socio-Economics 32(5), 521–548.
Bond, E. W. and J.-H. Park (2002). Gradualism in trade agreements with asymmetriccountries. The Review of Economic Studies 69(2), 379–406.
Bowman, D., D. Minehart, and M. Rabin (1999). Loss aversion in a consumption–savings model. Journal of Economic Behavior & Organization 38(2), 155–178.
Chan, J. M. (2019). Gradualism in the gatt: Strategic tariff bargaining and forward
manipulation. Review of International Economics 27(1), 220–239.
Datta, S. (1996). Building trust. Sticerd - theoretical economics paper series, Suntory
and Toyota International Centres for Economics and Related Disciplines, LSE.
DellaVigna, S., A. Lindner, B. Reizer, and J. F. Schmieder (2017).

Reference-

dependent job search: Evidence from hungary. The Quarterly Journal of Economics 132(4), 1969–2018.
Devereux, M. B. (1997). Growth, specialization, and trade liberalization. International Economic Review, 565–585.
Freund, C. and Ç. Özden (2008). Trade policy and loss aversion. American Economic
Review 98(4), 1675–1691.

27

Friedman, J. W. (1971). A non-cooperative equilibrium for supergames. The Review
of Economic Studies 38(1), 1–12.
Furusawa, T. and T. Kawakami (2008). Gradual cooperation in the existence ofoutside options. Journal of Economic Behavior & Organization 68(2), 378–389.
Furusawa, T. and E. L.-C. Lai (1999). Adjustment costs and gradual trade liberalization. Journal of International Economics 49(2), 333–361.
Ghosh, P. and D. Ray (1996). Cooperation in community interaction without information flows. The Review of Economic Studies 63(3), 491–519.
Green, E. J. and R. H. Porter (1984). Noncooperative collusion under imperfect priceinformation. Econometrica: Journal of the Econometric Society, 87–100.
Hua, X. and J. Watson (2022). Starting small in project choice: A discrete-time
setting with a continuum of types. Journal of Economic Theory 204, 105490.
Kahneman, D., J. L. Knetsch, and R. H. Thaler (1990). Experimental tests of theendowment effect and the coase theorem. Journal of Political Economy 98(6),1325–1348.
Kahneman, D. and A. Tversky (1979). Prospect theory: An analysis of decision underrisk. Econometrica 47(2), 263–292.
Kőszegi, B. and M. Rabin (2006). A model of reference-dependent preferences. The
Quarterly Journal of Economics 121(4), 1133–1165.
Kranton, R. E. (1996). The formation of cooperative relationships. The Journal ofLaw, Economics, and Organization 12(1), 214–233.
Lockwood, B. and J. P. Thomas (2002). Gradualism and irreversibility. The Review
of Economic Studies 69(2), 339–356.

28

Maggi, G. and A. Rodriguez-Clare (2007). A political-economy theory of trade agreements. American Economic Review 97(4), 1374–1406.
Post, T., M. J. Van den Assem, G. Baltussen, and R. H. Thaler (2008). Deal or no
deal? decision making under risk in a large-payoff game show. American Economic
Review 98(1), 38–71.
Rauch, J. E. and J. Watson (2003). Starting small in an unfamiliar environment.
International Journal of industrial organization 21(7), 1021–1042.
Sobel, J. (1985). A theory of credibility. The Review of Economic Studies 52(4),
557–573.
Staiger, R. W. (1994). A theory of gradual trade liberalization.
Tovar, P. (2009). The effects of loss aversion on trade policy: Theory and evidence.
Journal of International Economics 78(1), 154–167.
Tversky, A. and D. Kahneman (1991). Loss aversion in riskless choice: A referencedependent model. The Quarterly Journal of Economics 106(4), 1039–1061.
Watson, J. (1999). Starting small and renegotiation. Journal of economic Theory 85(1), 52–90.
Watson, J. (2002). Starting small and commitment. Games and Economic Behavior 38(1), 176–199.
Zissimos, B. (2007).

The gatt and gradualism.

nomics 71(2), 410–433.

29

Journal of International Eco-

Appendix
A

Proof of Lemma 1

I consider a subgame starting from period t. The utility for player i is given by
Ui,t ({αt+k }∞k=0 , ri,t ). The derivative of the utility with respect to αi,t+m is:∂Ui,t∂πi (αi,t+m , α−i,t+m )=δmΦ,∂αi,t+m∂αi,t+mwhereΦ = 1+1{π(αi,t+m , α−i,t+m ) < ri,t+m }η−∞X

δn 1{π(αi,t+m+n , α−i,t+m ) < ri,t+m+n }·η·n=1∂ri,t+m+n.
∂π(αi,t+m , α−i,t+m )Φ is positive as∞X∞X∂ri,t+m+nδη(1 − ρ)Φ ≥1 −δ η·= 1 − η δn ·(1 − ρ)ρn−1 = 1 −> 0,∂πi (αi,t+m , α−i,t+m )1 − δρ
n=1
n=1
nwhere the last inequality follows from η ∈ (0, 1] and δ, ρ ∈ (0, 1). Thus, ∂Ui,t /∂αi,t+m and∂πi (αi,t+m , α−i,t+m )/∂αi have the same sign, implying argmaxαi,t+m ∈A Ui,t ({αi,t+k , α−i,t+k }∞k=1 , ri,t ) =argmaxαi,t+m ∈A πi (αi,t+m , α−i,t+m ) for all m ∈ N. Therefore, (sN , sN ,. . . , sN ) is a Nash equilibrium in this subgame. This completes the proof.

B

Proof of Lemma 2

Suppose player i deviates in period t when the others play s({αct+k }∞k=0 , ri,t ). Then,the derivative of utility with respect to αi,t is ∂πi (αi,t , αct )/∂αi,t )Φ0 where Φ0 = 1 +P
1{πi (αi,t , αc−i,t ) < rt } · η − ∞k=1 δk · 1{π̃(αN ) < ri,t+k } · η · (∂ri,t+k /∂πi (αt )) evaluated at30

αt = (αi,t , αc−i,t ). I manipulate this Φ0 as:
0Φ ≥1 − η∞X
k=1

k ∂ri,t+kδ∂πi (αt ) |αt =(αi,t ,αc−i,t )

=1−η

∞
X

δk ·(1 − ρ)ρk−1 = 1 −k=1

δη(1 − ρ)> 0,
1 − δρ

where the last inequality follows from η ∈ (0, 1] and δ, ρ ∈ (0, 1). Thus, the sign of(∂πi (αi,t , αct )/∂αi,t )Φ0 equals to that of ∂πi (αi,t , αct )/∂αi,t , implying Lemma 2.

C

Proof of Lemma 4

I show the function v∗ satisfies (5). v∗ is bounded as: π/(1 − δ) ≤ v∗ ≤ π̄/(1 − δ). Since∞
πt = π is always feasible, Γ(rt , v∗ ) is nonempty for all rt . Let Ψ(rt ) ≡ {πk }∞k=t ∈ [π, π̄] :∗πk ∈ Γ(rk , v ), s.t. rk+1 = ρrk + (1 − ρ)πk be the set of feasible cooperation paths fromrt . Given ρrt + (1 − ρ)πot ∈ [π,π̄], there exists an optimized path {πk }∞k=t+1 such that


P∞
∗
o
∞
∞
k−t
v (ρrt +[1−ρ]πt ) = F({πk }k=t+1 ) where F({πk }k=t+1 ) = k=t+1 δ πk + 1{rk > πk }·η(rk −πk ) .
It follows that, given πt , rt ∈ [π,π̄],πot − 1{rt > πot } · η(rt − πot ) + δv∗ (ρrt + [1 − ρ]πot )= πot − 1{rt > πot } · η(rt − πot ) + δ
≤

max
∞

{πk }k=t ∈Ψ(rt )max{πk }∞∈Ψ([1−ρ]rt +ρπot )k=t+1F({πk }∞
k=t+1 )

 o∗πt − 1{rt > πot } · η(rt − πot ) + δF({πk }∞k=t+1 ) = v (rt )

(8)

∞
o
where I use the fact that {πot , {πk }∞k=t+1 } ∈ Ψ(rt ) when {πk }k=t+1 ∈ Ψ([1 − ρ]rt + ρπt ). At

the same time, given rt ∈ [π,π̄],v∗ (rt ) =max{πk }∞∈Ψ(rt )
k=t

≤ πot −

1

o
o
o
o ∞
F({πk }∞k=t ) = πt − 1{rt > πt } · η(rt − πt ) + δF({πk }k=t+1 ){rt > πot } · η(rt − πot ) + δv∗ (ρrt + [1 − ρ]πot )

31

(9)

∞
where {πot , {πok }∞k=t+1 } = argmax{πk }∞ ∈Ψ(rt ) F({πk }k=t ). Thus, it follows from inequalities (8)k=t

and (9) thatv∗ (rt ) = πot − 1{rt > πot } · η(rt − πot ) + δv∗ (ρrt + [1 − ρ]πot )v∗ satisfies the functional equation (5).

D

Proof of Lemma 5

I first obtain Lemma 7 that enables us to exclude the possibility of v∗ (π) = π/(1 − δ).
Lemma 7. There exists π0 > π such that v∗ (π) ≥ π0 /(1 − δ).
Proof. Consider a constant cooperation path {π}∞t=1 in problem 4. Given r1 = π, thefollowing inequality is suﬀicient for v∗ (π) ≥ π/(1 − π).
"
#
"
#
δ
δη
π
δη(1 − ρ)
δηρ
+
rt ≤
,
∀t, 1 −b(π) +
π−
1 − δρ
1 − δ 1 − δρ
1 − δρ
1−δ

(10)where the indicator function is absent because π ≥ rt for all t. Subsequently, inequality(10) for t = 1 becomes
"
#
#
δ
π
δη(1 − ρ)δη(1 − ρ)b(π) −++π ≤ 0.
f (π) ≡ 1 −1 − δρ
1−δ
1−δ
1 − δρ
"b(π) = π implies f (π) = 0. Taking derivative of f (π) yields"#
δη(1 − ρ) 0
1
f (π) = 1 −b (π) −
1 − δρ1−δ! X
δη(1 − ρ) ∂πi (αi , αc −i )
= 1−
1 − δρ
∂α j,t0j,i|αi =argmax πi (αi ,αc −i ) dαt dπ̃(αt )|αt =αc

−

1
,
1−δ

where the second equality follows from the first order condition for αi = argmax πi (αi , αc −i ).

32Evaluating this derivative at π = π yields

! X
N 

 dαt
∂π
(α
)
δη(1
−
ρ)
1

i


f 0 (π) = 1 −−
1 − δρ
∂α j,t dπ̃(αt ) |αt =αc 1 − δ
j,i

−1
!
δη(1 − ρ) X ∂πi (αN )  X ∂πi (αN ) 1  −= 1−
1 − δρ
∂α j,t∂α j,t
1−δ
j,i
j,i
=

1 − δρ − δη + δηρ1−< 0,
1 − δρ
1−δ

where the second equality follows from αN = argmax πi (αi , αN −i ). f (π) = 0 and f 0 (π) < 0
imply that there exists ϵ > 0 such that f (π + ϵ) < 0. Thus, {π + ϵ}∞
t=1 satisfy inequalities(10) for t = 1 and for t ≥ 2 because rt ≥ r1 . Let π0 = π + ϵ. Then, v∗ (π) ≥ π0 /(1 − δ).
□This completes the proof of Lemma 7.
Using π0 in Lemma 7, I define the interval X and the set of functions C(X) as
X : [π, π̄]C(X) : the set of bounded, continuous, and weakly increasing
functions f : X → R with the sup norm that are weakly concave on X




0


π /(1 − δ) ≤ f (x) ≤ π̄/(1 − δ) x ≤ γ,
s.t. 
 f (x) = π̄/(1 − δ)x ≥ γ,
where γ is implicitly defined by the following equation:
"

"
#
#
δ
δη
δηρ
π̄
δη(1 − ρ)b(π̄) +
+
π−
γ=
.
1−
1 − δρ
1 − δ 1 − δρ
1 − δρ
1−δ(11)Assumption 4 and the definition of Case 1 imply π < γ < π̄. On C(X), I define theoperator T by

T f (x) = max y − 1{x > y} · η(x − y) + δ f (ρx + [1 − ρ]y) ,y∈Γ(x; f )33

where"#δη(1 − ρ)Γ(x; f ) = x ∈ [π, π̄] : 1 −b(y) + 1{x > y} · η min{x, b(y)} − y
1 − δρ
"
#

δ
δη
δηρ
+
+
π−
x ≤ y + δ f (ρx + [1 − ρ]y) .
1 − δ 1 − δρ
1 − δρ
Given function f , I obtain properties of T f . The policy function, g(x; f ), is definedbyg(x; f ) = argmax y − 1{x > y} · η(x − y) + δ f (ρx + [1 − ρ]y).
y∈Γ(x; f )Since f is a weakly increasing function, g(x; f ) = max Γ(x; f ). By manipulating thecondition of Γ(x; f ), let h(x, y; f ) be"#δη(1 − ρ)h(x, y; f ) = 1 −b(y) + 1{x > y} · η min{x, b(y)} − y
1 − δρ
"
#
δηρ
δ
δη
π−
+
+
x − y − δ f (ρx + [1 − ρ]y).
1 − δ 1 − δρ
1 − δρ
y belongs to Γ(x; f ) if and only if h(x, y; f ) ≤ 0.
I eliminate the indicator function by showing g(x; f ) ≥ x. I define h̃(x; f ) by"#"#δη(1 − ρ)δδηδηρh̃(x; f ) = h(x, x; f ) = 1 −+π−x − x − δ f (x).
b(x) +
1 − δρ
1 − δ 1 − δρ
1 − δρ
h̃(x; f ) is negative at x = π as:
"

#
"
#
δ
δη(1 − ρ)δηδηρb(π) ++π−
π − π − δ f (π)
h̃(π; f ) = 1 −1 − δρ
1 − δ 1 − δρ
1 − δρ
"##"δη(1 − ρ)
δ
δη
δηρ
δπ0
≤ 1−
+
π+
π−
π−π−
1 − δρ
1 − δ 1 − δρ
1 − δρ
1−δδ=−(π0 − π) < 0,1−δwhere the inequality follows from b(π) = π and f (π) ≥ π0 /(1 − δ). Also, h̃(x; f ) is34

non-positive at π̄ by construction of Case 1. The strict convexity of b(·) and theweak concavity of f (·) imply h̃(x; f ) is strictly convex in x. These imply that, for allx ∈ X, h̃(x, f ) ≤ 0. In turn, this implies that g(x; f ) ≥ x for all x ∈ X, eliminating theindicator function.
h(x, g(x); f ) = 0 on [π, γ]I now show that the equality of h(x, y; f ) ≤ 0 holds with

y = g(x; f ) = max Γ(x; f ) for all x ≤ γ. From the result above, given x < γ, h(x, x; f ) =h̃(x; f ) < 0. On the other hand,"#"#δη(1 − ρ)δδηδηρh(x, π̄; f ) = 1 −π−b(π̄) ++
x − π̄ − δ f (ρx + [1 − ρ]π̄)
1 − δρ
1 − δ 1 − δρ
1 − δρ
δηρ(γ − π̄) + δ f (ργ + [1 − ρ]π̄) − δ f (ρx + [1 − ρ]π̄) > 0,=
1 − δρ
where the second inequality follows from eq. (11) and f is weakly increasing. Sinceb(·) and − f (·) are convex and continuous functions, given x, h(x, y; f ) is convex andcontinuous in y. This convexity, continuity, h(x, x; f ) < 0 and h(x, π̄; f ) > 0 imply
h(x, g(x; f ); f ) = 0.
T f and g are strictly increasing on [π, γ] T f (x) strictly increases in x on [π, γ] because
g(x) strictly increases on [π, γ], following from g(x) = max Γ(x; f ).
T f and g are strictly concave on [π, γ] T f (x) is strictly concave on (π, γ) because
g(x) is strictly concave on (π, γ), as below. For x1 , x2 ∈ [π,γ] such that x1 , x2 , g(x1 )and g(x2 ) satisfy the following equations, respectively.

35

"
#
"
#
δη(1 − ρ)
δ
δη
δηρ
1−
b(g(x1 )) ++π−x1 = g(x1 ) + δ f (ρx1 + [1 − ρ]g(x1 )),1 − δρ
1 − δ 1 − δρ
1 − δρ
(12)"#"#δη(1 − ρ)
δ
δη
δηρ
1−
b(g(x2 )) ++π−x2 = g(x2 ) + δ f (ρx2 + [1 − ρ]g(x2 )).
1 − δρ
1 − δ 1 − δρ
1 − δρ
Combining these two equations with a weight θ ∈ (0, 1) yields"#"#
δη(1 − ρ)
δ
δη
π
1−
+
{[1 − θ]b(g(x1 )) + θb(g(x2 ))} +1 − δρ
1 − δ 1 − δρδηρ([1 − θ]x1 + θx2 )−1 − δρ

= (1 − θ)g(x1 ) + θg(x2 ) + (1 − θ) δ f (ρx1 + [1 − ρ]g(x1 )) + θδ f (ρx2 + [1 − ρ]g(x2 )) .
Since g(x) strictly increases on [π, γ], g(x1 ) , g(x2 ). It follows from the convexity ofb(·) and the concavity of f (·) that:"##"δη(1 − ρ)δηδηρδ([1 − θ]x1 + θx2 )1−+
π−{b([1 − θ]g(x1 ) + θg(x2 ))} +1 − δρ
1 − δ 1 − δρ
1 − δρ

< (1 − θ)g(x1 ) + θg(x2 ) + δ f (ρ {[1 − θ]x1 + θx2 } + (1 − ρ) {[1 − θ]g(x1 ) + θg(x2 )}) . (13)Since [1 − θ]x1 + θx2 ∈ (π, γ), g([1 − θ]x1 + θx2 ) ∈ (π, γ) satisfies the following equality.
"#"#δη(1 − ρ)δδηδηρ([1 − θ]x1 + θx2 )
1−+
π−{b(g([1 − θ]x1 + θx2 ))} +1 − δρ
1 − δ 1 − δρ
1 − δρ
= g([1 − θ]x1 + θx2 ) + δ f (ρ {[1 − θ]x1 + θx2 } + (1 − ρ)g([1 − θ]x1 + θx2 )) .
(14)

Eq. (13) and (14) imply (1 − θ)g(x1 ) + θg(x2 ) < g([1 − θ]x1 + θx2 ). That is, g(x) is
strictly concave on (π,γ). Consequently, T f (x) is strictly concave on (π,γ).

36Properties on [γ, π̄] For the properties on [γ, π̄], h(x, π̄; f ) ≤ 0 for all x ≥ γ, following
from eq. (11), and π̄ ∈ Γ(x; f ). This immediately implies g(x; f ) = π̄ and T f (x) =π̄/(1 − δ). These also imply that g(x; f ) and T f (x) are weakly concave and weakly
increasing on X.
Continuity The weak concavities of g(x) and T f (x) imply their continuities on (π, π̄).
They are also continuous at π̄, following from the results on [γ, π̄]. As for the continuity at π, I prove that of g(x) first. I take a sequence, {xt } ⊂ [π,γ] , that converges
to π as t → ∞. Then, for all xt ,"#"#δη(1 − ρ)δδηδηρg(xt ) = 1 −b(g(xt )) ++π−xt − δ f (ρπ + [1 − ρ]g(xt )).
1 − δρ
1 − δ 1 − δρ
1 − δρ
The continuities of b(·) and f (·) imply
"
"
#
#
δ
δη(1 − ρ)δηb(lim g(xt )) ++πlim g(xt ) = 1 −
xt →π
xt →π
1 − δρ
1 − δ 1 − δρδηρ−
π − δ f (ρπ + [1 − ρ] lim g(xt )).
xt →π
1 − δρ
This equality implies lim xt →π g(xt ) = g(π).
Pointwise Convergence The results above imply T : C(X) → C 0 (X) where C 0 (x) is
defined as:

C 0 (X) : the set of bounded, continuous, weakly increasing, and weakly concave
functions f : X → R with the sup norm that arestrictly increasing and strictly concave on [π,γ]






π’/(1 − δ) ≤ f (x) ≤ π̄/(1 − δ) x ≤ γ,
s.t. 
 f (x) = π̄/(1 − δ)x ≥ γ,

37

which is a subset of C(X). Using this operator T , I define a sequence of functions
produced by T by




π̄



 f1 (x) = 1−δ for all x ∈ X,





 fn+1 = T fn for n ∈ N.

f1 is in C(X). It follows that






 f2 (x) < f1 (x)





 f2 (x) = f1 (x)

x<γ
x≥γ

For x < γ, given f (x) < f 0 (x), g(x; f ) = max Γ(x; f ) < max Γ(x; f 0 ) = g(x, f 0 ), implying
T f (x) < T f 0 (x). By this monotonicity of T , f3 = T f2 < T f1 = f2 . Repeating thisoperation yields







 fn (x) < · · · < f1 (x)





 fn (x) = · · · = f1 (x)x < γ,x ≥ γ.

Since { fn }n∈N is bounded, there exists f ∗ (x) such that fn (x) → f ∗ (x) as n → ∞.
Uniform Convergence I show the convergence is uniform. For all n+1, and y < x ∈ X,




0
x, y ≥ γ,






 π̄
| fn+1 (x) − fn+1 (y)| = |T fn (x) − T fn (y)| = − T fn (y)y < γ ≤ x,

1−δ







T fn (x) − T fn (y) y < x < γ.

38When y < x < γ,"#"#δη(1 − ρ)δδηδηρ| fn+1 (x) − fn+1 (y)| = 1 −b(g(x; fn )) ++π−x
1 − δρ
1 − δ 1 − δρ
1 − δρ
("#"#)δη(1 − ρ)
δ
δη
δηρ
− 1−
b(g(y; fn )) ++π−y1 − δρ
1 − δ 1 − δρ
1 − δρ
"#δη(1 − ρ) b(g(x; fn )) − b(g(y; fn ))δηρ=|x − y| · 1 −−1 − δρx−y1 − δρ
#"g(x) − g(y)δηρδη(1 − ρ) 0b (π̄) max−<|x − y| · max 1 −, (15)x,y1 − δρx−y1 − δρ
#"g(x) − g(y)δηρ δη(1 − ρ) 0
b (π) min
−
1−
.
x,y
1 − δρx−y1 − δρ
where the last inequality follows from the convexity of b(·). I show [g(x)−g(y)]/(x−y) on
the right-hand side (RHS) is bounded. g(x) strictly increasing implies h(x1 , g(x2 ); f ) −h(x1 , g(x1 ); f ) > 0 for x1 < x2 < γ. Thus,
#
"
#
δ
δη
δηρ
δη(1 − ρ)b(g(x1 )) ++π−x1 = g(x1 ) + δ f (ρx1 + [1 − ρ]g(x1 )).
1−
1 − δρ
1 − δ 1 − δρ
1 − δρ
""##δδη(1 − ρ)δηδηρ1−b(g(x2 )) ++π−x1 > g(x2 ) + δ f (ρx1 + [1 − ρ]g(x2 )).
1 − δρ
1 − δ 1 − δρ
1 − δρ
"It follows"#δη(1 − ρ) b(g(x2 )) − b(g(x1 ))f (ρx1 + [1 − ρ]g(x2 )) − f (ρx1 + [1 − ρ]g(x1 ))
1−
−1−δ
> 0.
1 − δρ
g(x2 ) − g(x1 )
g(x2 ) − g(x1 )
(16)

39

Subsequently, h(x2 , g(x2 ); f ) − h(x1 , g(x1 ); f ) = 0 implies"#δη(1 − ρ) b(g(x2 )) − b(g(x1 )) g(x2 ) − g(x1 )δηρg(x2 ) − g(x1 )1−−−1 − δρ
g(x2 ) − g(x1 )x2 − x1
1 − δρ
x2 − x1
f (ρx2 + [1 − ρ]g(x2 )) − f (ρx1 + [1 − ρ]g(x2 ))−δ
x2 − x1
f (ρx1 + [1 − ρ]g(x2 )) − f (ρx1 + [1 − ρ]g(x1 )) g(x2 ) − g(x1 )−δ=0g(x2 ) − g(x1 )x2 − x1#"δη(1 − ρ) b(g(x2 )) − b(g(x1 ))−1⇐⇒ 1 −
1 − δρ
g(x2 ) − g(x1 )f (ρx1 + [1 − ρ]g(x2 )) − f (ρx1 + [1 − ρ]g(x1 ))  g(x2 ) − g(x1 )−δ
g(x2 ) − g(x1 )x2 − x1
f (ρx2 + [1 − ρ]g(x2 )) − f (ρx1 + [1 − ρ]g(x2 ))
δηρ
−δ
=
.
x2 − x1
1 − δρ
The combination of ρx1 +[1−ρ]g(x2 ) > π and the concavity of f implies that the second
term on the left-hand side of the last equality is bounded. The RHS is also bounded.
Consequently, given inequality (16), [g(x2 ) − g(x1 )]/[x2 − x1 ] must be bounded. Consequently, there exists some Kn1 < ∞ such that | fn+1 (x) − fn+1 (y)| < Kn1 |x − y| for all
x, y < γ because of inequality (15).
When y < γ ≤ x,γ−y
(T fn (γ) − T fn (x))
x−yγ−y≤|x − y|x−y#"g(x) − g(y)δηρδη(1 − ρ) 0
· max 1 −b (π̄) max−,
x,y
1 − δρx−y1 − δρ
#"δη(1 − ρ) 0
g(x) − g(y)δηρ b (π) min
−
.
1−
x,y
1 − δρx−y1 − δρ

| fn+1 (x) − fn+1 (y)| = (x − y)Similar to the previous case, there exists some Kn2 < ∞ such that | fn+1 (x) − fn+1 (y)| <Kn2 |x − y| for all y < γ and x ≥ γ. These results imply each fn is Lipschitz continuouswith max{K11 , K12 , . . . Kn1 , Kn2 . . . } as the common Lipschitz constant. In turn, it implies { fn }n∈N is equicontinuous. Additionally, { fn }n∈N is bounded, and X is closed and40bounded. Thus, the convergence of { fn }n∈N is uniform by the Ascoli–Arzelà theorem.
Finally, (C(X), || · ||∞ ) is a compact set, implying f ∗ ∈ C(X).
f ∗ satisfies the functional equation

Next, I prove that f ∗ satisfies the functional

equation by showing that fn converges to T f ∗ . It follows from the uniform convergencethat for any ε > 0, there exists N such that k fn − f ∗ k < ε for all n > N. Additionally,f ∗ ≤ fn , implying fn < f ∗ + ε on X for n > N. By the monotonicity, T fn < T ( f ∗ + ε).
Subsequently,
kT fn − T f ∗ k = sup |T fn (x) − T f ∗ (x)|x∈X< sup |T f ∗ (x + ε) − T f ∗ (x)| (n > N)

(17)

x∈X

= sup l(x, f ∗ , ε),
x∈X

where
l(x, f ∗ , ε) =g(x; f ∗ + ε) + δ f ∗ (ρx + [1 − ρ]g(x; f ∗ + ε)) + δε
− {g(x; f ∗ ) + δ f ∗ (ρx + [1 − ρ]g(x; f ∗ ))}.
First, I show that l(x, f ∗ , ε) ≤ l(π, f ∗ , ε). I let ∆g(x) = g(x; f ∗ + ε) − g(x; f ∗ ) and∆ f ∗ (x) = f ∗ (ρx + [1 − ρ]g(x; f ∗ + ε)) − f ∗ (ρx + [1 − ρ]g(x; f ∗ )). Then, l(x, f ∗ , ε) increases
with ∆g(x) and ∆ f ∗ (x). To show that x = π maximizes l(x, f ∗ , ε), I show that, given ε,it maximizes both ∆g(x) and ∆ f ∗ (x). The following two equations hold for x ≤ γ − ϵ.
"#"#δη(1 − ρ)
δ
δη
δηρ
∗
1−
b(g(x; f + ε)) ++π−x
1 − δρ
1 − δ 1 − δρ
1 − δρ
= g(x0 f ∗ + ε) + δ f ∗ (ρx + [1 − ρ]g(x; f ∗ + ε)) + δε
#
"
#
"
δ
δη
δηρ
δη(1 − ρ)∗b(g(x; f )) +
+
π−
x
1−
1 − δρ
1 − δ 1 − δρ
1 − δρ
= g(x; f ∗ ) + δ f ∗ (ρx + [1 − ρ]g(x; f ∗ ))41

Combining these two equations yields#hδρ
Y(x, ∆g(x)) ≡ 1 −{b(g(x; f ∗ ) + ∆g(x)) − b (g(x; f ∗ ))}1 − δ(1 − ρ)"− ∆g(x) − δ { f ∗ (ρx + [1 − ρ]g(x; f ∗ ) + ρ∆g(x)) − f ∗ (ρx + [1 − ρ]g(x; f ∗ ))} = δεI let x̄ = argmax x∈{x∈X|Y(x,∆g(x)=δε} ∆g(x). Suppose x̄ > π. Then, the convexity of b, theweak concavity of f ∗ , and g(x) increasing in x implies δε = Y( x̄, ∆g( x̄)) > Y(π, ∆g( x̄)).
Y(π, ∆g(x)) is convex in ∆g(x), and Y(π, 0) = 0. Because of the convexity, there exits∆g > ∆g( x̄) such that Y(π, ∆g) = δε. This contradicts x̄ = argmax x∈{x∈X|Y(x,∆g(x)=δε} ∆g(x).
Thus, x̄ = π. As for ∆ f ∗ (x),∆ f ∗ (x) ≤ f ∗ (ρπ + [1 − ρ]g(π; f ∗ ) + ρ∆g(π)) − f ∗ (ρπ + [1 − ρ]g(π; f ∗ )) = ∆ f ∗ (π),
following the weak concavity of f ∗ , g(x; f ∗ ) weakly increasing in x, and max ∆g(x) =∆g(π). These results imply

kT fn − T f ∗ k < max l(x, f ∗ , ε) ≤ l(π, f ∗ , ε)x∈X

= g(π; f ∗ + ε) + δ f ∗ (ρπ + [1 − ρ]g(π; f ∗ + ε)) + δε
− {g(π; f ∗ ) + δ f ∗ (ρx + [1 − ρ]g(π; f ∗ ))}.

(18)

Next, I show that g(π, f ) is continuous in f at f = f ∗ . At x = π, the following
condition holds for small ε,
"
#
"
#
δ
δη
δηρ
δη(1 − ρ)1−b(g̃(ε)) ++π−
π
1 − δρ
1 − δ 1 − δρ
1 − δρ
= g̃(ε) + δ f ∗ (ρπ + [1 − ρ]g̃(ε)) + δε,

42

(19)

where g̃(ε) denotes g(π; f ∗ + ε). I rewrite this equation as Z(g̃(ε)) = ε where Z(x) =(1/δ) · {[1 − δη(1 − ρ)/(1 − δρ)]b(x) + {δ/(1 − δ) + δη/[1 − δρ]}π − δηρ/(1 − δρ)π − x −δ f (ρπ + [1 − ρ]x)}. Z(·) is a continuous and convex function, and Z(g̃(0)) = 0, implyingg̃(ε) is strictly increasing in ε and continuous in ε.
Finally, I can prove that T fn converges to T f ∗ . Since g̃ and f ∗ are both continuous,g̃(ε) + δ f ∗ ([1 − ρ]π + ρq(ε)) + δε is continuous in ε. Subsequently, for any ω > 0, there
exists ζ > 0 such that for all ε < ζ,kT fn − T f ∗ k < g̃(ε) + δ f ∗ (ρπ + [1 − ρ]g̃(ε)) + δε
− {g̃(0) + δ f ∗ (ρπ + [1 − ρ]g̃(0)) + δ · 0} < ω
Thus, T fn uniformly converges to T f ∗ , implying fn converging to T f ∗ . Since fn converges also to f ∗ , f ∗ = T f ∗ . Since f ∗ is the solution to the functional equation whose
value is bounded, v∗ = f ∗ . Additionally, since v∗ = f ∗ ∈ C(X), v∗ = T v∗ ∈ C 0 (X). The
policy function g∗ (x) = g(x; v∗ ) has the same properties as g(x; f ) for f ∈ C(X). This
completes the proof of Lemma 5.

E

Proof of Lemma 6

I prove Lemma 6 in three steps, each of which follows a procedure similar to thatin the proof of Lemma 5. First, I derive the properties of the value function w∗1 (r)and the policy function g∗w1 (r) of a modified problem that does not have the lossevading incentive or the future cooperation losses: both w∗1 (r) and g∗w1 (r) are strictlyincreasing and strictly concave on X. Second, I set up another modified problem inwhich only the loss-evading incentive is absent. I derive the properties of the valuefunction w∗2 (r) and the policy function g∗w2 (r), using the results of the first modified
problem: w∗2 (r) = w∗1 (r) and g∗w2 (r) = g∗w1 (r) on [π, z̄]; w∗2 (r) = z̄/(1 − δ) − η(r − z̄)/(1 − δρ)and g∗w2 (r) = z̄. Finally, using the results from the second modified problem, I derive
the properties of the value function and the policy function of the original problem
43(5) as in Lemma 6. See the Online Appendix for further details.

F

Proof of Proposition 3

I define X and C(X) as:
X :[π, π̄]C(X) :the set of bounded and continuous functions f : X → R
with the sup norm that are weakly concave on [z̄, π̄].
s.t. ∀x ∈ [π, z̄], f (x) = w∗1 (x),z̄η∀x ∈ [z̄, π̄], f (x) ≤−(x − z̄),1 − δ 1 − δρf (y) − f (x)1+η∀x, ∀y ∈ [z̄, π̄] s.t. x < y,≥−,y−x
δ(1 − ρ)where w∗1 is that of Appendix E. On C(X), I define the operator T by

T f (x) = max y − 1{x > y} · η(x − y) + δ f (ρx + [1 − ρ]y) ,y∈Γ(x, f )

where
#


δη(1 − ρ)Γ(x, f ) = x ∈ [π, π̄] : 1 −b(y) + 1{x > y} · η min{x, b(y)} − y
1 − δρ
"
#

δ
δη
δηρ
+
+
π−
x ≤ y + δ f (ρx + [1 − ρ]y) .
1 − δ 1 − δρ
1 − δρ
"

Following steps similar to those in the proof of Lemma 5 completes the proof. See
the Online Appendix for further details.

44