DP2016-08

Recu rsive Utility and the Solu tion to
the Bellm an Eq uation
R IEB Jun ior Re se ar ch F ellow

Masayuki YAO
February 29, 2016

* This Discussion Paper won the Kanematsu Fellowship Prize (FY 2013).

RECURSIVE UTILITY AND THE SOLUTION TO THE BELLMAN
EQUATION
MASAYUKI YAO

Abstract. This study infinite-horizon deterministic dynamic programming problems
based on recursive utility in discrete time. Under a small number of conditions, we showthat the Bellman operator has a fixed point using Knaster–Tarski’s fixed point theorem.
We also show the fixed point of the Bellman operator can be computed by iterationfrom the initial function between the lower boundary and the fixed point. To show theconvergence theorem, we use Tarski–Kantorovitch’s fixed point theorem.

1. Introduction
Dynamic programming has become an important tool in economic theory, particularly
since Stokey and Lucas (1989). Due to technical reasons, many dynamic economic models
are based on time additive separable utility with a constant discount rate. However,these models have been criticized with respect to consistency with economic situations.
Becker (1980) examines the long-run behavior of economy in a one-good model basedon time additive separable utility with a constant discount rate of dynamic equilibriumwith heterogeneous households. He discussed that unless all of the households have thesame discount rate, the household which has the lowest discount rate owns all the capitalin the long-run and all the others consume nothing using their labor income to service
their dept. So, Lucas and Stokey (1984) tackles this problem using recursive utility.
Recently, recursive utility has attracted attention, replacing time additive utility witha constant discount rate. Recursive utility comprises a wide class of representations ofutility including time additive utility with a constant discount rate. 1
Date: March 2, 2016.
Key words and phrases. Recursive utility, Fixed point theorem, Dynamic programming, Bellman equation.
JEL Cllasification: C61, O41
Financial support from the Japan Society for Promotion of Science, Keio University Doctorate Student Grant-in-Aid Program, 2013 and Keio Gijuku Academic Development Funds, 2014, 2015. The
author is grateful to Hiroyuki Ozaki for his constant support, guidance and encouragement. The author
is also grateful to Takako Fujiwara-Greve, Toru Hokari, Hidetoshi Komiya, Shinsuke Nakamura, Mikio
Nakayama, Toyotaka Sakai, Yukio Takeuchi and Chaowen Yu for their helpful comments and suggestions.
The author also thanks the participants of the 2013 Fall Semiannual Meeting of Japanese Economic Association, 8th Marco economics conference for young economists and Friday seminar in Waseda University.
.
1
The significance of recursive utility is discussed in Section 1.3 “Why Study Recursive utility?” in Becker
and Boyed (1997) with various economical examples.
1

2

M. YAO

Recursive utility as introduced by Koopmans (1960) formalizes a method used by Fisher(1907). Many researchers have proposed solutions to the problem of dynamic programming on recursive utility, often using an aggregator function constructed from recursive
utility functions to solve the problem. The aggregator method was first proposed by Lucasand Stoky (1984). Important studies with the aggregator function are the method basedon the contraction mapping theorem used by Lucas and Stokey (1984), the weighted contraction method used by Boyd (1990), the partial sum method used for all aggregators
by Le Van and Vailakis (2005), k-contraction used by Ricón-Zapatero and Rodrı́guezPalmero (2007) and Biconvergence condition used by Streufert (1990, 1992). Becker andBoyd (1997) and Boyd (2006) give a good exposition of these studies. With the exception of Streufert’s method, these approaches assume a variant of the Lipschitz condition
to the aggregator function. Streufert’s (1990, 1992) method is unique compared to theother approaches, as it does not use any fixed point theorem. Streufert (1998) provides adetailed exposition of these results and methods.
However, these contributions heavily rely on topological assumptions, such as continuity of a utility function and continuity of a feasible correspondence. Kamihigashi (2014)
establishes some elementary results on fixed points of the Bellman operator without topological assumptions in a dynamic economic model based on time additive separable utility
with a constant discount rate.
This paper is motivated by the method of Streufert (1992) and the idea of Kamihigashi(2014). The goal is to obtain some results on the solution to the Bellman equation – orfixed points of the Bellman operator – without topological assumptions in the dynamic
economic model based on recursive utility.
As in Kamihigashi’s (2014) approach, we use a fixed point theorem of an ordered
space instead of a variant of the contraction mapping theorem to solve the problem.
Kamihigashi (2014) uses the Knaster–Tarski fixed point theorem, which is a sort of a
fixed point theorem of an ordered space. However, the Knaster–Tarski fixed point theorem
only shows existence of a fixed point; it does not show that some sequence converges tothe fixed point. We show a fixed point theorem for the Bellman operator using TarskiKantrovitch’s fixed point theorem2 3,. Using the theorem, we show that a fixed point of
the Bellman operator can be computed by iteration starting from some boundary function
4.
Our main results are as follows. Given order interval of functions that is mapped
into itself by the Bellman operator, if the aggregator has certain properties, then (a) the
Bellman operator has a fixed point in the order interval and (b) the fixed point can becomputed iteratively starting from the lower boundary of the order interval. Under some
2In some research field, the fixed point theorem is called Kleene’s fixed point theorem, for example see

Baranga (1991)
3Knaster–Tarski fixed point theorem is sometimes used in macro-economic theory. See, for example,
Datta and Reﬀett (2006).
4Kamihigashi, Reﬀet and Yao (2015), Kamihigashi and Yao (2015a) and Kamihigashi and Yao (2015b)
use similar approach of this paper in the model of time addtive separable utility.

RECURSIVE UTILITY AND THE SOLUTION TO THE BELLMAN EQUATION

3

topological assumptions, if an upper boundary with certain properties exists, then (i)the Bellman operator has a fixed point in the order interval, (ii) this fixed point can becomputed iteratively starting from the upper boundary of the order interval. Using thisresult, we can construct an alternative proof of Theorem 2.3 (a) in Streufert (1998).
The remainder of this article is organized as follows. In Section 2, we prepare somemathematical tools. In Section 3, we describe the model based on Streufert’s (1992)
without topological assumptions of the state space and find a fixed point of Bellman
operator. In Section 4, we show that the fixed point operator can be computed iteratively
starting from the upper boundary of the order interval with some assumption about
continuity of the Bellman operator or topological assumptions. In Appendix A, we state
proofs on our results.
2. Preliminaries
Throughout this paper, we denote by N the set of positive integers, by Z+ the set of
nonnegative integers, by R the set of real numbers and by R+ the set of positive real
numbers.
A partially ordered set is pair (P, ≤), where P is nonempty set and ≤ is a relation in
P which is reflexive (p ≤ p for all p ∈ P ), antisymmetric (for p, q ∈ P , p ≤ q and q ≤ p
implies p = q) and transitive (for p, q, r ∈ P , p ≤ and q ≤ r implies p ≤ r). Let (P ; ≤) bea partially ordered set and M ⊂ P be a nonempty subset. An upper (resp. lower) boundfor M is an element p ∈ P with m ≤ p (resp. m ≤ p) for each m ∈ M . The least (resp.
greatest) element of M ⊂ P is an element p ∈ M satisfying p ≤ m (resp. m ≤ p) forallm ∈ M . The supremum (resp. infimum) of M ⊂ P denoted as sup M (resp. inf M ), is
the least upper bound (resp. greatest lower bound) of M . A partially ordered set (P ; ≤)is complete lattice if every subset M of P has both the supremum and infimum in (P, ≤).
M ⊂ P is said to be a chain if for p, q ∈ M , either p ≤ q or q ≤ p. A sequence {pn }n∈N
is called increasing (resp. decreasing) if pn ≤ pn+1 (resp. pn+1 ≤ pn ) for all n ∈ N. A
mapping F : P → P is said to be monotone if for p, q ∈ P , p ≤ q implies F (p) ≤ F (q).
Theorem 2.1 (Knaster–Tarski). Let (P, ≤) be a partially ordered set and T : P → P
monotone. Assume that there is a p0 ∈ P such that(i) p0 ≤ T (p0 ); and(ii) every chain in P0 ≡ {p ∈ P : p0 ≤ p} has a supremum.
Then T has a fixed point in P0 .
Proof. See Aliprantis and Border (2006), p16.

□

Using the inverse ordering and infimum, we have the following dual version of Theorem
2.1.
Corollary 2.1. Let (P, ≤) be a partially ordered set and T : P → P monotone. Assume
that there is a p0 ∈ P such that

4

M. YAO(i) T (p0 ) ≤ p0 ; and(ii) every chain in P0 ≡ {p ∈ P : p ≤ p0 } has a infimum.
Then T has a fixed point in P0 .
A mapping F from a partially ordered set (P, ≤) into itself is said to be s-order continuous (resp. i-order continuous) if for every countable chain C ⊂ P having a supremum(resp. infimum), the image F (C) has a supremum (resp. infimum) and(1)

sup F (C) = F (sup C) (resp. inf F (C) = F (inf C)).

It is clear that s-order continuous (resp. i-order continuous) is monotone.
Theorem 2.2 (Tarski–Kantorovitch). Let (P, ≤) be a partially ordered set and T : P → P
s–order continuous. Assume that there is a p0 ∈ P such that(i) p0 ≤ T (p0 ); and(ii) every countable chain in P0 ≡ {p ∈ P : p0 ≤ p} has a supremum.
Then T has a fixed point p∗ = supn∈N T n p0 and p∗ is the infimum of the set of the fixed
points of T in P0 .
Proof. See Granas and Dugundji (2003), p26.

□

Using the inverse ordering and infimum, we have the following dual version of Theorem
2.2.
Corollary 2.2. Let (P, ≤) be a partially ordered set and T : P → P i–order continuous.
Assume that there is a p0 ∈ P such that(i) T (p0 ) ≤ p0 ; and(ii) every countable chain in P0 ≡ {p ∈ P : p ≤ p0 } has a infimum.
Then T has a fixed point p∗ = inf n∈N T n p0 and p∗ is the supremum of the set of the fixed
points of T in P0 .
The following result is from Kamihigashi (2008).
Lemma 2.1. Let Y and Z be sets. Let Ω ⊂ Y × Z, and let f ; Ω → R. For any y ∈ Y
and z ∈ Z, define(2)Ωy = {z ∈ Z : (y, z) ∈ Ω},(3)Ωz = {y ∈ Y : (y, z) ∈ Ω}.

Then sup(y,z)∈Ω f (y, z) = supy∈Y supz∈Ωy f (y, z) = supy∈Z supy∈Ωz f (y, z).
Proof. See Lemma 1 in Kamihigashi (2008).

□

The following results are elementary but plays an important role in this paper.
Lemma 2.2. Let f be a monotone and lower semicontinuous function from R into itself
and let A ⊂ R. Then, we have(4)

sup f (A) = f (sup A).

RECURSIVE UTILITY AND THE SOLUTION TO THE BELLMAN EQUATION

5□

Proof. See Appendix A.
By similar proof, we have the following dual version of Lemma 2.2

Corollary 2.3. Let f be a monotone and upper semicontinuous function from R into
itself and let A ⊂ R. Then, we have(5)

inf f (A) = f (inf A).
3. The Model and Results

Let X be a nonempty set. An infinite sequence of elements of X will be denoted by∞{xt }∞
t=0 , with x ∈ X for all t ∈ Z+ . The space of these sequence is denoted by X . LetΓ be a nonempty correspondence from X to X. Let U : X ∞ → R where R = [−∞, ∞].
In the optimization problem introduced below, t ∈ Z+ is a discrete time period, X is astate space, Γ is the feasible correspondence and U is a utility function.
Example 3.1. In the one–sector growth model, today’s state (consuming and saving)x = (c, s) ∈ R2+ is feasible if c + s does not exceed the income F (s0 ) determined byyesterday’s saving s0 . There we define X = R2+ with x = (c, s), assume that F : R+ → R+ ,and define Γ : X → 2X by Γ(x0 ) = {x = (c, s) ∈ X : c + s ≤ F (x0 )}.
∞Let S denote the shift operator defined by S({xt }∞t=0 ) = {xt }t=1 .
We assume that there exists an aggregator W : X × R → R defined by∞(A1) ∀{xt }∞t=0 ∈ X ,(6)∞
U ({xt }∞t=0 ) = W (x0 , U (S({xt }t=0 )))We call such U a recursive utility function. We also assume that
(A2) W is weakly increasing in its second argument, that is, for all x ∈ X and ally, z ∈ R with y ≤ z, W (x, y) ≤ W (x, z).
The followings are the examples of a recursive utility function in one–sector growthmodel.
Example 3.2 (the discounted TAS utility). A simple example of a recursive utility function is the discounted time additive separable (TAS) form
(7)

U ({ct }∞t=0 ) =∞∑

β t u(ct )

t=0which has an aggregator W (c0 , U ({ct }∞t=1 )) = u(c0 ) + β

∑∞

t=1 β

t−1

u(ct ), a return function

u : R+ → [−∞, ∞) and a discount factor β ∈ (0, 1).
Example 3.3 (the CEIS aggregator, the additive CRRA utility). The aggregator W
exhibits constant elasticity of intertemporal substitution (CEIS) if(8)1

W (x, y) = (x1−1/γ + βy 1−1/γ ) 1−1/γ ,6

M. YAO

where γ > 0 is the elasticity of intertemporal substitution and β > 0 is the discount factor.
If the return function u(c) = c and γ = 1/δ, we have the standard additive CRRA utility
function as follows:(9)

U ({ct }∞t=0 ) = (∞∑1

β t c1−δ) 1−δ .
tt=0

Example 3.4 (the EHU aggregator). A non-TAS utility is given by 5
(10)

U ({ct }∞t=0 ) = −∞∑exp(−[t=0t∑u(cτ )]).

τ =0This function has return function u and aggregator(11)

W (x, y) = (−1 + y) exp(−u(x)).

We refer to it Epstein-Hynes-Uzawa (EHU) aggregator.
Let Π and Π(x0 ) denote the set of feasible paths and that the set of feasible paths froman initial state x0 ∈ X respectively.
(12)∞Π = {{xt }∞: xt+1 ∈ Γ(xt ) (∀t ∈ Z+ )},t=0 ∈ X(13)∞
Π(x0 ) = {{xt }∞: {xt }∞
t=1 ∈ X
t=0 ∈ Π}, ∀x0 ∈ X.

Given x0 ∈ X, consider the following optimization problem:
(14)

sup

{xt }∞
t=1 ∈Π(x0 )U ({xt }∞t=0 ).

The value function v ∗ : X → R is defined by(15)v ∗ (x0 ) =sup{xt }∞
t=1 ∈Π(x0 )U ({xt }∞t=0 ),∀x0 ∈ X.

Let V be the set of functions v : X → R, and let v, w ∈ V . We define the partial order≤ on V in the usual way:(16)v ≤ v ′ ⇒ v(x) ≤ v ′ (x),∀x ∈ X.

If v ≤ v ′ , we define the order interval [v, v ′ ] by(17)[v, v ′ ] = {f ∈ V : v ≤ f ≤ v ′ }.

The order interval [v, v ∞ ] means [v(x), v ∞ (x)] with v ∞ (x) = ∞ for all x ∈ X.
The Bellman operator B from V into itself is defined by(18)

Bv(x) = sup W (x, v(y)), ∀x ∈ X, v ∈ V.
y∈Γ(x)

From assumption (A2) and (18), B is a monotone operator:(19)v ≤ v ′ ⇒ Bv ≤ Bv ′5The utility is discrete-time version of the modified Uzawa (1964) utility by Epstein and Hynes (1983).

RECURSIVE UTILITY AND THE SOLUTION TO THE BELLMAN EQUATION

7

We now present some fixed point theorems for the Bellman operator.
Proposition 3.1. Assume that there exists v ∈ V such that(20)v ≤ Bv

Then, there exist a fixed point of B in [v, v ∞ ].
□Proof. See the Appendix A.

As a direct consequence of Proposition 3.1 and Corollary 2.1, we have the following
corollary:
Corollary 3.1. Assume that there exists v ∈ V such that(21)

Bv ≤ v

Then, there exist a fixed point of B in [v −∞ , v].
By Proposition 3.2 and Corollary 2.2, we have the following corollary.
Corollary 3.2. Assume that there exists v, v ∈ V such that(22)v ≤ Bv(23)

Bv ≤ v(24)v ≤ v.

Then, there exist a fixed point of B in [v, v].
The following proposition shows that the value function v ∗ is fixed point of B in V .
Proposition 3.2. Assume that
(LC) W is lower semicontinuous in its second argument.
Then v ∗ ∈ V is a fixed point of B.
□

Proof. See the Appendix A.

As stated above, we know that the Bellman operator has a fixed point if there exists
some boundary function and the value function is the fixed point of the Bellman operator.
These are merely the discussion about the existence of a fixed point. From a practicalperspective, we need a method to calculate the fixed point of the Bellman operator. In
the next result, we solve the problem to add some assumption to aggregator.
Theorem 3.1. Assume (LC), and that there exists v ∈ V such that(25)v ≤ Bv.

Then the following conclusions hold:(a) v ∗ ≡ supn∈N (B n v) is the least fixed point of B in [v, v ∞ ].
(b) The increasing sequence {B n v}n∈N convergences to v ∗ pointwise.

8

M. YAO□

Proof. See the Appendix A.
As a direct consequence of Theorem 3.1, we have the following corollary:
Corollary 3.3. Assume (LC), and that there exist v, v ∈ V such thatv ≤ Bv,(26)(27)

Bv ≤ v,(28)v ≤ v.

Then the following conclusions hold:(a) v ∗ ≡ supn∈N (B n v) is the least fixed point of B in [v, v].
(b) The increasing sequence {B n v}n∈N convergences to v ∗ pointwise.
Theorem 3.1 and Corollary 3.3 say that the fixed point of the Bellman operator canbe computed by iteration starting from the lower boundary v which satisfies suitableconditions. This approach has the applicability to a wide class of the dynamic optimization
problem. However, there is a problem that how should we find such boundary. The next
result is a partial answer of the problem.
Proposition 3.3. Assume (LC), and that there exists v ∈ V such thatv ≤ Bv.

(29)
Then the following conclusions hold:(a) v ∗ ≡ supn∈N (B n v) is the least fixed point of B in [v, v ∞ ].
(b’) For all v0 ∈ [v, v ∗ ], the increasing sequence {B n v0 }n∈N convergences to v ∗ pointwise.
Proof. See the Appendix A.

□

With respect to the precondition and the conclusion (a), Proposition 3.3 is the same as
Theorem 3.1. Conclusion (b’) in Proposition 3.3 say that an arbitrary function v0 between
the boundary function v and the least fixed point v ∗ converges to v ∗ pointwise. If there
exists the boundary function, it means that the initial function that you start iteration
can be chosen freely to some extent.
4. The convergence from the upper boundary
In the previous section, we show that the fixed point of the Bellman operator canbe computed by iteration starting from the lower boundary. The natural question is,
can the fixed point be computed by iteration starting from the upper boundary of theorder interval just as in Theorem 3.1. The answer is no under our assumptions, even if theaggregator function W has some assumption as (AS). In the standard case of the bounded
returns, the fixed point of the Bellman equation can be computed by iteration startingfrom any function. But, in this case, that sup and inf cannot be replaced in general is theRECURSIVE UTILITY AND THE SOLUTION TO THE BELLMAN EQUATION

9cause of iteration starting from the upper boundary of the order interval not holding. In
this section, we tackle the problem.
At first, we assume that B is i–order continuous. Then we have the following result.
Theorem 4.1. Assume that there exists v ∈ V such that
Bv ≤ v.

(30)

and that B is i–order continuous on [v −∞ , v]. Then the following conclusions hold:(a) v ∗ ≡ inf n∈N (B n v) is the greatest fixed point of B in [v −∞ , v].
(b) The decreasing sequence {B n v}n∈N convergences to v ∗ pointwise.
□

Proof. See the Appendix A.

The conclusions in Theorem 4.1 may be regarded as a dual version of the conclusion
in Theorem 3.1. However, i-order continuity of the Bellman operator does not hold ingeneral since it means that for all x ∈ X and all {vn }n∈N ⊂ [v −∞ , v],(31)

inf sup W (x, vn (y)) = sup inf W (x, vn (y)).

n∈N y∈Γ(x)y∈Γ(x) n∈N

That is, the cause that the dual version of Theorem 3.1 does not hold is due to the factthat the Bellman operator is defined by supremum.
We make the following topological assumptions:(i) the state space is a topological space;(ii) the feasible correspondence is upper hemicontinuous and compact-valued; and(iii) the aggregator function is upper semicontinuous for both arguments.
That is,Assumption 4.1. X is a topological space, Γ is upper hemicontinuous and compact-valued
and W is upper semicontinuous for both arguments.
To show the next theorem, we prepare the following two lemmas. The first lemma is
standard result.
Lemma 4.1. Let v be any upper semicontinuous function in V . Then, the following
conditions hold under assumption 4.1:(i) for all x ∈ X, maxy∈Γ(x) W (x, v(y)) exists;(ii) Bv is upper semicontinuous and Bv ∈ V ; and(iii) for all n ∈ N, B n is upper semicontinuous and B n ∈ V .
Proof. See the Appendix A.

□

The following lemma is a variant of Sion’s minimax theorem (Sion (1958)) and the proof
is inspired by Komiya’s elementary proof of Sion’s minimax theorem (Komiya (1988)).

10

M. YAO

Lemma 4.2. Let C be a nonempty compact subset of a topological space, and let {fn }n∈Nbe a wealy decreasing sequence of an upper semicontinuous functions. Then(32)

max inf fn (x) = inf max fn (x)x∈C n∈Nn∈N x∈C□

Proof. See the Appendix A.6

Let V u denote the set of upper semicontinous functions from X into R where X isa topological space. It is clear that V u ⊂ V . Using above lemmas, we can prove thefollowing fixed point theorem.
Theorem 4.2. Under Assumption 4.1, assume that there exists v ∈ V u such that
Bv ≤ v.

(33)
Then, the following conditions hold:(a) v ∗ ≡ inf n∈N (B n v) is the greatest fixed point of B in [v −∞ , v] and v ∗ is uppersemicontinuous.
(b) The decreasing sequence {B n v}n∈N convergences to v ∗ pointwise.
□

Proof. See the Appendix A.

Using Theorem 4.1, we can construct an an alternative proof of Theorem 2.3 (a) in
Streufert (1998).
Appendix A. Proofs
A.1. Proof of Lemma 2.2. At first we show that sup f (A) ≤ f (sup A). For all a ∈ A,a ≤ sup A. By monotonicity of f , we have f (x) ≤ f (sup A). Then we have sup f (A) ≤f (sup A).
To show that the inverse inequality, we proceed by cases.
Case 1: sup A ∈ R.
Let α = sup A. For all n ∈ N, there exists an ∈ A such that(34)

an > α + 1/n.

By definition of supremum, an ≤ α. Then, an → α as n → ∞. So,(35)

f (sup A) = f (α)(36)≤ lim inf f (an )(37)≤ sup f (an )

n→∞

n∈N

(38)

≤ sup f (A),

6After completing proof, we find similar result and it is often called Dini–Cartan theorem. See Dellacherie

and Meyer (2011), p98.

RECURSIVE UTILITY AND THE SOLUTION TO THE BELLMAN EQUATION

11

where (36) holds since f is a lower semicontinuous function. Then we have f (sup A) ≤sup f (A).
Case 2: sup A = ∞. Let α = sup A. For all n ∈ N, there exists an ∈ A such thatan > max{n, an−1 }.
Then an → α as n → ∞. By the same argument of the Case 1, we have f (sup A) ≤sup f (A).
Case 3: sup A = −∞. Then, a = −∞ for all a ∈ A. Form the left hand side of (4), we
have
(39)

sup f (A) = sup f (x) = f (−∞).
x∈A

Form the right hand side of (4), we have(40)

f (sup A) = f (−∞).

By (39) and (40), we obtain (4).
A.2. Proof of Proposition 3.1. Since [v, v ∞ ] is complete lattice, the subset has suprermum.in [v, v ∞ ]. By the monotonicity of B and (20), B maps [v, v ∞ ] into itself. Then
{B n v} is the countable chain in [v, v ∞ ]. Thus, [v, v ∞ ] has a at most one chain and every
chain has suprermum in [v, v ∞ ]. Therefore, B has fixed points in [v, v ∞ ] by Theorem 2.1.
A.3. Proof of Proposition 3.2. Let A ⊂ R. For all b ∈ A, b ≤ supa∈A a. By themonotonicity of B, we have W (x, b) ≤ W (x, supa∈A a) for all x ∈ X. Then, we obtainsupa∈A W (x, a) ≤ W (x, supa∈A a). By (LC), we have(41)

sup W (x, a) = W (x, sup a).
a∈A

a∈A

For all x0 ∈ X, we have
(42)
(43)

Bv ∗ (x0 ) ==sup W (x0 , v ∗ (x1 ))x1 ∈Γ(x0 )sup W (x0 ,x1 ∈Γ(x0 )

(44)

=

sup

(45)

=

(46)

= v ∗ (x0 )sup{xt }∞
t=2 ∈Π(x1 )sup

x1 ∈Γ(x0 ) {xt }∞t=2 ∈Π(x1 )sup{xt }∞
t=1 ∈Π(x0 )U ({xt }∞t=1 ))

W (x0 , U ({xt }∞t=1 ))

W (x0 , U ({xt }∞t=1 ))

where (43) follows by definition of V ∗ , (44) follows by (41) and that x1 is independent by∗∗∗{xt }∞
t=2 and (45) follows by Lemma 2.1. Since x0 is arbitrary in X, Bv = v . That is, v
is fixed point of B in V .

12

M. YAO

A.4. Proof of Theorem 3.1. By the monotonicity of B and (25), B maps [v, v ∞ ] into
itself. We apply Theorem 2.2 to B : [v, v ∞ ] → [v, v ∞ ]. For this purpose, it suﬃces to showthat every countable chain in [v, v ∞ ] has superemum and that B is s–order continuous on[v, v ∞ ].
Since [v, v ∞ ] is complete lattice, every subset of [v, v ∞ ] has supremum. Then To seethat every countable chain in [v, v ∞ ] has superemum.
To see that B is s–order continuous, let {vn }n∈N be a countable chain in [v, v ∞ ]. Letx ∈ X. By the monotonicity of B, we have W (x, vn (y)) ≤ W (x, supn∈N vn (y)) for alln ∈ N and all y ∈ X. Then, we obtain supn∈N W (x, supn∈N vn (y)) ≤ W (x, supn∈N vn (y)).
By (LC), we have(47)

sup W (x, vn (y)) = W (x, sup vn (y)).
n∈N

n∈N

Then, We have(48)

[B(sup vn )](x) = sup W (x, (sup vn )(y))
n∈N

(49)

y∈Γ(x)n∈N= sup sup W (x, vn (y))
y∈Γ(x) n∈N(50)= sup sup W (x, vn (y))
n∈N y∈Γ(x)(51)= [sup(Bvn )](x)n∈Nwhere (49) follows by (47), and (50) follows by Lemma 2.1. Since x is arbitrary, it follows
that B supn∈N vn = supn∈N Bvn . Thus, B is s–order continuous on [v, v ∞ ].
Now by Theorem 2.2, conclusion (a) follows. To see (b), by the monotonicity of B and(29), we have v ≤ Bv ≤ B 2 v ≤ · · · . Then {B n v}n∈N is increasing. By Theorem 2.2,
supn∈N B n v = v ∗ . Therefore, we have the desired result.
A.5. Proof of Proposition 3.3. To see (b’), let v0 ∈ [v, v ∗ ]. That is, v ≤ v0 ≤ v ∗ . By
the monotonicity of B, Bv ≤ Bv0 ≤ Bv ∗ = v ∗ , and by induction B n v ≤ B n v0 ≤ v ∗ .
Since {B n v}n∈N converges to v ∗ pointwise, {B n v0 }n∈N also converges to v ∗ pointwise by
the squeeze theorem.
A.6. Proof of Theorem 4.1. By the monotonicity of B and (30), B maps [v −∞ , v] into
itself. By the assumption, B is i–order continous. Using similar argument in the proof of
Theorem 3.1, we have that every countable chain in [v −∞ , v] has infimum Now by Corollary
2.2, conclusion (a) follows. To see (b), by the monotonicity of B and (30), we havev ≥ Bv ≥ B 2 v ≥ · · · . Then {B n v}n∈N is decreasing. By Corollary 2.2, inf n∈N B n v = v ∗ .
A.7. Proof of Lemma 4.1.
(i) By Assumption 4.1, W is upper semicontinuous for both arguments and Γ isnonempty, compact-valued. Then maxy∈Γ(x) W (x, v(y)) exists for all x ∈ X by the
Weierstrass maximization theorem.

RECURSIVE UTILITY AND THE SOLUTION TO THE BELLMAN EQUATION

13(ii) By (i), for all x ∈ X,
(52)

(Bv)(x) = max W (x, v(y)).
y∈Γ(x)

Using the Berge maximization theorem, (Bv)(x) is upper semicontinuous for allx ∈ X. Since x is arbitrary in X, Bv is upper semicontinuous. By the definition
of V , Bv ∈ V .
(iii) By the definition of V and the results of (i) and (ii), we have the desired result.
A.8. Proof of Lemma 4.2. It is obvious that maxx∈C inf n∈N fn (x) ≤ inf n∈N maxx∈C fn (x).
Hence, we show the reverse inequality. Let α ∈ R with α > maxx∈C inf n∈N fn (x). We
define Cn = {x ∈ C : fn (x) ≥ α} for all n ∈ N. By upper semicontinuity of fn , Cn is com∩∩pact. Then n∈N Cn = ∅, and hence there exist n1 , · · · , nm ∈ N such that m
i=1 Cni = ∅.
Thus we have α > maxx∈C min1≤i≤m f (x, ni ). Without loss of generality, n1 ≥ · · · ≥ nm .
Since f is weakly decreasing in its second argument, f (x, n1 ) ≥ · · · ≥ f (x, nm ) for allx ∈ C. Let nm = n0 . We have f (x, n0 ) = min1≤i≤m f (x, ni ) for all x ∈ C. Then, there
exists n0 ∈ N with α > maxx∈C f (x, n0 ). Thus we obtain α > inf n∈N maxx∈C f (x, n).
Therefore we have(53)

max inf f (x, n) ≥ inf max f (x, n).
x∈C n∈Nn∈N x∈C

A.9. Proof of Theorem 4.2. By the monotonicity of B and (33), B maps [v −∞ , v] into
itself. We apply Theorem 2.2 to B : [v −∞ , v] → [v −∞ , v]. For this purpose, it suﬃces toshow that every countable chain in [v −∞ , v] has infimum and that B is i–order continuouson [v −∞ , v].
Since [v −∞ , v] is complete lattice, every subset of [v −∞ , v] has supremum. Then to seethat every countable chain in [v −∞ , v] has superemum.
To see that B is i–order continuous, let {vn }n∈N be a countable chain in [v −∞ , v]. Since
B(vn ) ⊂ [v −∞ , v], inf n∈N B(vn ) ∈ [v −∞ , v]. Let x ∈ X. By Lemma 4.1 and 4.2, we have
(54)
(55)

inf [B(vn )](x) = inf max {W (x, vn (y))}

n∈Nn∈N y∈Γ(x)= max {W (x, inf vn (y))}
y∈Γ(x)(56)n∈N= [B(inf vn )](x).
n∈NSince x is arbitrary, it follows that B inf n∈N vn = inf n∈N Bvn . Thus, B is i–order continuouson [v −∞ , v].
Now by Corollary 2.2, conclusion (a) follows. To see (b), by the monotonicity of B and(33), we have v ≥ Bv ≥ B 2 v ≥ · · · . Then {B n v}n∈N is decreasing. By Corollary 2.2,
inf n∈N B n v = v ∗ . Therefore, we have the desired result.
References[1] C. D. Aliprantis and K. C. Border, Infinite Dimensional Analysis: A Hitchhiker’s Guide, 3rd Edition,
Springer-Verlag, Berlin, 2006.

14

M. YAO[2] A Baranga, “The contraction principle as a particular case of Kleene’s fixed point theorem”, Discrete
Mathematics, 1991, 98, 75–79.
[3] R A. Becker, “On the long-run steady state in a simple dynamic model of equilibrium with heterogeneous households.” The Quarterly Journal of Economics 95.2, 1980, 375–382.
[4] R.A. Becker and J.H. Boyd III, Capital Theory, Equilibrium Analysis and Recursive Utility, Blackwell
Publishers, Oxford, 1997.
[5] J.H. Boyd III, “Recursive utility and the Ramsey problem”, Journal of Economic Theory, 1990, 50,
326–245.
[6] J.H. Boyd III, “Discrete–Time Recursive Utility”, In: R.-A. Dana, C. Le Van and K. Nishimura,Ed., Handbook on Optimal Growth 1 Discrete Time, Springer Berlin, Heidelberg, 2006, pp. 251–272.
[7] M. Datta, K. L. Reﬀett “Isotone Recursive Methods: The Case of Homogeneous Agents”, In: R.A. Dana, C. Le Van and K. Nishimura, Ed., Handbook on Optimal Growth 1 Discrete Time, Springer
Berlin, Heidelberg, 2006, pp. 203–250.
[8] C. Dellacherie, , P-A. Meyer, Probabilities and Potential, C: Potential Theory for Discrete andContinuous Semigroups. Elsevier 2011.
[9] L.G. Epstein and J.A. Hynes. “The rate of time preference and dynamic economic analysis.” The
Journal of Political Economy 1983: 611-635.
[10] I. Fisher, The Rate of Interest: Its nature, determination and relation to economic phenomena,
Macmillan, 1907.
[11] A. Granas and J. Dugundji, “Fixed Point Theory”. Springer-Verlag, New York, 2003.
[12] T. Kamihigashi. “On the principle of optimality for nonstationary deterministic dynamic programming”, International Journal of Economic Theory, 2008, 4, 519–525.
[13] T. Kamihigashi. “Elementary Results on Solutions to the Bellman Equation of Dynamic Programming: Existence, Uniqueness, and Convergence”, No. DP2012-31, 2012.
[14] T. Kamihigashi, K. Reﬀett, and M. Yao. “An application of Kleene’s fixed point theorem to dynamic
programming.” International Journal of Economic Theory 11.4, 2015: 429-434.
[15] T. Kamihigashi and M. Yao. “Deterministic Dynamic Programming in Discrete Time: A Monotone
Convergence Principle. No. DP2015-15. 2015a.
[16] T. Kamihigashi and M. Yao. “Dynamic Programming for Upper semicontinous Returns”, mimeo
2015b[17] T. C. Koopmans, “Stationary ordinal utility and impatience.” Econometrica: Journal of the Econometric Society , 1960, 287-309.
[18] H. Komiya. “Elementary proof for Sion’s minimax theorem.” Kodai Mathematical Journal, 1988,
11.1, 5–7.
[19] C. Le Van and Y. Vailakis. “Recursive utility and optimal growth with bounded or unbounded
returns”, J. Econ. Theory, 2005, 123, 187–209.
[20] R. E. Lucas, Jr and N. Stokey. “Optimal growth with many consumers”, Journal of Economic Theory,
1984, 32, 139–171.
[21] H. Ozaki, and P. A. Streufert. “Dynamic programming for non-additive stochastic objectives.” Journal of Mathematical Economics 25.4, 1996: 391–442.
[22] J.P. Rincón-Zapatero and C. Rodrı́guez-Palmero. “Recursive utility with unbounded aggregators.”
Economic Theory 33.2 , 2007, 381-391.
[23] M. Sion, ”On general minimax theorems.” Pacific J. Math 8.1, 1958: 171–176.
[24] N. Stokey and R. E. Lucas, Jr with E. C. Prescott, Recursive Methods in Economic Dynamics,
Harvard University Press, Cambridge, MA, 1989.
[25] P. A. Streufert, “Stationary Recursive Utility and Dynamic Programming under the Assumption of
Biconvergence”, Review of Economic Studies, 1990, 57, 79–97.

RECURSIVE UTILITY AND THE SOLUTION TO THE BELLMAN EQUATION

15[26] P. A. Streufert, “An Abstract Topological Approach to Dynamic Programming”, Journal of Mathematical Economics, 1992, 21, 59–88.
[27] P. A. Streufert, “Recursive Utility and Dynamic Programming”, In: S. Barberá, P. Hammond and
C. Seidl, Ed., Handbook of Utility Theory, Kluwe Academic Publishers, Dordreccht, 199, 1998, pp.
93–121.
[28] H. Uzawa “Time preference, the consumption function, and optimum asset holdings.” Value, Capital,
and Growth: Papers in Honour of Sir John Hicks (University of Edinburgh Press, Edinburgh) 1968:
485-504.
(M. Yao) Department of Economics, Keio University, 2-15-45 Mita, Minato-ku, Tokyo
108-8345, Japan.
E-mail address: myao@gs.econ.keio.ac.jp