# A Stochastic Dynamic Programming Approach Based on Bounded Rationality and Application to Dynamic Portfolio Choice

Authors: Wenjie Bi; Liuqing Tian; Haiying Liu; Xiaohong Chen
Journal: Discrete Dynamics in Nature and Society
 (2014)

---

## Abstract

Dynamic portfolio choice is an important problem in finance, but the optimal strategy analysis is difficult when considering multiple stochastic volatility variables such as the stock price, interest rate, and income. Besides, recent research in experimental economics indicates that the agent shows limited attention, considering only the variables with high fluctuations but ignoring those with small ones. By extending the sparse max method, we propose an approach to solve dynamic programming problem with small stochastic volatility and the agent’s bounded rationality. This approach considers the agent’s behavioral factors and avoids effectively the “Curse of Dimensionality” in a dynamic programming problem with more than a few state variables. We then apply it to Merton dynamic portfolio choice model with stochastic volatility and get a tractable solution. Finally, the numerical analysis shows that the bounded rational agent may pay no attention to the varying equity premium and interest rate with small variance.


---

## Body

## 1. Introduction

In reality, how to choose an asset’s portfolio of consumption and investment is one of the most important decisions for many people. In modern portfolio choice field, Merton [1, 2] provides a general framework for understanding the portfolio demand of long-term investors when investment opportunities change over time. In a classical Merton model [1, 2], however, the riskless interest rate, the risky mean rate of return, and the volatility coefficient are usually assumed to be constant. These assumptions are lack of realism, particularly over long time intervals. A large volume of empirical researches in financial market which indicates the assumption that these variables are stochastic volatile and follow a certain stochastic process (e.g., Ornstein-Uhlenbeck process) is more realistic [3, 4]. But when introducing these stochastic variables into the Merton-style portfolio choice model, the problem becomes increasingly complicated and formidable to solve. Also, this will lead to the “Curse of Dimensionality.” Quite a lot of approaches have been developed to deal with this kind of problems, such as martingale methods [5–8] and various approximate numerical algorithms [9–12]. However, these methods have more restrictive assumptions and are too complex to get a tractable solution of strong explanations. Based on the control of small noise, Judd and Guu [13] proposed a method to solve dynamic programming problems with stochastic disturbance. He makes the simplifying assumption that uncertainty is small and obtains the first- and high-order solutions of complicated dynamic programming model. This method provides a quite suitable solution for dynamic portfolio choice model with stochastic volatility.On the other hand, a growing body of empirical studies indicate that the agent considers only the variables with high fluctuations but ignores those with small ones [14–16]. Bordalo et al. [17] showed that the agent rationally chooses to be inattentive to news. Kőszegi and Szeidl [18] analyzed the monetary policy and found out that when price is changed, the decision makers are usually unaware of it. There are also many literatures showing that the agent pays attention to salient factors. Sims [19] uses two empirical strategies to analyze how individuals optimize fully with respect to the incentives created by tax policies and shows that tax salience affects agents’ behavioral response. Peng and Xiong [20] study the allocation of investors’ attention among different information. They find out that investors with limited attention will focus on macroeconomic and industry information rather than that of a specific firm. Seasholes and Wu [21] demonstrate that attention-grabbing events will attract investors’ attention. In their model, they regard them as the proxy variables and their results empirically indicate that these events have a significant impact on the allocation of investor’s attention. Maćkowiak and Wiederholt [22] show that decision makers’ attention is usually drawn to salient payoffs.In recent years, Gabaix [23] provides a sparse max operator to model dynamic programming with bounded rationality. In the sparse max, the agent pays less or no attention to some features the fluctuations of which are smaller than some thresholds, and he tries to strike a good balance between the utility loss of inattention and the cognitive cost which can be regarded as the loss for taking time to think about the decisions rather than to enjoy oneself. The sparse max seems more realistic than traditional economic models since it has a very robust psychological foundation. Also, it can deal with problems of maximization with constraints easily and get a tractable solution in a parsimonious way.However, Gabaix [23] only studies the dynamic programming in a stationary environment without the stochastic volatility terms. But the financial market is strewn with numerous stochastic dynamic programming problems, and these problems are hard to solve due to multitudinous state variables. To address this issue, we extend the sparse max operator and develop a stochastic version of Gabaix’s method. The distinctive feature of this method is that it considers the agent’s behavioral factors (limited attention) and can effectively preclude the “Curse of Dimensionality” for multiple variables. To verify the validity and practicability of our model, we consider the Merton dynamic portfolio choice problem with stochastic volatility variables (e.g., [24, 25]) and get a tractable solution.The remainder of this paper is organized as follows. Section2 presents the sparse dynamic programming method proposed by Gabaix [23]. Section 3 extends this model and gives a general principle for solving continuous-time dynamic programming with stochastic variables. In Section 4, we apply our method to Merton dynamic portfolio choice. Finally, we discuss some implications of our findings and suggest topics for future research in Section 5.


## 2. The Sparse Max Operator without Constraints

We mainly introduce the sparse max operator proposed by Gabaix [23] in this section. In the traditional version, the agent faces a maximization problem:
(1)max⁡a⁡u(a,y)subjecttob(a,y)≥0,
where y=(y1,y2,…,yn), u is a utility function and b is a constraint. Variable a and function b have arbitrary dimensions. For any optimal decision, in principle, thousands of considerations are relevant to the agent. Since it would be too burdensome to take all of these variables into account, the agent is used to discarding most of them. At the same time, his attention is allocated purposefully to important variables.Hence, the agent might sensibly pick a “sparse” representation of the variables; namely, choose the attention vectorm=(m1,m2,…,mn) to replace variable yi with yis=miyii∈(1,2,…,n), where the superscript s of yis represents sparse. The optimal attention vector is obtained by weighing the utility losses for imperfect inattention against the cost savings without thinking too much.The utility losses from imperfect inattention can be expressed as follows [23]:
(2)E[v(m)-v(l)]=-12∑i,j(mi-1)Λij(mj-1)+o(∥y∥2),
where v(m):=u(a(ys(m)),y) is the utility for a sparse agent, ys(m)=(y1s,y2s,…,yns), l:=(1,1,…,1)T, and v(l) is the utility when the agent is fully attentive. o(∥y∥2) denotes the second-order infinitesimal of y. Λij=-σijayi2uaa, where σij=cov⁡(yi,yj), E(yi)=0, σi is the standard deviation of yi, and ayi=-uaa/uayi which indicates by how much a change yi should change the action for traditional agent. uaa is the second derivative of u with respect to a. All derivatives above are evaluated at y=0 and the default action ad=argmaxau(a,0).Gabaix [23] assumes the cognitive cost is c(mi)=κ|mi|β, where β≥0 and parameter κ≥0 is a penalty for lack of sparsity. If κ=0, the agent will be a traditional, rational agent.Based on above analysis, Gabaix [23] defines the sparse max operator as follows.Definition 1 (see [23] Sparse max operator without constraints).
The sparse max defined by the following procedure.Step 1.
Choose the attention vectorm*(3)m*=argmin⁡m12∑i·j=1n(1-mi)Λij(1-mj)+κ∑i=1n|mi|β.
Define yis=mi*yi as the sparse representation of yi.Step 2.
Choose the action(4)as=argmax⁡au(a,ys),
and set the resulting utility to be us=u(as,y).
Supposey is one-dimensional vector; formula (3) can be transformed into m*=min⁡m(1/2)(m-1)2σ2+κ|m|. Gabaix [23] defines a function A to represent the optimal attention vector, namely, m*=Aβ(σ2/κ)=inf⁡[argmin⁡m(1/2)(m-1)2(σ2/κ)+|m|β] and points out when β=1, the function Aβ(σ2/κ) satisfies the sparsity and continuity. When β=1 and κ=1, we have A1(σ2)=max⁡(1-1/σ2,0) as shown in Figure 1 [23].
From Figure1 we know that the agent will not consider the variable when 0≤σ2≤κ(κ=1).
When the vectory includes more than one variable and these variables perceived by the agent are uncorrelated, we have mi*=A1(σi2ayi2|uaa|/κ) through formula (3). To analyze the agent’s inattention expediently, Gabaix [23] defines the truncation function τ(s,p)=smax⁡(1-p2/s2,0), so we have mi*=τ(1,κσa/ayiσi). Truncation function has more intuitive economic implications: a one-standard-deviation change of the variable yi makes the agent change his action by ayi(σi/σa). When ayi(σi/σa) is small and satisfies |ayi(σi/σa)|≤κ, the agent will not consider this factor. Figure 2 shows the truncation function τ(s,p)=smax⁡(1-p2/s2,0) [23].
From Figure2, we know that the agent who seeks “sparsity” should sensibly drop relatively unimportant features. In addition, if the features are larger than that cutoff, they are still dampened: in Figure 2, τ(s,p) is below the 45 degree line (for positive s, in general, |τ(s,p)|<|s|).
Based on the analysis above, we can use the truncation function to represent the sparse agent’s optimal action.Figure 1
The attention functionA1.Figure 2
The truncation functionτ(s,p) with p=1.Remark 2 (see [23]).
If rational optimal action is(5)ar(y)=ad+∑iayiyi+o(∥y∥)2(rrepresentstherationality),
which is obtained by the Taylor expansion around the default action ad, then the sparse agent’s optimal action is
(6)as(y)=ad+∑iτ(ayi,κσaσi)yi+o(∥y∥2),
where σa is the standard deviation of a.


## 3. A Stochastic Dynamic Programming Approach Based on Sparse Max Operator

In order to effectively deal with stochastic dynamic programming in finance in this section, we extend Gabaix [23] sparse max operator and propose a bounded rational stochastic dynamic programming model in continuous time.The general model of stochastic dynamic programming in continuous time is(7)max⁡∫0∞e-ρtu(a,x,y)dtdx=g(x(t),y(t),a(t))dt+σ(x(t),a(t))dZx(t),dy=h(x(t),y(t))dt+σ(y(t))dBy(t),
whereρdenotes the discount factor, u is the utility function, a is the decision variable which has an arbitrary dimension, the vector x represents important factors which are always considered by the agent, and the vector y defined in Section 2 represents factors that that may not be considered by the sparse agent. g(x,y,a), h(x,y) are the state transition function of x and y, respectively. And σ(x,a), σ(y) represent the stochastic volatility of x and y, respectively. dZx, dBy are independent standard Brownian motions; namely dZxdBy=0. We define the value function as V(x,y)=∫0∞e-ρtu(a,x,y)dt.Assumption 3.
The utility functionu and value function V are n-order continuously differentiable (Cn, n≥3).Assumption 4.
All state variables are stochastic and they are independent of each other; stochastic volatility ofx is a function of x and a while stochastic volatility of y is uncorrelated with a.Assumption 5.
x is one dimensional; that is, only one variable would be always considered by the agent and other variables may not be considered by the agent.Assumption 6.
According to Judd and Guu [13], we assume the variance of each component of vector y is small and independent of one another.To facilitate analysis, we usex to replace x, denote the stochastic differential equation of yi by dyi=hi(x(t),yi(t))dt+σi(yi(t))dByi(t), i∈(1,2,…,n), and use the notation Dx[f]=∂x[f]+ax∂a[f] as the total derivative with respect to x (i.e., the full impact of a change in x, including the impact it has on a change in the action a).Based on Remark2 in Section 2, we have the following proposition.Proposition 7.
The optimal action in bounded rationality model (7) is
(8)as(x,y)=ad(x)+∑iτ(ayi,κσaσi)yi+o(∥y∥2),
where σa is the standard deviation of a.Proof.
See the  appendix.From Proposition7, we know that, in order to derive the optimal action as, we should get the default action ad which is related to x and ayi. The detail process of solving them is described as the following steps, which contain the main results of our method.Step 1.
Solve default actionad.
By substitutingy=0 into the basic model (7), we get the default model:
(9)max⁡∫0∞e-ρtu(a,x,0)dtdx=g(x(t),0,a(t))dt+σ(x(t),a(t))dZx(t).
This is a general dynamic programming model in continuous time and the state variable is one dimension, so we can get the optimal default action ad and the value function V(x,0)=∫0∞e-ρtu(a,x,0)dt easily [26].Step 2.
Solveayi.
The following Proposition8 and its proof in the appendix show the result and the process of obtaining ayi.Proposition 8.
The impact of a change inyi on the value function is
(10)Vxyi=(12σ2(x,a)fx(x)VyiDx[uayi+gyi(x,y,a)Vx]+∑i=1nhxyii(x,yi,a)Vyi+12σx2(x,a)f(x)Vyi+12σ2(x,a)fx(x)Vyi)×(ρ-∑i=1nhyii(x,yi,a)-12σ2(x,a)f(x))-1,
where
(11)Vyi=(uayi+gyi(x,y,a)Vx)×(ρ-∑i=1nhyii(x,yi,a)-12σ2(x,a)f(x))-1,f(x)=VxxV.
By implicit function theorem, the impact of a change inyion the optimal action is ayi=-ϕyi/ϕa, where
(12)ϕa=uaa+gaa(x,y,a)Vx+12σaa2(x,a)Dx[Vx],ϕyi=uayi+gayi(x,y,a)Vx+ga(x,y,a)Vxyi+12σa2(x,a)Dx[Vxyi].Proof.
See the appendix.Now we can get the optimal action based on the two steps above. By the analysis of Proposition7, we can see that τ(ayi,κσa/σi) represents the impact of variable yi on the action as. When ayi is smaller than κσa/σi, τ(ayi,κσa/σi)=0 which means the agent will discard this factor.


## 4. Application: Dynamic Portfolio Choice

### 4.1. Merton Portfolio Problem with Stochastic Volatility

In this section, we consider a Merton dynamic portfolio choice problem with stochastic volatility in continuous time [24]. In the traditional version of Merton model [1], the agent’s optimal problem is
(13)max⁡E∫0∞e-ρtu(c(t))dt,s.t.dw(t)=w(t)(r+(b-r)θ(t))dt+θ(t)σw(t)dW(t)-c(t)dt,
where ρ is the discount factor, u is the utility function, w(t) is the wealth at time t, σ is the standard deviation of w(t), and c(t) is the consumption at time t. The investment control θ(t) at time t is the fraction of the wealth invested in the risky asset, so (1-θ(t)) is the fraction of the wealth invested on the riskless asset. r is the riskless interest rate and b is the risky mean rate of return; dW follows standard Brownian motion. We assume the utility function (c)=c(t)1-γ/(1-γ), where γ(0<γ<1) is the parameter of risk preference. The goal is to choose consumption c(t) and investment θ(t) control processes to maximize long-term utility.In model (13), the riskless interest rate r and the risky mean rate of return b are assumed to be constant [1]. However, this assumption is unrealistic, particularly over long time intervals [27, 28]. Instead, now we assume that these two variables are stochastic and satisfying r(t)=r-+r^(t), b(t)=b-+b^(t), where r-, b- represent the long mean of the riskless interest rate and the risky rate of return, respectively, and r^(t), b^(t) are their volatile part. r(t) and b(t) depend on some “economic factor” η(t) [24]; namely,
(14)r(t)=R(η(t)),b(t)=M(η(t)).
And η(t) satisfies the stochastic differential formula [24]:
(15)dη=g(η(t))dt+σ~[λdW(t)+(1-λ2)1/2dW~(t)],
where W~(t) and W(t) are independent standard Brownian motions. The parameter λ with |λ|<1 allows a correlation between the Brownian motion W(t) driving the short rate and its volatility. σ~ is the standard deviation of η.The budget equation of model (13) now becomes
(16)dw=w(t)(r(t)+(b(t)-r(t))θ(t))dt+θ(t)σw(t)dW(t)-c(t)dt.We further assume that the stochastic differential equation ofr(t)=η(t) follows an Ornstein-Uhlenbeck process [25, 29]:
(17)dr=-ζr(r--r(t))dt+σr[λrdW(t)+(1-λr2)1/2dR(t)],
where ζr is the degree of mean reversion in expected excess returns, σr is the standard deviation of r(t), R(t) is a Brownian motion and independent of W(t), and λr is the coefficient of w and r. To simplify our model, we assume λr=0 [30]. By substituting r(t)=r-+r^(t) into (17), we obtain d(r-+r^(t))=-ζrr^(t)dt+σrdR(t). Since dr-=0, it follows that dr^=-ζrr^(t)dt+σrdR(t).We assume thatb(t) follows an Ornstein-Uhlenbeck process too, so we obtain db^=-ζbb^(t)dt+σbdB(t) where ζb has the same meaning with ζr. σb is the standard deviation of b(t), and we assume σr and σb are small [13]. B(t) is a Brownian motion and independent of R(t) and W(t). Then we get the following model:
(18)max⁡E∫0∞e-ρtc(t)1-γ1-γdt(19)s.t.dw=w(t)((r(t)+θ(t)(b(t)-r(t))))dt+θ(t)σw(t)dW(t)-c(t)dt,r(t)=r-+r^(t),b(t)=b-+b^(t),dr^=-ζrr^(t)dt+σrdR(t),db^=-ζbb^(t)dt+σbdB(t).From Section3, we know that, for the bounded rational agent, the optimal consumption cs and the optimal fraction of wealth allocated to risky market θs in model (18) can be expressed as
(20)cs=cd+τ(cr^,κσcσr)r^+τ(cb^,κσcσb)b^,θs=θd+τ(θr^,κσθσr)r^+τ(θb^,κσθσb)b^,
where cd and θd are the default actions when r^(t)=0, b^(t)=0. σc, σθ are the standard deviation of c and θ, respectively. cr^, cb^, θr^, and θb^ are the impact of r^ and b^ on c and θ, respectively. Next we will give the process of solving them using the approach described in Section 3.Step 1.
Solve the default decisioncd and θd.
By substitutingr^(t)=0, b^(t)=0 into the basic model (18), we get the default model:
(21)max⁡E∫0∞e-ρtc(t)1-γ1-γdtdw=w(t)(r-+θ(t)(b--r-))dt+θ(t)σw(t)dW(t)-c(t)dt.
This is the same Merton model as the model in (13); many scholars have solved this problem [1, 2]. We define the value function to be V(w)=∫0∞e-ρt(c(t)1-γ/(1-γ))dt; then we have [2]
(22)cd=w(R(1-γ))-1/γ,θd=b--r-γσ2,V(w)=Rw1-γ1-γ,
where R=[(ρ-r-(1-γ)-(b--r-)2(1-γ)/2σ2γ)/γ]-γ/(1-γ).Step 2.
Solvecr^, cb^, θr^, and θb^.
Next we will give the results ofcr^, cb^, θr^, and θb^. Proposition 9 shows their expressions, and the proof is the solution process.Proposition 9.
Based on the results of (22) and implicit function theorem, we have
(23)cr^=-(1-γ)(1-θd)cd(ρ+ζr+(1/2)γ(1-γ)θd2σ2)γ,cb^=-(1-γ)cd(ρ+ζb+(1/2)γ(1-γ)θd2σ2)γ,θr^=-1γσ2,θb^=1γσ2.
And the final results of model (18) are
(24)cs=cd+τ(-1-θdM,κσcσr)r^+τ(-1N,κσcσb)b^,θs=θd+τ(-1γσ2,κσθσr)r^+τ(1γσ2,κσθσb)b^,
where
(25)M=(1-γ)cd(ρ+ζr+(1/2)γ(1-γ)θd2σ2)γ,N=(1-γ)cd(ρ+ζu+(1/2)γ(1-γ)θd2σ2)γ,cd=w(R(1-γ))-1/γ,R=[(ρ-r-(1-γ)-(b--r-)2(1-γ)/2σ2γ)/γ]-γ(1-γ),θd=b--r-γσ2.Proof.
See the appendix.Proposition9 makes predictions about the sparse agent’s choice. When κ=0, the agent is the traditional, perfectly rational agent. And when κ>0, it is a policy of a sparse agent. Larger κ indicates that the agent is less sensitive to fluctuations of both the riskless interest rate and the risky mean rate of return.


### 4.2. Numerical Example

The purpose of this numerical analysis is to intuitively understand how the boundedly rational agent changes its decisions with the changing of the variances of factors. Firstly, we setcr^s=τ(-(1-θd)/M,κσc/σr), cb^s=τ(-1/N,κσc/σb), θr^s=τ(-1/γσ2,κσθ/σr), and θb^s=τ(1/γσ2,κσθ/σb). Let γ=0.6, σ=2.2, σr=1.5, σb=1.7, σθ=0.5, σc=1.3, b-=3.2, r-=18, ρ=0.95, ζr=0.2, ζb=0.4, and w=5; then we have cr^s, cb^s as shown in Figures 3 and 4. In these figures, the horizontal axis κ is an index of bounded rationality and κ∈[0,1] which is also applied to Figures 5 and 6.(a) Impact of a change inr^ on cs with σr=1.5, σb=1.7. (b) Impact of a change in b^ on cs with σr=1.5, σb=1.7.
(a)
(b)(a) Impact of a change inr^ on θs with σr=1.5, σb=1.7. (b) Impact of a change in b^ on θs with σr=1.5, σb=1.7.
(a)
(b)(a) Impact of a change inr^ on cs with σr=0.15, σb=0.25. (b) Impact of a change in b^ on cs with σr=0.15, σb=0.25.
(a)
(b)(a) Impact of a change inr^ on θs with σr=0.15, σb=0.25. (b) Impact of a change in b^ on θs with σr=0.15, σb=0.25.
(a)
(b)From Figure3, we know that whatever κ is, |cr^s|>0 and |cb^s|>0, which means when the variances of r^ and b^ are big, the agent will consider them in the process of making a decision.Figure3(a) shows that if κ=0, then the agent reacts like the rational agent: when r^ goes up by 1%, cr^s will fall by −2.19% (the agent saves more). For κ=1, if r^ goes up by 1%, cr^s falls by −1.85%. This result indicates that the greater the cognitive cost about the factor is, the less attention will be paid to this factor by the boundedly rational agent. From Figure 3(b), we can reach a similar conclusion.Figure4 also shows the agent will always consider r^ and b^, that is, |θr^s|>0 and |θb^s|>0, whatever κ is. It addition, we can obtain that if κ=0, θr^s=-0.34% and θb^s=0.34%, which means the rational agent has the same sensitivity about the r^ and b^ when deciding θs. With the increasing of κ, the absolute values of θr^s and θb^s both will decrease, which means that the agent will pay less attention to them. In other words, the impact of r^ and b^ on θs will decrease for the increasing cognitive cost.Next, we assume that the standard deviation of riskless interest rate and the risky mean rate of return is smaller,σr=0.15, σb=0.25. By keeping other parameters fixed, we get results shown in Figures 5 and 6.Figure5(a) shows that when κ≥0.26, cr^s=0 which means that if the fluctuation is small the agent may discard r^ when he decides the optimal consumption. We can get a similar conclusion from Figure 5(b): when κ≥0.95, cr^s=0 with σb=0.25.From Figure5(a), we know that if κ=0, cr^s=-2.19% while Figure 3(a) also shows if κ=0, cr^s=-2.19% which means, for the rational agent, the sensitivity of cs to r^ has nothing to do with r^’s variance. However, the boundedly rational agents have different reactions to r^ as κ increases, such as when κ=0.26, cr^s=0 with σr=0.15 in Figure 5(a) while cr^s=-2.1% with σr=1.5 in Figure 3(a). This disparity indicates that when the cognitive costs are the same and κ>0, that is, the agents have the same boundedly rational degree, more volatile factors will be considered while the factor with smaller variance may be neglected.Additionally, we can know that when0.26≤κ<0.95, the agent does not react to r^, namely, cr^s=0 (in Figure 5(a)), but will react to a change in b^ (in Figure 5(b)), which is more important: the sensitivity of cs to b^ remains high even for a high cognitive friction κ. Note that this “feature by feature” selective attention could not be rationalized by just a fixed cost to consumption, which is not feature dependent. But when κ≥0.95, cr^s=cb^s=0, which indicates that the agent will pay no attention to both r^ and b^ once their thinking costs are beyond some thresholds.Considering Figure6(a), we can see that when κ≥0.11, θr^s=0, while Figure 4(a) shows whatever κ is, θr^s>0 with σr=1.5 which means the smaller the variance of a factor is, the more likely the agent will ignore it. From Figure 6(b), we can also obtain the same conclusion.


## 4.1. Merton Portfolio Problem with Stochastic Volatility

In this section, we consider a Merton dynamic portfolio choice problem with stochastic volatility in continuous time [24]. In the traditional version of Merton model [1], the agent’s optimal problem is
(13)max⁡E∫0∞e-ρtu(c(t))dt,s.t.dw(t)=w(t)(r+(b-r)θ(t))dt+θ(t)σw(t)dW(t)-c(t)dt,
where ρ is the discount factor, u is the utility function, w(t) is the wealth at time t, σ is the standard deviation of w(t), and c(t) is the consumption at time t. The investment control θ(t) at time t is the fraction of the wealth invested in the risky asset, so (1-θ(t)) is the fraction of the wealth invested on the riskless asset. r is the riskless interest rate and b is the risky mean rate of return; dW follows standard Brownian motion. We assume the utility function (c)=c(t)1-γ/(1-γ), where γ(0<γ<1) is the parameter of risk preference. The goal is to choose consumption c(t) and investment θ(t) control processes to maximize long-term utility.In model (13), the riskless interest rate r and the risky mean rate of return b are assumed to be constant [1]. However, this assumption is unrealistic, particularly over long time intervals [27, 28]. Instead, now we assume that these two variables are stochastic and satisfying r(t)=r-+r^(t), b(t)=b-+b^(t), where r-, b- represent the long mean of the riskless interest rate and the risky rate of return, respectively, and r^(t), b^(t) are their volatile part. r(t) and b(t) depend on some “economic factor” η(t) [24]; namely,
(14)r(t)=R(η(t)),b(t)=M(η(t)).
And η(t) satisfies the stochastic differential formula [24]:
(15)dη=g(η(t))dt+σ~[λdW(t)+(1-λ2)1/2dW~(t)],
where W~(t) and W(t) are independent standard Brownian motions. The parameter λ with |λ|<1 allows a correlation between the Brownian motion W(t) driving the short rate and its volatility. σ~ is the standard deviation of η.The budget equation of model (13) now becomes
(16)dw=w(t)(r(t)+(b(t)-r(t))θ(t))dt+θ(t)σw(t)dW(t)-c(t)dt.We further assume that the stochastic differential equation ofr(t)=η(t) follows an Ornstein-Uhlenbeck process [25, 29]:
(17)dr=-ζr(r--r(t))dt+σr[λrdW(t)+(1-λr2)1/2dR(t)],
where ζr is the degree of mean reversion in expected excess returns, σr is the standard deviation of r(t), R(t) is a Brownian motion and independent of W(t), and λr is the coefficient of w and r. To simplify our model, we assume λr=0 [30]. By substituting r(t)=r-+r^(t) into (17), we obtain d(r-+r^(t))=-ζrr^(t)dt+σrdR(t). Since dr-=0, it follows that dr^=-ζrr^(t)dt+σrdR(t).We assume thatb(t) follows an Ornstein-Uhlenbeck process too, so we obtain db^=-ζbb^(t)dt+σbdB(t) where ζb has the same meaning with ζr. σb is the standard deviation of b(t), and we assume σr and σb are small [13]. B(t) is a Brownian motion and independent of R(t) and W(t). Then we get the following model:
(18)max⁡E∫0∞e-ρtc(t)1-γ1-γdt(19)s.t.dw=w(t)((r(t)+θ(t)(b(t)-r(t))))dt+θ(t)σw(t)dW(t)-c(t)dt,r(t)=r-+r^(t),b(t)=b-+b^(t),dr^=-ζrr^(t)dt+σrdR(t),db^=-ζbb^(t)dt+σbdB(t).From Section3, we know that, for the bounded rational agent, the optimal consumption cs and the optimal fraction of wealth allocated to risky market θs in model (18) can be expressed as
(20)cs=cd+τ(cr^,κσcσr)r^+τ(cb^,κσcσb)b^,θs=θd+τ(θr^,κσθσr)r^+τ(θb^,κσθσb)b^,
where cd and θd are the default actions when r^(t)=0, b^(t)=0. σc, σθ are the standard deviation of c and θ, respectively. cr^, cb^, θr^, and θb^ are the impact of r^ and b^ on c and θ, respectively. Next we will give the process of solving them using the approach described in Section 3.Step 1.
Solve the default decisioncd and θd.
By substitutingr^(t)=0, b^(t)=0 into the basic model (18), we get the default model:
(21)max⁡E∫0∞e-ρtc(t)1-γ1-γdtdw=w(t)(r-+θ(t)(b--r-))dt+θ(t)σw(t)dW(t)-c(t)dt.
This is the same Merton model as the model in (13); many scholars have solved this problem [1, 2]. We define the value function to be V(w)=∫0∞e-ρt(c(t)1-γ/(1-γ))dt; then we have [2]
(22)cd=w(R(1-γ))-1/γ,θd=b--r-γσ2,V(w)=Rw1-γ1-γ,
where R=[(ρ-r-(1-γ)-(b--r-)2(1-γ)/2σ2γ)/γ]-γ/(1-γ).Step 2.
Solvecr^, cb^, θr^, and θb^.
Next we will give the results ofcr^, cb^, θr^, and θb^. Proposition 9 shows their expressions, and the proof is the solution process.Proposition 9.
Based on the results of (22) and implicit function theorem, we have
(23)cr^=-(1-γ)(1-θd)cd(ρ+ζr+(1/2)γ(1-γ)θd2σ2)γ,cb^=-(1-γ)cd(ρ+ζb+(1/2)γ(1-γ)θd2σ2)γ,θr^=-1γσ2,θb^=1γσ2.
And the final results of model (18) are
(24)cs=cd+τ(-1-θdM,κσcσr)r^+τ(-1N,κσcσb)b^,θs=θd+τ(-1γσ2,κσθσr)r^+τ(1γσ2,κσθσb)b^,
where
(25)M=(1-γ)cd(ρ+ζr+(1/2)γ(1-γ)θd2σ2)γ,N=(1-γ)cd(ρ+ζu+(1/2)γ(1-γ)θd2σ2)γ,cd=w(R(1-γ))-1/γ,R=[(ρ-r-(1-γ)-(b--r-)2(1-γ)/2σ2γ)/γ]-γ(1-γ),θd=b--r-γσ2.Proof.
See the appendix.Proposition9 makes predictions about the sparse agent’s choice. When κ=0, the agent is the traditional, perfectly rational agent. And when κ>0, it is a policy of a sparse agent. Larger κ indicates that the agent is less sensitive to fluctuations of both the riskless interest rate and the risky mean rate of return.


## 4.2. Numerical Example

The purpose of this numerical analysis is to intuitively understand how the boundedly rational agent changes its decisions with the changing of the variances of factors. Firstly, we setcr^s=τ(-(1-θd)/M,κσc/σr), cb^s=τ(-1/N,κσc/σb), θr^s=τ(-1/γσ2,κσθ/σr), and θb^s=τ(1/γσ2,κσθ/σb). Let γ=0.6, σ=2.2, σr=1.5, σb=1.7, σθ=0.5, σc=1.3, b-=3.2, r-=18, ρ=0.95, ζr=0.2, ζb=0.4, and w=5; then we have cr^s, cb^s as shown in Figures 3 and 4. In these figures, the horizontal axis κ is an index of bounded rationality and κ∈[0,1] which is also applied to Figures 5 and 6.(a) Impact of a change inr^ on cs with σr=1.5, σb=1.7. (b) Impact of a change in b^ on cs with σr=1.5, σb=1.7.
(a)
(b)(a) Impact of a change inr^ on θs with σr=1.5, σb=1.7. (b) Impact of a change in b^ on θs with σr=1.5, σb=1.7.
(a)
(b)(a) Impact of a change inr^ on cs with σr=0.15, σb=0.25. (b) Impact of a change in b^ on cs with σr=0.15, σb=0.25.
(a)
(b)(a) Impact of a change inr^ on θs with σr=0.15, σb=0.25. (b) Impact of a change in b^ on θs with σr=0.15, σb=0.25.
(a)
(b)From Figure3, we know that whatever κ is, |cr^s|>0 and |cb^s|>0, which means when the variances of r^ and b^ are big, the agent will consider them in the process of making a decision.Figure3(a) shows that if κ=0, then the agent reacts like the rational agent: when r^ goes up by 1%, cr^s will fall by −2.19% (the agent saves more). For κ=1, if r^ goes up by 1%, cr^s falls by −1.85%. This result indicates that the greater the cognitive cost about the factor is, the less attention will be paid to this factor by the boundedly rational agent. From Figure 3(b), we can reach a similar conclusion.Figure4 also shows the agent will always consider r^ and b^, that is, |θr^s|>0 and |θb^s|>0, whatever κ is. It addition, we can obtain that if κ=0, θr^s=-0.34% and θb^s=0.34%, which means the rational agent has the same sensitivity about the r^ and b^ when deciding θs. With the increasing of κ, the absolute values of θr^s and θb^s both will decrease, which means that the agent will pay less attention to them. In other words, the impact of r^ and b^ on θs will decrease for the increasing cognitive cost.Next, we assume that the standard deviation of riskless interest rate and the risky mean rate of return is smaller,σr=0.15, σb=0.25. By keeping other parameters fixed, we get results shown in Figures 5 and 6.Figure5(a) shows that when κ≥0.26, cr^s=0 which means that if the fluctuation is small the agent may discard r^ when he decides the optimal consumption. We can get a similar conclusion from Figure 5(b): when κ≥0.95, cr^s=0 with σb=0.25.From Figure5(a), we know that if κ=0, cr^s=-2.19% while Figure 3(a) also shows if κ=0, cr^s=-2.19% which means, for the rational agent, the sensitivity of cs to r^ has nothing to do with r^’s variance. However, the boundedly rational agents have different reactions to r^ as κ increases, such as when κ=0.26, cr^s=0 with σr=0.15 in Figure 5(a) while cr^s=-2.1% with σr=1.5 in Figure 3(a). This disparity indicates that when the cognitive costs are the same and κ>0, that is, the agents have the same boundedly rational degree, more volatile factors will be considered while the factor with smaller variance may be neglected.Additionally, we can know that when0.26≤κ<0.95, the agent does not react to r^, namely, cr^s=0 (in Figure 5(a)), but will react to a change in b^ (in Figure 5(b)), which is more important: the sensitivity of cs to b^ remains high even for a high cognitive friction κ. Note that this “feature by feature” selective attention could not be rationalized by just a fixed cost to consumption, which is not feature dependent. But when κ≥0.95, cr^s=cb^s=0, which indicates that the agent will pay no attention to both r^ and b^ once their thinking costs are beyond some thresholds.Considering Figure6(a), we can see that when κ≥0.11, θr^s=0, while Figure 4(a) shows whatever κ is, θr^s>0 with σr=1.5 which means the smaller the variance of a factor is, the more likely the agent will ignore it. From Figure 6(b), we can also obtain the same conclusion.


## 5. Conclusion

Dynamic portfolio choice is an important but complex problem in modern financial field, but extant methods always generate complicated numerical calculations due to numerous state variables. Hence, to address this problem, this paper extends the sparse max operator proposed by Gabaix [23] and proposes a new approach to deal with dynamic programming under stochastic terms under the assumption of the agent’s limited attention. We apply this method to Merton dynamic portfolio choice problem and find that it effectively simplifies the model’s solution process and avoids the “Curse of Dimensionality.” Finally, numerical example shows that this method has significant economic implications and clearly interprets the agent’s economic behavior when he makes a portfolio choice.Our study can be extended in several directions. Future research should consider the condition when the stochastic factors are correlated with each other for it is more realistic. Besides, information faced by the agent is always imprecise and incomplete, and the fuzzy set theory is an important approach to deal with this kind of problem [31–33]. Hence, using fuzzy set theory to handle imprecise values in dynamic programming may be another direction for further research.



---

*Source: 840725-2014-05-22.xml*