# A General Setting and Solution of Bellman Equation in Monetary Theory

Authors: Xiaoli Gan; Wanbo Lu
Journal: Journal of Applied Mathematics
 (2014)

---

## Abstract

As an important tool in theoretical economics, Bellman equation is very powerful in solving optimization problems of discrete time and is frequently used in monetary theory. Because there is not a general method to solve this problem in monetary theory, it is hard to grasp the setting and solution of Bellman equation and easy to reach wrong conclusions. In this paper, we discuss the rules and problems that should be paid attention to when incorporating money into general equilibrium models. A general setting and solution of Bellman equation in monetary theory are provided. The proposed method is clear, is easy to grasp, is generalized, and always leads to the correct results.


---

## Body

## 1. Introduction

In recent years, many economists applied business cycle approaches to macroeconomic modeling so that monetary factors could be modeled into dynamic general equilibrium models. As an important method of monetary economics modeling, infinite horizon representative-agent models provide a close link between theory and practice; its research framework can guide practice behavior and be tested by actual data. This method can link monetary economics and other popular models for studying business cycle phenomena closely. There are three basic monetary economics approaches introducing money to economic general equilibrium models in the infinite horizon representative-agent framework. First, it is assumed that utility could be yielded by money directly so that the money variable has been incorporated into utility function of the representative-agent models [1]. Second, assuming asset exchanges are costly, transaction costs of some form give rise to a money demand [2, 3]. Clower [4] considers that money is used for some types of transactions. Brock [5], McCallum and Goodfriend [6], and Croushore [7] assume that time and money can be combined to yield transaction services. Third, money is treated as an asset that can be used to transfer resources between generations [8].Dynamic optimization is the main involved issue during the modeling process. It is represented and solved by Bellman equation method, namely, the value function method. The method will obtain a forward-looking household’s path to maximize lifetime utility through the optimal behavior and further relevant conclusions. The setting of Bellman equation is the first and crucial step to solve dynamic programming problems. It is hard to grasp the setting and solution of Bellman equation and easy to reach wrong conclusions since there is not a general method to set Bellman equation or the settings of Bellman equation are excessively flexible. Walsh [9] used Bellman equation to set and solve dynamic general equilibrium models of money. However he does not give a general law of Bellman equation setting. The setting and solution of the equation in his book are ambiguous and not clear. To the best of the authors’ knowledge, there have been no studies of uniform setting of Bellman equation.In this paper, we provide a set of general setting and solution methods for Bellman equation with multipliers. It is very clear and easy to grasp. The most important thing is that the proposed method can always lead to correct results. We apply our method to monetary general equilibrium models that are in the framework of the first two basic monetary economic approaches which incorporate money into economic general equilibrium model and make some extensions. At the same time, we compare our method with other current methods of setting and solution for Bellman equation to display the clarity and correctness of the proposed method. It is assumed that all the relevant assumptions of applying Bellman equation are satisfied. This is to ensure the feasibility of analysis and solution. For the convenience of discussion and suitable length of the paper, we mainly discuss the certainty linear programming problems. The Bellman equation’s setting and solution of uncertainty problems are similar to those with certainty problems essentially.


## 2. A New Method of Setting and Solution of Bellman Equation

First of all, we provide the theoretical details of the new general method and steps when applying Bellman equation to solve problems. It has many advantages. The relationship of the results is very clear through the connection of multipliers. There are no tedious expansions during the derivations. It differs from the expansion method because it does not need to consider which control variable should be replaced. The technical details about the equivalence of Bellman equations and dynamic programming problems and the solvability of set problems can be found in [10]. Actually, it is easy to reach wrong conclusions using other settings and solutions of Bellman equation with multipliers, or it has to find some particular skills to get the right results. But our method does not meet the problem. The proposed method can be described as the following five steps.Step 1.
List the expression of target problem.
Consider the dynamic programming problem as(1)H=max⁡∑t=0∞βthx1t,x2t,…,xpt(2)s.t.g1x1t,x2t,…,xpt,xp+1,t,…,xqt=0,p≤q,hhhhhhhhhhhhhhh⋮gdx1t,x2t,…,xpt,xp+1,t,…,xqt=0,p≤q,
where the objective function (1) represents the maximum sum of present value of a forward-looking behavioral agent’s every period objective function h(x1t,x2t,…,xpt). In monetary theory, h(·) usually represents the utility function of a family and β is a subjective rate of discount. Equation (2) is constraints; g1(·),…,gd(·) are dynamic constraint functions and usually are intertemporal budget constraints. Let control variables x1t(c),x2t(c),…,xmt(c)∈x1t,x2t,…,xpt,xp+1,t,…,xqt; the remaining variables x1t,x2t,…,xpt,xp+1,t,…,xqt-x1t(c),x2t(c),…,xmt(c)=x1t(s),x2t(s),…,xnt(s) are state variables.Step 2.
Set up Bellman equation with multipliers to express dynamic optimization problem in Step1:
(3)Vx1ts,x2ts,…,xnts=max⁡+βVx1,t+1s,x2,t+1s,…,xn,t+1shx1t,x2t,…,xpthhhhhhhh+βVx1,t+1s,x2,t+1s,…,xn,t+1s+λ1tg1x1t,…,xpt,xp+1,t,…,xqt+⋯+λdtgdx1t,…,xpt,xp+1,t,…,xqt,
where V(·) is the value function and λit is the multiplier of the ith constraint gi(·), i=1,2,…,d.Step 3.
Compute the partial derivatives of all control variables on the right side of the equation at Step2 to derive first-order conditions:
(4)∂Vx1ts,x2ts,…,xnts∂xjtc=∂hx1t,x2t,…,xpt∂xjtchhhh+β∂Vx1,t+1s,…,xn,t+1s∂x1,t+1s·∂x1,t+1s∂xjtchhhhhhhhh+⋯+∂Vx1,t+1s,…,xn,t+1s∂xn,t+1s·∂xn,t+1s∂xjtc+λ1t∂g1x1t,…,xqt∂xjtc+λdt∂gdx1t,…,xqt∂xjtc=0,hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhj=1,2,…,m.
During the derivation, it should be taken into account that next period state variables can be represented by other control variables according to the constraints, that is, to expandx1,t+1(s),x2,t+1(s),…,xn,t+1(s). Because the constraints are dynamic and are usually intertemporal budget constraints, those control variables x1,t+1(s),x2,t+1(s),…,xn,t+1(s) at time t+1 can often be represented as expressions of the previous period control variables. Attention should be paid to compute the derivatives of control variables hiding in V(x1,t+1(s),x2,t+1(s),…,xn,t+1(s)). But there is no need to expand x1,t+1(s),x2,t+1(s),…,xn,t+1(s) after the multipliers during the derivation. Other than that, no particular attention should be paid to other current skills; for example, some control variables should be replaced after x1,t+1(s),x2,t+1(s),…,xn,t+1(s) represented by other control variables in V(x1,t+1(s),x2,t+1(s),…,xn,t+1(s)) and so on.Step 4.
By the envelope theorem, take the partial derivatives of control variables at timet on both sides of Bellman equation to derive the remaining first-order conditions:
(5)∂Vx1ts,…,xnts∂xjts=λ1t∂gd(x1t,…,xqt)∂xjt(s)+⋯+λdt∂gdx1t,…,xqt∂xjts,hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhj=1,2,…,m.
Still, it does not need to expandx1,t+1(s),x2,t+1(s),…,xn,t+1(s) after the multipliers during the derivations. Requirements are the same as Step 3.Step 5.
Obtain the new relevant results about target problem through recursion and substitution according to the above results.
We could combine several state variables to one as you need according to specific economic significance and constraints when there is no need to describe the relevant economic significance of state variables one by one. For example,x1t(s), x2t(s) could be combined as a(x1t(s),x2t(s))=At. At is a state variable in value function V(a(x1t(s),x2t(s)),x3t(s),…,xnt(s)), that is, V(At,x2t(s),…,xnt(s)). And then we follow the above five steps to solve specific problem. The rest of state variables could be operated similarly. But combinations should not be overlapped over state or combined state variables. For instance, suppose that x1t(s), x2t(s) have been combined as a(x1t(s),x2t(s))=At. It is inflexible to take x2t(s) as an independent state variable in value function V(At,x2t(s),…,xnt(s)) or put the combination of x2t(s), x3t(s), b(x1t(s),x2t(s))=Bt in value function V(At,Bt,…,xnt(s)) as a state variable.The steps above give a general setting and solution of Bellman equation. These can be summarized as follows: first, set Bellman equation with multipliers of target dynamic optimization problem under the requirement of no overlaps of state variables; second, extend the late period state variables inV(·) on the right side of Bellman equation and there is no need to expand these variables after the multipliers; third, let the derivatives of state variables of time t equal zero and take the partial derivatives of these variables on both sides of Bellman equation to derive first-order conditions; finally, get more needed results for analysis from these conditions.Different from some current settings which allow overlap of state variables in value function, our method does not permit overlaps. In fact, overlap of state variable is easy to reach wrong conclusions or it has to find some particular skills to get the right results [9]. These methods are hard to generalize. More details will be seen in the following sections.Now we will take several discrete time dynamic optimization problems that are under framework of the basic two methods of incorporating money into general equilibrium models as examples to show the applications of our setting and solution method and compare with current popular methods.


## 3. Setting and Solution of Bellman Equation in Basic Money-In-Utility Model

### 3.1. Model and Solution Using Previous Approach

The basic Money-In-Utility model has few features. The labor-leisure options of families are ignored temporarily in utility function. Only family consumption and real money balances are involved in utility function; that is, real money balances yield utility directly. We ignore the uncertainty of currency impact and technological changes temporarily for convenience.The total present utility value of family life cycle isU=∑t=0∞βtu(ct,mt). Usually the per capita version of intertemporal budget constraint is
(6)ωt≡fkt-11+n+τt+1-δ1+nkt-1+1+it-1bt-1+mt-11+πt1+n=ct+kt+mt+bt,
where ct, bt, kt, mt, τt, it-1, and πt are the per capita consumption, bonds, stock of capital, money balances, net lump-sum transfer received from the government, nominal interest rate, and inflation rate, respectively, at time t. δ is the rate of depreciation of physical capital, n is the population growth rate (assumed to be constant), and f(kt-1/(1+n)) is the production function assumed to be continuously differentiable and to satisfy the usual Inada conditions (see [11]). Note that control variables can be changed and affect the total discounted value of utility through consumption ct in period t; state variables cannot be changed. Determination of state and control variables is crucial to derive the first-order conditions and other results. The control variables for this problem are ct, bt, kt and, mt and the state variables are bt-1, kt-1, mt-1, τt, and ωt, the household’s initial level of resources.This problem can be solved by the expansion method below according to Walsh [9]. It is started with setting up value function
(7)Vωt=max⁡uct,mt+βVωt+1.Using (6), value function can be expanded as
(8)Vωt=max⁡+τt+1+1-δ1+nkt+1+itbt+mt1+πt+11+nuct,mthhhhhhhhh+βV+1+itbt+mt1+πt+11+nfkt1+nhhhhhhhhhhihhhhh+τt+1+1-δ1+nkthhhhhhhhhhhhhh+1+itbt+mt1+πt+11+n.Replacekt as ωt-ct-mt-bt; according to (6), (8) can be written as an expansion equation as follows:
(9)Vωt=max⁡+1+itbt+mt1+πt+11+nuct,mthhhhhhh+βV+1+itbt+mt1+πt+11+nfωt-ct-mt-bt1+n+τt+1hhhhhhhhhhhhh+1-δ1+nωt-ct-mt-bthhhhhhhhhhhhh+1+itbt+mt1+πt+11+n.Compute the partial derivatives of control variables to derive first-order conditions:(10)∂Vωt∂ct=ucct,mt-Vωωt+1·β1+n·fkt+1-δ=0,∂Vωt∂bt=βVωωt+1·1+it1+πt+11+n-βVωωt+1fkkt+1-δ1+n=0,∂V(ωt)∂mt=um(ct,mt)+βVω(ωt+1)1+πt+11+n-βVω(ωt+1)fk(kt)+1-δ1+n=0,
and the envelope theorem yields
(11)Vωωt=ucct,mt.Another expansion method will replacect as ωt-kt-mt-bt; according to (6), (8) can be written as
(12)Vωt=max⁡+1+itbt+mt1+πt+11+nuωt-kt-mt-bt,mthhhhhhh+βV+1+itbt+mt1+πt+11+nfkt1+n+τt+1+1-δ1+nkthhhhhhhhhhhhhh+1+itbt+mt1+πt+11+n.Compute the partial derivatives of control variables to derive first-order conditions as(13)∂Vωt∂bt=-ucωt-kt-mt-bt,mt+βVωωt+1·1+it1+πt+11+n=0,∂Vωt∂kt=-ucωt-kt-mt-bt,mt+Vωωt+1·β1+n·fkkt+1-δ=0,∂Vωt∂mt=-uc(ωt-kt-mt-bt,mt)+um(ωt-kt-mt-bt,mt)+βVωωt+11+πt+11+n=0.Using the envelope theorem and computing the derivative with respect to state variableωt, we get
(14)Vωωt=ucωt-kt-mt-bt,mt.


### 3.2. Our Solving Approach

Now, we use our proposed steps of setting and solution of Bellman equation to solve the above basic Money-In-Utility problem. First, let the Bellman equation with multiplierλt be
(15)Vωt=max⁡u(ct,mt)+βV(ωt+1)+λtωt-ct-kt-mt-bt.Second, computing the partial derivatives for the control variables, we obtain the first-order conditions as(16)∂Vωt∂ct=ucct,mt=λt,∂Vωt∂bt=βVωωt+1·1+it1+πt+11+n=λt,∂Vωt∂kt=βVωωt+1·11+n·fkkt+1-δ=λt,∂Vωt∂mt=um(ct,mt)+βVω(ωt+1)·11+πt+11+n=λt.Third, using the envelope theorem and computing the derivatives of both sides of Bellman equation with respect to state variableωt, we get
(17)Vωωt=λt.Finally, based on the above results, it is easy to get(18)ucct,mt=λt=umct,mt+βVωωt+1·11+πt+11+n.This expression indicates that the marginal benefit of increased money holdings should be equal to the marginal utility of consumption on periodt. Similarly, we can easily obtain the allocation of initial level of resources ωt among consumption, capital, bonds, and money balances, and the same marginal benefit must be yielded by each use at an optimum allocation.These results with multiplier are open-and-shut, and it is easy to find the economic significance of marginal utility of consumption. Comparing with current approach mentioned above, during the process of solving problem, there is no need to consider which variable should be replaced. There are no tedious expanded expressions. Using expansion method, if we replacekt, the derived first-order conditions are seen to be messy and it is not easy to find the relationships of results. If we replace ct, the derived results are a bit more clear, but we might not think of replacing ct at the beginning. So expansion methods are not easy to operate, and it is not clear to replace variable.


## 3.1. Model and Solution Using Previous Approach

The basic Money-In-Utility model has few features. The labor-leisure options of families are ignored temporarily in utility function. Only family consumption and real money balances are involved in utility function; that is, real money balances yield utility directly. We ignore the uncertainty of currency impact and technological changes temporarily for convenience.The total present utility value of family life cycle isU=∑t=0∞βtu(ct,mt). Usually the per capita version of intertemporal budget constraint is
(6)ωt≡fkt-11+n+τt+1-δ1+nkt-1+1+it-1bt-1+mt-11+πt1+n=ct+kt+mt+bt,
where ct, bt, kt, mt, τt, it-1, and πt are the per capita consumption, bonds, stock of capital, money balances, net lump-sum transfer received from the government, nominal interest rate, and inflation rate, respectively, at time t. δ is the rate of depreciation of physical capital, n is the population growth rate (assumed to be constant), and f(kt-1/(1+n)) is the production function assumed to be continuously differentiable and to satisfy the usual Inada conditions (see [11]). Note that control variables can be changed and affect the total discounted value of utility through consumption ct in period t; state variables cannot be changed. Determination of state and control variables is crucial to derive the first-order conditions and other results. The control variables for this problem are ct, bt, kt and, mt and the state variables are bt-1, kt-1, mt-1, τt, and ωt, the household’s initial level of resources.This problem can be solved by the expansion method below according to Walsh [9]. It is started with setting up value function
(7)Vωt=max⁡uct,mt+βVωt+1.Using (6), value function can be expanded as
(8)Vωt=max⁡+τt+1+1-δ1+nkt+1+itbt+mt1+πt+11+nuct,mthhhhhhhhh+βV+1+itbt+mt1+πt+11+nfkt1+nhhhhhhhhhhihhhhh+τt+1+1-δ1+nkthhhhhhhhhhhhhh+1+itbt+mt1+πt+11+n.Replacekt as ωt-ct-mt-bt; according to (6), (8) can be written as an expansion equation as follows:
(9)Vωt=max⁡+1+itbt+mt1+πt+11+nuct,mthhhhhhh+βV+1+itbt+mt1+πt+11+nfωt-ct-mt-bt1+n+τt+1hhhhhhhhhhhhh+1-δ1+nωt-ct-mt-bthhhhhhhhhhhhh+1+itbt+mt1+πt+11+n.Compute the partial derivatives of control variables to derive first-order conditions:(10)∂Vωt∂ct=ucct,mt-Vωωt+1·β1+n·fkt+1-δ=0,∂Vωt∂bt=βVωωt+1·1+it1+πt+11+n-βVωωt+1fkkt+1-δ1+n=0,∂V(ωt)∂mt=um(ct,mt)+βVω(ωt+1)1+πt+11+n-βVω(ωt+1)fk(kt)+1-δ1+n=0,
and the envelope theorem yields
(11)Vωωt=ucct,mt.Another expansion method will replacect as ωt-kt-mt-bt; according to (6), (8) can be written as
(12)Vωt=max⁡+1+itbt+mt1+πt+11+nuωt-kt-mt-bt,mthhhhhhh+βV+1+itbt+mt1+πt+11+nfkt1+n+τt+1+1-δ1+nkthhhhhhhhhhhhhh+1+itbt+mt1+πt+11+n.Compute the partial derivatives of control variables to derive first-order conditions as(13)∂Vωt∂bt=-ucωt-kt-mt-bt,mt+βVωωt+1·1+it1+πt+11+n=0,∂Vωt∂kt=-ucωt-kt-mt-bt,mt+Vωωt+1·β1+n·fkkt+1-δ=0,∂Vωt∂mt=-uc(ωt-kt-mt-bt,mt)+um(ωt-kt-mt-bt,mt)+βVωωt+11+πt+11+n=0.Using the envelope theorem and computing the derivative with respect to state variableωt, we get
(14)Vωωt=ucωt-kt-mt-bt,mt.


## 3.2. Our Solving Approach

Now, we use our proposed steps of setting and solution of Bellman equation to solve the above basic Money-In-Utility problem. First, let the Bellman equation with multiplierλt be
(15)Vωt=max⁡u(ct,mt)+βV(ωt+1)+λtωt-ct-kt-mt-bt.Second, computing the partial derivatives for the control variables, we obtain the first-order conditions as(16)∂Vωt∂ct=ucct,mt=λt,∂Vωt∂bt=βVωωt+1·1+it1+πt+11+n=λt,∂Vωt∂kt=βVωωt+1·11+n·fkkt+1-δ=λt,∂Vωt∂mt=um(ct,mt)+βVω(ωt+1)·11+πt+11+n=λt.Third, using the envelope theorem and computing the derivatives of both sides of Bellman equation with respect to state variableωt, we get
(17)Vωωt=λt.Finally, based on the above results, it is easy to get(18)ucct,mt=λt=umct,mt+βVωωt+1·11+πt+11+n.This expression indicates that the marginal benefit of increased money holdings should be equal to the marginal utility of consumption on periodt. Similarly, we can easily obtain the allocation of initial level of resources ωt among consumption, capital, bonds, and money balances, and the same marginal benefit must be yielded by each use at an optimum allocation.These results with multiplier are open-and-shut, and it is easy to find the economic significance of marginal utility of consumption. Comparing with current approach mentioned above, during the process of solving problem, there is no need to consider which variable should be replaced. There are no tedious expanded expressions. Using expansion method, if we replacekt, the derived first-order conditions are seen to be messy and it is not easy to find the relationships of results. If we replace ct, the derived results are a bit more clear, but we might not think of replacing ct at the beginning. So expansion methods are not easy to operate, and it is not clear to replace variable.


## 4. Setting and Solution of Bellman Equation in Shopping-Time Model

### 4.1. Model and Solution Using Previous Approach

In Shopping-Time Model, shopping time is a function of consumption and money balances. Because consumption needs shopping time, leisure is reduced. Household utility is assumed to depend on consumption and leisure. Consumption can not only yield utility directly but also decrease utility indirectly. In this section,n is time spent in market employment and ns is time spent shopping. ns is the function of consumption and money balances; that is, ns=g(c,m), gc>0, gm≤0. Total time available is normalized to equal 1. Growth rate of population is assumed to be 0 for convenience. Let l be the leisure time. The utility function is u(c,m,l)=v[c,1-n-g(c,m)]. It donates utility as a function of consumption, labor supply, and money holdings.The household’s intertemporal objective is maximum discounted utility subject to resource constraint as(19)max⁡∑i=0∞βivct+i,1-nt+i-gct+i,mt+ihhhhh=max⁡∑i=0∞βivct+i,lt+is.t.fkt-1,nt+τt+1-δkt-1hhhhh+1+it-1bt-1+mt-11+πt=ct+kt+mt+bt.Because controllable factors, labor supply, and consumption affect labor supply, output is affected not only by state variablekt-1 but also by these controllable factors. The left side of the budget constraint is unsuitable to be combined as a state variable ωt. Combining the money holdings, bonds, and transfers to at=τt+((1+it-1bt-1+mt-1)/(1+πt)), the household’s financial assets, value function can be written as
(20)Vat,kt-1=max⁡vct,lt+βVat+1,kt.According to Walsh [9], after replacing at+1, kt, value function can be expanded as
(21)Vat,kt-1=max⁡+1-δkt-1+at-mt-bt-ctτt+1+1+itbt+mt1+πt+1,fkt-1,ntvct,1-nt-gct,mthhhhhhhh+βVτt+1+1+itbt+mt1+πt+1,fkt-1,nthhhhhhhhhhhhhh+1-δkt-1+at-mt-bt-ctτt+1+1+itbt+mt1+πt+1,fkt-1,nt.Computing the partial derivatives with respect to control variables, we get(22)∂Vat,kt-1∂ct=vcct,1-nt-gct,mt-vlct,1-nt-gct,mtgcct,mt-βVkat+1,kt=0,∂V(at,kt-1)∂kt=-vl[ct,1-nt-g(ct,mt)]+βVk(at+1,kt)·fn(kt-1,nt)=0,∂Vat,kt-1∂mt=-vlct,1-nt-gct,mtgmct,mt+βVaat+1,kt·11+πt+1-βVkat+1,kt=0.Computing the partial derivatives of both sides of the value equation with respect to the state variablesat, kt-1, we get
(23)Va(at,kt-1)=βVk(at+1,kt)Vkat,kt-1=βVkat+1,ktfkkt-1,nt+1-δ.


### 4.2. Our Solving Approach

Now, we use our proposed method to solve the above Shopping-Time Model problem. First, let the Bellman equation with multiplierλt be
(24)Vat,kt-1=max⁡vct,lt+βVat+1,kthhh+λtat+fkt-1,nt+1-δkt-1hhhhhhhh-ct-kt-mt-bt,
where state variable at+1=τt+1+(((1+it)bt+mt)/(1+πt+1)). Then, compute the partial derivatives with respect to control variables to derive first-order conditions,
(25)∂Vat,kt-1∂ct=vcct,lt-vlct,ltgcct,mt=λt,∂Vat,kt-1∂bt=βVaat+1,kt·1+it1+πt+1=λt,∂Vat,kt-1∂kt=βVkat+1,kt=λt,∂Vat,kt-1∂mt=-vlct,ltgmct,mt+βVaat+1,kt1+πt+1=λt.Using the envelope theorem and computing the derivatives with respect to the state variablesat, kt-1, we get
(26)Vaat,kt-1=λt,Vk(at,kt-1)=λtfkkt-1,nt+1-δ.Now, we use the above results to compute the opportunity cost of holding money. Since the utility function is(27)uc,m,l=vc,l-n-gc,m,
and based on (6), (25), and (26), we obtain
(28)umct,mt,ltucct,mt,lt=-vlct,ltgmct,mtvcct,lt-vlct,ltgcct,mt=λt-βVaat+1,kt/1+πt+1λt=1-βVaat+1,kt/1+πt+1βVa(at+1,kt)·1+it/1+πt+1=1-11+it=it1+it.Obviously, the expressions of the derived first-order conditions by previous method seem to be tedious and messy, and it is not so easy to compute the relevant results such as the opportunity cost of holding money. Our proposed method is comparatively neat and can easily obtain relevant results correctly.


## 4.1. Model and Solution Using Previous Approach

In Shopping-Time Model, shopping time is a function of consumption and money balances. Because consumption needs shopping time, leisure is reduced. Household utility is assumed to depend on consumption and leisure. Consumption can not only yield utility directly but also decrease utility indirectly. In this section,n is time spent in market employment and ns is time spent shopping. ns is the function of consumption and money balances; that is, ns=g(c,m), gc>0, gm≤0. Total time available is normalized to equal 1. Growth rate of population is assumed to be 0 for convenience. Let l be the leisure time. The utility function is u(c,m,l)=v[c,1-n-g(c,m)]. It donates utility as a function of consumption, labor supply, and money holdings.The household’s intertemporal objective is maximum discounted utility subject to resource constraint as(19)max⁡∑i=0∞βivct+i,1-nt+i-gct+i,mt+ihhhhh=max⁡∑i=0∞βivct+i,lt+is.t.fkt-1,nt+τt+1-δkt-1hhhhh+1+it-1bt-1+mt-11+πt=ct+kt+mt+bt.Because controllable factors, labor supply, and consumption affect labor supply, output is affected not only by state variablekt-1 but also by these controllable factors. The left side of the budget constraint is unsuitable to be combined as a state variable ωt. Combining the money holdings, bonds, and transfers to at=τt+((1+it-1bt-1+mt-1)/(1+πt)), the household’s financial assets, value function can be written as
(20)Vat,kt-1=max⁡vct,lt+βVat+1,kt.According to Walsh [9], after replacing at+1, kt, value function can be expanded as
(21)Vat,kt-1=max⁡+1-δkt-1+at-mt-bt-ctτt+1+1+itbt+mt1+πt+1,fkt-1,ntvct,1-nt-gct,mthhhhhhhh+βVτt+1+1+itbt+mt1+πt+1,fkt-1,nthhhhhhhhhhhhhh+1-δkt-1+at-mt-bt-ctτt+1+1+itbt+mt1+πt+1,fkt-1,nt.Computing the partial derivatives with respect to control variables, we get(22)∂Vat,kt-1∂ct=vcct,1-nt-gct,mt-vlct,1-nt-gct,mtgcct,mt-βVkat+1,kt=0,∂V(at,kt-1)∂kt=-vl[ct,1-nt-g(ct,mt)]+βVk(at+1,kt)·fn(kt-1,nt)=0,∂Vat,kt-1∂mt=-vlct,1-nt-gct,mtgmct,mt+βVaat+1,kt·11+πt+1-βVkat+1,kt=0.Computing the partial derivatives of both sides of the value equation with respect to the state variablesat, kt-1, we get
(23)Va(at,kt-1)=βVk(at+1,kt)Vkat,kt-1=βVkat+1,ktfkkt-1,nt+1-δ.


## 4.2. Our Solving Approach

Now, we use our proposed method to solve the above Shopping-Time Model problem. First, let the Bellman equation with multiplierλt be
(24)Vat,kt-1=max⁡vct,lt+βVat+1,kthhh+λtat+fkt-1,nt+1-δkt-1hhhhhhhh-ct-kt-mt-bt,
where state variable at+1=τt+1+(((1+it)bt+mt)/(1+πt+1)). Then, compute the partial derivatives with respect to control variables to derive first-order conditions,
(25)∂Vat,kt-1∂ct=vcct,lt-vlct,ltgcct,mt=λt,∂Vat,kt-1∂bt=βVaat+1,kt·1+it1+πt+1=λt,∂Vat,kt-1∂kt=βVkat+1,kt=λt,∂Vat,kt-1∂mt=-vlct,ltgmct,mt+βVaat+1,kt1+πt+1=λt.Using the envelope theorem and computing the derivatives with respect to the state variablesat, kt-1, we get
(26)Vaat,kt-1=λt,Vk(at,kt-1)=λtfkkt-1,nt+1-δ.Now, we use the above results to compute the opportunity cost of holding money. Since the utility function is(27)uc,m,l=vc,l-n-gc,m,
and based on (6), (25), and (26), we obtain
(28)umct,mt,ltucct,mt,lt=-vlct,ltgmct,mtvcct,lt-vlct,ltgcct,mt=λt-βVaat+1,kt/1+πt+1λt=1-βVaat+1,kt/1+πt+1βVa(at+1,kt)·1+it/1+πt+1=1-11+it=it1+it.Obviously, the expressions of the derived first-order conditions by previous method seem to be tedious and messy, and it is not so easy to compute the relevant results such as the opportunity cost of holding money. Our proposed method is comparatively neat and can easily obtain relevant results correctly.


## 5. Setting and Solution of Bellman Equation in Cash-In-Advance Model

### 5.1. Model and Solution Using Previous Approach

In basic Cash-In-Advance model, money is used to purchase goods. Money cannot yield utility itself, but the consumption of future can yield utility. Svensson [12] assumed that agents are available for spending only the cash carried over from the previous period. This is essentially different from Money-In-Utility model. Consider a simple form of discounted household utility value ∑t=0∞βtu(ct). More complicated utility function will be discussed in the following sections. Take Cash-In-Advance constraint form as ct≤(mt-1/1+πt)+τt due to Svensson [12]. This means that the agent enters the period with money holdings mt-1 and receives a lump-sum transfer τt (in real currency terms) for consumption goods. Bonds and capital may not be purchased by currency. If capital is assumed to be purchased by money, the Cash-In-Advance constraint will become (mt-1/(1+πt))+τt≥ct+kt.The budget constraint is rewritten in real terms as(29)fkt-1+τt+1-δkt-1+1+it-1bt-1+mt-11+πt≥ct+kt+mt+bt.In monetary theory, constraints are expressed as inequality frequently. This constraint describes that the representative agent’s timet real resources should be more than or equal to the use of it, that is, purchasing consumption, capital, bonds, and money holdings that are then carried into period t+1. Because of the assumption of rational agents, the certainty problems we are discussing, and the positive opportunity cost of money holdings, the constraints become equations in equilibriums.Output is only affected by state variablekt-1 without considering the effect of labor supply. Following the solution methods of Walsh [9], let the left side of the budget constraint be a state variable ωt=f(kt-1)+τt+(1-δ)kt-1+(((1+it-1)bt-1+mt-1)/(1+πt)). It will be very tedious adopting expansion method with two constraints. Setting Bellman equation with multipliers will be better. Let state variable mt-1 in value function get the economic significance of money in first-order conditions. However, there is an overlapped setting with ωt,
(30)Vωt,mt-1=max⁡uct+βVωt+1,mt+λtωt-ct-kt-mt-bt+μtmt-11+πt+τt-ct.


### 5.2. Our Solving Approach

Letξt=f(kt-1)+τt+(1-δ)kt-1+(((1+it-1)bt-1)/(1+πt)). There is no overlap with mt in the value function. The Bellman equation is set as
(31)Vξt,mt-1=max⁡uct+βVξt+1,mt+λtξt+mt-11+πt-ct-kt-mt-bt+μtmt-11+πt+τt-ct.The solution of this problem is similar to the application of the proposed method in Sections3 and 4. We will not give unnecessary details here.Note that Walsh [9] puts overlapping state variables mt-1 and ωt in value function; it is easy to get wrong results in those problems with Cash-In-Advance constraints although it is no problem setting V(ωt,mt-1) with no more results required now. This will be seen in the following example of Section 6.


## 5.1. Model and Solution Using Previous Approach

In basic Cash-In-Advance model, money is used to purchase goods. Money cannot yield utility itself, but the consumption of future can yield utility. Svensson [12] assumed that agents are available for spending only the cash carried over from the previous period. This is essentially different from Money-In-Utility model. Consider a simple form of discounted household utility value ∑t=0∞βtu(ct). More complicated utility function will be discussed in the following sections. Take Cash-In-Advance constraint form as ct≤(mt-1/1+πt)+τt due to Svensson [12]. This means that the agent enters the period with money holdings mt-1 and receives a lump-sum transfer τt (in real currency terms) for consumption goods. Bonds and capital may not be purchased by currency. If capital is assumed to be purchased by money, the Cash-In-Advance constraint will become (mt-1/(1+πt))+τt≥ct+kt.The budget constraint is rewritten in real terms as(29)fkt-1+τt+1-δkt-1+1+it-1bt-1+mt-11+πt≥ct+kt+mt+bt.In monetary theory, constraints are expressed as inequality frequently. This constraint describes that the representative agent’s timet real resources should be more than or equal to the use of it, that is, purchasing consumption, capital, bonds, and money holdings that are then carried into period t+1. Because of the assumption of rational agents, the certainty problems we are discussing, and the positive opportunity cost of money holdings, the constraints become equations in equilibriums.Output is only affected by state variablekt-1 without considering the effect of labor supply. Following the solution methods of Walsh [9], let the left side of the budget constraint be a state variable ωt=f(kt-1)+τt+(1-δ)kt-1+(((1+it-1)bt-1+mt-1)/(1+πt)). It will be very tedious adopting expansion method with two constraints. Setting Bellman equation with multipliers will be better. Let state variable mt-1 in value function get the economic significance of money in first-order conditions. However, there is an overlapped setting with ωt,
(30)Vωt,mt-1=max⁡uct+βVωt+1,mt+λtωt-ct-kt-mt-bt+μtmt-11+πt+τt-ct.


## 5.2. Our Solving Approach

Letξt=f(kt-1)+τt+(1-δ)kt-1+(((1+it-1)bt-1)/(1+πt)). There is no overlap with mt in the value function. The Bellman equation is set as
(31)Vξt,mt-1=max⁡uct+βVξt+1,mt+λtξt+mt-11+πt-ct-kt-mt-bt+μtmt-11+πt+τt-ct.The solution of this problem is similar to the application of the proposed method in Sections3 and 4. We will not give unnecessary details here.Note that Walsh [9] puts overlapping state variables mt-1 and ωt in value function; it is easy to get wrong results in those problems with Cash-In-Advance constraints although it is no problem setting V(ωt,mt-1) with no more results required now. This will be seen in the following example of Section 6.


## 6. Setting and Solution of Bellman Equation in Model considering Both Labor Time and Cash-In-Advance Constraints

### 6.1. Model and Solution Using Previous Approach

Assuming that money is used to purchase consumption goods and investments, the Cash-In-Advance constraint becomes(mt-1/(1+πt))+τt≥ct+xt, where x is investment and xt=kt-(1-δ)kt-1. In considering of the effects of labor supply on output and leisure on utility, we have lt=1-nt. The agent’s objective becomes
(32)max⁡∑t=0∞uct,lts.t.fkt-1+τt+1-δkt-1+1+it-1bt-1+mt-11+πt≥ct+kt+mt+bt,mt-11+πt+τt≥ct+xt.Output is affected not only by state variablekt-1 but also by controllable factor nt; the left side of the budget constraint is unsuitable to be combined as a state variable. Thinking of the setting of the Shopping-Time Model and Cash-In-Advance model above, it is plausible to put overlapped state variables at=τt+(((1+it-1)bt-1+mt-1)/(1+πt)) and mt-1 in value function following the solution methods of Walsh [9]. The Bellman equation will be
(33)Vat,kt-1,mt-1=max⁡uct,lt+βVat+1,kt,mt+λtat+fkt-1,nt+1-δkt-1hhhhhhhh-ct-kt-mt-bt+μtat-ct-kt+1-δkt-1.The first-order conditions are(34)∂Vat,kt-1,mt-1∂ct=ucct,lt=λt+μt,∂Vat,kt-1,mt-1∂bt=βVaat+1,kt,mt·1+it1+πt+1=λt,∂Vat,kt-1,mt-1∂kt=βVkat+1,kt,mt=λt+μt,∂Vat,kt-1,mt-1∂mt=βVm(at+1,kt,mt)+βVa(at+1,kt,mt)·11+πt+1=λt,∂Vat,kt-1,mt-1∂nt=-ulct,lt+λtfnkt-1,nt=0.Using the envelope theorem and computing the derivatives with respect to state variables, we get(35)∂V(at,kt-1,mt-1)∂at=Va(at,kt-1,mt-1)=λt+μt,∂Vat,kt-1,mt-1∂kt-1=Vkat,kt-1,mt-1=λtfkkt-1,nt+1-δ+μt1-δ,∂Vat,kt-1,mt-1∂mt-1=Vmat,kt-1,mt-1=λt+μt11+πt.When the question is to derive the effect of inflation rate on the steady-state capital-labor ratio, that is, the steady-state relationship ofkss/nss and πss, assume that the aggregate production takes the form yt=f(kt-1,nt)=Akt-1αnt1-α. Using (34) and (35),
(36)ulct,ltucct,lt=λtfnkt-1,ntλt+μt=βVmat+1,kt,mt+βVaat+1,kt,mt·11+πt+1hhhhh×fnkt-1,ntβVmat+1,kt,mt+βVaat+1,kt,mt·11+πt+1λt+μt-1=βλt+1+μt+111+πt+1+βλt+1+μt+111+πt+1hhhhh×fnkt-1,ntβλt+1+μt+111+πt+1+βλt+1+μt+111+πt+1λt+μt-1=2β11+πt+1fnkt-1,nt·λt+1+μt+1λt+μt.Then(37)λtλt+μt=2β11+πt+1λt+1+μt+1λt+μt.From (35),
(38)fkkt-1,nt=Vkat,kt-1,mt-1-1-δλt+μtλt=1β-1-δλt+μtλt.Rewriting the aggregate production asfk=αAkt-1/ntα-1, we have
(39)λtλt+μt=αA-1kt-1nt1-α1β-1-δ.Using (37), we obtain(40)αA-1kt-1nt1-α1β-1-δ=2β11+πt+1λt+1+μt+1λt+μt.Rewriting this equation, we get(41)kt-1nt=2βαA1+πt+11β-1+δ-1λt+1+μt+1λt+μt1/(1-α).In steady-state,λt=λt+1=λss, μt=μt+1=μss, πt+1=πss, kt-1=kss, nt=nss, from (41), we have
(42)kssnss=1+πss2αβ1β-1+δ1/(α-1).This steady-state capital-labor ratio is derived by using current prevailing methods. However, it is a wrong result. We will provide the correct result by using the proposed method of this paper.


### 6.2. Our Solving Approach

Output is affected by state variablekt-1 and control variable nt; it is unsuitable to let the left side of the budget constraint be combined as a state variable. We should separate kt-1 from it. Let state variable mt-1 in value function alone get the economic significance of money in first-order conditions. Because overlaps of state variables are not allowed according to our proposed method, we put ζt=τt+(((1+it-1)bt-1)/(1+πt)) in value function as a state variable.First, set up value function with multipliers:(43)Vζt,kt-1,mt-1=max⁡uct,lt+βVζt+1,kt,mt+λtζt+mt-11+πt+fkt-1,nthhhhhhhhh+1-δkt-1-ct-kt-mt-btζt+mt-11+πt+fkt-1,nt+1-δkt-1+μtζt-1+it-1bt-11+πt+mt-11+πthhhhhhhhh-ct-kt+1-δkt-1ζt-1+it-1bt-11+πt+mt-11+πt.Second, compute the derivatives of state variables and derive first-order conditions:(44)∂Vat,kt-1,mt-1∂ct=ucct,lt=λt+μt,∂Vat,kt-1,mt-1∂bt=βVζζt+1,kt,mt·1+it1+πt+1=λt,∂Vat,kt-1,mt-1∂kt=βVkζt+1,kt,mt=λt+μt,∂Vat,kt-1,mt-1∂mt=βVmζt+1,kt,mt=λt,∂Vat,kt-1,mt-1∂nt=-ulct,lt+λtfnkt-1,nt=0.Third, compute the partial derivatives with respect toζt, kt-1, and mt-1 and the envelope theorem yields
(45)∂Vat,kt-1,mt-1∂ζt=Vζζt,kt-1,mt-1=λt+μt,∂Vat,kt-1,mt-1∂kt-1=Vk(ζt,kt-1,mt-1)=λtfkkt-1,nt+1-δ+μt(1-δ),∂Vat,kt-1,mt-1∂mt-1=Vmζt,kt-1,mt-1=λt+μt11+πt.Finally, derive the steady-state capital-labor ratio by the results above:(46)kssnss=1+πssαβ1β-1+δ1/(α-1).The process of deriving this ratio is similar to the deriving process of (42). We will not give unnecessary details here. Compare (46) with (42); the result of (42) is 1/21/(α-1) times that of (46). Equation (46) is the correct result. It is the different partial derivatives with respect to mt in (34) and (44) that cause this deviation. In (34), βVm(at+1,kt,mt)+βVa(at+1,kt,mt)·(1/(1+πt+1))=λt. In (44), βVm(ζt+1,kt,mt)=λt. Fundamentally speaking, the reason for the deviation is the overlapped setting of state variables at and mt-1 versus the nonoverlapped setting of state variables ζt, kt-1, and mt-1. One has to use the particular skills in [9] that is hard to think about and grasp to obtain the right answer or get a wrong result adopting the overlapped setting V(at,kt-1,mt-1). For this reason, the setting of value function by Walsh could not be generalized. In this model considering both Labor Time and Cash-In-Advance constraints, the basic methods of incorporating money into general equilibrium models are generalized, and our proposed method of setting and solution of Bellman equation demonstrates its clarity and validity and corrects the defect that some other current methods of setting and solution are easy to get wrong results.


## 6.1. Model and Solution Using Previous Approach

Assuming that money is used to purchase consumption goods and investments, the Cash-In-Advance constraint becomes(mt-1/(1+πt))+τt≥ct+xt, where x is investment and xt=kt-(1-δ)kt-1. In considering of the effects of labor supply on output and leisure on utility, we have lt=1-nt. The agent’s objective becomes
(32)max⁡∑t=0∞uct,lts.t.fkt-1+τt+1-δkt-1+1+it-1bt-1+mt-11+πt≥ct+kt+mt+bt,mt-11+πt+τt≥ct+xt.Output is affected not only by state variablekt-1 but also by controllable factor nt; the left side of the budget constraint is unsuitable to be combined as a state variable. Thinking of the setting of the Shopping-Time Model and Cash-In-Advance model above, it is plausible to put overlapped state variables at=τt+(((1+it-1)bt-1+mt-1)/(1+πt)) and mt-1 in value function following the solution methods of Walsh [9]. The Bellman equation will be
(33)Vat,kt-1,mt-1=max⁡uct,lt+βVat+1,kt,mt+λtat+fkt-1,nt+1-δkt-1hhhhhhhh-ct-kt-mt-bt+μtat-ct-kt+1-δkt-1.The first-order conditions are(34)∂Vat,kt-1,mt-1∂ct=ucct,lt=λt+μt,∂Vat,kt-1,mt-1∂bt=βVaat+1,kt,mt·1+it1+πt+1=λt,∂Vat,kt-1,mt-1∂kt=βVkat+1,kt,mt=λt+μt,∂Vat,kt-1,mt-1∂mt=βVm(at+1,kt,mt)+βVa(at+1,kt,mt)·11+πt+1=λt,∂Vat,kt-1,mt-1∂nt=-ulct,lt+λtfnkt-1,nt=0.Using the envelope theorem and computing the derivatives with respect to state variables, we get(35)∂V(at,kt-1,mt-1)∂at=Va(at,kt-1,mt-1)=λt+μt,∂Vat,kt-1,mt-1∂kt-1=Vkat,kt-1,mt-1=λtfkkt-1,nt+1-δ+μt1-δ,∂Vat,kt-1,mt-1∂mt-1=Vmat,kt-1,mt-1=λt+μt11+πt.When the question is to derive the effect of inflation rate on the steady-state capital-labor ratio, that is, the steady-state relationship ofkss/nss and πss, assume that the aggregate production takes the form yt=f(kt-1,nt)=Akt-1αnt1-α. Using (34) and (35),
(36)ulct,ltucct,lt=λtfnkt-1,ntλt+μt=βVmat+1,kt,mt+βVaat+1,kt,mt·11+πt+1hhhhh×fnkt-1,ntβVmat+1,kt,mt+βVaat+1,kt,mt·11+πt+1λt+μt-1=βλt+1+μt+111+πt+1+βλt+1+μt+111+πt+1hhhhh×fnkt-1,ntβλt+1+μt+111+πt+1+βλt+1+μt+111+πt+1λt+μt-1=2β11+πt+1fnkt-1,nt·λt+1+μt+1λt+μt.Then(37)λtλt+μt=2β11+πt+1λt+1+μt+1λt+μt.From (35),
(38)fkkt-1,nt=Vkat,kt-1,mt-1-1-δλt+μtλt=1β-1-δλt+μtλt.Rewriting the aggregate production asfk=αAkt-1/ntα-1, we have
(39)λtλt+μt=αA-1kt-1nt1-α1β-1-δ.Using (37), we obtain(40)αA-1kt-1nt1-α1β-1-δ=2β11+πt+1λt+1+μt+1λt+μt.Rewriting this equation, we get(41)kt-1nt=2βαA1+πt+11β-1+δ-1λt+1+μt+1λt+μt1/(1-α).In steady-state,λt=λt+1=λss, μt=μt+1=μss, πt+1=πss, kt-1=kss, nt=nss, from (41), we have
(42)kssnss=1+πss2αβ1β-1+δ1/(α-1).This steady-state capital-labor ratio is derived by using current prevailing methods. However, it is a wrong result. We will provide the correct result by using the proposed method of this paper.


## 6.2. Our Solving Approach

Output is affected by state variablekt-1 and control variable nt; it is unsuitable to let the left side of the budget constraint be combined as a state variable. We should separate kt-1 from it. Let state variable mt-1 in value function alone get the economic significance of money in first-order conditions. Because overlaps of state variables are not allowed according to our proposed method, we put ζt=τt+(((1+it-1)bt-1)/(1+πt)) in value function as a state variable.First, set up value function with multipliers:(43)Vζt,kt-1,mt-1=max⁡uct,lt+βVζt+1,kt,mt+λtζt+mt-11+πt+fkt-1,nthhhhhhhhh+1-δkt-1-ct-kt-mt-btζt+mt-11+πt+fkt-1,nt+1-δkt-1+μtζt-1+it-1bt-11+πt+mt-11+πthhhhhhhhh-ct-kt+1-δkt-1ζt-1+it-1bt-11+πt+mt-11+πt.Second, compute the derivatives of state variables and derive first-order conditions:(44)∂Vat,kt-1,mt-1∂ct=ucct,lt=λt+μt,∂Vat,kt-1,mt-1∂bt=βVζζt+1,kt,mt·1+it1+πt+1=λt,∂Vat,kt-1,mt-1∂kt=βVkζt+1,kt,mt=λt+μt,∂Vat,kt-1,mt-1∂mt=βVmζt+1,kt,mt=λt,∂Vat,kt-1,mt-1∂nt=-ulct,lt+λtfnkt-1,nt=0.Third, compute the partial derivatives with respect toζt, kt-1, and mt-1 and the envelope theorem yields
(45)∂Vat,kt-1,mt-1∂ζt=Vζζt,kt-1,mt-1=λt+μt,∂Vat,kt-1,mt-1∂kt-1=Vk(ζt,kt-1,mt-1)=λtfkkt-1,nt+1-δ+μt(1-δ),∂Vat,kt-1,mt-1∂mt-1=Vmζt,kt-1,mt-1=λt+μt11+πt.Finally, derive the steady-state capital-labor ratio by the results above:(46)kssnss=1+πssαβ1β-1+δ1/(α-1).The process of deriving this ratio is similar to the deriving process of (42). We will not give unnecessary details here. Compare (46) with (42); the result of (42) is 1/21/(α-1) times that of (46). Equation (46) is the correct result. It is the different partial derivatives with respect to mt in (34) and (44) that cause this deviation. In (34), βVm(at+1,kt,mt)+βVa(at+1,kt,mt)·(1/(1+πt+1))=λt. In (44), βVm(ζt+1,kt,mt)=λt. Fundamentally speaking, the reason for the deviation is the overlapped setting of state variables at and mt-1 versus the nonoverlapped setting of state variables ζt, kt-1, and mt-1. One has to use the particular skills in [9] that is hard to think about and grasp to obtain the right answer or get a wrong result adopting the overlapped setting V(at,kt-1,mt-1). For this reason, the setting of value function by Walsh could not be generalized. In this model considering both Labor Time and Cash-In-Advance constraints, the basic methods of incorporating money into general equilibrium models are generalized, and our proposed method of setting and solution of Bellman equation demonstrates its clarity and validity and corrects the defect that some other current methods of setting and solution are easy to get wrong results.


## 7. Conclusions

As an important tool in theory economics, Bellman equation is very powerful in solving optimization problems of discrete time and is frequently used in monetary theory. It is hard to grasp the setting and solution of Bellman equation and easy to reach wrong conclusions since there is no general method to set Bellman equation or the settings of Bellman equation are excessively flexible. In this paper, we provide a set of general setting and solution methods for Bellman equation with multipliers. In the processes of solving monetary problems, comparing with other current methods in classic reference, our proposed method demonstrates its features of clarity, validity, correct results, easy operation, and generalization. Bellman equation is used not only in monetary problems but also in almost every dynamic programming problem associated with discrete time optimization. Our future work is to study the applicability of the proposed method in this paper in other areas.



---

*Source: 495089-2014-12-21.xml*