# On the Hamilton-Jacobi-Bellman Equation by the Homotopy Perturbation Method

Authors: Abdon Atangana; Aden Ahmed; Suares Clovis Oukouomi Noutchie
Journal: Abstract and Applied Analysis
 (2014)

---

## Abstract

Our concern in this paper is to use the homotopy decomposition method to solve the Hamilton-Jacobi-Bellman equation (HJB). The approach is obviously extremely well organized and is an influential procedure in obtaining the solutions of the equations. We portrayed particular compensations that this technique has over the prevailing approaches. We presented that the complexity of the homotopy decomposition method is of orderO
(
n
). Furthermore, three explanatory examples established good outcomes and comparisons with exact solution.


---

## Body

## 1. Introduction

Theory and application of optimal control have been widely used in different fields such as biomedicine, aircraft systems, and robotics. However, optimal control of nonlinear systems is a challenging task which has been studied extensively for decades. In the past two decades, the indirect methods have been extensively developed. It is well known that the nonlinear optimal control leads to a nonlinear two-point boundary value problem or a Hamilton-Jacobi-Bellman (HJB) partial differential equation. Many recent researches have been devoted to solving these two problems. In general, the HJB equation is a nonlinear partial differential equation that is hard to solve in most cases. An excellent literature review on the methods for solving the HJB equation is provided in [1], where a successive Galerkin approximation method is also considered. In the Galerkin approximation method a sequence of generalized Hamilton-Jacobi-Bellman equations is solved iteratively to obtain a sequence of approximations approaching the solution of Hamilton-Jacobi-Bellman equation. However, the proposed sequence may converge very slowly or even diverge. There are various efficient methods such as those reported in [2, 3] for the computation of open-loop optimal controls. However, feedback controls are much preferred in many engineering applications. Many other numerical methods [4–6]. In those, a sequence of nonhomogeneous linear time-varying TPBVPs is solved instead of directly solving the nonlinear TPBVP derived from the maximum principle. However, solving time-varying equations is much more difficult than solving time-invariant ones. Thus, it is required to solve HJB equation by an approximate-analytic method. Some of these equations were also evaluated in [7]; they used the so-called piecewise homotopy perturbation method to derive approximate solution for (HJB). Their method was a simple modification of the well-known homotopy perturbation method; the modification was based upon an algorithm in a sequence of small interval meaning time step for finding more accurate approximate solutions to the corresponding (HJB) equation.In this paper we use homotopy decomposition method that was recently proposed in [8] to solve the Hamilton-Jacobi-Bellman equation. The method was first used to solve the groundwater flow equation [9]. To show the efficiency of the method, the following three problems are solved.Problem 1.
Consider a single-input scalar system as follows [10]:
(1)
d
x
(
t
)
d
t
=
-
x
(
t
)
+
u
(
t
)
,
J
=
1
2
∫
0
1
(
x
(
t
)
)
2
+
(
u
(
t
)
)
2
d
t
.
The corresponding Hamiltonian function will be
(2)
H
(
x
,
u
,
V
x
,
t
)
=
1
2
x
(
t
)
2
+
1
2
u
(
t
)
2
+
∂
V
(
x
,
t
)
∂
x
[
-
x
(
t
)
+
u
(
t
)
]
.Problem 2.
Consider the following purely mathematical optimal control problem:(3)
d
x
(
t
)
d
t
=
x
(
t
)
+
u
(
t
)
,
J
=
x
(
t
f
)
2
+
∫
0
t
f
u
2
(
t
)
d
t
.
The corresponding Hamiltonian function will be
(4)
H
(
x
,
u
,
V
x
,
t
)
=
u
(
t
)
2
+
V
x
(
x
,
t
)
(
x
(
t
)
+
u
(
t
)
)
.Problem 3.
Consider the following nonlinear optimal control problem [11]:
(5)
d
x
(
t
)
d
t
=
1
2
x
(
t
)
2
sin
⁡
(
x
(
t
)
)
+
u
(
t
)
,
t
∈
[
0,1
]
,
x
(
0
)
=
0
,
x
(
1
)
=
0.5
,
J
=
∫
0
1
u
(
t
)
2
d
t
.
The corresponding Hamiltonian function will be
(6)
H
(
x
,
u
,
V
x
,
t
)
=
u
(
t
)
2
+
V
x
(
x
,
t
)
(
1
2
x
(
t
)
2
sin
⁡
(
x
(
t
)
)
+
u
(
t
)
)
.
The paper is organized as follows. In Section2, the basics idea of homotopy decomposition method is presented. In Section 3, the advantages of the chosen method are presented. In Section 4, nonlinear time-variant HJB equation is presented. Conclusion is made in Section 5.


## 2. Homotopy Decomposition Method [12, 13]

To exemplify the primitive thought of this approach we study an overall nonlinear nonhomogeneous partial differential equation with initial conditions of the following form:(7)
∂
m
U
(
x
,
t
)
∂
t
m
=
L
(
U
(
x
,
t
)
)
+
N
(
U
(
x
,
t
)
)
+
f
(
x
,
t
)
,
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
m
=
1,2
,
3
,
…
subject to the initial condition
(8)
∂
i
U
(
x
,
0
)
∂
t
i
=
f
m
(
x
)
,
∂
m
-
1
U
(
x
,
0
)
∂
t
m
-
1
=
0
,
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
i
=
0,1
,
2
,
…
,
m
-
2
.
Somewhere, f is a known function, N is the general nonlinear differential operator, and L represents a linear differential operator. The first step of the method here is to apply the inverse operator ∂
m
/
∂
t
m of both sides of (7) to obtain
(9)
U
(
x
,
t
)
=
∑
k
=
0
m
-
1
t
k
k
!
d
k
u
(
x
,
t
)
d
t
k
∣
t
=
0
+
∫
0
t
∫
0
t
1
⋯
∫
0
t
m
-
1
L
(
U
(
x
,
τ
)
)
+
N
(
U
(
x
,
τ
)
)
hhhhhhhhhhhhhhhhh
+
f
(
x
,
τ
)
d
τ
⋯
d
t
.
The multi-integral in (9) can be distorted to
(10)
∫
0
t
∫
0
t
1
⋯
∫
0
t
m
-
1
L
(
U
(
x
,
τ
)
)
+
N
(
U
(
x
,
τ
)
)
+
f
(
x
,
τ
)
d
τ
⋯
d
t
=
1
(
m
-
1
)
!
∫
0
t
(
t
-
τ
)
m
-
1
L
(
U
(
x
,
τ
)
)
hhhhhhhhhhhh
+
N
(
U
(
x
,
τ
)
)
+
f
(
x
,
τ
)
d
τ
,
so that (9) can be reformulated as
(11)
U
(
x
,
t
)
=
∑
k
=
0
m
-
1
t
k
k
!
{
d
k
u
(
x
,
0
)
d
t
k
}
+
1
(
m
-
1
)
!
×
∫
0
t
(
t
-
τ
)
m
-
1
L
(
U
(
x
,
τ
)
)
+
N
(
U
(
x
,
τ
)
)
+
f
(
x
,
τ
)
d
τ
.
Employing the homotopy arrangement the resolution of the overhead integral equation is assumed in series form as
(12)
U
(
x
,
t
,
p
)
=
∑
n
=
0
∞
p
n
U
n
(
x
,
t
)
,
U
(
x
,
t
)
=
lim
⁡
p
→
1
⁡
U
(
x
,
t
,
p
)
,
and the nonlinear term can be decomposed as
(13)
N
U
(
x
,
t
)
=
∑
n
=
1
∞
p
n
H
n
(
U
)
,
where p
ϵ
(
0,1
] is an embedding parameter. H
n
(
U
) is He’s polynomials [14] that can be generated by
(14)
H
n
(
U
0
,
…
,
U
n
)
=
1
n
!
∂
n
∂
p
n
[
N
(
∑
j
=
0
n
p
j
U
j
(
x
,
t
)
)
]
,
n
=
0,1
,
2
,
…
.
The homotopy decomposition method is obtained by the beautiful coupling of decomposition method with He’s polynomials and is given by
(15)
∑
n
=
0
∞
p
n
U
n
(
x
,
t
)
=
T
(
x
,
t
)
+
p
1
(
m
-
1
)
!
×
∫
0
t
(
t
-
τ
)
m
-
1
[
f
(
x
,
τ
)
+
L
(
∑
n
=
0
∞
p
n
U
n
(
x
,
τ
)
)
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
+
∑
n
=
0
∞
p
n
H
n
(
U
)
]
d
τ
with
(16)
T
(
x
,
t
)
=
∑
k
=
0
m
-
1
t
k
k
!
{
d
k
u
(
x
,
0
)
d
t
k
}
.
Comparing the terms of same powers of p, give solutions of various orders. The initial guess of the approximation is T
(
x
,
t
) [13]. It is important to point out that the initial guess T
(
x
,
t
) is Taylor series of order m of the exact solution of the main problem [13].


### 2.1. Advantage of the Method

The homotopy decomposition method is chosen to solve this nonlinear problem because of the following advantages the method has over the existing methods.(1)
The method does not require the linearization or assumptions of weak nonlinearity.
(2)
The solutions are not generated in the form of general solution as in Adomian decomposition method. With ADM the recursive formula allows repetition of terms in the case of nonhomogeneous partial differential equation and this leads to the noisy solution [15].
(3)
The solution obtained is noise-free compared to the variational iteration method [16].
(4)
No correctional function is required as in the case of the variational homotopy decomposition method.
(5)
No Lagrange multiplier is required in the case of the variational iteration method [16].
(6)
It is more realistic compared to the method of simplifying the physical problems.
(7)
If the exact solution of the partial differential equation exists, the approximated solution via the method converges to the exact solution [8].
(8)
A construction of a homotopyv
(
r
,
p
)
:
Ω
×
[
0,1
] is not needed as in the case of the homotopy perturbation method [14].


## 2.1. Advantage of the Method

The homotopy decomposition method is chosen to solve this nonlinear problem because of the following advantages the method has over the existing methods.(1)
The method does not require the linearization or assumptions of weak nonlinearity.
(2)
The solutions are not generated in the form of general solution as in Adomian decomposition method. With ADM the recursive formula allows repetition of terms in the case of nonhomogeneous partial differential equation and this leads to the noisy solution [15].
(3)
The solution obtained is noise-free compared to the variational iteration method [16].
(4)
No correctional function is required as in the case of the variational homotopy decomposition method.
(5)
No Lagrange multiplier is required in the case of the variational iteration method [16].
(6)
It is more realistic compared to the method of simplifying the physical problems.
(7)
If the exact solution of the partial differential equation exists, the approximated solution via the method converges to the exact solution [8].
(8)
A construction of a homotopyv
(
r
,
p
)
:
Ω
×
[
0,1
] is not needed as in the case of the homotopy perturbation method [14].


## 3. Complexity of the Homotopy Decomposition Method

It is very important to test the computational complexity of the method or algorithm. Complexity of an algorithm is the study of how long a program will take to run, depending on the size of its input and length of loops made inside the code [13]. We compute a numerical example which is solved by the homotopy decomposition method. The code has been presented with Mathematica 8 according to the following code [13].Step 1.
Setm
←
0.Step 2.
Calculating the recursive relation after the comparison of the terms of the same power is done.Step 3.
If∥
U
n
+
1
(
x
,
t
)
-
U
n
(
x
,
t
)
∥
<
r with r being the ratio of the neighbourhood of the exact solution [8] then go to Step 4; else m
←
m
+
1 and go to Step 2.Step 4.
Print out(17)
U
(
x
,
t
)
=
∑
n
=
0
∞
U
n
(
x
,
t
)
as the approximation of the exact solution.Lemma 1.
If the exact solution of the partial differential equation (7) exists, then
(18)
∥
U
n
+
1
(
x
,
t
)
-
U
n
(
x
,
t
)
∥
<
r
∀
(
x
,
t
)
∈
X
×
T
.Proof.
Let(
x
,
t
)
∈
X
×
T; since the exact solution exists, then we have the following:
(19)
∥
U
n
+
1
(
x
,
t
)
-
U
n
(
x
,
t
)
∥
=
∥
U
n
+
1
(
x
,
t
)
-
U
(
x
,
t
)
+
U
(
x
,
t
)
-
U
n
(
x
,
t
)
∥
≤
∥
U
n
+
1
(
x
,
t
)
-
U
(
x
,
t
)
∥
+
∥
-
U
n
(
x
,
t
)
+
U
(
x
,
t
)
∥
≤
r
2
+
r
2
=
r
.
The last inequality follows from [13].Lemma 2.
The complexity of the homotopy decomposition method is of orderO
(
n
).Proof.
The number of computations including product, addition, subtraction, and division is in Step2:U
0: 0 because, obtains directly from the initial guess [13],
U
1: 3,
⋮
U
n: 3.
Now in Step4 the total number of computations is equal to ∑
j
=
0
n
U
j
(
x
,
t
)
=
3
n
=
O
(
n
).


## 4. Nonlinear Time-Variant Hamilton-Jacobi-Bellman Equation

In this paper, we consider a general nonlinear control system described by(20)
x
˙
(
t
)
=
L
(
x
(
t
)
,
u
(
t
)
,
t
)
,
where a state is x
(
t
) and vector  u
(
t
) is a control signal. The objective is to find the optimal control law u
*
(
t
), which minimizes the following cost function:
(21)
J
=
k
(
x
(
t
f
)
,
t
f
)
+
∫
0
t
f
f
(
x
(
t
)
,
u
(
t
)
,
t
)
d
t
.
In this cost function, k and f are arbitrary convex functions and t
f is final time of system operation. It is supposed [4] that
(22)
V
(
x
(
t
)
,
t
)
=
J
*
(
x
(
t
)
,
t
)
=
min
⁡
u
(
τ
)
t
≤
τ
≤
t
f
{
∫
t
t
f
f
k
(
x
(
t
f
)
,
t
f
)
hhhhhhh
+
∫
t
t
f
f
(
x
(
τ
)
,
u
(
τ
)
,
τ
)
d
τ
}
,
-
∂
V
∂
t
=
H
(
x
,
u
*
(
x
,
V
x
,
t
)
,
V
x
,
t
)
.Problem 4.
Consider a single-input scalar system as follows [9]:
(23)
d
x
(
t
)
d
t
=
-
x
(
t
)
+
u
(
t
)
,
J
=
1
2
∫
0
1
(
x
(
t
)
)
2
+
(
u
(
t
)
)
2
d
t
.
The corresponding Hamiltonian function will be
(24)
H
(
x
,
u
,
V
x
,
t
)
=
1
2
x
(
t
)
2
+
1
2
u
(
t
)
2
+
∂
V
(
x
,
t
)
∂
x
[
-
x
(
t
)
+
u
(
t
)
]
.
Our concern here is to find u
*, that is, stationary point for the Hamiltonian function. Therefore, differentiating (24) with respect to u, we obtain
(25)
∂
H
∂
u
=
u
(
t
)
+
∂
V
(
x
,
t
)
∂
x
=
0
⟹
u
*
(
t
)
=
-
∂
V
(
x
,
t
)
∂
x
.
Applying the second derivative test, we obtain ∂
2
H
/
∂
u
2
=
1
>
0, since the second derivative is positive for all u; it follows that our turning point is a minimum, which is acceptable because our concern is to find the minimum value. Thus, by substituting u
*
(
t
) in Hamilton-Jacobi-Bellman equation we obtain
(26)
∂
V
∂
t
=
-
1
2
x
(
t
)
2
+
1
2
(
∂
V
(
x
,
t
)
∂
x
)
2
+
x
(
t
)
∂
V
(
x
,
t
)
∂
x
,
d
x
(
t
)
d
t
=
-
x
(
t
)
-
∂
V
(
x
,
t
)
∂
x
.
To solve the above system of equation via the homotopy decomposition method, we transform it to the integral equation as follows:
(27)
x
(
t
)
=
x
(
0
)
-
∫
0
t
x
(
τ
)
+
∂
V
(
x
,
τ
)
∂
x
d
τ
,
(28)
V
(
x
,
t
)
=
V
(
x
,
0
)
+
∫
0
t
[
-
1
2
x
2
+
1
2
(
∂
V
(
x
,
τ
)
∂
x
)
2
+
x
∂
V
(
x
,
τ
)
∂
x
]
d
τ
.
Using the homotopy scheme the solution of the above integral equation is given in series form as
(29)
V
(
x
,
t
,
p
)
=
∑
n
=
0
∞
p
n
V
n
(
x
,
t
)
,
V
(
x
,
t
)
=
lim
⁡
p
→
1
⁡
V
(
x
,
t
,
p
)
verifying
(30)
∑
n
=
0
∞
p
n
V
n
(
x
,
t
)
=
V
(
x
,
0
)
+
p
∫
0
t
[
-
1
2
x
2
+
1
2
∂
∂
x
(
∑
n
=
0
∞
p
n
V
n
(
x
,
τ
)
)
2
hhhhhhhhhhhhhhhh
+
x
∂
∂
x
(
∑
n
=
0
∞
p
n
V
n
(
x
,
τ
)
)
(
∑
n
=
0
∞
p
n
V
n
(
x
,
τ
)
)
2

]
d
τ
.
Comparing the terms of same powers of p, we obtain the following system of integral equations:
(31)
p
0
:
V
0
(
x
,
t
)
=
V
(
x
,
0
)
=
0
,
p
1
:
V
1
(
x
,
t
)
=
∫
1
t
[
-
1
2
x
2
+
1
2
(
∂
V
0
(
x
,
τ
)
∂
x
)
2
+
x
∂
V
0
(
x
,
τ
)
∂
x
]
d
τ
,
V
1
(
x
,
0
)
=
0
,
p
2
:
V
2
(
x
,
t
)
=
∫
1
t
[
+
1
2
2
∂
V
0
(
x
,
τ
)
∂
x
∂
V
1
(
x
,
τ
)
∂
x
+
x
∂
V
1
(
x
,
τ
)
∂
x
]
d
τ
,
V
2
(
x
,
0
)
=
0
,
p
n
:
V
n
(
x
,
t
)

=

∫
1
t
[
1
2
∑
j
=
0
n
-
1
∂
V
j
(
x
,
τ
)
∂
x
∂
V
n
-
j
-
1
(
x
,
τ
)
∂
x

+

x
∂
V
n
-
1
(
x
,
τ
)
∂
x
]
d
τ
,

V
n
(
x
,
0
)
=
0
.Lemma 3.
Ifx
(
t
)
≠
0 over its domain, then u
*
(
x
(
t
)
,
t
)
=
-
l
(
t
)
x
(
t
).Proof.
Notice that(32)
∂
V
(
x
,
t
)
∂
t
=
∂
V
(
x
,
t
)
∂
x
∂
x
∂
t
=
-
u
*
(
x
,
t
)
x
˙
(
t
)
=
-
u
*
(
x
,
t
)
[
-
x
(
t
)
+
u
(
t
)
]
.
Replacing (32) in (26) we obtain
(33)
-
u
*
(
x
,
t
)
[
-
x
(
t
)
+
u
(
t
)
]
=
-
1
2
x
(
t
)
2
+
1
2
(
u
*
(
x
,
t
)
)
2
-
x
(
t
)
u
*
(
x
,
t
)
.
Arranging this we obtain
(34)
(
u
*
(
x
,
t
)
)
2
-
2
u
*
(
x
,
t
)
[
2
x
(
t
)
-
u
(
t
)
]
-
x
2
(
t
)
=
0
.
Solving the above, we obtain
(35)
u
*
(
x
(
t
)
,
t
)
=
-
x
(
t
)
[
-
2
+
u
(
t
)
x
(
t
)
+
1
x
(
t
)
(
2
x
(
t
)
-
u
(
t
)
)
2
+
x
2
(
t
)
]
=
-
l
(
t
)
x
(
t
)
.
Solving the integral equations we obtain the following terms:
(36)
V
0
(
x
,
t
)
=
V
(
x
,
0
)
=
0
,
V
1
(
x
,
t
)
=
-
x
2
2
(
t
-
1
)
,
V
2
(
x
,
t
)
=
-
x
2
(
1
2
-
t
+
t
2
2
)
,
V
3
(
x
,
t
)
=
-
x
2
(
t
2
-
1
6
-
t
2
2
+
t
3
6
)
,
V
4
(
x
,
t
)
=
-
x
2
(
2
3
t
-
1
6
-
t
2
+
2
3
t
3
-
t
4
6
)
,
V
5
(
x
,
t
)
=
-
x
2
(
-
1
12
+
t
6
+
t
2
6
-
2
t
3
3
+
7
t
4
12
-
t
5
6
)
,
V
6
(
x
,
t
)
=
-
x
2
(
-
8
45
+
t
2
-
t
2
2
+
t
3
3
-
t
4
3
+
7
t
5
30
-
t
6
18
)
⋮
Using the package Mathematica, in the same manner, one can obtain the rest of the components. But, here, 16 terms were computed and the asymptotic solution is given by
(37)
u
N
=
7
*
(
x
,
t
)
=
-
∂
x
V
N
=
16
(
x
,
t
)
=
-
x
l
(
t
)
.
This verifies Lemma 3.
The analytical solution for this problem of statex
(
t
) and the control u
(
t
) is given as [4]
(38)
x
(
t
)
=
cosh
⁡
⁡
(
2
t
)
+
α
sinh
⁡
⁡
(
2
t
)
u
(
t
)
=
(
1
+
2
α
)
cosh
⁡
⁡
(
2
t
)
+
(
2
+
α
)
sinh
⁡
⁡
(
2
t
)
with
(39)
α
=
-
cosh
⁡
⁡
(
2
)
+
2
sinh
⁡
⁡
(
2
)
2
cosh
⁡
⁡
(
2
)
+
sinh
⁡
⁡
(
2
)
,
u
*
(
x
,
t
)
=
-
l
(
t
)
x
(
t
)
,
where, for simplicity,
(40)
l
(
t
)
=
-
(
1
+
2
α
)
cosh
⁡
⁡
(
2
t
)
+
(
2
+
α
)
sinh
⁡
⁡
(
2
t
)
cosh
⁡
⁡
(
2
t
)
+
α
sinh
⁡
⁡
(
2
t
)
.
To access the accuracy of the homotopy decomposition method, we determine numerically the absolute errors |
l
analytic
(
t
)
-
l
HDM
(
t
)
| for n
=
16 as shown in Figure 1 that is showing the comparison between the two.Figure 1
Approximated solution.Problem 5.
Consider the following purely mathematical optimal control problem:(41)
d
x
(
t
)
d
t
=
x
(
t
)
+
u
(
t
)
,
J
=
x
(
t
f
)
2
+
∫
0
t
f
u
2
(
t
)
d
t
.
The corresponding Hamiltonian function will be(42)
H
(
x
,
u
,
V
x
,
t
)
=
u
(
t
)
2
+
V
x
(
x
,
t
)
(
x
(
t
)
+
u
(
t
)
)
.
Our concern here is to find u
*, that is, stationary point for the Hamiltonian function. Therefore, differentiating (42) with respect to u we obtain
(43)
∂
H
∂
u
=
2
u
(
t
)
+
∂
V
(
x
,
t
)
∂
x
=
0
⟹
u
*
(
t
)
=
-
∂
V
(
x
,
t
)
2
∂
x
.
Applying the second derivative test we obtain ∂
2
H
/
∂
u
2
=
2
>
0, since the second derivative is positive for all u; it follows that our turning point is a minimum, which is acceptable because our aim is to minimise. Thus, by substituting u
*
(
t
) in Hamilton-Jacobi-Bellman equation we obtain
(44)
∂
V
(
x
,
t
)
∂
t
=
1
4
V
2
(
x
,
t
)
∂
x
-
x
∂
V
(
x
,
t
)
∂
x
subjected to the following condition:
(45)
V
(
x
(
t
f
)
,
t
f
)
=
x
2
(
t
f
)
.
The exact solution of this form was proposed in [4] as
(46)
V
(
x
,
t
,
t
f
)
=
2
x
2
1
+
e
2
(
t
-
t
f
)
.
Shadowing the homotopy decomposition structure and equaling the relations of the same powers of p, we obtain the following system of integral equations:
(47)
p
0
:
V
0
(
x
,
t
)
=
V
(
x
,
0
)
=
0
,
p
1
:
V
1
(
x
,
t
)
=
∫
t
f
t
[
1
4
(
∂
V
0
(
x
,
τ
)
∂
x
)
2
-
x
∂
V
0
(
x
,
τ
)
∂
x
]
d
τ
,
V
1
(
x
,
0
)
=
0
,
p
2
:
V
2
(
x
,
t
)
=
∫
t
f
t
[
1
4
2
∂
V
0
(
x
,
τ
)
∂
x
∂
V
1
(
x
,
τ
)
∂
x
-
x
∂
V
1
(
x
,
τ
)
∂
x
]
d
τ
,
V
2
(
x
,
0
)
=
0
,
p
n
:
V
n
(
x
,
t
)
=
∫
t
f
t
[
1
4
∑
j
=
0
n
-
1
∂
V
j
(
x
,
τ
)
∂
x
∂
V
n
-
j
-
1
(
x
,
τ
)
∂
x
h
h
h
h
h
h
h
+
x
∂
V
n
-
1
(
x
,
τ
)
∂
x
∑
j
=
0
n
-
1
∂
V
j
(
x
,
τ
)
∂
x
∂
V
n
-
j
-
1
(
x
,
τ
)
∂
x
]
d
τ
,
V
n
(
x
,
0
)
=
0
.
The following solutions are obtained. Here we chose the first term to be -
x
2 so that
(48)
V
1
(
x
,
t
)
=
-
x
2
(
t
-
1
)
,
V
2
(
x
,
t
)
=
0
,
V
3
(
x
,
t
)
=
-
x
2
(
1
3
-
t
+
t
2
-
t
3
3
)
,
V
4
(
x
,
t
)
=
0
,
V
5
(
x
,
t
)
=
-
x
2
(
-
2
15
+
2
3
t
-
4
t
2
3
+
4
t
3
3
-
2
t
4
3
+
2
t
5
15
)
,
V
6
(
x
,
t
)
=
0
,
V
7
(
x
,
t
)
=
-
x
2
(
17
315
-
17
45
+
17
t
2
15
-
17
t
3
9
+
17
t
4
9
-
17
t
5
15
h
h
h
h
h
h
h
h
h
+
17
t
6
45
-
17
t
7
315
)
⋮
With the similar routine one can acquire the remaining terms of the components. But 8 terms were computed and the asymptotic solution is given by
(49)
V
(
x
,
t
)
=
V
0
(
x
,
t
)
+
V
1
(
x
,
t
)
+
⋯
,
V
(
x
,
t
)
=
-
x
2
(
t
)
f
(
t
)
⟹
u
*
(
x
(
t
)
,
t
)
=
1
2
x
(
t
)
f
(
t
)
.
To access the accuracy of the homotopy decomposition method, we determine numerically the absolute errors |
f
analytic
(
t
)
-
f
HDM
(
t
)
| for n
=
7 as shown in Figure 4. Figures 2 and 3 show approximate solution of equation and numerical error respectively.Figure 2
Exact solution.Figure 3
Numerical error.Figure 4
Numerical error.Problem 6.
Consider the following nonlinear optimal control problem [14]:
(50)
d
x
(
t
)
d
t
=
1
2
x
(
t
)
2
sin
⁡
(
x
(
t
)
)
+
u
(
t
)
,
t
∈
[
0,1
]
,
x
(
0
)
=
0
,
x
(
1
)
=
0.5
,
J
=
∫
0
1
u
(
t
)
2
d
t
.
The corresponding Hamiltonian function will be
(51)
H
(
x
,
u
,
V
x
,
t
)

=

u
(
t
)
2

+

V
x
(
x
,
t
)
(
1
2
x
(
t
)
2
sin
⁡
(
x
(
t
)
)

+

u
(
t
)
)
.
Following the discussion presented earlier, we obtain
(52)
∂
H
∂
u
=
2
u
(
t
)
+
∂
V
(
x
,
t
)
∂
x
=
0
⟹
u
*
(
t
)
=
-
∂
V
(
x
,
t
)
2
∂
x
.
Applying the second derivative test we obtain ∂
2
H
/
∂
u
2
=
2
>
0, since the second derivative is positive for all u; it follows that our turning point is a minimum, which is acceptable because our aim is to minimise. Thus, by substituting u
*
(
t
) in Hamilton-Jacobi-Bellman equation we obtain
(53)
∂
V
(
x
,
t
)
∂
t
=
1
4
(
∂
V
(
x
,
t
)
∂
x
)
2
-
1
2
x
2
sin
⁡
(
x
)
∂
V
(
x
,
t
)
∂
x
subjected to the following condition:
(54)
V
(
x
(
1
)
,
1
)
=
0
.
Following the homotopy decomposition scheme and comparing the terms of the same powers of p, we obtain the following system of integral equations:
(55)
p
0
:
V
0
(
x
,
t
)
=
V
(
x
,
0
)
=
0
,
p
1
:
V
1
(
x
,
t
)
=
∫
t
f
t
[
1
4
(
∂
V
0
(
x
,
τ
)
∂
x
)
2
-
x
sin
(
x
)
∂
V
0
(
x
,
τ
)
∂
x
]
d
τ
,
V
1
(
x
,
0
)
=
0
,
p
2
:
V
2
(
x
,
t
)
=

∫
t
f
t
[
1
4
2
∂
V
0
(
x
,
τ
)
∂
x
∂
V
1
(
x
,
τ
)
∂
x

-

x
sin
(
x
)
∂
V
1
(
x
,
τ
)
∂
x
]
d
τ
,
V
2
(
x
,
0
)
=
0
,
p
n
:
V
n
(
x
,
t
)
=
∫
t
f
t
[
1
4
∑
j
=
0
n
-
1
∂
V
j
(
x
,
τ
)
∂
x
∂
V
n
-
j
-
1
(
x
,
τ
)
∂
x
h
h
h
h
h
h
h
+
x
sin
(
x
)
∂
V
n
-
1
(
x
,
τ
)
∂
x
∑
j
=
0
n
-
1
∂
V
j
(
x
,
τ
)
∂
x
∂
V
n
-
j
-
1
(
x
,
τ
)
∂
x
]
d
τ
,
V
n
(
x
,
0
)
=
0
.
The following solutions are obtained. Here we chose the first term to be -
x so that
(56)
V
1
(
x
,
t
)
=
t
4
+
x
2
t
sin
(
x
)
,
V
2
(
x
,
t
)
=
-
t
2
x
2
4
cos
⁡
⁡
(
x
)
-
t
2
x
sin
(
x
)
2
-
t
3
x
4
2
cos
⁡
⁡
(
x
)
sin
⁡
(
x
)
-
t
2
x
3
sin
2
(
x
)
.
The asymptotic solution is given by
(57)
V
(
x
,
t
)
=
V
0
(
x
,
t
)
+
V
1
(
x
,
t
)
+
⋯
.Figure 5 shows the graphical representation of u
*
(
x
,
t
).Figure 5
Numerical approximation ofV
(
x
,
t
).


## 5. Conclusion

The purpose of this paper was to use the homotopy decomposition method to solve the Hamilton-Jacobi-Bellman equation. The method is clearly a very efficient and powerful technique in finding the solutions of the equations. We presented some advantages the method has over the existing methods. We showed that the complexity of the homotopy decomposition method is of orderO
(
n
). In addition, three illustrative examples demonstrated good results and comparisons with exact solution. Although the technique used in this paper is very accurate, we have to mention that there exist more appropriate techniques that can be used to handle this type of nonlinear equation.



---

*Source: 436362-2014-06-22.xml*