# General LQG Homing Problems in One Dimension

Authors: Mario Lefebvre; Foued Zitouni
Journal: International Journal of Stochastic Analysis
 (2012)

---

## Abstract

Optimal control problems for one-dimensional diffusion processes in the interval (d1,d2) are considered. The aim is either to maximize or to minimize the time spent by the controlled processes in (d1,d2). Exact solutions are obtained when the processes are symmetrical with respect to d∗∈(d1,d2). Approximate solutions are derived in the asymmetrical case. The one-barrier cases are also treated. Examples
are presented.


---

## Body

## 1. Introduction

Let{X(t),t≥0} be a one-dimensional controlled diffusion process defined by the stochastic differential equation
(1.1)dX(t)=m[X(t)]dt+b0Xk(t)u[X(t)]dt+{v[X(t)]}1/2dB(t),
where u(·) is the control variable, m(·) and v(·)>0 are Borel measurable functions, b0≠0 and k∈{0,1,…} are constants, and {B(t),t≥0} is a standard Brownian motion. The set of admissible controls consists of Borel measurable functions.Remark 1.1.
We assume that the solution of (1.1) exists for all t∈[0,∞) and is weakly unique.We define the first-passage time(1.2)T(x)=inf{t>0:X(t)=d1ord2∣X(0)=x},
where x∈(d1,d2). We want to find the control u* that minimizes the expected value of the cost function
(1.3)J(x)=∫0T(x)‍{12q0u2[X(t)]+λ}dt,
where q0>0 and λ≠0 are constants. Notice that if λ is negative, then the optimizer wants to maximize the survival time of the controlled process in the interval (d1,d2), taking the quadratic control costs into account. In general, there is a maximal value that the parameter λ can take. Otherwise, the expected reward becomes infinite.When the relation(1.4)αv[X(t)]=b02q0X2k(t)
holds for some positive constant α, using a theorem in Whittle [1, p. 289], we can express the value function
(1.5)F(x):=infu[X(t)],0≤t≤T(x)E[J(x)]
in terms of a mathematical expectation for the uncontrolled process obtained by setting u[X(t)]≡0 in (1.1). Actually, for the result to hold, ultimate entry of the uncontrolled process into the stopping set must be certain, which is not a restrictive condition in the case of one-dimensional diffusion processes considered in finite intervals.In practice, the theorem in Whittle [1] gives a transformation that enables us to linearize the differential equation satisfied by the function F(x).In Lefebvre [2], using symmetry, the author was able to obtain an explicit and exact expression for the optimal control u* when {X(t),t≥0} is a one-dimensional controlled standard Brownian motion process (so that m[X(t)]≡0 and v[X(t)]≡1), d2=-d1=d and k=1. Notice that the relation in (1.4) does not hold in that case. The author assumed that the parameter λ in the cost function is negative, and he found the maximal value that this parameter can take.Previously, Lefebvre [3] had computed the value of u* when k=0, but with the cost function
(1.6)J1(x)=∫0T(x)‍{12q0X2(t)u2[X(t)]+λ}dt
rather than the function J(x) defined above. We cannot appeal to the theorem in Whittle [1] in that case either. However, the author expressed the function F(x) in terms of a mathematical expectation for an uncontrolled geometric Brownian motion.In Section2, we will generalize the results in Lefebvre [2] to one-dimensional diffusion processes for which the functions m[X(t)] and v[X(t)] are symmetrical with respect to zero and d1=-d2. Important particular cases will be considered.Next, in Section3, we will treat the general symmetrical case when d1 is not necessarily equal to -d2. In Section 4, we will consider processes for which the functions m[X(t)] and v[X(t)] are not symmetrical with respect to a certain d*∈(d1,d2). An approximate solution will then be proposed. In Section 5, we will present possible extensions, including the case of a single barrier. Finally, we will make some concluding remarks in Section 6.


## 2. Optimal Control in the Symmetrical Case withd1=-d2

In this section, we taked2=-d1=d. Assuming that it exists and that it is twice differentiable, we find that the value function F(x) defined in (1.5) satisfies the dynamic programming equation
(2.1)infu(x){12q0u2(x)+λ+[m(x)+b0xku(x)]F′(x)+12v(x)F'’(x)}=0,
where x=X(0). Differentiating with respect to u(x) and equating to zero, we deduce that the optimal control u*(x) can be expressed as follows:
(2.2)u*(x)=-b0xkq0F′(x).
Substituting the optimal control into the dynamic programming equation (2.1), we obtain that the function F(x) satisfies the nonlinear second-order ordinary differential equation
(2.3)λ+m(x)F′(x)-b02x2k2q0[F′(x)]2+12v(x)F'’(x)=0,
subject to the boundary conditions
(2.4)F(d)=F(-d)=0.Now, in general solving nonlinear second-order differential equations is not an easy task. As mentioned previously, when the relation in (1.4) holds, there exists a transformation that enables us to linearize (2.3). Notice, however, that in order to obtain an explicit expression for the optimal control u*(x), one only needs the derivative of the value function F(x). Hence, if we can find a boundary condition in terms of F'(x), rather than the boundary conditions in (2.4), then we could significantly simplify our problem, since we would only have to solve the first-order nonlinear (Riccati) differential equation:
(2.5)λ+m(x)G(x)-b02x2k2q0G2(x)+12v(x)G′(x)=0,
where
(2.6)G(x):=F′(x).Proposition 2.1.
Assume that the functionm(x) is odd and that the function v(x) is even. Then the optimal control u*(x) is given by
(2.7)u*(x)=-b0xkq0G(x),
where G(x) satisfies (2.5), subject to the condition
(2.8)G(0)=0.Proof.
The condition (2.8) follows from the fact that, by symmetry, when the parameter λ is positive, then 0 is the value of x for which the function F(x) has a maximum, whereas F(x) has a minimum at x=0 when λ is negative. Indeed, the origin is the worst (resp., best) position possible when the optimizer is trying to minimize (resp., maximize), taking the quadratic control costs into account, the time spent by X(t) in the interval (-d,d).Remarks 2.2.
(i) The solution to (2.5), subject to (2.8), might not be unique.
(ii) Notice that the symmetrical case includes the one whenm(x) is identical to 0, and v(x) is a constant, so that the uncontrolled process is a Wiener process with zero drift.The previous proposition can be generalized as follows.Corollary 2.3.
IfXk(t) is replaced by h[X(t)] in (1.1), where h2(x) is even, and if the hypotheses in Proposition 2.1 are satisfied, then the optimal control u*(x) can be expressed as
(2.9)u*(x)=-b0h(x)q0G(x),
where G(x) is a solution of
(2.10)λ+m(x)G(x)-b02h2(x)2q0G2(x)+12v(x)G′(x)=0
that satisfies the condition G(0)=0.We will now present an example for which we can determine the optimal controlu* explicitly.We consider the case when{X(t),t≥0} is a controlled Bessel process, so that
(2.11)dX(t)=θ-12X(t)dt+b0Xk(t)u[X(t)]dt+dB(t).
Moreover, we assume that the parameter θ belongs to the interval (0,2). The origin is then a regular boundary (see [4, p. 239]) for the uncontrolled process {X0(t),t≥0} obtained by setting u[X(t)]≡0 in the stochastic differential equation above. Notice that if the parameter θ is equal to 1, then {X0(t),t≥0} becomes a standard Brownian motion, which is the process considered in Lefebvre [2]. Therefore, this example generalizes the results in Lefebvre’s paper.Here, the relation in (1.4) holds if there exists a positive constant α such that
(2.12)α=b02q0X2k(t).
Hence, we can appeal to the theorem in Whittle [1] when k is equal to zero. We will treat the case when k>0 instead.The differential equation that we must solve is(2.13)λ+θ-12xG(x)-b02x2k2q0G2(x)+12G′(x)=0.
We find that
(2.14)G(x)=-2λq0b0xk{Jν(cxk+1)+c0Yν(cxk+1)Jν-1(cxk+1)+c0Yν-1(cxk+1)},
where Jν and Yν are Bessel functions and c0 is an arbitrary constant,
(2.15)ν:=θ2(k+1),c:=b0-2λ/q0k+1.The expression above for the functionG(x) is appropriate when the parameter λ is negative. However, when λ>0, it is better to rewrite it as follows:
(2.16)G(x)=2λq0b0xk{c0Kν(c*xk+1)-Iν(c*xk+1)c0Kν-1(c*xk+1)+Iν-1(c*xk+1)},
where Iν and Kν are modified Bessel functions and
(2.17)c*:=b02λ/q0k+1=ic.In order to determine the value of the constantc0, we will use the condition G(0)=0. First, we consider the special case when k=1, θ=1 and λ is negative. Then, we have that ν=1/4 and
(2.18)G(x)=-2λq0b0x{J1/4(cx2)+c0Y1/4(cx2)J-3/4(cx2)+c0Y-3/4(cx2)},
where c=b0-λ/(2q0).Next, whenν≠-1,-2,…, we have the formula (see [5, p. 358])
(2.19)Yν(z)=Jν(z)cos(νπ)-J-ν(z)sin(νπ).
It follows, with ν=1/4 and ν=-3/4, that
(2.20)G(x)=-2λq0b0x{(1+c0)J1/4(cx2)-2c0J-1/4(cx2)(1+c0)J-3/4(cx2)+2c0J3/4(cx2)}.Finally, making use of the limiting form of the functionJν(z) when z→0 (see [5, p. 360]):
(2.21)Jν(z)~(z2)ν1Γ(ν+1)(ifν≠-1,-2,…),
we obtain that
(2.22)limx→0G(x)=--2λq0b0Γ(1/4)Γ(3/4)c1/2c01+c0.
It follows that we must set the constant c0 equal to 0, which implies that
(2.23)G(x)=-2λq0b0xJ1/4(cx2)J-3/4(cx2).
We then deduce from (2.7) (with k=1) that the optimal control is given by
(2.24)u*(x)=--2λq0J1/4((-λ/2q0)b0x2)J-3/4((-λ/2q0)b0x2)for-d<x<d.
This formula for the optimal control is the same as the one obtained by Lefebvre [2].Now, in the general case, proceeding as previously we find that whenλ<0, the function G(x) defined in (2.14) is such that when x decreases to zero,
(2.25)G(x)~-2λq0b0xk×{(cxk+1/2)νΓ(ν+1)(1+c0cos(νπ)sin(νπ))-c0sin(νπ)(cxk+1/2)-νΓ(-ν+1)}{(cxk+1/2)ν-1Γ(ν)(1+c0cos[(ν-1)π]sin[(ν-1)π])-c0sin[(ν-1)π](cxk+1/2)-(ν-1)Γ(-ν+2)}.
This expression may be rewritten as follows:
(2.26)G(x)~-2λq0b0c1xθ/2+c2x-θ/2c3x(θ/2)-1+c4x-(θ/2)+2k+1,
where ci is a constant, for i=1,2,3,4. Multiplying the numerator and the denominator by xθ/2, we obtain that
(2.27)G(x)~-2λq0b0c1xθ+c2c3xθ-1+c4x2k+1.
Hence, we deduce that if θ∈(1,2), we must set the constant c2 equal to 0. This implies that c0=0, so that the constant c4 is equal to 0 as well. It follows that the function G(x) is given by
(2.28)G(x)=-2λq0b0xk{Jν(cxk+1)Jν-1(cxk+1)}.
This expression is valid as long as the denominator is positive. This is tantamount to saying that the parameter λ, which represents the instantaneous reward given for survival in the interval (-d,d), must not be too large.Finally, the optimal control is(2.29)u*(x)=--2λq0{Jν(cxk+1)Jν-1(cxk+1)}.Now, ifθ∈(0,1), it turns out that limx→0G(x)=0 for any constant c0, so that the solution is not unique. However, this does not entail that we can choose any c0. For instance, in the particular case when θ=1/2, k=1, λ=-1, b0=1 and q0=1/2, we find that
(2.30)G(x)=1x{J1/8(x2)+c0Y1/8(x2)J-7/8(x2)+c0Y-7/8(x2)}.
If we choose c0=0, as in the case when θ∈(1,2), then the expression for the optimal control is
(2.31)u*(x)=-2J1/8(x2)J-7/8(x2).
One can check that if d=1/2, then u*(x)<0 for 0<|x|<1/2, which is logical because the optimizer wants to maximize the survival time in (-1/2,1/2). But if we let c0 tend to infinity, the optimal control becomes
(2.32)u*(x)=-2Y1/8(x2)Y-7/8(x2),
which is strictly positive for 0<|x|<1/2. Thus, when the solution to (2.5), (2.8) is not unique, one must use other arguments to find the optimal control. One can obviously check whether the expression obtained for the optimal control does indeed correspond to a minimum (or a maximum in absolute value). In the particular case considered previously, if we let
(2.33)F(x)=∫-dx‍1yJ1/8(y2)J-7/8(y2)dy,
we find that this function satisfies all the conditions of the optimal control problem set up in Section 1 and leads to a valid expression for the optimal control.Next, if the parameterλ is positive, we deduce from the function G(x) in (2.16) that the optimal control is given by
(2.34)u*(x)=-2λq0{Iν(c*xk+1)Iν-1(c*xk+1)}
when θ∈(1,2). However, when θ∈(0,1), again we do not obtain a unique solution to (2.5) and (2.8).Moreover, contrary to the case whenλ is negative, there is no constraint on this parameter when it is positive. That is, we can give as large a penalty as we want for survival in the continuation region.


## 3. Optimal Control in the Symmetrical General Case

In this section, we assume thatd1 and d2 are not necessary such that d1=-d2. Moreover, we assume that there is a transformation Y(t)=g[X(t)] of the stochastic process {X(t),t≥0} such that the functions m[Y(t)] and v[Y(t)] are symmetrical with respect to zero. Then the optimal control problem is reduced to the one presented in the previous section.A simple example of such a situation is the case when{X(t),t≥0} is a one-dimensional controlled standard Brownian motion and d1≠-d2. Then one can simply define
(3.1)Y(t)=aX(t)+b,
with
(3.2)a=2dd2-d1,b=-d(d2+d1)d2-d1
to obtain a controlled Brownian motion with zero drift and variance parameter σ2=a2 in the interval (-d,d). We can then apply Proposition 2.1 to find the optimal control.A more interesting example is the following one: assume that the controlled process{X(t),t≥0} is defined by
(3.3)dX(t)=12X(t)dt+b0[X(t)]ku[X(t)]dt+X(t)dB(t).
That is, {X(t),t≥0} is a controlled geometric Brownian motion. Since this process is strictly positive, we cannot have d1=-d2. Let us define
(3.4)T(x)=inf{t>0:X(t)=1dord∣X(0)=x}forx∈(1d,d),
where d>1.Notice that the relation in (1.4) only holds in the case when k=1. To obtain the control that minimizes the expected value of the cost function defined in (1.3), we will transform the geometric Brownian motion process into a Wiener process by setting
(3.5)Y(t)=ln[X(t)].The infinitesimal parameters of the process{Y(t),t≥0} are given by (see, e.g., [6, p. 64])
(3.6)mY(y)=(12x+b0xku)1x+(12x2)(-1x2)=b0xk-1u=b0e(k-1)yu,vY(y)=x2(1x)2≡1.Hence, we can write thatY(t) satisfies the stochastic differential equation
(3.7)dY(t)=b0e(k-1)Y(t)u[Y(t)]dt+dB(t).
That is, {Y(t),t≥0} is a controlled standard Brownian motion. Moreover, the random variable T(x) becomes
(3.8)TY(y)=inf{t>0:|Y(t)|=ln(d)∣Y(0)=y},
where y∈(-ln(d),ln(d)).We can find the functionG(y), from which the optimal control u* is obtained at once, for any choice of k∈{0,1,…}. We will present the solution in the case when k=0. Furthermore, we let λ=-1, b0=1 and q0=1/2. We then must solve the nonlinear ordinary differential equation
(3.9)12G′(y)-e-2yG2(y)-1=0.
The solution that satisfies the condition G(0)=0 is
(3.10)G(y)=eyY0(2)J0(2e-y)-J0(2)Y0(2e-y)Y0(2)J1(2e-y)-J0(2)Y1(2e-y).
Hence, from Corollary 2.3, we can state that the optimal control is given by u*=-2e-yG(y). In terms of the original process, we have that
(3.11)u*(x)=-2Y0(2)J0(2/x)-J0(2)Y0(2/x)Y0(2)J1(2/x)-J0(2)Y1(2/x).
Since λ<0, this solution is only valid as long as it remains finite. That is, because we chose the value of λ, the constant d must not be too large. The optimal control is plotted in Figure 1 when d=3/2. Notice that u*(x) is positive when x<1 and negative when x>1, which is logical because the optimizer wants to maximize the survival time in the interval (2/3,3/2). However, the optimal control is not symmetrical with respect to 1.Figure 1
Optimal control whenk=0, λ=-1, b0=1, q0=1/2 and d=3/2.


## 4. Approximate Optimal Control in the Asymmetrical Case

We will now consider the case when the infinitesimal parameters of the controlled process{X(t),t≥0} do not satisfy the hypotheses in Proposition 2.1. In order to obtain the optimal control without having to find the function F(x) explicitly, we need a condition on G(x). If we could determine the value x0 of x in the interval (d1,d2) for which the function G(x) has a maximum or a minimum, then we would set G(x0)=0.An approximate solution can be obtained by finding the value ofx that maximizes the expected value of the time it takes the uncontrolled process that corresponds to {X(t),t≥0} to leave the interval (d1,d2). Let e(x) denote this expected value. This function satisfies the ordinary differential equation (see [6, p. 220])
(4.1)12v(x)e'’(x)+m(x)e'(x)=-1.
The boundary conditions are obviously
(4.2)e(d1)=e(d2)=0.We can state the following proposition.Proposition 4.1.
Letx0 be the value of x that maximizes the function e(x) defined previously. The optimal control u*(x) is approximately given by (2.7), where the function G(x) satisfies (2.5), subject to the condition G(x0)=0.To illustrate this result, we will present an example for which we can find the exact optimal control. We will then be able to assess the quality of the approximation proposed previously.Let{X(t),t≥0} be the controlled Wiener process with drift μ≠0 and variance parameter σ2 defined by
(4.3)dX(t)=μdt+b0u[X(t)]dt+σdB(t).
Because the relation in (1.4) holds with
(4.4)α=b02q0σ2>0,
and ultimate entry of the uncontrolled process into the set {d1,d2} is certain, we can indeed appeal to Whittle’s theorem to obtain the control that minimizes the expected value of the cost function J(x) defined in (1.3).Assume that the parameterλ is positive, so that the optimizer wants X(t) to leave the interval (d1,d2) as soon as possible. We deduce from Whittle’s theorem that the value function F(x) can be expressed as follows:
(4.5)F(x)=-1αln[M(x)],
where
(4.6)M(x):=E[e-αλτ(x)],
in which τ(x) is the same as the random variable T(x) in (1.2), but for the uncontrolled process {ξ(t),t≥0} defined by
(4.7)dξ(t)=μdt+σdB(t).It is a simple matter to find that(4.8)M(x)∝f-(d1)-f-(d2)+f+(d1)-f+(d2),
where
(4.9)f±(di):=exp{-1σ2[(x+di)μ±(x-di)Δ]}
for i=1,2, and
(4.10)Δ:=(μ2+2αλσ2)1/2.
We then obtain that the exact optimal control is given by
(4.11)u*(x)=-μb0-Δb0{f-(d1)-f-(d2)+f+(d1)-f+(d2)f-(d2)-f-(d1)+f+(d1)-f+(d2)}.Now, the functione(x):=E[τ(x)] satisfies the ordinary differential equation
(4.12)σ22e′′(x)+μe′(x)=-1.
The unique solution for which e(d1)=e(d2)=0 is
(4.13)e(x)∝(d2-d1)e-2μx/σ2-(x-d1)e-2μd2/σ2-(d2-x)e-2μd1/σ2.
The value of x that maximizes e(x) is obtained by differentiation:
(4.14)e′(x)=0⟺x=-σ22μln{σ22μe-2μd2/σ2-e-2μd1/σ2d2-d1}:=x0.Next, from (2.2), the optimal control is given by
(4.15)u*(x)=-b0q0F′(x),
where G(x):=F′(x) satisfies the nonlinear differential equation
(4.16)λ-12b02q0G2(x)+μG(x)+σ22G′(x)=0.
We find that the solution of this equation is the following:
(4.17)G(x)=μq0b02-2λq0b02+μ2q02b02tanh{2λq0b02+μ2q02σ2q0(x+c1)}.
The constant c1 is uniquely determined from the condition G(x0)=0. We have that
(4.18)G(x)≃μq0b02-2λq0b02+μ2q02b02×tanh{2λq0b02+μ2q02σ2q0(x-x0)+arctanh(μq02λq0b02+μ2q02)}.The expression that we obtain for the approximate optimal control by multiplying the functionG(x) by -b0/q0 is quite different from the exact optimal control. To compare the two solutions, we consider the special case when μ=σ=b0=q0=λ=1, d1=0 and d2=2. We then find that the constant Δ is equal to 3, and the value that maximizes G(x) is approximately x0≃ 0,7024. We plotted the two controls in Figure 2. Notice how close the two curves are.Figure 2
Exact (solid line) and approximate (dotted line) optimal controls whenμ=σ=b0=q0=λ=1, d1=0, and d2=2.


## 5. Extensions

To complete this work, we will consider two possible extensions of the results presented. First, suppose that the random variableT(x) defined in (1.2) is replaced by
(5.1)Td(x)=inf{t>0:X(t)=d∣X(0)=x}.
That is, we want to solve a one-barrier, rather than a two-barrier problem. To do so, we can introduce a second barrier, at x=d*. In general, it will be necessary to find a transformation Y(t)=g[X(t)] for which the infinitesimal parameters of the uncontrolled process {Y0(t),t≥0} that corresponds to {Y(t),t≥0} satisfy the hypotheses in Proposition 2.1. If we can find such a transformation, then we can try to obtain the optimal control u*(y) for the transformed process. Finally, we must express the optimal control in terms of the original variable x and take the limit as d* tends to ∞ (resp., -∞) if d*>d (resp., d*<d).Remark 5.1.
If there is a natural boundary at the origin, for example, and ifd*<d, then we would take the limit as d* decreases to zero.We will now present an example where the technique described previously is used. We consider the controlled geometric Brownian motion defined in Section3 by
(5.2)dX(t)=12X(t)dt+b0[X(t)]ku[X(t)]dt+X(t)dB(t),
and we assume that X(0)=x>d∈(0,1) and that b0>0. Let
(5.3)Xa(t)=aX(t),
where
(5.4)a:=1dd*.
Notice that the boundaries x=d and x=d*>d become, respectively:
(5.5)dd*:=1δ,d*d=δ.Next, we setY(t)=ln[Xa(t)]. We then find that {Y(t),t≥0} is a controlled standard Brownian motion, and the first-passage time
(5.6)Td,d*(x):=inf{t>0:X(t)=dord*∣X(0)=x}
becomes
(5.7)Tδ(y):=inf{t>0:|Y(t)|=ln(δ)∣Y(0)=y}.
Hence, we can appeal to Proposition 2.1 to determine the optimal value of the control u(y).Assume thatk=1 in (5.2). Then Whittle’s theorem applies with
(5.8)α=b02q0>0.
The optimal control is given by
(5.9)u*(x)=-b0q0xF′(x),
and, as in Section 4, the value function F(x) can be expressed as
(5.10)F(x)=-1αln[M(x)],
where
(5.11)M(x):=E[e-αλτd(x)],
in which τd(x) is the same as Td(x) for the uncontrolled process that corresponds to {X(t),t≥0}.The functionM(x) satisfies the second-order ordinary differential equation
(5.12)12x2M′′(x)+12xM′(x)=αλM(x).
The general solution of this equation can be written as
(5.13)M(x)=c1x2αλ+c2x-2αλ.
We assume that the parameter λ is positive. Then, we can write that
(5.14)limx→∞M(x)=0.
It follows that we must choose the constant c1=0 in the general solution. Finally, making use of the boundary condition M(d)=1, we obtain that
(5.15)M(x)=(dx)2αλ,
from which we deduce that the optimal control is constant:
(5.16)u*(x)≡-2λq0.If we do not appeal to Whittle’s theorem, we must solve the nonlinear first-order differential equation (see Section3)
(5.17)λ-b022q0G2(y)+12G′(y)=0,
subject to the boundary condition G(0)=0. We find that
(5.18)G(y)=-2λq0b0tanh(2λb0yq0).
It follows that
(5.19)u*(y)=-b0q0G(y)=2λq0tanh(2λb0yq0).In terms of the original variablex=ey/a=dd*ey, we can write that
(5.20)u*(x)=2λq0tanh(2λb0ln(x/dd*)q0).
Since
(5.21)limc→∞tanh[c0ln(xc)]=-1
for any positive constant c0, we obtain that
(5.22)limd*→∞u*(x)=-2λq0.
Thus, we retrieve the formula for the optimal control.Next, we will treat the case whenk=0 in (5.2), so that Whittle’s theorem does not apply. The optimal control becomes
(5.23)u*(x)=-b0q0G(x).In terms of the transformed variabley, we have that
(5.24)u*(y)=-b0q0e-yG(y),
where G(y) is a solution of (see Section 3)
(5.25)12G′(y)-b022q0e-2yG2(y)+λ=0.
When λ is positive, the solution that satisfies the condition G(0)=0 is
(5.26)G(y)=-q0b02κeyI0(κ)K0(κe-y)-K0(κ)I0(κe-y)I0(κ)K1(κe-y)+K0(κ)I1(κe-y),
where
(5.27)κ:=2λb0q0,
from which we deduce that
(5.28)u*(y)=κb0I0(κ)K0(κe-y)-K0(κ)I0(κe-y)I0(κ)K1(κe-y)+K0(κ)I1(κe-y).
It follows that
(5.29)u*(x)=κb0I0(κ)K0(κdd*/x)-K0(κ)I0(κdd*/x)I0(κ)K1(κdd*/x)+K0(κ)I1(κdd*/x).
Finally, using the asymptotic expansions for large arguments of the functions Iν(z) and Kν(z) (see [5, p. 377]), we find that
(5.30)limd*→∞u*(x)≡-κb0=-2λq0,
which is the same optimal control as in the case when k=1.Remarks 5.2.
(i) If we take the limit asd* decreases to zero in u*(x) instead, then making use of the formulas (see [5, p. 375])
(5.31)I0(z)~1,I1(z)~z2,K0(z)~-ln(z),K1(z)~1zasz↓0,
we obtain that
(5.32)limd*↓0u*(x)≡0.
(ii) We can also try to solve the differential equation
(5.33)x22G′(x)+x2G(x)-b022q0G2(x)+λ=0
satisfied by the function G(x) directly. However, the solution that we are looking for must be such that G(dd*)=0, because x=dd* is the value of the original variable x that corresponds to y=0.Now, in Corollary2.3 we mentioned that Proposition 2.1 could be generalized by replacing Xk(t) by h[X(t)] in (1.1). Another extension of Proposition 2.1 is to generalize the cost function J(x) defined in (1.3) to
(5.34)J(x)=∫0T(x)‍{12q[X(t)]u2[X(t)]+λ}dt,
where the function q(·)≥0 is even.To illustrate this result, we consider a particular controlled Ornstein-Uhlenbeck process defined by(5.35)dX(t)=-X(t)dt+u[X(t)]dt+dB(t),
and we take
(5.36)J(x)=∫0T(x)‍{u2[X(t)]|X(t)|+1+1}dt,
in which
(5.37)T(x)=inf{t>0:|X(t)|=2∣X(0)=x∈(-2,2)}.
Then, we find that the optimal control is given by
(5.38)u*(x)=-12(|x|+1)G(x),
and that the function G(x) satisfies the nonlinear differential equation
(5.39)1-xG(x)-(|x|+1)G2(x)+12G′(x)=0,
subject to the condition G(0)=0.Next, by symmetry, we can write thatG(-x)=-G(x) (and that u*(-x)=-u*(x)). Hence, we can restrict ourselves to the interval [0,d]. The solution of the differential equation that is such that G(0)=0 is
(5.40)G(x)=π[erf(x+2)-erf(2)]π[erf(2)-erf(x+2)]-e-(x+2)2,
where “erf” is the error function. It follows that the optimal control is given by
(5.41)u*(x)=12(x+1)π[erf(2)-erf(x+2)]π[erf(2)-erf(x+2)]-e-(x+2)2
for 0≤x≤2. This function is plotted in Figure 3.Figure 3
Optimal control in the interval[0,2].


## 6. Conclusion

We have shown that when the LQG homing problem that we want to solve possesses a certain symmetry, then it is not necessary to obtain the value functionF(x) explicitly; only the derivative of F(x) is needed to determine the optimal control. Using this result, we were able to solve various problems for which Whittle’s theorem does not apply. In Section 4, we proposed an approximate solution in the case when the infinitesimal parameters of the controlled processes are not symmetrical with respect to the origin.Many papers have been written on LQG homing problems, in particular by the first author (see, e.g., [7]) and recently by Makasu [8]. In most cases, the problems considered were only for one-dimensional processes, because to apply Whittle’s theorem a certain relation must hold between the noise and control terms. This relation is generally not verified in two or more dimensions. Furthermore, even if the relation in question holds, we still must solve a nontrivial probability problem. More precisely, we need to evaluate the moment-generating function of a first-passage time. To do so, we must find the solution of a Kolmogorov backward equation that satisfies the appropriate boundary conditions.Proceeding as we did in this paper, we could simplify, at least in the symmetrical case, the differential equation problem, even in more than one dimension. Therefore, we should be able to solve more realistic problems. Such problems will also have interesting applications.Finally, in order to be able to treat real-life applications, we should try to find a way to solve problems that are not symmetrical and for which Whittle’s theorem does not apply. This could be achieved by finding a transformation that linearizes the differential equations that we need to solve.



---

*Source: 803724-2012-10-02.xml*