# Stochastic Linear Quadratic Control Problem on Time Scales

Authors: Yingjun Zhu; Guangyan Jia
Journal: Discrete Dynamics in Nature and Society
 (2021)

---

## Abstract

This paper addresses a version of the stochastic linear quadratic control problem on time scalesSΔLQ, which includes the discrete time and continuous time as special cases. Riccati equations on time scales are given, and the optimal control can be expressed as a linear state feedback. Furthermore, we present the uniqueness and existence of the solution to the Riccati equation on time scales. Furthermore, we give an example to illustrate the theoretical results.


---

## Body

## 1. Introduction

The linear quadratic control problem is one of the most important issues for the optimal control problem. It has received much attention [1–3] and has a wide range of applications in engineering and finance. Until now, the linear quadratic control problem is well understood from both continuous and discrete points of view. In this paper, the stochastic linear quadratic control problem is studied in the version of time scales.Time scales were first introduced by Hilger [4] in 1988 in order to unite and extend the continuous and discrete analysis into a general framework. Time scales theory has been extensively studied in many works [5–10]. It is well known that the optimal control problem on time scales is an important field for both theory and applications. Since the calculus of variations on time scales was studied by Bohner [11], results on related topics and their applications have become more and more [12, 13]. The existence of optimal control for the dynamic systems on time scales was considered in [14, 15]. Subsequently, the maximum principle and dynamic programming in the time scale setting were studied [16, 17] for a deterministic dynamic system. In [18, 19], some results were obtained for the deterministic linear quadratic control problems on time scales. Recently, in Poulsen’s Ph.D. thesis [20, 21], the authors developed an interesting theory of dynamic equations on time scales, in which the time scale is a discrete and stochastic one depending on a sequence of i.i.d. positive random variables, but the associated dynamic equation does not include stochastic terms; they also studied the associated control and stability problems.In this paper, we are interested in the stochastic linear quadratic control problem on time scales (SΔLQ for short). Very similar to continuous and discrete cases, we can also obtain the associated Riccati equations (see [22, 23] for continuous and discrete cases) on time scales. Meanwhile, we discuss the existence and uniqueness of the solution to Riccati equations on time scales (RΔE for short). The difference from [24] is that the control system does not contain the expectation term, but includes the inhomogeneous term.The organization of this paper is as follows. In Section2, we introduce some preliminaries about the time scale theory and SΔLQ problem. We will study the stochastic linear quadratic control problem on time scales and the associated Riccati equations on time scales in Section 3. Finally, an example is given.


## 2. Preliminaries

A time scaleT is a nonempty closed subset of the real numbers ℝ, and we denote 0,TT=0,T∩T. In this paper, we fix T∈T. The forward jump operator σ is defined by(1)σt=infs∈T:s>t,supplemented by inf∅≔supT, where ∅ denotes the empty set. If σt=t and t<supT, the point t is called right-dense, while if σt>t, the point t is called right-scattered. For a function f, we denote fσt=fσt to represent the compositions of the functions f and σ. The backward jump operator ρ is defined by(2)ρt=sups∈T:s<t,supplemented by sup∅≔infT. If ρt=t and t>infT, the point t is called left-dense, while if ρt<t, the point t is called left-scattered. Moreover, a point is called isolated if it is both left-scattered and right-scattered. The definition of the graininess function μ is as follows:(3)μt=σt−t.We now present some basic concepts and properties about time scales (see [6, 7]).Definition 1.
Letf be a function on T; f is called a right-dense continuous function if f is continuous at every right-dense point and has finite left-sided limits at every left-dense point. For an n×n matrix-valued function A, we say that A is right-dense continuous if each entry of A is right-dense continuous.Definition 2.
A functionf:T⟶ℝ is called continuous at the point t0∈T if ftn⟶ft0 for any sequence tn∈T such that tn⟶t0 as n⟶∞.
Define the setTκ as(4)Tκ=TρsupT,supT,if supT<∞,T,if supT=∞.Definition 3.
Letf: T⟶ℝ be a function and t∈Tκ; if for all ε>0, there exists a neighborhood U of t such that(5)fσt−fs−fΔtσt−s≤εσt−s,for alls∈U.
We callfΔt the Δ-derivative of f at t.
If the functionsf and g are differentiable at t, then the product fg is also differentiable at t, and the product rule is given by(6)fgΔt=fΔtgt+fσtgΔt.Proposition 1.
If a functionf is right-dense continuous, then f has antiderivative F.Definition 4.
A functionp is said to be regressive if 1+μtpt≠0 for all t∈T. The set of all regressive and right-dense continuous functions p:T⟶ℝ is denoted by ℛ. We define ℛ+=p∈ℛ:1+μtpt>0for allt∈T. For an n×n matrix-valued function A, if I+μtAt is invertible for all t∈T, we say that A is regressive. Similar to the scalar case, the regressive and right-dense continuous matrix-valued function A is denoted by A∈ℛ.
Notations: the following notations will be used:M′: the transpose of any vector or matrix MM:  = ∑i,jmij2 for any matrix or vector M=mijX,Yt: the quadratic covariation process of X and YSn: the space of all n×n symmetric matricesS+n: the space of all n×n positive semidefinite symmetric matricesL∞0,TT;ℝ: the space of bounded, Δ-Lebesgue-integrable, and ℝ-valued functions on 0,TTCrd0,TT;ℝ: the space of ℝ-valued right-dense continuous functions on 0,TTC0,TT;ℝ: the space of ℝ-valued continuous functions on 0,TTC10,TT;ℝ: the family of all ℝ-valued continuous functions ft defined on 0,TT such that they are Δ-differentiable in t
We now introduce two known results, which will be used in the following.Lemma 1 (see [6]).
Supposey∈Crd0,TT;ℝ and α∈ℛ+. Then,(7)yt≤∫tTαyσsΔs+yTimplies(8)yt≤yTeαT,t.Lemma 2 (Schur’s lemma; see [25]).
LetU and V be symmetric matrices and Y be given with appropriate dimensions. The following conditions are equivalent:(i)
U−YV−1Y′≥0,V>0,(ii)
UYY′V≥0,V>0.In this paper, we adopt the stochastic integral defined by Bohner et al. [10]. Let Ω,ℱ,ℱtt∈0,TT,ℙ be a complete probability space with an increasing and continuous filtration ℱtt∈0,TT. We define that Lℱ20,TT;ℝ is the set of all ℱt-adapted, ℝ-valued measurable processes Xt such that E∫0TXt2Δt<∞.A Brownian motion indexed by time scaleT was defined by Grow and Sanyal [9]. Although the Brownian motion on time scales is very similar to that on continuous time, there are also some differences between them. For example, the quadratic variation of a Brownian motion on time scales (see [26]) is an increasing process yet, but it is not deterministic. In fact, Wt=λ0,tT+∑s≤tWsσ−Ws2, where λ is the Lebesgue measure.Similar to Definition4, we give the definition of stochastic regressive.Definition 5 (see [27]).
A functionq is said to be stochastic regressive if 1+qtWσt−Wt≠0 almost everywhere for all t∈T, denoted by q∈ℛW. For an n×n matrix-valued function B, if I+BtWσt−Wt is invertible almost everywhere for all t∈T, we say that B is stochastic regressive, denoted by B∈ℛW.
Now, we give the definition of the stochasticΔ-integral and its properties.Definition 6 (see [10]).
The random processXt is stochastic Δ-integrable on 0,TT if the corresponding X˜t is integrable. Furthermore, we define the integral value of Xt as(9)∫0TXtΔWt≔∫0TX˜tdWt,where(10)X˜t=Xsup0,tTfor allt∈0,T,and the Brownian motion on the right side of (9) is indexed by continuous time.
We also have the following properties.
LetXt,Yt∈Lℱ20,TT;ℝ and α,β∈ℝ. Then,(i)
∫0TαXt+βYtΔWt=α∫0TXtΔWt+β∫0TYtΔWt,(ii)
E∫0TXtΔWt=0,(iii)
E∫0TXtΔWt2=E∫0TXt2ΔWt=E∫0TXt2Δt,where the integral X with respect to the quadratic variation of Brownian motion Wt is defined by the Stieltjes integral as ∫0TXtωΔWtω.
In addition, we have the following result about the stochasticΔ-differential equation which is easy to be proved.Lemma 3.
LetΦ⋅ be the solution of the following stochastic Δ-differential equation on Ω,ℱ,ℱtt∈0,TT,ℙ,(11)ΔΦt=AtΦtΔt+CtΦtΔWt,t∈0,TT,Φ0=I,A∈ℛ, and I+μA−1C∈ℛW; then, Φ−1⋅ exists.Proof.
For the continuous closed intervalt1,t2, we introduce the following equation:(12)ΔΨ=Ψ−A+C2Δt−ΨCΔW,Ψt1=I,and then ΔΨΦ=0 and ΨtΦt=Φt1 for all t∈t1,t2. If Φ−1t1 exists, we can obtain that Φ−1t exists for all t∈t1,t2. For a right-scattered point t, we have(13)Φσt=I+μtAtI+I+μtAt−1CtWσt−WtΦt.
IfΦ−1t exists, then Φ−1σt exists. So, Φ−1t exists due to Φ0=I.
Finally, we introduce ourSΔLQ problem. Consider the following stochastic linear quadratic control problem:(14)minimizeJu⋅=E∫0Tx1X′tQtXt+2u′tLtxt+u′tRtutΔt+X′THXT,where the control u∈Lℱ20,TT;Rm and X satisfies the linear Δ-differential equation:(15)ΔXt=AtXt+Btut+ftΔt+∑j=1dDjtutΔWjt,t∈0,TT,X0=x0∈Rn.
Here,W=W1,…,Wd is a d-dimensional standard Brownian motion on time scales. And the associated coefficients satisfy A∈L∞0,TT;ℝn×n,B,Dj,L′∈L∞0,TT;ℝn×m,f∈L∞0,TT;ℝ,Q∈L∞0,TT;Sn,R∈L∞0,TT;Sm,H∈Sn. Obviously, equation (15) admits a unique solution under this condition (see Bohner et al. [10]).


## 3. Main Results and the Riccati Equations

By some simple calculations, it is not hard for us to obtain the following product rule for stochastic processes on time scales, which is very similar to that of Du and Dieu [8].Lemma 4.
For any twon-dimensional stochastic processes X1 and X2 with(16)ΔXit=bit,XitΔt+σit,XitΔWt,t∈T,Xi0=ξi,where bi,σi:T×ℝn⟶ℝn, we have(17)ΔX1′tX2t=X1′tΔX2t+ΔX1′tX2t+ΔX1,X2t,where(18)ΔX1,X2t=μtb1′b2Δt+σ1′σ2ΔWt+μtb1′σ2+b2′σ1ΔWt.Remark 1.
We can also obtain another form of the above product rule as follows:(19)ΔX1′tX2t=X1′tΔX2t+ΔX1′tX2t+ΔX1′tΔX2t,where ΔtΔt=μtΔt,ΔtΔW=ΔWΔt=μtΔW, and ΔWΔW=ΔWt. When T=ℝ, it is consistent with It o^’s formulas.Remark 2.
As mentioned before, since the quadratic variation of a process depends on not only the process itself but also the structure of the time scale, the quadratic variation of a process becomes a little more complicated than the classical one. For example, the quadratic variation of a deterministic continuous process is no longer zero. Therefore, we can have different forms of the product rule on time scales. For example, product rule (6) is equivalent to(20)fgΔt=fΔtgt+ftgΔt+μtfΔtgΔt.
Now, we introduce the followingRΔE for our problem:(21)−PΔt=Qt+A′tPσt+PσtAt+μtA′tPσtAt−I+μtA′tPσtBt+L′tK−1t×B′tPσtI+μtAt+Lt,Kt=Rt+μtB′tPσtBt+∑j=1dDj′tPσtDjt>0,t∈0,TT,Pt=H,along with an equation on time scales:(22)Δgt=ΦtPσtft+ΨtgσtΔt,t∈0,TT,gT=0,where 0 is the zero vector and(23)Φt=μtI+μtA′tPσtBt+L′tK−1tB′t−I+μtA′t,(24)Ψt=I+μtA′tPσtBt+L′tK−1tB′t−A′t.
Clearly, equation (22) has a unique solution since it is a linear equation (see Bohner and Peterson [7]). RΔE (21) is very similar to the classical Riccati equation in continuous and discrete time (see [22, 28]). The only difference is that RΔE includes graininess function μ. When T=ℝ+ or T=ℤ+, (21) can degenerate into the classical ones.
A matrix-valued functionP∈C10,TT;Sn is called a solution to RΔE if it satisfies (21). A function g∈C10,TT;ℝn is a solution to (22) if it satisfies the Δ-differential equation. Here, we use the square completion technique to show that the solutions of equations (21) and (22) can give a state-feedback optimal control.Theorem 1.
Suppose thatP∈C10,TT;Sn and g∈C10,TT;ℝn are solutions of (21) and (22), respectively; then, SΔLQ problem (14) and (15) have an optimal control u¯:(25)u¯t=−K−1tB′tPσtI+μtAt+LtXt+μtB′tPσtft+B′tgσt,and the optimal cost functional J¯ is(26)J¯=x0′P0x0+2x0′g0−∫0Tgσt′BtK−1tB′tgσtΔt+∫0Tμtf′tI−μtPσtBtK−1tB′tPσtftΔt+2∫0Tf′tI−μtPσtBtK−1tB′tgσtΔt.Proof.
Applying the product rule toXt′PtXt and X′tgt, we can obtain that(27)ΔX′tPtXt=X′tI+μtA′tPσtBt+L′tK−1t×B′tPσtI+μtAt+Lt−QtXt+μtu′tB′tPσtBtut+2u′tB′tPσt×I+μtAtXt+μtft+2X′tI+μtA′tPσtft+μtf′tPσtftΔt+∑i=1d∑j=1du′tDi′tPσtDjtutΔWi,Wjt+2Xt+μtAtXt+Btut+ft′×Pσt∑j=1dDjtutΔWjt,t∈0,TT,ΔX′tgt=X′tΦtPσtft+Ψtgσt+AtXt+Btut+ft′gσtΔt+∑j=1du′tDj′tgσtΔWjt,t∈0,TT.
Integrating from 0 toT, taking expectation on both sides of the above two equations, and then adding them into the cost functional (14) yield(28)Ju⋅=x0′P0x0+2x0′g0−∫0TgσtBtK−1tB′tgσtΔt+∫0Tμtf′tI−μtPσtBtK−1tB′tPσtftΔt+2∫0Tf′tI−μtPσtBtK−1tB′tgσtΔt+E∫0Tut+K−1tB′tPσtI+μtAt+LtXt+μtB′tPσtft+B′tgσt′Kt×ut+K−1tB′tPσtI+μtAt+LtXt+μtB′tPσtft+B′tgσtΔt.
SinceK>0, it follows that the optimal control is the form of (25), and the minimum value of the cost functional is (26).Remark 3.
Obviously, iff≡0, we just need to think about RΔE (21). And we can see from the above proof that the uniqueness of the solutions to (21) and (22) is not necessary for the establishment of (25) and (26).
It is worth pointing out that the state feedback matrix relies on not only the solution of RΔE but also the graininess function μ. In addition to this, (25) and (26) are very similar to the classical cases.
Now, we discuss the existence and uniqueness of the solution toRΔE. We will apply a technique given in [29].Theorem 2.
Assume thatP∈C10,TT;Sn is a solution to RΔE (21); then, P is unique.Proof.
Suppose thatP˜∈C10,TT;Sn is another solution of RΔE (21). We consider a new SΔLQ problem as follows:(29)minimizeJs,y;u⋅=E∫sTX′tQtXt+2u′tLtXt+u′tRtutΔt+X′THXT,(30)s.t.ΔXt=AtXt+BtutΔt+∑j=1dDjtutΔWjt,t∈s,TT,Xs=y.
According to the proof of Theorem1, y′Psy and y′P˜sy are both minimum values of the cost functional Js,y;u⋅. Therefore, we obtain that Ps=P˜s. Due to the arbitrariness of s,y, it follows that P=P˜.
For the existence of the solution toRΔE (21), we first consider a special case:(31)H1:R>0,H≥0,andQ−L′R−1L≥0.
In this case, we study a linear equation required in the following.Proposition 2.
The linear matrix-valuedΔ-differential equation(32)−PΔt=A′tPσt+PσtAt+μtA′tPσtAt+∑j=1dCj′tPσtCjt+Qt,t∈0,TT,PT=H,where A,Cj∈L∞0,TT;ℝn×n, Q∈L∞0,TT;Sn, and H∈Sn, has a unique solution P∈C10,TT;Sn.Proof.
LetP0t≡H and(33)Pk+1t=H+∫tTA′sPkσs+PkσsAs+μsA′sPkσsAs+∑j=1dCj′sPkσsCjs+QsΔs,t∈0,TT.
Then,(34)Pk+1t−Pkt=∫tTA′sPkσs−Pk−1σs+Pkσs−Pk−1σsAs+μsA′sPkσs−Pk−1σsAs+∑j=1dCj′sPkσs−Pk−1σsCjsΔs,t∈0,TT,(35)P1t−P0t=∫tTA′sHs+HsAs+μsA′sHsAs+∑j=1dCj′sHsCjs+QsΔs,t∈0,TT.
Because of the boundedness ofA,Cj,H, and Q, there exists a constant M>0 such that(36)Pk+1t−Pkt≤M∫tTPkσs−Pk−1σsΔs,t∈0,TT,(37)P1t−P0t≤MT−t.
Consequently, we have that(38)Pk+1t−Pkt≤Mk+1hk+1T,t,t∈0,TT,where hk is the generalized monomials defined in [30]. Thus, ∑kPk+1t−Pkt is uniformly convergent. Moreover, Pk uniformly converges to P, P is a solution of (32), and we get the existence of a solution to (32). If P˜ is another solution of (32), then(39)−P^Δt=A′tP^σt+P^σtAt+μtA′tP^σtAt+∑j=1dCj′tP^σtCjt,t∈0,TT,P^T=0,where P^=P−P˜. Based on some properties of matrix norms, there is a constant α such that(40)P^t≤∫tTαP^σsΔs+P^T.
By Lemma1, it is easy to obtain P^=0. This implies the uniqueness of a solution to (32).
For linear equation (32), we also have the following property.Proposition 3.
LetQ and H be positive semidefinite symmetric matrices; then, the solution P of (32) is also a positive semidefinite symmetric matrix.Proof.
For the continuous closed intervalt1,t2∈0,TT and t1,t2∈0,TT, by (11) and (32) and applying product rules to Φ′tPtΦt, we get(41)ΔΦ′tPtΦt=−Φ′t∑j=1dCj′tPσtCjt+QtΦtΔ+∑j=1dCj′tPσtCjtΦtΔWtt+∑j=1dΦ′tCj′tPσt+PσtCjt+μtA′tPσtCjt+μtCj′tPσtAtΦtΔWt,t∈t1,t2T.
Therefore,(42)Pt=EΦ′t−1Φ′t2Pt2Φt2Φ′t−1+∫tTΦ′t−1Φ′sQsΦsΦ′t−1Δs,t∈t1,t2T.
IfPt2≥0, we have Pt≥0 for all t∈t1,t2T. For right-scattered point t∈0,TT, we have(43)Pt=I+μtA′tPσtI+μtAt+μt∑j=1dCj′tPσtCjt+μtQt.
IfPσt≥0, then Pt≥0. From the above, since PT=H≥0, we have Pt≥0 for all t∈0,TT.
Furthermore, we have an equivalent form of RΔE (21) under assumption (H1).Lemma 5.
Assume (H1) holds;RΔE (21) is equivalent to the following equation:(44)−PΔt=A^′tPσt+PσtA^t+μtA^′tPσtA^t+∑j=1dC^j′tPσtC^jt+Q^t,t∈0,TT,PT=H,where(45)A^t=At−BtΨt,C^jt=−DjtΨt,Q^t=Ψt−R−1tLt′RtΨt−R−1tLt+Qt−L′tR−1tLt,Ψt=Rt+μtB′tPσtBt+∑j=1dDjtPσtDjt−1×B′tPσtI+μtAt+Ltt∈0,TT.Proof.
By (45), we have(46)Ψ′tRt+μtB′tPσtBt+∑j=1dDjtPσtDjtΨt=Ψ′tB′tPσtI+μtAt+Lt=I+μtA′tPσtBt+L′tΨt,(47)−PΔt=A′tPσt+PσtAt+μtA′tPσtAt+Qt−I+μtA′tPσtBt+LtK−1tB′tPσtI+μtAt+Lt=A^′tPσt+PσtA^t+μtA^tPσtA^t+Qt+I+μtA′tPσtBt+L′tΨt−L′tΨt+Ψ′tB′tPσtI+μtAt+Lt−Ψ′tLt−Ψ′tRt+μtB′tPσtBt+∑j=1dDjtPσtDjtΨt−μtΨ′tB′tPσtBtΨt=A^′tPσt+PσtA^t+μtA^tPσtA^t+Ψ′tRtΨt+Ψ′t∑j=1dDj′tPσtDjtΨt−L′tΨt−Ψ′tLt+Qt=A^′tPσt+PσtA^t+μtA^tPσtA^t+∑j=1dC^j′tPσtC^jt+Q^t.
The proof is completed.
Consequently, the existence of the solution to (44) is equivalent to the existence of the solution to (21) under assumption (H1).Theorem 3.
Let (H1) hold; then RΔE (21) admits a positive semidefinite symmetric solution.Proof.
LetPi+1 be the solution of(48)−Pi+1Δt=A^i′tPi+1σt+Pi+1σtA^it+μtA^i′tPi+1σtA^it+∑j=1dC^ij′tPi+1σtC^ijt+Q^it,t∈0,TT,Pi+1T=H,where(49)P0t=H,A^it=At−BtΨit,C^ijt=−DjtΨit,Q^it=Ψit−R−1tLt′RtΨit−R−1tLt+Qt−L′tR−1tLt,Ψit=Rt+μtB′tPiσtBt+∑j=1dDj′tPiσtDjt−1×B′tPiσtI+μtAt+Lt,t∈0,TT.
By Proposition3, we can see that Pi≥0.
We will claim thatPi≥Pi+1. Set Mi=Pi−Pi+1; then, Mi satisfies(50)−MiΔt=A^i′tMiσt+MiσtA^it+μtA^i′tMiσtA^it+∑j=1dC^ij′tMiσtC^ijt+Nt,t∈0,TT,MiT=0,where(51)Nt=∑j=1dC^i−1,j′tPiσtC^i−1,jt−∑j=1dC^ij′tPiσtC^ijt+Q^i−1t−Q^it+A^i−1′t−A^i′tPiσt+PiσtA^i−1t−A^it+μtA^i−1′tPiσtA^i−1t−μtA^i′tPiσtA^it.
An equivalent form ofNt can also be written as follows:(52)Nt=Ψit−Ψi−1t′Rt+μtB′tPiσtBt+∑j=1dDj′tPiσtDjt×Ψit−Ψi−1t≥0,t∈0,TT.
Noting thatMiT=0, by Proposition 3 once again, we get Mi≥0. Thus, Pii≥0 is a decreasing sequence and has a limit P. Clearly, P is the solution to (44). Hence, P is also the solution to RΔE (21) by Lemma 5.
Now, we prove the following sufficient and necessary conditions of the existence of the solution toRΔE (21).Theorem 4.
RΔE (21) admits a solution if and only if there exists P˜ which satisfies the following conditions:(i)
P˜T≤H(ii)
Rt+μtB′tP˜σtBt+∑j=1dDj′tP˜σtDjt>0,t∈0,TT(iii)
P˜satisfies the linear matrix inequalityLMIcondition:P˜Δ+A′P˜σ+P˜σA+μA′P˜σA+QI+μA′P˜σB+LB′P˜σI+μA+L′R+μB′P˜σB+∑j=1dDj′P˜σDj≥0Proof.
If RΔE (21) has a solution P, then PT=H and R+μB′PσB+∑j=1dDj′PσDj>0. By Lemma 2, where U=PΔ+A′Pσ+PσA+μA′PσA+Q, Y=I+μA′PσB+L, and V=R+μB′PσB+∑j=1dDj′PσDj, we have(53)PΔ+A′Pσ+PσA+μA′PσA+QI+μA′PσB+LB′PσI+μA+L′R+μB′PσB+∑j=1dDj′PσDj≥0,namely, P satisfies conditions (i)–(iii).
Conversely, suppose there existsP˜ which satisfies (i)–(iii), and define(54)Q^t=P˜Δt+A′tP˜σt+P˜σtAt+μtA′tP˜σtAt+Qt,R^t=Rt+μtB′tP˜σtBt+∑j=1dDj′tP˜σtDjt,L^t=B′tP˜σtI+μtAt+Lt,t∈0,TT.
Thus,R^>0. Moreover, Lemma 2 yields Q^−L^′R^L^≥0. Now, consider a new Riccati equation:(55)−ZΔt=Q^t+A′tZσt+ZσtAt+μtA′tZσtAt−I+μtA′tZσtBt+L^′t×R^t+μB′tZσtBt+∑j=1dDj′tZσtDjt−1×B′tZσtI+μtAt+L^t,R^t+μtB′tZσtBt+∑j=1dDj′tZσtDjt>0,t∈0,TT,ZT=H−P˜T.
According to Theorem3, the above equation has a solution Z≥0. Let P=Z+P˜; it is obvious that R+μB′PσB+∑j=1dDj′PσDj>0 and PT=H. By some manipulations, we can easily verify that P is a solution to RΔE (21). This completes the proof.Remark 4.
Note that ifμ≡0, conditions (i)–(iii) are consistent with the result of Theorem 3.3 in [29].


## 4. Example

In this section, in order to illustrate our result, we give an example. Consider a simple Mayer problem on time scaleT=0,1/4∪1/2∪3/4,1 as follows:(56)minimizeJ=Ex21,subject toΔxt=utΔt+utΔWt,x0=1.The corresponding dynamic Riccati equation is(57)ΔPt=11+μtPσtΔt,P1=1,1+μtPσt>0.We havePt=e1/1+μt1,t. The optimal control u¯t=−1/1+μtxt (see Figure 1).Figure 1
Optimal control of the dynamic system on time scaleT=0,1/4∪1/2∪3/4,1.We can see that the optimal strategy depends on the structure of time scaleT=0,1/4∪1/2∪3/4,1. Furthermore, the example implies that we should take an impulsive control when t=1/4 in the time scale setting. This interesting result is hidden in the classical continuous and discrete formulation. It reveals that we should use an impulsive control method at the right-scattered points. In addition, the impulsive control relies on the time gap.


## 5. Conclusion

The linear quadratic optimal control problems for stochastic differential equations on time scales are studied. They unify and extend the optimal control problems in continuous- and discrete-time formulations. Through the Riccati equation on time scales, we get the corresponding optimal control with the state feedback representation. The optimal control problems established in this paper offer a more practical scheme in directly tackling the issue on the mixture of continuous time and discrete time.



---

*Source: 5743014-2021-03-30.xml*