# One Form of Lyapunov Operator for Stochastic Dynamic System with Markov Parameters

Authors: Taras Lukashiv
Journal: Journal of Mathematics
 (2016)

---

## Abstract

The form of weak infinitesimal operator of Lyapunov type on solutions of stochastic dynamic systems of random structure with constant delay which exist under the action of Markov perturbations is obtained.


---

## Body

## 1. Introduction

The explicit view of the Lyapunov operator is necessary for solving the problem of stabilization for stochastic dynamic systems. The view of the Lyapunov operator for different stabilization problems can be found in [1–7]. In particular, in [6], the view of the Lyapunov operator for stochastic diffusion dynamical system of random structure with Markov switching is obtained, in [7], for controlled stochastic dynamical system with impulse Markov switching and parameters.The explicit view of infinitesimal operator (of Lyapunov type) for the solution of dynamic systems of random structure with Markov switching and constant delay is obtained in this article.


## 2. Formulation of the Problem

On the probabilistic basis(Ω,F,F≡{Ft⊂F,t≥0},P) [8] consider a random process x(t), t≥0, given by stochastic differential-difference equation (SDDE)(1)dxt=at,ξt,xt,xt-rdt+bt,ξt,xt,xt-rdwt,t∉S,with Markov switching(2)Δxtk=xtk-xtk-=gtk-,ξtk-,ηk,xtk-,tk∈S≔tk↑,k∈N,limk→∞tk=+∞and initial conditions(3)xt=φt,-r≤t≤0,r>0,φ∈D,ξ0=y∈Y,η0=h∈H.Hereξ(t), t≥0, is Markov process with values in dimensional space Y; ηk, k≥0, is Markov chain with values in dimensional space H; w is one-dimensional standard Wiener process [8–10]; w(t), ξ(t), t≥0, and   ηk, k≥0 are Ft-mesurable and independent of each other; D≡D[-r,0],Rm is Skorokhod’s space of continuous functions affair with left sided border [11] with the norm(4)φ=sup-r≤θ≤0φθ.Suppose that measurable mapsa:R+×Y×Rm×Rm→Rm; b:R+×Y×Rm×Rm→Rm; g:R+×Y×H×Rm→Rm satisfy Lipschitz condition (5)at,y,φ1,φ2-at,y,ψ1,ψ22+bt,y,φ1,φ2-bt,y,ψ1,ψ22+gt,y,h,φ3-gt,y,h,ψ32≤Lφ1-ψ12+φ2-ψ22+φ3-ψ32for ∀t≥0, y∈Y, h∈H, φi,ψi∈Rm, i=1,2,3; and next condition hold(6)at,y,0,0+bt,y,0,0+gt,y,h,0=c<∞,∀t≥0, y∈Y, h∈H, moreover for ∀T<+∞(7)supy∈Y,φ1,φ2∈Rm∫0Tbt,y,φ1,φ22dt<+∞.The existence and uniqueness of the solution of problem (1)–(3) are proved in [12] under conditions (5)–(7).


## 3. Calculation Infinitesimal Operator

LetU:R+×Y×H×D→R1 be continuous in the set of variables and continuously differentiable in t functional.One can prove [13] that the pair {xt,ξ(t)}, where xt={x(t+θ),θ∈[-r,0]}, is a Markov process and we can consider the weak infinitesimal operator [9](8)QUt,y,h,φ≔limΔt↓01ΔtEy,h,φtUt+Δt,ξt+Δt,ηt+Δt,xt+Δt-Ut,y,h,φ,where Ey,h,φ(t)v≔E{v/ξ(t)=y,η(t)=h,xt=φ}, η(t)≔ηk, and tk≤t<tk+1, k≥0. It is natural to assume that the functional U is in the domain of the operator Q, if the limit (8) exists in sense of uniform convergence in a neighborhood (y,h,φ) uniformly on h∈H that D(Q)={φ∈C1([-r;0];Rm)}.Let us introduce the operatorL0, which is connected with Markov switching (2) at time tk, k≥0,(9)L0Ut,y,h,φ≔It∈S∫HUt,y,z,φPkh,dz-Ut,y,h,φ,where Pk(h,dz) is transition probability of Markov chain η on kth step and I is indicator of set S.Calculate the operator(QU)(t,y,h,φ) on solutions of system (1)–(3).Consider the following cases.Case 1.
Letξ(t), t≥0, be a Markov chain with a finite number of states Y≔{y1,y2,…,yN} and generator Q~={q~ij}, i,j=1,N¯.
At the timeτ of changing of a structure of parameter ξ of system yi→yj is the stepwise changing of vector xτ with transition probability Pij(τ,φ,A): (10)Pijτ,φ,A≔Pxτ∈A∣xτ-=φ,ξτ-0=yi,ξτ=yj,A⊂D.Theorem 1.
The weak infinitesimal operatorQ on solutions of system (1)–(3) of functional U is calculated by the formula(11)QUt,y,h,φ=LtUt,y,h,φ+LxUt,y,h,φ+LyUt,y,h,φ+L0Ut,y,h,φ,where (LtU)(t,y,h,φ)=∂U(t,y,h,φ)/∂t;(12)LxUt,y,h,φ=∇xU,at,y,h,φ+12Sp∇xx2Ubt,y,h,φ,bTt,y,h,φ,(13)LyUt,y,h,φ=∑j≠i∫DUt,yj,h,ζPijt,φ,dζ-Ut,yi,h,φqij,where (·,·) is scalar product, (∇U)=∂U/∂x1,…,∂U/∂xmT, ∂U/∂xi is Frechet derivative at ith coordinate of the vector x∈D, ∇xx2U=∂2U/∂xi∂xji,j=1m is Frechet second derivative matrix, Sp is trace of matrix, qij=-q~ij/q~i, (L0U)(t,y,h,φ) is calculated by formula (9), and U is differentiable in t functional that has Frechet derivative of the 1st and 2nd orders by the last variable.Proof.
By definition (see (8)) (14)QUt,y,h,φ≔limΔt↓01ΔtEy,h,φtUt+Δt,ξt+Δt,ηt+Δt,xt+Δt-Ut,y,h,φ.
Further,(15)QUt,y,h,φ=limΔt↓01ΔtEy,h,φtUt+Δt,ξt+Δt,ηt+Δt,xt+Δt-Ut,y,h,φ±Ut,ξt+Δt,ηt+Δt,xt+Δt±Ut,y,ηt+Δt,xt+Δt±Ut,y,h,xt+Δt.
That,Q can be represented as (16)QUt,y,h,φ=limΔt↓01ΔtEy,h,φtUt+Δt,ξt+Δt,ηt+Δt,xt+Δt-Ut,ξt+Δt,ηt+Δt,xt+Δt+limΔt↓01ΔtEy,h,φtUt,ξt+Δt,ηt+Δt,xt+Δt-Ut,y,ηt+Δt,xt+Δt+limΔt↓01ΔtEy,h,φtUt,y,ηt+Δt,xt+Δt-Ut,y,h,xt+Δt+limΔt↓01ΔtEy,h,φtUt,y,h,xt+Δt-Ut,y,h,φ=LtUt,y,h,φ+LyUt,y,h,φ+L0Ut,y,h,φ+LxUt,y,h,φ.
Consider each term separately. For the first termLtU image is obvious.
Establish an explicit form of termLxU. Consider a complete group of disjoin events constructed as follows: denote by Hi event, when in the time interval (t,t+Δt] structure of (1) is not changing; that is, ξ(τ)=yi for τ∈(t,t+Δt]. Then without o(Δt) terms [14] we obtain following equality: (17)PHi=1-qiΔt.
Then denote byHij event, when in the time interval (t,t+Δt] there is a change on yi→yj≠yi. Then without o(Δt) terms we will have (18)PHij=qijΔt.
Denote byΔiU≔U(t+Δt,ξ(t+Δt),h,xt+Δt)-U(t,yi,h,xt) and by ΔijU increment ΔU by doing Hij. Calculate increments ΔiU and ΔijU of functional U when the events Hi, Hij, j≠i, occurrence, ignoring the terms order o(Δt):(19)ΔiU=∂U∂t+∇xU,at,yi,h,φ+12Sp∇xx2Ubt,yi,h,φ,bTt,yi,h,φΔt+oΔt.Here, all Frechet partial derivatives are calculated in (t,yi,h,x), where x is solution of (1) with initial conditions xt=φ, ξ(t)=yi, and s>t≥0.
Further, forLyU, in case of change of structure yi→yj on the time interval (t,t+Δt] we will have an increment(20)ΔijU=Ut+Δt,yj,h,xt+Δt-Ut,yi,h,φwith probability qijΔt.
The terms which illustrate the possibility of changing of structure of parameterξ are not included in last equality and there is no Markov switching. After averaging these terms will be equal to o(Δt), and we can ignore them.
To calculateE{ΔU∣ξt=yi,η0=h,xt=φ} use the law of total probability (21)EΔU∣ξt-=yi,η0=h,xt=φ=EEΔU∣ξt=yj,ξt-=yi,η0=h,xt=φ,where external expectation on the right side is calculated by variable ξ in t.
Ignoringo(Δt) terms, from (19) and (20), we get equality(22)EΔU∣ξt=yi,η0=h,xt=φ=∂U∂t+∇xU,at,y,h,φ+12Sp∇xx2Ubt,y,h,φ,bTt,y,h,φ1-qiΔtΔt+∑j≠i∫DUt,yj,h,ζPijt,φ,dζ-Ut,yi,h,φqijΔt+oΔt.In calculation of the third term we used property x′Ax=Sp(Axx′) and property of Wiener process about covariance of increment [8, 9].
Dividing onΔt and passing to the limit on Δt↓0 we obtained the first, second, and forth terms in (11). The idea of calculating the third term L0U can be found in [15, pp. 163-164]. Theorem 1 is proved.Case 2.
Let in the time of changing of structureyi→yj phase vector change by determined law x(τ)=ψij(x(τ-0)), i≠j.Theorem 2.
Let the conditions of Theorem1 hold; then the operator (QU)(t,y,h,φ) is calculated by the formula (23)QUt,y,h,φ=∂U∂t+∇xU,at,y,h,φ+12Sp∇xx2Ubt,y,h,φ,bTt,y,h,φ+∑j≠iUt,yj,h,ψijφ-Ut,yi,h,φqij.
The proof is carried out under the scheme proof of Theorem1.
If at the time of change of structureyi→yj phase vector changes continuously x(τ-0)=x(τ)=x.Corollary 3.
Under the conditions of Theorem1(24)QUt,y,h,φ=∂U∂t+∇xU,at,y,h,φ+12Sp∇xx2Ubt,y,h,φ,bTt,y,h,φ+∑j≠iUt,yj,h,φ-Ut,yi,h,φqij.Case 3.
Letξ(t), t≥0, be purely discontinuous Markov processes, allowing the decomposition of probability (25)Pξt+Δt∈β,β+Δβ∣ξt=α≠β=pt,α,βΔβΔt+oΔt,Pξτ=α,t<τ<t+Δt∣ξt=α=1-pt,αΔt+oΔt,and at the time of jump ξ phase vector changes continuously, leading to calculations (26)QUt,y,h,φ=LtUt,y,h,φ+LxUt,y,h,φ+L0Ut,y,h,φ+∫YUt,β,h,φ-Ut,α,h,φpt,α,βdβ,where LtU, LxU, and L0U are, respectively, calculated by formulas (9), (11), and (12).


## 4. Model Example

Consider SDDE(27)dxt=xt-xt-2ξtdt+4xtξtdwt,t∈R+∖N,with switching,(28)Δxtk=x2tk-ηk,tk∈N,and initial conditions,(29)xθ=-θ,-2≤θ≤0,η0=-1,ξ0=12.Here x(t)∈Rm; ξ is Markov process with values in the space Y≔{1/2,1/4,1/8} with generator Q~ξ=-1101-2134-7; ηk, k≥0, is Markov chain with values in the space H≔{-1;1} with matrix of transition probabilities Pη=1/32/31/43/4.We assume that the phase vectorx changes continuously during t∈R+∖N, and functional U:R+×Y×H×D→R1 is defined as (30)Ut,y,h,φ=φ0+φ-22hy+gt,g(t)≥0, t≥0, g∈C1(R+).Then WIO based system (27)–(29) have following form: (31)QUt,y,h,φ=g˙t+2hφ0+φ-2+8hφ0+hφ0+φ-22∑j≠i1yj-1yiqij+L0Ut,y,h,φ,where (32)L0Ut,y,-1,φ=It∈N23Ut,y,1,φ-Ut,y,-1,φ,L0Ut,y,1,φ=It∈N34Ut,y,-1,φ-Ut,y,1,φ.


## 5. Conclusion

(1)
Parameterξ is continuous Markov chain with a finite number of states and at the time of changing the structure is jumping phase vector which is known for the transition probability (Case 1). (2)
At the time of changing the structure of the phase vector changes determined by law (Case2) or continuously (Corollary 3). (3)
Parameterξ is purely discontinuous Markov processes and is continuously changing phase vector (Case 3).



---

*Source: 1694935-2016-09-29.xml*