# Spectral Perspective on the Stability of Discrete-Time Markov Jump Systems with Multiplicative Noise

Authors: Ting Hou; Weihai Zhang; Hongji Ma
Journal: Mathematical Problems in Engineering
 (2014)

---

## Abstract

We apply the spectrum analysis approach to address the stability of discrete-time Markov jump systems with state-multiplicative noise. In terms of the spectral distribution of a generalized Lyapunov operator, spectral criteria are presented to testify three different kinds of stochastic stabilities: asymptotic mean square stability, critical stability, and essential instability.


---

## Body

## 1. Introduction

Since the pioneering research of [1], Markov jump systems have received an increasing attention from the control community. The main impetus for studying this kind of systems arises from the widespread dynamics that are subject to abrupt structural changes, such as random component failures or repairs, sudden variation in operating points of nonlinear plants, the switching between the economic scenarios, and temporary loss of communication signal. By now, the study of Markov jump systems has become one of the most active areas in the control theory and great progress has been made in the relevant analysis and synthesis. Among many contributions, we would like to mention that structural properties and quadratic optimal control of continuous-time Markov jump linear systems have been considered in [2]. Stability and robust control design have been performed in [3] for discrete-time Markov jump linear systems. Moreover, robustly asymptotic stabilization for a class of stochastically nonlinear singular jump systems has been tackled in [4] based on the Lyapunov-Krasovskii functional. Besides, stability analyses for Markovian jumping neural networks with delays have also been elaborately addressed; see [5–10] and the references therein. Recently, many researchers are attracted to the study of stochastic linear systems subject to both multiplicative noise perturbations and Markov jump, which are commonly viewed as a powerful mathematical tool to investigate the financial phenomena and engineering problems. For instance, the indefinite stochastic linear quadratic (LQ) optimal control was tackled in [11] for discrete-time multiplicative noise systems with Markov jump parameters. Robust H
∞ filtering has been reported in [12] for stochastic nonlinear Itô systems with Markov jumps. In addition, by means of four coupled matrix recursions, mixed H
2
/
H
∞ control problem has been settled in [13] for a class of discrete-time Markov jump systems with multiplicative noise. For more details of recent developments, interested readers are referred to [14–16] and the references cited therein.It is well known that stability is one of the most fundamental notions in the modern control theory. It is the primal factor to be taken into account in the controller design. For stochastic Itô systems and Markov jump systems, various stability concepts such as stochastic stability, exponential stability [17–19], asymptotic stability in probability [20], finite-time stability [21], global asymptotical stability [22, 23], and absolute stability [24] have been studied extensively. It can be remarked that almost all existing works about the stability analysis of stochastic systems are based on the Lyapunov functional method. However, our main purpose is to provide a spectral perspective about the stability of discrete-time Markov jump systems with state-multiplicative noise. Concretely speaking, we will employ the spectral analysis technique to classify three different kinds of stabilities: asymptotic mean square stability, critical stability, and essential instability. For them, spectral criteria are derived according to a generalized Lyapunov operator generated from the system coefficients and transition probability matrix of Markov jump parameter.The rest of this paper is organized as follows. In Section2, we introduce a generalized Lyapunov operator which serves as the fundamental tool in the stability analysis. In Section 3, necessary and sufficient conditions, including spectral criteria, are supplied for three kinds of stochastic stabilities, respectively. Section 4 ends this paper with a brief concluding remark.Throughout this paper, the following notations are adopted:R
n: n-dimensional Euclidean space with the usual 2-norm ∥
·
∥; R
n
×
m: the space of all n
×
m real matrices; S
n: the set of all n
×
n symmetric matrices; M
>
0
(
≥

0
): M that is positive definite (positive semidefinite) symmetric matrix; R
n
×
m
N
(
S
n
N
): the set of all N sequences V
=
(
V
1
,
…
,
V
N
) with V
i
∈
R
n
×
m
(
S
n
); S
n
+
N: the set of all N sequences V
=
(
V
1
,
…
,
V
N
) where V
i
∈
S
n and V
i
≥
0; A
′: the transpose of a matrix (vector) A; A
⊗
B: the Kronecker product of matrices A and B; I
n: the n
×
n identity matrix; N
:
=
{
0,1
,
…
}, N
t
:
=
{
0
,
…
,
t
}, and N
¯
:
=
{
1
,
…
,
N
}; I
{
A
}: the indicator function of a set A.


## 2. Lyapunov-Type Operator and Its Spectra

On a given probability space(
Ω
,
F
,
P
), we consider the following linear system with Markovian jumps and multiplicative noise:
(1)
x
(
k
+
1
)
=
A
(
θ
k
)
x
(
k
)
+
C
(
θ
k
)
x
(
k
)
w
(
k
)
,
x
(
0
)
=
x
0
∈
R
n
,
k
∈
N
,
where x
(
k
)
∈
R
n represents the state of the system. θ
(
k
) is a Markov chain taking values in N
¯ and the transition probability matrix is P
=
[
p
i
j
]
N
×
N, p
i
j
=
P
(
θ
k
+
1
=
j
∣
θ
k
=
i
). w
(
k
)
(
k
∈
N
T
) is a sequence of real random variables with E
(
w
(
k
)
)
=
0 and E
(
w
(
k
)
w
(
s
)
)
=
δ
k
s (Kronecker function). The random variables {
w
(
k
)
,
k
=
0,1
,
…
} are independent of the Markov chain {
θ
k
,
k
=
0,1
,
…
}. The initial value θ
0 is independent of the noise w
(
k
)
(
k
∈
N
T
). Set F
k the σ-field generated by {
(
θ
t
,
x
(
t
)
)
,
t
=
0,1
,
…
,
k
}. Fixing θ
k
=
i, we denote A
(
θ
k
)
=
A
i, C
(
θ
k
)
=
C
i, where A
i and C
i  (
i
∈
N
¯
) are matrix-valued functions of suitable dimensions. Let
(2)
A
=
(
A
1
,
A
2
,
…
,
A
N
)
,
C
=
(
C
1
,
C
2
,
…
,
C
N
)
.
In the sequel, system (1) is also written as [
A
,
C
∣
P
] for short.First of all, we show a useful representation which characterizes the state evolution of system (1).Theorem 1.
For[
A
,
C
∣
P
], one sets that X
i
(
k
)
=
E
[
x
(
k
)
x
(
k
)
′
I
{
θ
k
=
i
}
]. Then, X
i
(
k
)
(
i
∈
N
¯
) satisfies
(3)
X
i
(
k
+
1
)
=
∑
j
=
1
N
‍
p
j
i
A
j
X
j
(
k
)
A
j
′
+
∑
j
=
1
N
‍
p
j
i
C
j
X
j
(
k
)
C
j
′
,
X
i
(
0
)
=
E
[
x
0
x
0
′
I
{
θ
0
=
i
}
]
.Proof.
Sincex
(
k
)
x
(
k
)
′ and θ
k are F
k-measurable and {
θ
k
,
k
=
0,1
,
…
} are independent of {
w
(
k
)
,
k
=
0,1
,
…
}, we have
(4)
X
i
(
k
+
1
)
=
E
[
x
(
k
+
1
)
x
(
k
+
1
)
′
I
{
θ
k
+
1
=
i
}
]
=
E
{
e
e
[
A
(
θ
k
)
x
(
k
)
+
C
(
θ
k
)
x
(
k
)
w
(
k
)
]
×
[
A
(
θ
k
)
x
(
k
)
+
C
(
θ
k
)
x
(
k
)
w
(
k
)
]
′
I
{
θ
k
+
1
=
i
}
e
e
}
=
E
{
[
e
e
A
(
θ
k
)
x
(
k
)
x
(
k
)
′
A
(
θ
k
)
′
+
C
(
θ
k
)
x
(
k
)
x
(
k
)
′
C
(
θ
k
)
′
w
2
(
k
)
+
A
(
θ
k
)
x
(
k
)
x
(
k
)
′
C
(
θ
k
)
′
w
(
k
)
+
C
(
θ
k
)
x
(
k
)
x
(
k
)
′
A
(
θ
k
)
′
w
(
k
)
e
e
]
I
{
θ
k
+
1
=
i
}
}
=
∑
j
=
1
N
‍
E
{
E
{
[
e
e
A
j
x
(
k
)
x
(
k
)
′
A
j
′
+
A
j
x
(
k
)
x
(
k
)
′
C
j
′
w
(
k
)
+
C
j
x
(
k
)
x
(
k
)
′
C
j
′
w
2
(
k
)
+
C
j
x
(
k
)
x
(
k
)
′
A
j
′
w
(
k
)
e
e
]
×
I
{
θ
k
+
1
=
i
}
I
{
θ
k
=
j
}
∣
F
k
e
e
}
}
=
∑
j
=
1
N
‍
E
{
{
[
e
e
A
j
x
(
k
)
x
(
k
)
′
A
j
′
+
A
j
x
(
k
)
x
(
k
)
′
C
j
′
w
(
k
)
+
C
j
x
(
k
)
x
(
k
)
′
C
j
′
w
2
(
k
)
+
C
j
x
(
k
)
x
(
k
)
′
A
j
′
w
(
k
)
e
e
]
I
{
θ
k
=
j
}
}
×
E
{
I
{
θ
k
+
1
=
i
}
∣
σ
{
θ
k
:
k
=
0,1
,
…
,
k
}
}
e
e
}
=
∑
j
=
1
N
‍
E
{
[
e
e
A
j
x
(
k
)
x
(
k
)
′
A
j
′
+
A
j
x
(
k
)
x
(
k
)
′
C
j
′
w
(
k
)
+
C
j
x
(
k
)
x
(
k
)
′
C
j
′
w
2
(
k
)
+
C
j
x
(
k
)
x
(
k
)
′
A
j
′
w
(
k
)
e
e
]
I
{
θ
k
=
j
}
}
×
E
{
I
{
θ
k
+
1
=
i
}
∣
σ
{
θ
k
:
k
=
0,1
,
…
,
k
}
}
=
∑
j
=
1
N
‍
E
{
[
e
e
A
j
x
(
k
)
x
(
k
)
′
A
j
′
+
A
j
x
(
k
)
x
(
k
)
′
C
j
′
w
(
k
)
+
C
j
x
(
k
)
x
(
k
)
′
C
j
′
w
2
(
k
)
+
C
j
x
(
k
)
x
(
k
)
′
A
j
′
w
(
k
)
e
e
]
I
{
θ
k
=
j
}
}
×
P
{
θ
k
+
1
=
i
∣
θ
k
=
j
}
=
∑
j
=
1
N
‍
p
j
i
A
j
X
j
(
k
)
A
j
′
+
∑
j
=
1
N
‍
p
j
i
C
j
X
j
(
k
)
C
j
′
,
X
i
(
0
)
=
E
[
x
(
0
)
x
(
0
)
′
I
{
θ
0
=
i
}
]
=
E
[
x
0
x
0
′
I
{
θ
0
=
i
}
]
.
The proof is completed.GivenX
=
(
X
1
,
X
2
,
…
,
X
N
)
∈
S
n
N, we introduce the following Lyapunov operator:
(5)
L
A
,
C
i
(
X
)
=
∑
j
=
1
N
‍
p
j
i
A
j
X
j
A
j
′
+
∑
j
=
1
N
‍
p
j
i
C
j
X
j
C
j
′
,
L
A
,
C
(
X
)
=
(
L
A
,
C
1
(
X
)
,
L
A
,
C
2
(
X
)
,
…
,
L
A
,
C
N
(
X
)
)
.
According to the formulation of X
i
(
k
)
(
i
∈
N
¯
) derived in Theorem 1, the spectra of L
A
,
C are defined as follows.Definition 2.
For[
A
,
C
∣
P
], let L
A
,
C be the linear operator from S
n
N to S
n
N. Then, the spectral set of L
A
,
C is
(6)
σ
(
L
A
,
C
)
:
=
{
λ
∈
C
:
L
A
,
C
(
X
)
=
λ
X
,
X
∈
S
n
N
,
X
≠
0
}
.
Let us see an illustrative example which shows how to compute all the spectra ofL
A
,
C.Example 3.
In[
A
,
C
∣
P
], the jump parameter θ
k takes values in N
¯
=
{
1,2
} and the transition probability matrix is given by
(7)
P
=
[
1
0
1
10
9
10
]
.
When θ
k
=
1,
(8)
A
1
=
[
1
2
-
1
4
1
4
0
]
,
C
1
=
[
-
1
4
1
2
0
-
1
4
]
.
In the case of θ
k
=
2,
(9)
A
2
=
[
1
2
0
0
1
]
,
C
2
=
[
1
2
1
2
0
-
1
2
]
.
By (5) and noting the symmetry of X, we get
(10)
L
A
,
C
1
(
X
)
=
[
5
16
x
1
11
-
1
2
x
1
12
+
5
16
x
1
22
*
1
8
x
1
11
-
1
8
x
1
22
1
16
x
1
11
+
1
16
x
1
22
]
+
[
1
20
x
2
11
+
1
20
x
2
12
+
1
40
x
2
22
*
1
40
x
2
12
-
1
40
x
2
22
1
8
x
2
22
]
,
(11)
L
A
,
C
2
(
X
)
=
[
9
20
x
2
11
+
9
20
x
2
12
+
9
40
x
2
22
*
9
40
x
2
12
-
9
40
x
2
22
9
8
x
2
22
]
,
where * is the ellipsis of the symmetric terms. The eigenvalue equation in (6) can be equivalently expressed as
(12)
L
M
[
x
1
11
x
1
12
x
1
22
x
2
11
x
2
12
x
2
22
]
=
λ
[
x
1
11
x
1
12
x
1
22
x
2
11
x
2
12
x
2
22
]
.
Further, we can derive
(13)
L
M
=
[
5
16
-
1
2
5
16
1
20
1
20
1
40
1
8
0
-
1
8
0
1
40
-
1
40
1
16
0
1
16
0
0
1
8
0
0
0
9
20
9
20
9
40
0
0
0
0
9
40
9
40
0
0
0
0
0
9
8
]
.
It is easy to obtain that the spectral set of L
A
,
C is
(14)
σ
(
L
A
,
C
)
=
{
1
4
,
9
40
,
9
20
,
9
8
,
1
16
±
7
16
j
}
,
j
2
=
-
1
.Remark 4.
The matrixL
M that appeared in Example 3 is called the induced matrix of L
A
,
C. Clearly, L
M is unique and has the same eigenvalues as the operator L
A
,
C.
Similar to [25, 26], we define two linear operators ψ
→ and φ
~ as follows: ∀
X
=
(
X
1
,
…
,
X
N
)
∈
S
n
N,
(15)
ψ
→
(
X
)
=
[
ψ
→
(
X
1
)
⋮
ψ
→
(
X
N
)
]
,
φ
~
(
X
)
=
[
φ
~
(
X
1
)
⋮
φ
~
(
X
N
)
]
,
ψ
→
(
X
i
)
=
(
x
i
11
,
…
,
x
i
1
n
,
x
i
21
,
…
,
x
i
2
n
,
…
,
x
i
n
1
,
…
,
x
i
n
n
)
′
,
φ
~
(
X
i
)
=
(
x
i
11
,
…
,
x
i
1
n
,
x
i
22
,
…
,
x
i
2
n
,
…
,
x
i
n
-
1
,
n
-
1
,
x
i
n
-
1
,
n
,
x
i
n
n
)
′
,
where x
i
j
k  (
j
,
k
=
1,2
,
…
,
n
) is the element of X
i. That is, ψ
→
(
X
i
) is a column vector consisting of all elements of X
i, while φ
~
(
X
i
) is the column vector generated by all upper diagonal elements of X
i. The relation between them can be dominated by the unique matrix H
n
2
N
,
(
n
(
n
+
1
)
/
2
)
N that is of full column rank:
(16)
ψ
→
(
X
)
=
H
n
2
N
,
(
n
(
n
+
1
)
/
2
)
N
φ
~
(
X
)
.
In fact, if we take that
(17)
Θ
(
H
n
2
N
,
(
n
(
n
+
1
)
/
2
)
N
,
A
,
C
)
=
(
H
n
2
N
,
(
n
(
n
+
1
)
/
2
)
N
′
H
n
2
N
,
(
n
(
n
+
1
)
/
2
)
N
)
-
1
×
H
n
2
N
,
(
n
(
n
+
1
)
/
2
)
N
′
·
{
(
P
′
⊗
I
)
Diag
(
A
i
⊗
A
i
+
C
i
⊗
C
i
)
}
H
n
2
N
,
(
n
(
n
+
1
)
/
2
)
N
,
where
(18)
Diag
(
A
i
⊗
A
i
+
C
i
⊗
C
i
)
=
[
Π
1
0
⋯
0
0
Π
2
⋯
0
⋮
⋮
⋱
⋮
0
0
⋯
Π
N
]
,
Π
i
=
A
i
⊗
A
i
+
C
i
⊗
C
i
,
i
∈
N
¯
,
then the expression of L
M turns out to be
(19)
L
M
=
Θ
(
H
n
2
N
,
(
n
(
n
+
1
)
/
2
)
N
,
A
,
C
)
.
By Definition2, the operator L
A
,
C is a linear operator defined on the Hilbert space S
n
N with the inner product 〈
A
,
B
〉
=
∑
i
=
1
N
‍
Tr
⁡
(
A
i
B
i
), where A
,
B
∈
S
n
N. The adjoint operator of L
A
,
C induced by the definition of inner production and the property of trace is given as follows:
(20)
L
A
,
C
i
,
*
(
X
)
=
∑
j
=
1
N
‍
p
i
j
A
i
′
X
j
A
i
+
∑
j
=
1
N
‍
p
i
j
C
i
′
X
j
C
i
,
L
A
,
C
*
(
X
)
=
(
L
A
,
C
1
,
*
(
X
)
,
L
A
,
C
2
,
*
(
X
)
,
…
,
L
A
,
C
N
,
*
(
X
)
)
.
We can show that L
A
,
C
* and L
A
,
C have the same spectral radius. Particularly, when the coefficients of [
A
,
C
∣
P
] are all real matrices, we have σ
(
L
A
,
C
)
=
σ
(
L
A
,
C
*
).


## 3. Stability Analysis

Stability is one of the core concepts in the modern control theory. It is a common sense that stability is a necessary prerequisite for the system to behave well. Next, we will make use of the spectrum technique to discuss the stability of system (1).


### 3.1. Asymptotic Mean Square Stability

Above all, we recall the well-known asymptotic mean square stability of stochastic Markov jump systems.Definition 5.
The system described by the state equation (1) is asymptotically mean square stable (AMSS), if for any x
0
∈
R
n and θ
0
∈
N
¯, the corresponding state satisfies lim
⁡
k
→
∞
E
∥
x
(
k
)
∥
2
=
0.
In terms of the spectra of the operatorL
A
,
C, several necessary and sufficient conditions can be obtained for the asymptotic mean square stability of [
A
,
C
∣
P
].Theorem 6.
The following assertions are equivalent to AMSS of[
A
,
C
∣
P
]:(i)
σ
(
L
A
,
C
)
⊂
D
(
0,1
)
:
=
{
λ
:
λ
∈
C
,
|
λ
|
<
1
};
(ii)
for any given∀
W
=
(
W
1
,
W
2
,
…
,
W
N
)
∈
S
n
+
N with W
>
0, there exists V
=
(
V
1
,
V
2
,
…
,
V
N
)
∈
S
n
+
N with V
>
0 such that L
A
,
C
(
V
)
-
V
=
-
W;
(iii)
for any given∀
W
=
(
W
1
,
W
2
,
…
,
W
N
)
∈
S
n
+
N with W
>
0, there exists V
=
(
V
1
,
V
2
,
…
,
V
N
)
∈
S
n
+
N with V
>
0 such that L
A
,
C
*
(
V
)
-
V
=
-
W;
(iv)
there existsV
=
(
V
1
,
V
2
,
…
,
V
N
)
∈
S
n
+
N with V
>
0 such that L
A
,
C
(
V
)
-
V
<
0;
(v)
there existsV
=
(
V
1
,
V
2
,
…
,
V
N
)
∈
S
n
+
N with V
>
0 such that L
A
,
C
*
(
V
)
-
V
<
0.Proof.
We only show the validity of (i). The rigorous arguments of (ii)–(v) can be found in [3]. Set that X
i
(
k
)
=
E
[
x
(
k
)
x
(
k
)
′
I
{
θ
k
=
i
}
]
(
i
∈
N
¯
) and X
(
k
)
=
(
X
1
(
k
)
,
…
,
X
N
(
k
)
). Based on the state equation of [
A
,
C
∣
P
], Theorem 1 and Definition 2 give that
(21)
X
(
k
+
1
)
=
L
A
,
C
(
X
(
k
)
)
,
X
(
0
)
=
X
0
=
(
E
[
x
0
x
0
′
I
{
θ
0
=
1
}
]
,
…
,
E
[
x
0
x
0
′
I
{
θ
0
=
N
}
]
)
.
Recalling Remark 4 and using the operator φ
~, (21) can be equivalently expressed as
(22)
φ
~
(
X
(
k
+
1
)
)
=
L
M
φ
~
(
X
(
k
)
)
,
φ
~
(
X
(
0
)
)
=
φ
~
(
X
0
)
.
From Definition 5, we have [
A
,
C
∣
P
] that is asymptotically mean square stable if and only if lim
⁡
k
→
∞
⁡
E
∥
x
(
k
)
∥
2
=
0; that is, lim
⁡
k
→
∞
E
[
x
(
k
)
x
(
k
)
′
I
{
θ
k
=
i
}
]
=
0, i
∈
N
¯. By (22) and the stability theory of discrete-time linear systems, the desired result follows.Example 7.
In[
A
,
C
∣
P
], the jump parameter θ
k takes values in N
¯
=
{
1,2
} and the transition probability matrix is given by
(23)
P
=
[
1
0
1
2
1
2
]
.
Let
(24)
A
1
=
[
1
2
-
1
4
1
4
0
]
,
A
2
=
[
1
2
0
0
1
]
,
C
1
=
[
-
1
4
1
2
0
-
1
4
]
,
C
2
=
[
1
2
1
2
0
-
1
2
]
.
There exists
(25)
V
1
=
[
6.7347
-
0.3543
-
0.3543
5.1088
]
,
V
2
=
[
2.8953
-
0.6119
-
0.6119
4.1560
]
,
satisfying the following LMIs:
(26)
p
11
(
A
1
V
1
A
1
′
+
C
1
V
1
C
1
′
)
+
p
21
(
A
2
V
2
A
2
′
+
C
2
V
2
C
2
′
)
-
V
1
<
0
,
p
12
(
A
1
V
1
A
1
′
+
C
1
V
1
C
1
′
)
+
p
22
(
A
2
V
2
A
2
′
+
C
2
V
2
C
2
′
)
-
V
2
<
0
,
V
1
>
0
,
V
2
>
0
.
It thus follows by Theorem 6 that the system denoted by this example is asymptotically mean square stable.


### 3.2. Critical Stability

In this subsection, we are concerned with the critical stability, which was first proposed in [27] and played a key role in defining the strong solution of a class of generalized Riccati equations arising from stochastic linear quadratic control problem. The following definition can be regarded as an extension of Definition 2.3 [27].Definition 8.
[
A
,
C
∣
P
] is called critically stable, if σ
(
L
A
,
C
)
⊂
D
[
0,1
]
:
=
{
λ
:
λ
∈
C
,
|
λ
|
≤
1
}.
The following result presents useful criteria for verifying the critical stability of[
A
,
C
∣
P
]. Its proof can be accomplished by using Theorem 6 and Definition 8. So, only conclusions are listed below.Theorem 9.
[
A
,
C
∣
P
] is critically stable if and only if one of the following conditions is satisfied:(i)
for any given∀
ɛ
>
1, σ
(
L
A
/
ɛ
,
C
/
ɛ
)
⊂
D
(
0,1
);
(ii)
for any given∀
ɛ
>
1 and W
=
(
W
1
,
W
2
,
…
,
W
N
)
∈
S
n
+
N with W
>
0, there exists V
=
(
V
1
,
V
2
,
…
,
V
N
)
∈
S
n
+
N with V
>
0 such that L
A
/
ɛ
,
C
/
ɛ
(
V
)
-
V
=
-
W;
(iii)
for any given∀
ɛ
>
1 and W
=
(
W
1
,
W
2
,
…
,
W
N
)
∈
S
n
+
N with W
>
0, there exists V
=
(
V
1
,
V
2
,
…
,
V
N
)
∈
S
n
+
N with V
>
0 such that L
A
/
ɛ
,
C
/
ɛ
*
(
V
)
-
V
=
-
W;
(iv)
for any given∀
ɛ
>
1, there exists V
=
(
V
1
,
V
2
,
…
,
V
N
)
∈
S
n
+
N with V
>
0 such that L
A
/
ɛ
,
C
/
ɛ
(
V
)
-
V
<
0;
(v)
for any given∀
ɛ
>
1, there exists V
=
(
V
1
,
V
2
,
…
,
V
N
)
∈
S
n
+
N with V
>
0 such that L
A
/
ɛ
,
C
/
ɛ
*
(
V
)
-
V
<
0.


### 3.3. Essential Instability

Essential instability for stochastic Itô systems was first proposed in [25]. In what follows, we generalize this concept to Markov jump systems.Definition 10.
[
A
,
C
∣
P
] is called essentially instable, if there exists λ
∈
σ
(
L
A
,
C
) such that |
λ
|
>
1.Next, we give a spectral criterion for essential instability of system (1). The detailed proof can be given by following the similar procedure of Theorem 1 [25].Theorem 11.
[
A
,
C
∣
P
] is essentially instable if and only if for any given ∀
W
=
(
W
1
,
W
2
,
…
,
W
N
)
∈
S
n
+
N such that W
>
0, there exists a constant γ
>
1 and a unique V
=
(
V
1
,
V
2
,
…
,
V
N
)
∈
S
n
N satisfying that L
A
/
γ
,
C
/
γ
(
V
)
-
V
=
W. Moreover, there must exist an i
∈
N
¯ such that V
i admits at least one positive eigenvalue.Example 12.
LetN
¯
=
{
1,2
}. Take the same transition probability matrix and the same coefficient matrices as those in Example 3. According to Definition 10, we claim that [
A
,
C
∣
P
] is essentially instable because of 9
/
8
∈
σ
(
L
A
,
C
).


## 3.1. Asymptotic Mean Square Stability

Above all, we recall the well-known asymptotic mean square stability of stochastic Markov jump systems.Definition 5.
The system described by the state equation (1) is asymptotically mean square stable (AMSS), if for any x
0
∈
R
n and θ
0
∈
N
¯, the corresponding state satisfies lim
⁡
k
→
∞
E
∥
x
(
k
)
∥
2
=
0.
In terms of the spectra of the operatorL
A
,
C, several necessary and sufficient conditions can be obtained for the asymptotic mean square stability of [
A
,
C
∣
P
].Theorem 6.
The following assertions are equivalent to AMSS of[
A
,
C
∣
P
]:(i)
σ
(
L
A
,
C
)
⊂
D
(
0,1
)
:
=
{
λ
:
λ
∈
C
,
|
λ
|
<
1
};
(ii)
for any given∀
W
=
(
W
1
,
W
2
,
…
,
W
N
)
∈
S
n
+
N with W
>
0, there exists V
=
(
V
1
,
V
2
,
…
,
V
N
)
∈
S
n
+
N with V
>
0 such that L
A
,
C
(
V
)
-
V
=
-
W;
(iii)
for any given∀
W
=
(
W
1
,
W
2
,
…
,
W
N
)
∈
S
n
+
N with W
>
0, there exists V
=
(
V
1
,
V
2
,
…
,
V
N
)
∈
S
n
+
N with V
>
0 such that L
A
,
C
*
(
V
)
-
V
=
-
W;
(iv)
there existsV
=
(
V
1
,
V
2
,
…
,
V
N
)
∈
S
n
+
N with V
>
0 such that L
A
,
C
(
V
)
-
V
<
0;
(v)
there existsV
=
(
V
1
,
V
2
,
…
,
V
N
)
∈
S
n
+
N with V
>
0 such that L
A
,
C
*
(
V
)
-
V
<
0.Proof.
We only show the validity of (i). The rigorous arguments of (ii)–(v) can be found in [3]. Set that X
i
(
k
)
=
E
[
x
(
k
)
x
(
k
)
′
I
{
θ
k
=
i
}
]
(
i
∈
N
¯
) and X
(
k
)
=
(
X
1
(
k
)
,
…
,
X
N
(
k
)
). Based on the state equation of [
A
,
C
∣
P
], Theorem 1 and Definition 2 give that
(21)
X
(
k
+
1
)
=
L
A
,
C
(
X
(
k
)
)
,
X
(
0
)
=
X
0
=
(
E
[
x
0
x
0
′
I
{
θ
0
=
1
}
]
,
…
,
E
[
x
0
x
0
′
I
{
θ
0
=
N
}
]
)
.
Recalling Remark 4 and using the operator φ
~, (21) can be equivalently expressed as
(22)
φ
~
(
X
(
k
+
1
)
)
=
L
M
φ
~
(
X
(
k
)
)
,
φ
~
(
X
(
0
)
)
=
φ
~
(
X
0
)
.
From Definition 5, we have [
A
,
C
∣
P
] that is asymptotically mean square stable if and only if lim
⁡
k
→
∞
⁡
E
∥
x
(
k
)
∥
2
=
0; that is, lim
⁡
k
→
∞
E
[
x
(
k
)
x
(
k
)
′
I
{
θ
k
=
i
}
]
=
0, i
∈
N
¯. By (22) and the stability theory of discrete-time linear systems, the desired result follows.Example 7.
In[
A
,
C
∣
P
], the jump parameter θ
k takes values in N
¯
=
{
1,2
} and the transition probability matrix is given by
(23)
P
=
[
1
0
1
2
1
2
]
.
Let
(24)
A
1
=
[
1
2
-
1
4
1
4
0
]
,
A
2
=
[
1
2
0
0
1
]
,
C
1
=
[
-
1
4
1
2
0
-
1
4
]
,
C
2
=
[
1
2
1
2
0
-
1
2
]
.
There exists
(25)
V
1
=
[
6.7347
-
0.3543
-
0.3543
5.1088
]
,
V
2
=
[
2.8953
-
0.6119
-
0.6119
4.1560
]
,
satisfying the following LMIs:
(26)
p
11
(
A
1
V
1
A
1
′
+
C
1
V
1
C
1
′
)
+
p
21
(
A
2
V
2
A
2
′
+
C
2
V
2
C
2
′
)
-
V
1
<
0
,
p
12
(
A
1
V
1
A
1
′
+
C
1
V
1
C
1
′
)
+
p
22
(
A
2
V
2
A
2
′
+
C
2
V
2
C
2
′
)
-
V
2
<
0
,
V
1
>
0
,
V
2
>
0
.
It thus follows by Theorem 6 that the system denoted by this example is asymptotically mean square stable.


## 3.2. Critical Stability

In this subsection, we are concerned with the critical stability, which was first proposed in [27] and played a key role in defining the strong solution of a class of generalized Riccati equations arising from stochastic linear quadratic control problem. The following definition can be regarded as an extension of Definition 2.3 [27].Definition 8.
[
A
,
C
∣
P
] is called critically stable, if σ
(
L
A
,
C
)
⊂
D
[
0,1
]
:
=
{
λ
:
λ
∈
C
,
|
λ
|
≤
1
}.
The following result presents useful criteria for verifying the critical stability of[
A
,
C
∣
P
]. Its proof can be accomplished by using Theorem 6 and Definition 8. So, only conclusions are listed below.Theorem 9.
[
A
,
C
∣
P
] is critically stable if and only if one of the following conditions is satisfied:(i)
for any given∀
ɛ
>
1, σ
(
L
A
/
ɛ
,
C
/
ɛ
)
⊂
D
(
0,1
);
(ii)
for any given∀
ɛ
>
1 and W
=
(
W
1
,
W
2
,
…
,
W
N
)
∈
S
n
+
N with W
>
0, there exists V
=
(
V
1
,
V
2
,
…
,
V
N
)
∈
S
n
+
N with V
>
0 such that L
A
/
ɛ
,
C
/
ɛ
(
V
)
-
V
=
-
W;
(iii)
for any given∀
ɛ
>
1 and W
=
(
W
1
,
W
2
,
…
,
W
N
)
∈
S
n
+
N with W
>
0, there exists V
=
(
V
1
,
V
2
,
…
,
V
N
)
∈
S
n
+
N with V
>
0 such that L
A
/
ɛ
,
C
/
ɛ
*
(
V
)
-
V
=
-
W;
(iv)
for any given∀
ɛ
>
1, there exists V
=
(
V
1
,
V
2
,
…
,
V
N
)
∈
S
n
+
N with V
>
0 such that L
A
/
ɛ
,
C
/
ɛ
(
V
)
-
V
<
0;
(v)
for any given∀
ɛ
>
1, there exists V
=
(
V
1
,
V
2
,
…
,
V
N
)
∈
S
n
+
N with V
>
0 such that L
A
/
ɛ
,
C
/
ɛ
*
(
V
)
-
V
<
0.


## 3.3. Essential Instability

Essential instability for stochastic Itô systems was first proposed in [25]. In what follows, we generalize this concept to Markov jump systems.Definition 10.
[
A
,
C
∣
P
] is called essentially instable, if there exists λ
∈
σ
(
L
A
,
C
) such that |
λ
|
>
1.Next, we give a spectral criterion for essential instability of system (1). The detailed proof can be given by following the similar procedure of Theorem 1 [25].Theorem 11.
[
A
,
C
∣
P
] is essentially instable if and only if for any given ∀
W
=
(
W
1
,
W
2
,
…
,
W
N
)
∈
S
n
+
N such that W
>
0, there exists a constant γ
>
1 and a unique V
=
(
V
1
,
V
2
,
…
,
V
N
)
∈
S
n
N satisfying that L
A
/
γ
,
C
/
γ
(
V
)
-
V
=
W. Moreover, there must exist an i
∈
N
¯ such that V
i admits at least one positive eigenvalue.Example 12.
LetN
¯
=
{
1,2
}. Take the same transition probability matrix and the same coefficient matrices as those in Example 3. According to Definition 10, we claim that [
A
,
C
∣
P
] is essentially instable because of 9
/
8
∈
σ
(
L
A
,
C
).


## 4. Conclusions

In this paper, we have applied the technique of operator spectrum to demonstrate some new aspects on the stability of discrete-time Markov jump systems with multiplicative noise. With the help of the spectra of a generalized Lyapunov operator, we have distinguished three kinds of stochastic stabilities and obtained their spectral criteria which are easy to test. As one of our future works, we will focus on how to use the proposed stability theories to devise the stochastic LQ optimal control with convergence rate constraints, which still remains open to date.



---

*Source: 769302-2014-07-10.xml*