# Stability and Linear Quadratic Differential Games of Discrete-Time Markovian Jump Linear Systems with State-Dependent Noise

Authors: Huiying Sun; Meng Li; Shenglin Ji; Long Yan
Journal: Mathematical Problems in Engineering
 (2014)

---

## Abstract

We mainly consider the stability of discrete-time Markovian jump linear systems with state-dependent noise as well as its linear quadratic (LQ) differential games. A necessary and sufficient condition involved with the connection between stochasticT
n-stability of Markovian jump linear systems with state-dependent noise and Lyapunov equation is proposed. And using the theory of stochastic T
n-stability, we give the optimal strategies and the optimal cost values for infinite horizon LQ stochastic differential games. It is demonstrated that the solutions of infinite horizon LQ stochastic differential games are concerned with four coupled generalized algebraic Riccati equations (GAREs). Finally, an iterative algorithm is presented to solve the four coupled GAREs and a simulation example is given to illustrate the effectiveness of it.


---

## Body

## 1. Introduction

In this paper we discuss linear systems with Markovian jump and state-dependent noise. Here the discrete-time stochastic linear systems subject to abrupt parameter changes can be modeled by a discrete-time finite-state Markov chain. They are a special sort of hybrid systems with both modes and state variables. Since the class of systems was firstly introduced in early 1960s, the hybrid systems driven by continuous-time Markov chains have been broadly employed to model many practical systems which may experience abrupt changes in system structure and parameters such as solar-powered systems, power systems, battle management in command, and control and communication systems [1–4]. In the past several decades, considerable attention has been focused on the analysis and synthesis of linear systems with Markovian jump, including stability analysis, state feedback, and output feedback controller design, filter design, and so forth [5–11].The stability theory of linear systems with Markovian jump and state-dependent noise, here we also say Markovian jump stochastic linear systems (MJSLS for short), is rather complex in that there exist some stability concepts. Particularly, the study of stability about these systems has attracted the attention of many researchers [12–17]. The very important stability notions are mean-square stability, moment stability, and almost sure stability. Mean-square stability deals with the asymptotic convergence to zero of the second moment of the state norm. There are some necessary and sufficient conditions for mean-square stability involving either the solution of the coupled Lyapunov equations or the location in the complex plane of the eigenvalues of suitable augmented matrices [12, 13]. Moment stability, δ-moment stability, requires the convergence to zero of δ-moment of the state norm (mean-square stability is just a particular case for δ
=
2). Although there exist some practical sufficient conditions, a simple necessary and sufficient condition testing δ-moment stability is not available (except for δ
=
2). Almost sure stability holds if the sample path of the state converges to zero with probability one. The checking about almost sure stability involves the determination of the sign of the top Lyapunov exponent, which is usually a rather difficult task [14, 15]. Contrary to deterministic systems, for which all moments are stable whenever the sample path is stable, the moment stability for stochastic systems implies almost sure stability, but not vice versa, as pointed out by [16].On the other hand, the LQ differential games have many applications in the economy, military, and intelligent robots. Since the book [18], entitled “Differential Games,” written by Dr. Isaacs came out, theory and application of differential games have been developed greatly. A differential game is a mathematical model that represents a conflict between different agents which control a dynamical system and each of them is trying to minimize his individual objective function by giving a control to the system. In fact, many situations in industry, economies, management, and elsewhere are characterized by multiple decision makers and enduring consequences of decisions which can be treated as dynamic games. Particularly applications of differential games are widely researched in LQ control problem [19–23]. By solving the LQ control problems, players can avoid most of the additional cost incurred by this perturbation. The authors consider the zero-sum, infinite-horizon, and LQ differential games in [19]. A sufficient condition for the LQ differential games is applied to the H
∞ optimization problem in [20]. The authors in [21] study the problem of designing suboptimal LQ differential games with multiple players. In [22], the authors study the LQ nonzero sum stochastic differential games problem with random jump. A leader-follower stochastic differential game is considered with the state equation being a linear Itô-type stochastic differential equation and the cost functionals being quadratic [23].As far as we know, there are few researchers paying attention to the discrete-time stochastic LQ differential games, especially MJSLS. Thus, it is significant to consider these systems. In [24], we have considered stochastic differential games in infinite-time horizon. By introducing stochastic exact observability and stochastic exact detectability, the optimal strategies (Nash equilibrium strategies) and the optimal cost values have been given. In [25], we have considered LQ differential games in finite horizon for discrete-time stochastic systems with Markovian jump parameters and multiplicative noise. Furthermore, a suboptimal solution of the LQ differential games is proposed based on a convex optimization approach. On the basis of [26], in the paper we further investigate the LQ differential games for discrete-time MJSLS with a finite number of jump times. It gives the optimal strategies and the optimal cost values for infinite horizon LQ stochastic differential games which are associated with the four coupled GAREs. Generally, it is difficult to solve the four coupled GAREs analytically. Here, we will solve the LQ differential games by means of stochastic T
n-stability and we will employ a recursive procedure to solve the coupled GAREs.The paper is organized as follows. In Section2, some basic definitions are recalled. A necessary and sufficient condition in relation to the connection between the T
n-stability for MJSLS and Lyapunov equation is presented. In Section 3, we formulate LQ differential games with quadratic cost function for the MJSLS and propose the stochastic T
n-stability of discrete-time MJSLS with a finite number of jump times, which is essential to obtain the main results. An iterative algorithm for solving the four coupled GAREs is put forward, and an illustrative example is also displayed in Section 4. Section 5 ends this paper with some concluding remarks.For convenience, the paper adopts the following basic notations. It usesR
n to denote the linear space of all n-dimensional real vectors. R
m
×
n denotes the linear space of all m
×
n real matrices. N
=
{
0,1
,
2
,
…
}. U
′ indicates the transpose of matrix U and U
≥
0  (
U
>
0
) represents a nonnegative definite matrix (positive definite). The standard vector norm in R
n is indicated by ∥
·
∥ and the corresponding induced norm of matrix U by U. L
2
(
∞
,
R
k
) represents the space of R
k-valued, square integrable random vectors and δ
{
·
} is the Dirac measure. Finally, we write E
k
[
·
] instead of E
[
·
∣
x
k
,
θ
k
], and we define the following operator: ε
i
(
U
)
=
∑
j
≠
i
‍
p
i
j
U
j for U
=
U
′
≥
0.


## 2. StochasticT
n-Stability for Discrete-Time MJSLS with a Finite Number of Jump Times

Let(
Ω
,
F
,
P
;
{
F
k
}
) be a given fundamental probability space where there exist a Markov chain θ
k and a sequence of real random variables w
(
k
). F
k denotes the σ-algebra generated by θ
k and w
(
k
); that is, F
k
=
σ
{
θ
s
,
w
(
s
)
∣
s
=
0,1
,
2
,
…
,
k
}
⊂
F. Consider the following MJSLS defined on the space (
Ω
,
F
,
P
;
{
F
k
}
):
(1)
x
(
k
+
1
)
=
A
θ
k
x
(
k
)
+
A
¯
θ
k
x
(
k
)
w
(
k
)
,
x
0
=
x
0
∈
R
n
,
k
∈
N
,
where {
x
(
k
)
,
θ
k
;
k
∈
N
} are the states of process with values in R
n
×
X; {
θ
k
;
k
∈
N
} is a time homogeneous Markov chain taking values in a finite set X
=
{
1,2
,
…
,
N
}, with initial distribution μ and transition probability matrix P
=
[
p
i
j
], where
(2)
p
i
j
:
=
P
θ
k
+
1
=
j
∣
θ
k
=
i
,
∀
i
,
j
∈
X
,
k
∈
N
.
The set X comprises the various operation modes of the system (1). For each θ
k
=
i
∈
X, the matrices A
θ
k
∈
R
n
×
n and A
¯
θ
k
∈
R
n
×
n (associated with “ith” mode) will be assigned as A
θ
k
:
=
A
i and A
¯
θ
k
:
=
A
¯
i in someplace of the paper. {
w
(
k
)
,
k
∈
N
} is a sequence of real random variables, which is also a wide sense stationary, second-order process with E
[
w
(
k
)
]
=
0 and E
[
w
(
i
)
w
(
j
)
]
=
δ
i
j, where δ
i
j refers to a Kronecker function; that is, δ
i
j
=
1 if i
=
j and δ
i
j
=
0 if i
≠
j. The MJSLS as defined is trivially a strong Markov process. We assume that θ
k is independent of w
(
k
). To make the operation more convenient, let x
(
k
)
=
x
k. When system (1) is stable, we also say (
A
i
,
A
¯
i
) stable for short.Although, several concepts of stochastic stability can be found in the literature, in this paper, the stochastic stability concept associated with the stopping times for MJSLS is researched. The stopping times in relation to jump times are defined as follows:(3)
T
0
=
0
,
T
n
=
min
⁡
k
>
T
n
-
1
:
θ
k
≠
θ
T
n
-
1
,
n
=
1,2
,
…
,
N
.
The stopping time may represent interesting situations from the point of view of applications. For instance, it can be the accumulated nth failure and repair of the system. In another situation, the stopping time can represent the occurrence of a “crucial failure” (which may happen after a random number of failures).This class of stochastic systems is associated with systems subject to failures in their components or connections according to a Markov chain. The situation that we are interested in arises when one wishes to study the stability of such a system until the occurrence of a fixed numberN of failures and repairs. The paper recognizes the sequence of the stopping times containing the successive times of the occurrence of such failures and then it studies the stability of system (1) according to these stopping times.Definition 1 (see [27]).
Consider a stopping timeT
n. The MJSLS (1) is(i)
stochasticallyT
n-stable if for each initial condition x
0 and initial distribution μ
(4)
E
∑
k
=
0
∞
‍
x
k
2
δ
{
T
n
≥
k
}
<
∞
,
(ii)
mean-squareT
n-stable if for each initial condition x
0 and initial distribution μ
(5)
lim
⁡
k
→
∞
E
x
k
2
δ
{
T
n
≥
k
}
=
0
.Lemma 2 (see [27]).
For allm
≥
1 and i
,
j
∈
X
(6)
P
(
T
1
=
m
,
θ
m
=
j
∣
θ
0
=
i
)
=
p
i
j
δ
{
m
=
1
}
,
w
h
e
n
p
i
i
=
0
,
p
i
i
m
-
1
p
i
j
δ
{
m
>
1
}
,
w
h
e
n
0
<
p
i
i
<
1
.Remark 3.
P
(
T
1
=
1
∣
θ
0
=
i
)
=
1 and P
(
T
1
=
+
∞
∣
θ
0
=
i
)
=
1, whenever p
i
i
=
0 and p
i
i
=
1, respectively. That is in any case system will jump to another state.Next, we will give an important theorem that will be used later.Theorem 4.
The MJSLS (1) is T
n-stable if and only if, for any given set of matrices W
i
>
0, there exists a unique set of matrices L
i
>
0, satisfying the Lyapunov equations
(7)
p
i
i
A
i
′
L
i
A
i
+
A
¯
i
′
L
i
A
¯
i
-
L
i
+
W
i
=
0
,
i
∈
X
.Proof.
Sufficiency. In the proof we employ an induction argument on the stopping times T
n. First, define the function
(8)
V
k
(
x
,
i
)
:
=
x
k
′
P
i
δ
T
1
>
k
+
G
i
δ
{
T
1
=
k
}
x
k
,
where G
i
>
0, and P
i
>
0 is the solution of
(9)
p
i
i
A
i
′
P
i
A
i
+
A
¯
i
′
P
i
A
¯
i
-
P
i
+
W
i
+
A
i
′
ε
i
(
G
)
A
i
+
A
¯
i
′
ε
i
(
G
)
A
¯
i
=
0
,
i
∈
X
.
The existence of such P
i
>
0 relies on (7). Hence, to functional V
k
(
x
,
i
) in the following operation, we can derive
(10)
E
k
[
V
k
+
1
(
x
k
+
1
,
θ
k
+
1
)
-
V
k
(
x
k
,
θ
k
)
]
=
E
k
x
k
+
1
′
P
θ
k
+
1
δ
T
1
>
k
+
1
+
G
θ
k
+
1
δ
{
T
1
=
k
+
1
}
x
k
+
1
vvvvvvi
-
x
k
′
P
θ
k
δ
T
1
≥
k
+
1
+
G
θ
k
δ
{
T
1
=
k
}
x
k
=
E
k
x
k
+
1
′
P
θ
k
+
1
x
k
+
1
-
x
k
′
P
θ
k
x
k
δ
{
T
1
>
k
+
1
}
+
E
k
x
k
+
1
′
G
θ
k
+
1
x
k
+
1
-
x
k
′
P
θ
k
x
k
δ
{
T
1
=
k
+
1
}
-
x
k
′
G
θ
k
x
k
δ
{
T
1
=
k
}
=
E
k
w
k
′
x
k
′
A
¯
θ
k
′
P
θ
k
+
1
A
¯
θ
k
x
k
w
k
-
x
k
′
P
θ
k
x
k
x
k
′
A
θ
k
′
P
θ
k
+
1
A
θ
k
x
k
+
x
k
′
A
θ
k
′
P
θ
k
+
1
A
¯
θ
k
x
k
w
k
vvvvvvv
+
w
k
′
x
k
′
A
¯
θ
k
′
P
θ
k
+
1
A
θ
k
x
k
vvvvvvv
+
w
k
′
x
k
′
A
¯
θ
k
′
P
θ
k
+
1
A
¯
θ
k
x
k
w
k
-
x
k
′
P
θ
k
x
k
δ
{
T
1
>
k
+
1
}
+
E
k
w
k
′
x
k
′
A
¯
θ
k
′
G
θ
k
+
1
A
¯
θ
k
x
k
w
k
x
k
′
A
θ
k
′
G
θ
k
+
1
A
θ
k
x
k
+
x
k
′
A
θ
k
′
G
θ
k
+
1
A
¯
θ
k
x
k
w
k
vvvvvvvvv
+
w
k
′
x
k
′
A
¯
θ
k
′
G
θ
k
+
1
A
θ
k
x
k
vvvvvvvvv
+
w
k
′
x
k
′
A
¯
θ
k
′
G
θ
k
+
1
A
¯
θ
k
x
k
w
k
-
x
k
′
P
θ
k
x
k
δ
T
1
=
k
+
1
-
x
k
′
G
θ
k
x
k
δ
{
T
1
=
k
}
.
We know that E
[
w
(
k
)
]
=
0 and E
[
w
(
i
)
w
(
j
)
]
=
δ
i
j, where δ
i
j
=
1 if i
=
j and δ
i
j
=
0 if i
≠
j. Calculating the expected values above, we can obtain that
(11)
E
k
V
k
+
1
x
k
+
1
,
θ
k
+
1
-
V
k
x
k
,
θ
k
=
x
k
′
p
θ
k
θ
k
A
θ
k
′
P
θ
k
A
θ
k
+
A
¯
θ
k
′
P
θ
k
A
¯
θ
k
+
A
θ
k
′
ε
θ
k
(
G
)
A
θ
k
vvvvvvv
+
A
¯
θ
k
′
ε
θ
k
G
A
¯
θ
k
-
P
θ
k
x
k
δ
T
1
>
k
-
x
k
′
G
θ
k
x
k
δ
T
1
=
k
,
due to P
(
T
1
=
k
+
1
∣
θ
k
)
=
1
-
p
θ
k
θ
k
,
P
(
T
1
>
k
+
1
∣
θ
k
)
=
p
θ
k
θ
k, and ε
θ
k
(
G
)
=
Σ
θ
k
+
1
≠
θ
k
p
θ
k
θ
k
+
1
G
θ
k
+
1.
The above relation can be rewritten as(12)
E
k
V
k
+
1
x
k
+
1
,
θ
k
+
1
-
V
k
x
k
,
θ
k
=
-
x
k
′
W
θ
k
δ
T
1
>
k
+
G
θ
k
δ
T
1
=
k
x
k
,
where
(13)
-
W
θ
k
=
p
θ
k
θ
k
A
θ
k
′
P
θ
k
A
θ
k
+
A
¯
θ
k
′
P
θ
k
A
¯
θ
k
+
A
θ
k
′
ε
θ
k
(
G
)
A
θ
k
+
A
¯
θ
k
′
ε
θ
k
(
G
)
A
¯
θ
k
-
P
θ
k
.
Now, let us observe that
(14)
∑
k
=
0
N
‍
E
0
V
k
+
1
x
k
+
1
,
θ
k
+
1
-
V
k
x
k
,
θ
k
=
∑
k
=
0
N
‍
E
0
E
k
V
k
+
1
x
k
+
1
,
θ
k
+
1
-
V
k
x
k
,
θ
k
.
By applying (12) and considering that W
i
>
0
,
G
i
>
0 for each initial condition x
0 and initial distribution μ, then we have
(15)
E
0
V
k
+
1
x
k
+
1
,
θ
k
+
1
-
V
0
x
0
,
θ
0
=
-
∑
k
=
0
N
‍
E
0
[
x
k
′
W
θ
k
δ
T
1
>
k
+
G
θ
k
δ
T
1
=
k
x
k
δ
{
T
1
≥
k
}
]
≤
-
∑
k
=
0
N
‍
γ
E
0
[
x
k
2
δ
{
T
1
≥
k
}
]
,
for some γ
>
0. Because E
0
[
V
k
(
x
k
,
θ
k
)
]
≥
0
,
∀
k
≥
0, then lim
⁡
k
→
∞
E
0
[
V
k
+
1
(
x
k
+
1
,
θ
k
+
1
)
]
=
0 by (5). Finally, it is easy to verify that, for any T
1,
(16)
limsup
⁡
N
→
∞
∑
k
=
0
N
‍
E
0
[
x
k
2
δ
{
T
1
≥
k
}
]
≤
1
γ
V
0
(
x
0
,
θ
0
)
<
∞
holds from (15). Therefore, for any T
1, MJSLS (1) is stable according to (i) of Definition 1.
Now, using an induction argument, we assume that for somen the inequality
(17)
limsup
⁡
N
→
∞
∑
k
=
0
N
‍
E
x
k
′
Q
θ
k
x
k
δ
T
n
≥
k
<
x
0
′
P
θ
0
x
0
holds and thus, by setting Q
≡
I, E
[
x
T
n
2
δ
{
T
n
≥
k
}
]
<
∞. However,
(18)
limsup
⁡
N
→
∞
∑
k
=
0
N
‍
E
[
x
k
′
Q
θ
k
x
k
δ
{
T
n
+
1
≥
k
}
]
=
limsup
⁡
N
→
∞
E
∑
k
=
T
n
N
‍
x
k
′
Q
θ
k
x
k
δ
T
n
≤
k
≤
T
n
+
1
∑
k
=
0
N
‍
x
k
′
Q
θ
k
x
k
δ
T
n
>
k
vvvvvvvvvvvv
+
∑
k
=
T
n
N
‍
x
k
′
Q
θ
k
x
k
δ
{
T
n
≤
k
≤
T
n
+
1
}
.
Notice that using the strong Markov property and the homogeneity property, the second term conditioned to the knowledge of (x
T
n
,
θ
T
n) can be written as
(19)
limsup
⁡
N
→
∞
E
∑
k
=
T
n
N
‍
x
k
′
Q
θ
k
x
k
δ
{
T
n
+
1
≥
k
}
∣
x
T
n
,
θ
T
n
=
limsup
⁡
N
→
∞
E
∑
k
=
0
N
-
T
n
‍
x
k
′
Q
θ
k
x
k
δ
{
k
≤
T
1
}
∣
x
0
=
x
T
n
,
θ
0
=
θ
T
n
<
x
T
n
′
P
θ
T
n
x
T
n
.
So one can conclude from (18) and (19) that
(20)
limsup
⁡
N
→
∞
∑
k
=
0
N
‍
E
[
x
k
′
Q
θ
k
x
k
δ
{
T
n
+
1
≥
k
}
]
<
limsup
⁡
N
→
∞
∑
k
=
0
N
‍
E
[
x
k
′
Q
θ
k
x
k
δ
T
n
>
k
+
x
T
n
′
P
θ
T
n
x
T
n
]
=
limsup
⁡
N
→
∞
∑
k
=
0
N
‍
E
[
x
k
′
Q
θ
k
x
k
δ
T
n
≥
k
+
x
T
n
′
(
P
θ
T
n
-
Q
θ
T
n
)
x
T
n
]
<
2
x
0
′
P
θ
0
x
0
.
Therefore, for any T
n,
(21)
limsup
⁡
N
→
∞
∑
k
=
0
N
‍
E
[
x
k
2
δ
{
T
n
≥
k
}
]
<
∞
indicate that the MJSLS (1) is T
n-stable.
Necessity. As in the previous part define the functional
(22)
x
0
′
P
θ
0
x
0
:
=
E
0
∑
k
=
0
∞
‍
x
k
′
W
θ
k
x
k
δ
T
1
>
k
+
x
T
1
′
G
θ
T
1
x
T
1
,
for all (
x
0
,
θ
0
)
∈
R
n
×
X. Therefore,
(23)
x
1
′
P
θ
1
x
1
=
E
T
1
∑
k
=
1
∞
‍
x
k
′
W
θ
k
x
k
δ
{
T
1
>
k
}
+
x
T
1
′
G
θ
T
1
x
T
1
δ
{
T
1
>
1
}
.
The right-hand side of (22) can be expressed as
(24)
E
0
x
0
′
W
θ
0
x
0
+
E
T
1
∑
k
=
1
∞
‍
x
k
′
W
θ
k
x
k
δ
T
1
>
k
vvvvvvvvvvvvvvvvv
+
x
T
1
′
G
θ
T
1
x
T
1
∑
k
=
1
∞
‍
δ
{
T
1
≥
1
}
.
In addition,
(25)
E
T
1
∑
k
=
1
∞
‍
x
k
′
W
θ
k
x
k
δ
{
T
1
>
k
}
+
x
T
1
′
G
θ
T
1
x
T
1
δ
{
T
1
≥
1
}
=
E
T
1
∑
k
=
1
∞
‍
x
k
′
W
θ
k
x
k
δ
{
T
1
>
k
}
+
x
T
1
′
G
θ
T
1
x
T
1
δ
{
T
1
>
1
}
vvvvvvvv
+
x
T
1
′
G
θ
T
1
x
T
1
δ
{
T
1
=
1
}
∑
k
=
1
∞
‍
x
k
′
W
θ
k
x
k
δ
T
1
>
k
+
x
T
1
′
G
θ
T
1
x
T
1
δ
T
1
>
1
.
Thus, based on the strong Markov property, applying homogeneity in (25) and introducing it in (24), we arrive at
(26)
x
0
′
P
θ
0
x
0
=
x
0
′
W
θ
0
x
0
+
E
0
x
1
′
P
θ
1
x
1
δ
T
1
>
1
+
x
T
1
′
G
θ
T
1
x
T
1
δ
{
T
1
=
1
}
.
Since x
0 is arbitrary, and calculating the expected values above, (26) implies that
(27)
p
i
i
A
i
′
P
i
A
i
+
A
¯
i
′
P
i
A
¯
i
-
P
i
+
A
i
′
ε
i
(
G
)
A
i
+
A
¯
i
′
ε
i
(
G
)
A
¯
i
=
-
W
i
,
using the fact that P
(
T
1
=
k
+
1
∣
θ
k
=
i
)
=
1
-
p
i
i and P
(
T
1
>
k
+
1
∣
θ
k
=
i
)
=
p
i
i. Thus, from the Lyapunov stability theory, the existence of the set L
i
>
0 satisfying (7) is guaranteed, completing the proof for n
=
1.
Now, for the general case, from the stochasticallyT
n-stable of the system we can obtain that
(28)
E
∑
k
=
0
∞
‍
x
k
′
W
θ
k
x
k
δ
{
T
n
>
k
}
+
x
T
n
′
G
θ
T
n
x
T
n
<
∞
.
And from the strong Markov property, we can deduce that
(29)
E
T
n
∑
k
=
T
n
∞
‍
x
k
′
W
θ
k
x
k
δ
T
n
+
1
>
k
+
x
T
n
+
1
′
G
θ
T
n
+
1
x
T
n
+
1
<
∞
,
for n
=
0,1
,
…
,
N
-
1. By the homogeneity property, it follows that (29) is equivalent to (22) with x
0
=
x
T
n and θ
0
=
θ
T
n, and the existence of a set of matrices L
i
>
0 satisfying (7) is assured. Then, the proof of Theorem 4 is completed.


## 3. LQ Differential Games for MJSLS with a Finite Number of Jump Times

### 3.1. Problem Formulation

Now we study the LQ differential games for discrete-time MJSLS. Comparing with system (1), consider the following discrete-time MJSLS with a finite number of jump times:
(30)
x
k
+
1
=
A
θ
k
x
(
k
)
+
B
θ
k
u
(
k
)
+
C
θ
k
v
(
k
)
+
[
A
¯
θ
k
x
(
k
)
+
B
¯
θ
k
u
(
k
)
+
C
¯
θ
k
v
(
k
)
]
w
(
k
)
,
x
(
0
)
=
x
0
∈
R
n
,
k
∈
N
,
y
τ
(
k
)
=
Q
θ
k
τ
x
(
k
)
,
τ
=
1,2
.
y
τ
(
k
)
∈
R
m are the measurement outputs for each player. Here (
u
(
k
)
,
v
(
k
)
)
∈
R
r
×
R
r represent the system control inputs. The matrices (
B
θ
k
,
B
¯
θ
k
,
C
θ
k
,
C
¯
θ
k
,
Q
θ
k
τ
)
∈
R
n
×
r
×
R
n
×
r
×
R
n
×
r
×
R
n
×
r
×
R
m
×
n (associated with “ith” mode) will be assigned as (
B
i
,
B
¯
i
,
C
i
,
C
¯
i
,
Q
i
τ
) for each θ
k
=
i
∈
X.Throughout this paper, we choose the infinite horizon quadratic cost functions associated with each player:(31)
J
τ
u
,
v
=
∑
k
=
0
∞
‍
E
x
k
′
Q
θ
k
τ
′
Q
θ
k
τ
x
(
k
)
vvvvvvvv
+
u
k
′
R
θ
k
τ
u
(
k
)
+
v
k
′
S
θ
k
τ
v
k
x
k
′
Q
θ
k
τ
′
Q
θ
k
τ
x
k
,
τ
=
1,2
.
The weighting matrices Q
θ
k
τ
=
Q
i
τ
≥
0
,
R
θ
k
τ
=
R
i
τ
>
0
∈
R
r
×
r, and S
θ
k
τ
=
S
i
τ
>
0
∈
R
r
×
r.So we are looking for actions that satisfy simultaneously(32)
J
1
(
u
*
,
v
*
)
≤
J
1
(
u
*
,
v
)
,
J
2
(
u
*
,
v
*
)
≤
J
2
(
u
,
v
*
)
,
where (
u
*
(
k
)
,
v
*
(
k
)
)
∈
L
2
(
∞
,
R
r
u
)
×
L
2
(
∞
,
R
r
v
).To ensure the finiteness of the infinite-time cost function, we restrain the admissible control set to the constant linear feedback strategies; that is,u
(
k
)
=
K
θ
k
1
x
(
k
), v
(
k
)
=
K
θ
k
2
x
(
k
), where K
θ
k
1 and K
θ
k
2 are constant matrices of appropriate dimensions, and K
θ
k
1
,
K
θ
k
2 belong to
(33)
K
∶
=
K
=
K
θ
k
1
,
K
θ
k
2
∣
system
30
can
be
stabilized
vvvvvvvvvvvvvvvvv
with
u
(
k
)
=
K
θ
k
1
x
(
k
)
,
vvvvvvvvvvvvvvvvv
v
(
t
)
=
K
θ
k
2
x
(
k
)
.We say that the optimization problem is well posed and theu
(
k
) and v
(
k
) have the following two additional properties:
(34)
E
u
k
2
<
∞
,
E
v
k
2
<
∞
,
k
∈
N
.The optimal strategiesu
* and v
* determined by (32) are also called the Nash equilibrium strategies u
*
,
v
*. In order to guarantee the unique global Nash game solutions, both the players are only allowed to take constant feedback controls. Next we focus on finding the optimal strategies.


### 3.2. Main Results

First, we give an important lemma that will be used later. If the system (1) is T
n-stable, we can obtain the following result for the discrete-time MJSLS (30).Lemma 5.
IfA
θ
k
,
A
¯
θ
k is T
n-stable, then so is A
θ
k
+
B
θ
k
K
θ
k
1
+
C
θ
k
K
θ
k
2
,
A
¯
θ
k
+
B
¯
θ
k
K
θ
k
1
+
C
¯
θ
k
K
θ
k
2, where (
K
θ
k
1
,
K
θ
k
2
)
∈
K.Proof.
Sufficiency. The proof employs an induction argument on the stopping times T
n. First, define the function
(35)
V
k
(
x
,
i
)
∶
=
x
k
′
(
P
i
δ
T
1
>
k
+
G
i
δ
{
T
1
=
k
}
)
x
k
,
where G
i
>
0, and P
i
>
0 is the solution of
(36)
p
i
i
(
A
i
′
P
i
A
i
+
A
¯
i
′
P
i
A
¯
i
)
-
P
i
+
W
i
+
A
i
′
ε
i
(
G
)
A
i
+
A
¯
i
′
ε
i
(
G
)
A
¯
i
=
0
,
i
∈
X
.
The existence of such P
i
>
0 relies on (7). Hence, to the function V
k
(
x
,
i
) and the system (30) in the following operation, we acquire that
(37)
E
k
[
V
k
+
1
(
x
(
k
+
1
)
,
θ
k
)
-
V
k
(
x
(
k
)
,
θ
k
)
]
=
x
k
′
p
θ
k
θ
k
A
θ
k
+
B
θ
k
K
θ
k
1
+
C
θ
k
K
θ
k
2
′
vvvvvvvvvvvvvv
×
P
θ
k
A
θ
k
+
B
θ
k
K
θ
k
1
+
C
θ
k
K
θ
k
2
vvvvvvvvvvvvvv
+
A
¯
θ
k
+
B
¯
θ
k
K
θ
k
1
+
C
¯
θ
k
K
θ
k
2
′
vvvvvvvvvvvvvv
×
P
θ
k
A
¯
θ
k
+
B
¯
θ
k
K
θ
k
1
+
C
¯
θ
k
K
θ
k
2
A
θ
k
+
B
θ
k
K
θ
k
1
+
C
θ
k
K
θ
k
2
′
vvvvvvvvvv
+
A
θ
k
+
B
θ
k
K
θ
k
1
+
C
θ
k
K
θ
k
2
′
ε
θ
k
G
vvvvvvvvvv
×
(
A
θ
k
+
B
θ
k
K
θ
k
1
+
C
θ
k
K
θ
k
2
)
vvvvvvvvvv
+
A
¯
θ
k
+
B
¯
θ
k
K
θ
k
1
+
C
¯
θ
k
K
θ
k
2
′
ε
θ
k
G
vvvvvvvvvv
×
A
¯
θ
k
+
B
¯
θ
k
K
θ
k
1
+
C
¯
θ
k
K
θ
k
2
vvvvvvvvvv
-
P
θ
k
p
θ
k
θ
k
A
θ
k
+
B
θ
k
K
θ
k
1
+
C
θ
k
K
θ
k
2
′
x
(
k
)
δ
T
1
>
k
-
x
k
′
G
θ
k
x
k
δ
T
1
=
k
.
Compared with (12), we know
(38)
E
k
V
k
+
1
x
k
+
1
,
θ
k
+
1
-
V
k
x
k
,
θ
k
=
-
x
k
′
W
θ
k
δ
T
n
>
k
+
G
θ
k
δ
T
n
=
k
x
k
,
where
(39)
-
W
θ
k
=
-
P
θ
k
+
p
θ
k
θ
k
A
θ
k
+
B
θ
k
K
θ
k
1
+
C
θ
k
K
θ
k
2
′
vvvvvvvvvvvv
×
P
θ
k
(
A
θ
k
+
B
θ
k
K
θ
k
1
+
C
θ
k
K
θ
k
2
)
vvvvvvvvvvvv
+
A
¯
θ
k
+
B
¯
θ
k
K
θ
k
1
+
C
¯
θ
k
K
θ
k
2
′
vvvvvvvvvvvv
×
P
θ
k
A
¯
θ
k
+
B
¯
θ
k
K
θ
k
1
+
C
¯
θ
k
K
θ
k
2
A
θ
k
+
B
θ
k
K
θ
k
1
+
C
θ
k
K
θ
k
2
′
+
A
θ
k
+
B
θ
k
K
θ
k
1
+
C
θ
k
K
θ
k
2
′
ε
θ
k
(
G
)
×
A
θ
k
+
B
θ
k
K
θ
k
1
+
C
θ
k
K
θ
k
2
+
A
¯
θ
k
+
B
¯
θ
k
K
θ
k
1
+
C
¯
θ
k
K
θ
k
2
′
ε
θ
k
G
×
A
¯
θ
k
+
B
¯
θ
k
K
θ
k
1
+
C
¯
θ
k
K
θ
k
2
.
Considering that W
θ
k
>
0
,
G
θ
k
>
0, we can obtain (19). And because E
[
V
k
(
x
(
k
)
,
θ
k
)
]
≥
0
,
∀
k
≥
0 for each initial condition x
0, from (40), it is easy to verify (21). Therefore, the MJSLS (30) is T
n-stable.Theorem 6.
For system (30), suppose the following coupled equations admit the solutions (
L
i
1
,
L
i
2
;
K
i
1
,
K
i
2
) with L
i
1
>
0, L
i
2
>
0:
(40)
-
L
i
1
+
p
i
i
A
i
+
B
i
K
i
1
′
L
i
1
A
i
+
B
i
K
i
1
vvvvvvvvv
+
A
¯
i
+
B
¯
i
K
i
1
′
L
i
1
A
¯
i
+
B
¯
i
K
i
1
+
Q
i
1
′
Q
i
1
,
+
K
i
1
′
R
i
1
K
i
1
-
K
i
3
′
H
i
1
L
i
1
-
1
K
i
3
=
0
,
H
i
1
L
i
1
>
0
,
(41)
K
i
1
=
-
H
i
2
L
i
2
-
1
K
i
4
,
(42)
-
L
i
2
+
p
i
i
A
i
+
C
i
K
i
2
′
L
i
2
A
i
+
C
i
K
i
2
vvvvvvvvv
+
A
¯
i
+
C
¯
i
K
i
2
′
L
i
2
A
¯
i
+
C
¯
i
K
i
2
+
Q
i
2
′
Q
i
2
+
K
i
2
′
S
i
2
K
i
2
-
K
i
4
′
H
i
2
L
i
2
-
1
K
i
4
=
0
,
H
i
2
L
i
2
>
0
,
(43)
K
i
2
=
-
H
i
1
L
i
1
-
1
K
i
3
,
where
(44)
H
i
1
(
L
i
1
)
=
S
i
1
+
p
i
i
C
i
′
L
i
1
C
i
+
C
¯
i
′
L
i
1
C
¯
i
,
K
i
3
=
p
i
i
C
i
′
L
i
1
A
i
+
B
i
K
i
1
+
C
¯
i
′
L
i
1
A
¯
i
+
B
¯
i
K
i
1
,
H
i
2
(
L
i
2
)
=
R
i
2
+
p
i
i
B
i
′
L
i
2
B
i
+
B
¯
i
′
L
i
2
B
¯
i
,
K
i
4
=
p
i
i
B
i
′
L
i
2
A
i
+
C
i
K
i
2
+
B
¯
i
′
L
i
2
A
¯
i
+
C
¯
i
K
i
2
.
If (
A
i
,
A
¯
i
) is T
n-stable, then(i)
(
K
i
1
,
K
i
2
)
∈
K;
(ii)
the problem of infinite horizon stochastic differential games admits a pair of solutions(
u
*
(
k
)
,
v
*
(
k
)
) with u
*
(
k
)
=
K
i
1
x
(
k
), v
*
(
k
)
=
K
i
2
x
(
k
);
(iii)
the optimal cost functions incurred by playing strategies(
u
*
(
k
)
,
v
*
(
k
)
) are J
τ
=
x
0
′
L
i
τ
x
0
(
τ
=
1,2
).Proof.
In the deduction of Lemma5, we can prove that (i) is correct. Next what we have to do is to prove (ii) and (iii). In the light of the Lyapunov equation (7) and any given set of matrices W
i in Theorem 4, it is easy to get the following equations for system (30):
(45)
p
i
i
A
i
+
B
i
K
i
1
+
C
i
K
i
2
′
L
i
1
A
i
+
B
i
K
i
1
+
C
i
K
i
2
vvv
+
A
¯
i
+
B
¯
i
K
i
1
+
C
¯
i
K
i
2
′
L
i
1
A
¯
i
+
B
¯
i
K
i
1
+
C
¯
i
K
i
2
+
Q
i
1
′
Q
i
1
+
K
i
1
′
R
i
1
K
i
1
+
K
i
2
′
S
i
1
K
i
2
=
L
i
1
,
S
i
1
+
p
i
i
C
i
′
L
i
1
C
i
+
C
¯
i
′
L
i
1
C
¯
i
>
0
,
(46)
p
i
i
A
¯
i
+
B
¯
i
K
i
1
+
C
¯
i
K
i
2
′
L
i
2
A
¯
i
+
B
¯
i
K
i
1
+
C
¯
i
K
i
2
A
i
+
B
i
K
i
1
+
C
i
K
i
2
′
L
i
2
(
A
i
+
B
i
K
i
1
+
C
i
K
i
2
)
vvv
+
A
¯
i
+
B
¯
i
K
i
1
+
C
¯
i
K
i
2
′
L
i
2
A
¯
i
+
B
¯
i
K
i
1
+
C
¯
i
K
i
2
+
Q
i
2
′
Q
i
2
+
K
i
1
′
R
i
2
K
i
1
+
K
i
2
′
S
i
2
K
i
2
=
L
i
2
,
R
i
2
+
p
i
i
B
i
′
L
i
2
B
i
+
B
¯
i
′
L
i
2
B
¯
i
>
0
.
By rearranging (45) and (46), (40) and (42) can be obtained, respectively.
Notingu
*
(
k
)
=
K
i
1
x
(
k
), and by substituting u
*
(
k
) into (30), it is easy to get the following system:
(47)
x
k
+
1
=
(
A
θ
k
+
B
θ
k
K
θ
k
1
)
x
(
k
)
+
C
θ
k
v
(
k
)
+
[
(
A
¯
θ
k
+
B
¯
θ
k
K
θ
k
1
)
x
(
k
)
+
C
¯
θ
k
v
(
k
)
]
w
(
k
)
,
x
0
=
x
0
∈
R
n
,
k
∈
N
.
Then, considering the scalar function Z
(
x
k
)
:
=
x
k
′
L
θ
k
1
x
k, we have
(48)
E
k
Δ
Z
x
k
=
E
k
Z
x
k
+
1
-
Z
x
k
=
E
k
[
x
k
+
1
′
L
θ
k
+
1
1
x
k
+
1
-
x
k
′
L
θ
k
1
x
k
]
=
E
k
-
x
k
′
L
θ
k
1
x
k
+
p
i
i
A
θ
k
+
B
θ
k
K
θ
k
1
x
k
+
C
θ
k
v
k
′
vvvvvvv
×
L
θ
k
1
A
θ
k
+
B
θ
k
K
θ
k
1
x
k
+
C
θ
k
v
k
vvvvvvv
+
p
i
i
A
¯
θ
k
+
B
¯
θ
k
K
θ
k
1
x
k
+
C
¯
θ
k
v
k
′
vvvvvvv
×
L
θ
k
1
A
¯
θ
k
+
B
¯
θ
k
K
θ
k
1
x
k
+
C
¯
θ
k
v
k
-
x
k
′
L
θ
k
1
x
k
+
p
i
i
A
θ
k
+
B
θ
k
K
θ
k
1
x
k
+
C
θ
k
v
k
′
.
Due to
(49)
∑
k
=
0
∞
‍
E
k
Δ
Z
x
k
=
E
k
∑
k
=
0
∞
‍
Δ
Z
x
k
=
E
k
Z
x
∞
-
Z
x
0
=
-
x
0
′
L
i
1
x
0
,
by (40) and a completing squares technique, (31) can be derived that
(50)
J
1
(
u
*
,
v
)
=
∑
k
=
0
∞
‍
E
k
x
k
′
Q
θ
k
1
′
Q
θ
k
1
+
K
θ
k
1
′
R
θ
k
1
K
θ
k
1
x
k
+
v
k
′
S
θ
k
1
v
k
+
∑
k
=
0
∞
‍
E
k
Δ
Z
x
k
+
x
0
′
L
i
1
x
0
=
x
0
′
L
i
1
x
0
+
∑
k
=
0
∞
‍
E
k
x
k
′
-
L
i
1
+
p
i
i
A
i
+
B
i
K
i
1
′
vvvvvvvvvvvvvvvv
×
L
i
1
A
i
+
B
i
K
i
1
+
p
i
i
A
¯
i
+
B
¯
i
K
i
1
′
vvvvvvvvvvvvvvvv
×
L
i
1
A
¯
i
+
B
¯
i
K
i
1
+
Q
i
1
′
Q
i
1
vvvvvvvvvvvvvvvv
+
K
i
1
′
R
i
1
K
i
1
x
k
+
x
k
′
K
i
3
′
v
k
+
v
k
′
K
i
3
x
k
vvvvvvvvvvvvv
+
v
k
′
(
S
i
1
+
p
i
i
C
i
′
L
i
1
C
i
+
p
i
i
C
¯
i
′
L
i
1
C
¯
i
)
v
k
=
x
0
′
L
i
1
x
0
+
∑
k
=
0
∞
‍
E
k
x
k
′
K
i
3
′
H
i
1
L
i
1
-
1
K
i
3
x
k
vvvvvvvvvvvvvvvvi
+
x
k
′
K
i
3
′
v
k
+
v
k
′
K
i
3
x
k
+
v
k
′
H
i
1
L
i
1
v
k
x
k
′
K
i
3
′
H
i
1
L
i
1
-
1
K
i
3
x
k
=
x
0
′
L
i
1
x
0
+
∑
k
=
0
∞
‍
E
k
v
k
-
K
i
2
x
k
′
H
i
1
L
i
1
v
k
-
K
i
2
x
k
≥
x
0
′
L
i
1
x
0
,
τ
=
1
.
Then by (32), it follows that v
*
(
k
)
=
K
i
2
x
(
k
) and J
1
(
u
*
,
v
*
)
=
x
0
′
L
i
1
x
0. Finally, by substituting v
*
(
k
) into (30), in the same way as before, we have u
*
(
k
)
=
K
i
1
x
(
k
) and J
2
(
u
*
,
v
*
)
=
x
0
′
L
i
2
x
0.Theorem 7.
If(
A
i
,
A
¯
i
) is T
n-stable, and, for system (30), assume that (40)–(43) admit the solution (
L
i
1
,
L
i
2
;
K
i
1
,
K
i
2
) with (
K
i
1
,
K
i
2
)
∈
K, then(i)
L
i
1
>
0, L
i
2
>
0,
(ii)
the problem of infinite horizon stochastic differential games admits a pair of solutions(
u
*
(
k
)
,
v
*
(
k
)
) with u
*
(
k
)
=
K
i
1
x
(
k
), v
*
(
k
)
=
K
i
2
x
(
k
),
(iii)
the optimal cost functions incurred by playing strategies(
u
*
(
k
)
,
v
*
(
k
)
) are J
τ
=
x
0
′
L
i
τ
x
0
(
τ
=
1,2
).Remark 8.
Whenw
(
k
)
≡
0, these results still hold in the paper. Only for the reason of simplicity, in (1) and (30), we assume the state x
(
t
) and control inputs (
u
(
t
)
,
v
(
t
)
) depend on the same noise w
(
k
). If they rely on the different noises w
1
(
k
)
,
w
2
(
k
), then new results will be yielded. The discussion is omitted.


## 3.1. Problem Formulation

Now we study the LQ differential games for discrete-time MJSLS. Comparing with system (1), consider the following discrete-time MJSLS with a finite number of jump times:
(30)
x
k
+
1
=
A
θ
k
x
(
k
)
+
B
θ
k
u
(
k
)
+
C
θ
k
v
(
k
)
+
[
A
¯
θ
k
x
(
k
)
+
B
¯
θ
k
u
(
k
)
+
C
¯
θ
k
v
(
k
)
]
w
(
k
)
,
x
(
0
)
=
x
0
∈
R
n
,
k
∈
N
,
y
τ
(
k
)
=
Q
θ
k
τ
x
(
k
)
,
τ
=
1,2
.
y
τ
(
k
)
∈
R
m are the measurement outputs for each player. Here (
u
(
k
)
,
v
(
k
)
)
∈
R
r
×
R
r represent the system control inputs. The matrices (
B
θ
k
,
B
¯
θ
k
,
C
θ
k
,
C
¯
θ
k
,
Q
θ
k
τ
)
∈
R
n
×
r
×
R
n
×
r
×
R
n
×
r
×
R
n
×
r
×
R
m
×
n (associated with “ith” mode) will be assigned as (
B
i
,
B
¯
i
,
C
i
,
C
¯
i
,
Q
i
τ
) for each θ
k
=
i
∈
X.Throughout this paper, we choose the infinite horizon quadratic cost functions associated with each player:(31)
J
τ
u
,
v
=
∑
k
=
0
∞
‍
E
x
k
′
Q
θ
k
τ
′
Q
θ
k
τ
x
(
k
)
vvvvvvvv
+
u
k
′
R
θ
k
τ
u
(
k
)
+
v
k
′
S
θ
k
τ
v
k
x
k
′
Q
θ
k
τ
′
Q
θ
k
τ
x
k
,
τ
=
1,2
.
The weighting matrices Q
θ
k
τ
=
Q
i
τ
≥
0
,
R
θ
k
τ
=
R
i
τ
>
0
∈
R
r
×
r, and S
θ
k
τ
=
S
i
τ
>
0
∈
R
r
×
r.So we are looking for actions that satisfy simultaneously(32)
J
1
(
u
*
,
v
*
)
≤
J
1
(
u
*
,
v
)
,
J
2
(
u
*
,
v
*
)
≤
J
2
(
u
,
v
*
)
,
where (
u
*
(
k
)
,
v
*
(
k
)
)
∈
L
2
(
∞
,
R
r
u
)
×
L
2
(
∞
,
R
r
v
).To ensure the finiteness of the infinite-time cost function, we restrain the admissible control set to the constant linear feedback strategies; that is,u
(
k
)
=
K
θ
k
1
x
(
k
), v
(
k
)
=
K
θ
k
2
x
(
k
), where K
θ
k
1 and K
θ
k
2 are constant matrices of appropriate dimensions, and K
θ
k
1
,
K
θ
k
2 belong to
(33)
K
∶
=
K
=
K
θ
k
1
,
K
θ
k
2
∣
system
30
can
be
stabilized
vvvvvvvvvvvvvvvvv
with
u
(
k
)
=
K
θ
k
1
x
(
k
)
,
vvvvvvvvvvvvvvvvv
v
(
t
)
=
K
θ
k
2
x
(
k
)
.We say that the optimization problem is well posed and theu
(
k
) and v
(
k
) have the following two additional properties:
(34)
E
u
k
2
<
∞
,
E
v
k
2
<
∞
,
k
∈
N
.The optimal strategiesu
* and v
* determined by (32) are also called the Nash equilibrium strategies u
*
,
v
*. In order to guarantee the unique global Nash game solutions, both the players are only allowed to take constant feedback controls. Next we focus on finding the optimal strategies.


## 3.2. Main Results

First, we give an important lemma that will be used later. If the system (1) is T
n-stable, we can obtain the following result for the discrete-time MJSLS (30).Lemma 5.
IfA
θ
k
,
A
¯
θ
k is T
n-stable, then so is A
θ
k
+
B
θ
k
K
θ
k
1
+
C
θ
k
K
θ
k
2
,
A
¯
θ
k
+
B
¯
θ
k
K
θ
k
1
+
C
¯
θ
k
K
θ
k
2, where (
K
θ
k
1
,
K
θ
k
2
)
∈
K.Proof.
Sufficiency. The proof employs an induction argument on the stopping times T
n. First, define the function
(35)
V
k
(
x
,
i
)
∶
=
x
k
′
(
P
i
δ
T
1
>
k
+
G
i
δ
{
T
1
=
k
}
)
x
k
,
where G
i
>
0, and P
i
>
0 is the solution of
(36)
p
i
i
(
A
i
′
P
i
A
i
+
A
¯
i
′
P
i
A
¯
i
)
-
P
i
+
W
i
+
A
i
′
ε
i
(
G
)
A
i
+
A
¯
i
′
ε
i
(
G
)
A
¯
i
=
0
,
i
∈
X
.
The existence of such P
i
>
0 relies on (7). Hence, to the function V
k
(
x
,
i
) and the system (30) in the following operation, we acquire that
(37)
E
k
[
V
k
+
1
(
x
(
k
+
1
)
,
θ
k
)
-
V
k
(
x
(
k
)
,
θ
k
)
]
=
x
k
′
p
θ
k
θ
k
A
θ
k
+
B
θ
k
K
θ
k
1
+
C
θ
k
K
θ
k
2
′
vvvvvvvvvvvvvv
×
P
θ
k
A
θ
k
+
B
θ
k
K
θ
k
1
+
C
θ
k
K
θ
k
2
vvvvvvvvvvvvvv
+
A
¯
θ
k
+
B
¯
θ
k
K
θ
k
1
+
C
¯
θ
k
K
θ
k
2
′
vvvvvvvvvvvvvv
×
P
θ
k
A
¯
θ
k
+
B
¯
θ
k
K
θ
k
1
+
C
¯
θ
k
K
θ
k
2
A
θ
k
+
B
θ
k
K
θ
k
1
+
C
θ
k
K
θ
k
2
′
vvvvvvvvvv
+
A
θ
k
+
B
θ
k
K
θ
k
1
+
C
θ
k
K
θ
k
2
′
ε
θ
k
G
vvvvvvvvvv
×
(
A
θ
k
+
B
θ
k
K
θ
k
1
+
C
θ
k
K
θ
k
2
)
vvvvvvvvvv
+
A
¯
θ
k
+
B
¯
θ
k
K
θ
k
1
+
C
¯
θ
k
K
θ
k
2
′
ε
θ
k
G
vvvvvvvvvv
×
A
¯
θ
k
+
B
¯
θ
k
K
θ
k
1
+
C
¯
θ
k
K
θ
k
2
vvvvvvvvvv
-
P
θ
k
p
θ
k
θ
k
A
θ
k
+
B
θ
k
K
θ
k
1
+
C
θ
k
K
θ
k
2
′
x
(
k
)
δ
T
1
>
k
-
x
k
′
G
θ
k
x
k
δ
T
1
=
k
.
Compared with (12), we know
(38)
E
k
V
k
+
1
x
k
+
1
,
θ
k
+
1
-
V
k
x
k
,
θ
k
=
-
x
k
′
W
θ
k
δ
T
n
>
k
+
G
θ
k
δ
T
n
=
k
x
k
,
where
(39)
-
W
θ
k
=
-
P
θ
k
+
p
θ
k
θ
k
A
θ
k
+
B
θ
k
K
θ
k
1
+
C
θ
k
K
θ
k
2
′
vvvvvvvvvvvv
×
P
θ
k
(
A
θ
k
+
B
θ
k
K
θ
k
1
+
C
θ
k
K
θ
k
2
)
vvvvvvvvvvvv
+
A
¯
θ
k
+
B
¯
θ
k
K
θ
k
1
+
C
¯
θ
k
K
θ
k
2
′
vvvvvvvvvvvv
×
P
θ
k
A
¯
θ
k
+
B
¯
θ
k
K
θ
k
1
+
C
¯
θ
k
K
θ
k
2
A
θ
k
+
B
θ
k
K
θ
k
1
+
C
θ
k
K
θ
k
2
′
+
A
θ
k
+
B
θ
k
K
θ
k
1
+
C
θ
k
K
θ
k
2
′
ε
θ
k
(
G
)
×
A
θ
k
+
B
θ
k
K
θ
k
1
+
C
θ
k
K
θ
k
2
+
A
¯
θ
k
+
B
¯
θ
k
K
θ
k
1
+
C
¯
θ
k
K
θ
k
2
′
ε
θ
k
G
×
A
¯
θ
k
+
B
¯
θ
k
K
θ
k
1
+
C
¯
θ
k
K
θ
k
2
.
Considering that W
θ
k
>
0
,
G
θ
k
>
0, we can obtain (19). And because E
[
V
k
(
x
(
k
)
,
θ
k
)
]
≥
0
,
∀
k
≥
0 for each initial condition x
0, from (40), it is easy to verify (21). Therefore, the MJSLS (30) is T
n-stable.Theorem 6.
For system (30), suppose the following coupled equations admit the solutions (
L
i
1
,
L
i
2
;
K
i
1
,
K
i
2
) with L
i
1
>
0, L
i
2
>
0:
(40)
-
L
i
1
+
p
i
i
A
i
+
B
i
K
i
1
′
L
i
1
A
i
+
B
i
K
i
1
vvvvvvvvv
+
A
¯
i
+
B
¯
i
K
i
1
′
L
i
1
A
¯
i
+
B
¯
i
K
i
1
+
Q
i
1
′
Q
i
1
,
+
K
i
1
′
R
i
1
K
i
1
-
K
i
3
′
H
i
1
L
i
1
-
1
K
i
3
=
0
,
H
i
1
L
i
1
>
0
,
(41)
K
i
1
=
-
H
i
2
L
i
2
-
1
K
i
4
,
(42)
-
L
i
2
+
p
i
i
A
i
+
C
i
K
i
2
′
L
i
2
A
i
+
C
i
K
i
2
vvvvvvvvv
+
A
¯
i
+
C
¯
i
K
i
2
′
L
i
2
A
¯
i
+
C
¯
i
K
i
2
+
Q
i
2
′
Q
i
2
+
K
i
2
′
S
i
2
K
i
2
-
K
i
4
′
H
i
2
L
i
2
-
1
K
i
4
=
0
,
H
i
2
L
i
2
>
0
,
(43)
K
i
2
=
-
H
i
1
L
i
1
-
1
K
i
3
,
where
(44)
H
i
1
(
L
i
1
)
=
S
i
1
+
p
i
i
C
i
′
L
i
1
C
i
+
C
¯
i
′
L
i
1
C
¯
i
,
K
i
3
=
p
i
i
C
i
′
L
i
1
A
i
+
B
i
K
i
1
+
C
¯
i
′
L
i
1
A
¯
i
+
B
¯
i
K
i
1
,
H
i
2
(
L
i
2
)
=
R
i
2
+
p
i
i
B
i
′
L
i
2
B
i
+
B
¯
i
′
L
i
2
B
¯
i
,
K
i
4
=
p
i
i
B
i
′
L
i
2
A
i
+
C
i
K
i
2
+
B
¯
i
′
L
i
2
A
¯
i
+
C
¯
i
K
i
2
.
If (
A
i
,
A
¯
i
) is T
n-stable, then(i)
(
K
i
1
,
K
i
2
)
∈
K;
(ii)
the problem of infinite horizon stochastic differential games admits a pair of solutions(
u
*
(
k
)
,
v
*
(
k
)
) with u
*
(
k
)
=
K
i
1
x
(
k
), v
*
(
k
)
=
K
i
2
x
(
k
);
(iii)
the optimal cost functions incurred by playing strategies(
u
*
(
k
)
,
v
*
(
k
)
) are J
τ
=
x
0
′
L
i
τ
x
0
(
τ
=
1,2
).Proof.
In the deduction of Lemma5, we can prove that (i) is correct. Next what we have to do is to prove (ii) and (iii). In the light of the Lyapunov equation (7) and any given set of matrices W
i in Theorem 4, it is easy to get the following equations for system (30):
(45)
p
i
i
A
i
+
B
i
K
i
1
+
C
i
K
i
2
′
L
i
1
A
i
+
B
i
K
i
1
+
C
i
K
i
2
vvv
+
A
¯
i
+
B
¯
i
K
i
1
+
C
¯
i
K
i
2
′
L
i
1
A
¯
i
+
B
¯
i
K
i
1
+
C
¯
i
K
i
2
+
Q
i
1
′
Q
i
1
+
K
i
1
′
R
i
1
K
i
1
+
K
i
2
′
S
i
1
K
i
2
=
L
i
1
,
S
i
1
+
p
i
i
C
i
′
L
i
1
C
i
+
C
¯
i
′
L
i
1
C
¯
i
>
0
,
(46)
p
i
i
A
¯
i
+
B
¯
i
K
i
1
+
C
¯
i
K
i
2
′
L
i
2
A
¯
i
+
B
¯
i
K
i
1
+
C
¯
i
K
i
2
A
i
+
B
i
K
i
1
+
C
i
K
i
2
′
L
i
2
(
A
i
+
B
i
K
i
1
+
C
i
K
i
2
)
vvv
+
A
¯
i
+
B
¯
i
K
i
1
+
C
¯
i
K
i
2
′
L
i
2
A
¯
i
+
B
¯
i
K
i
1
+
C
¯
i
K
i
2
+
Q
i
2
′
Q
i
2
+
K
i
1
′
R
i
2
K
i
1
+
K
i
2
′
S
i
2
K
i
2
=
L
i
2
,
R
i
2
+
p
i
i
B
i
′
L
i
2
B
i
+
B
¯
i
′
L
i
2
B
¯
i
>
0
.
By rearranging (45) and (46), (40) and (42) can be obtained, respectively.
Notingu
*
(
k
)
=
K
i
1
x
(
k
), and by substituting u
*
(
k
) into (30), it is easy to get the following system:
(47)
x
k
+
1
=
(
A
θ
k
+
B
θ
k
K
θ
k
1
)
x
(
k
)
+
C
θ
k
v
(
k
)
+
[
(
A
¯
θ
k
+
B
¯
θ
k
K
θ
k
1
)
x
(
k
)
+
C
¯
θ
k
v
(
k
)
]
w
(
k
)
,
x
0
=
x
0
∈
R
n
,
k
∈
N
.
Then, considering the scalar function Z
(
x
k
)
:
=
x
k
′
L
θ
k
1
x
k, we have
(48)
E
k
Δ
Z
x
k
=
E
k
Z
x
k
+
1
-
Z
x
k
=
E
k
[
x
k
+
1
′
L
θ
k
+
1
1
x
k
+
1
-
x
k
′
L
θ
k
1
x
k
]
=
E
k
-
x
k
′
L
θ
k
1
x
k
+
p
i
i
A
θ
k
+
B
θ
k
K
θ
k
1
x
k
+
C
θ
k
v
k
′
vvvvvvv
×
L
θ
k
1
A
θ
k
+
B
θ
k
K
θ
k
1
x
k
+
C
θ
k
v
k
vvvvvvv
+
p
i
i
A
¯
θ
k
+
B
¯
θ
k
K
θ
k
1
x
k
+
C
¯
θ
k
v
k
′
vvvvvvv
×
L
θ
k
1
A
¯
θ
k
+
B
¯
θ
k
K
θ
k
1
x
k
+
C
¯
θ
k
v
k
-
x
k
′
L
θ
k
1
x
k
+
p
i
i
A
θ
k
+
B
θ
k
K
θ
k
1
x
k
+
C
θ
k
v
k
′
.
Due to
(49)
∑
k
=
0
∞
‍
E
k
Δ
Z
x
k
=
E
k
∑
k
=
0
∞
‍
Δ
Z
x
k
=
E
k
Z
x
∞
-
Z
x
0
=
-
x
0
′
L
i
1
x
0
,
by (40) and a completing squares technique, (31) can be derived that
(50)
J
1
(
u
*
,
v
)
=
∑
k
=
0
∞
‍
E
k
x
k
′
Q
θ
k
1
′
Q
θ
k
1
+
K
θ
k
1
′
R
θ
k
1
K
θ
k
1
x
k
+
v
k
′
S
θ
k
1
v
k
+
∑
k
=
0
∞
‍
E
k
Δ
Z
x
k
+
x
0
′
L
i
1
x
0
=
x
0
′
L
i
1
x
0
+
∑
k
=
0
∞
‍
E
k
x
k
′
-
L
i
1
+
p
i
i
A
i
+
B
i
K
i
1
′
vvvvvvvvvvvvvvvv
×
L
i
1
A
i
+
B
i
K
i
1
+
p
i
i
A
¯
i
+
B
¯
i
K
i
1
′
vvvvvvvvvvvvvvvv
×
L
i
1
A
¯
i
+
B
¯
i
K
i
1
+
Q
i
1
′
Q
i
1
vvvvvvvvvvvvvvvv
+
K
i
1
′
R
i
1
K
i
1
x
k
+
x
k
′
K
i
3
′
v
k
+
v
k
′
K
i
3
x
k
vvvvvvvvvvvvv
+
v
k
′
(
S
i
1
+
p
i
i
C
i
′
L
i
1
C
i
+
p
i
i
C
¯
i
′
L
i
1
C
¯
i
)
v
k
=
x
0
′
L
i
1
x
0
+
∑
k
=
0
∞
‍
E
k
x
k
′
K
i
3
′
H
i
1
L
i
1
-
1
K
i
3
x
k
vvvvvvvvvvvvvvvvi
+
x
k
′
K
i
3
′
v
k
+
v
k
′
K
i
3
x
k
+
v
k
′
H
i
1
L
i
1
v
k
x
k
′
K
i
3
′
H
i
1
L
i
1
-
1
K
i
3
x
k
=
x
0
′
L
i
1
x
0
+
∑
k
=
0
∞
‍
E
k
v
k
-
K
i
2
x
k
′
H
i
1
L
i
1
v
k
-
K
i
2
x
k
≥
x
0
′
L
i
1
x
0
,
τ
=
1
.
Then by (32), it follows that v
*
(
k
)
=
K
i
2
x
(
k
) and J
1
(
u
*
,
v
*
)
=
x
0
′
L
i
1
x
0. Finally, by substituting v
*
(
k
) into (30), in the same way as before, we have u
*
(
k
)
=
K
i
1
x
(
k
) and J
2
(
u
*
,
v
*
)
=
x
0
′
L
i
2
x
0.Theorem 7.
If(
A
i
,
A
¯
i
) is T
n-stable, and, for system (30), assume that (40)–(43) admit the solution (
L
i
1
,
L
i
2
;
K
i
1
,
K
i
2
) with (
K
i
1
,
K
i
2
)
∈
K, then(i)
L
i
1
>
0, L
i
2
>
0,
(ii)
the problem of infinite horizon stochastic differential games admits a pair of solutions(
u
*
(
k
)
,
v
*
(
k
)
) with u
*
(
k
)
=
K
i
1
x
(
k
), v
*
(
k
)
=
K
i
2
x
(
k
),
(iii)
the optimal cost functions incurred by playing strategies(
u
*
(
k
)
,
v
*
(
k
)
) are J
τ
=
x
0
′
L
i
τ
x
0
(
τ
=
1,2
).Remark 8.
Whenw
(
k
)
≡
0, these results still hold in the paper. Only for the reason of simplicity, in (1) and (30), we assume the state x
(
t
) and control inputs (
u
(
t
)
,
v
(
t
)
) depend on the same noise w
(
k
). If they rely on the different noises w
1
(
k
)
,
w
2
(
k
), then new results will be yielded. The discussion is omitted.


## 4. Iterative Algorithm and Simulation

### 4.1. An Iterative Algorithm

In this section, an iterative algorithm is proposed to solve the four coupled GAREs (40)–(43). Infinite horizon Riccati equations are hard to be solved; hence the particular problems can be solved via finite horizon equations. N represents the finite number of iterations in the following equations:
(51)
L
i
1
N
k
=
p
i
i
A
i
+
B
i
K
i
1
N
k
′
L
i
1
N
k
+
1
×
A
i
+
B
i
K
i
1
N
k
+
p
i
i
A
¯
i
+
B
¯
i
K
i
1
N
k
′
×
L
i
1
N
k
+
1
A
¯
i
+
B
¯
i
K
i
1
N
k
+
Q
i
1
′
Q
i
1
+
K
i
1
N
k
′
R
i
1
K
i
1
N
k
-
K
i
3
N
k
′
H
i
1
L
i
1
N
k
+
1
-
1
K
i
3
N
(
k
)
,
L
i
1
N
(
k
+
1
)
=
0
,
H
i
1
L
i
1
N
k
+
1
>
0
,
(52)
K
i
1
N
(
k
)
=
-
H
i
2
L
i
2
N
k
+
1
-
1
K
i
4
N
(
k
)
,
(53)
L
i
2
N
k
=
p
i
i
A
i
+
C
i
K
i
2
N
k
′
L
i
2
N
(
k
+
1
)
×
A
i
+
C
i
K
i
2
N
k
+
p
i
i
A
¯
i
+
C
¯
i
K
i
2
N
k
′
×
L
i
2
N
k
+
1
A
¯
i
+
C
¯
i
K
i
2
N
k
+
Q
i
2
′
Q
i
2
+
K
i
2
N
k
′
S
i
2
K
i
2
N
k
-
K
i
4
N
k
′
H
i
2
L
i
2
N
k
+
1
-
1
K
i
4
N
k
,
L
i
2
N
k
+
1
=
0
,
H
i
2
L
i
2
N
k
+
1
>
0
,
(54)
K
i
2
N
(
k
)
=
-
H
i
1
L
i
1
N
k
+
1
-
1
K
i
3
N
(
k
)
,
where
(55)
H
i
1
L
i
1
N
k
+
1
=
S
i
1
+
p
i
i
C
i
′
L
i
1
N
k
+
1
C
i
vvvvvvvvvvv
+
C
¯
i
′
L
i
1
N
k
+
1
C
¯
i
,
H
i
2
(
L
i
2
N
(
k
+
1
)
)
=
R
i
2
+
p
i
i
B
i
′
L
i
2
N
k
+
1
B
i
vvvvvvvvvvv
+
B
¯
i
′
L
i
2
N
k
+
1
B
¯
i
,
K
i
3
N
k
=
p
i
i
C
i
′
L
i
1
N
k
+
1
A
i
+
B
i
K
i
1
N
k
+
1
vvvvv
+
C
¯
i
′
L
i
1
N
k
+
1
A
¯
i
+
B
¯
i
K
i
1
N
k
+
1
,
K
i
4
N
k
=
p
i
i
B
i
′
L
i
2
N
k
+
1
A
i
+
C
i
K
i
2
N
k
+
1
vvvvv
+
B
¯
i
′
L
i
2
N
k
+
1
A
¯
i
+
C
¯
i
K
i
2
N
k
+
1
.
An iterative process for solving (40)–(43) based on the above recursions is presented as follows.(a)
Given appropriate natural numberN and the initial conditions L
i
1
N
(
N
+
1
)
=
0, L
i
2
N
(
N
+
1
)
=
0, K
i
1
N
(
N
+
1
)
=
0, and K
i
2
N
(
N
+
1
)
=
0.
(b)
Through the numerical values ofL
i
1
N
(
N
+
1
), L
i
2
N
(
N
+
1
), K
i
1
N
(
N
+
1
), and K
i
2
N
(
N
+
1
), we have H
i
1
(
L
i
1
N
(
N
+
1
)
), H
i
2
(
L
i
2
N
(
N
+
1
)
), K
i
3
N
(
N
), and K
i
4
N
(
N
) according to (55).
(c)
K
i
1
N
(
N
) and K
i
2
N
(
N
) can be, respectively, computed by (52) and (54). Then, L
i
1
N
(
N
) and L
i
2
N
(
N
) can also be, respectively, obtained by (51) and (53).
(d)
LetL
i
1
N
(
N
+
1
)
=
L
i
1
N
(
N
), L
i
2
N
(
N
+
1
)
=
L
i
2
N
(
N
), K
i
1
N
(
N
+
1
)
=
K
i
1
N
(
N
), and K
i
2
N
(
N
+
1
)
=
K
i
2
N
(
N
).
(e)
Then,N
:
=
N
-
1. Repeat steps (b)–(d) until the number of iterations is N
+
1. We can finally obtain the numerical values of L
i
1
N
(
0
), L
i
2
N
(
0
), K
i
1
N
(
0
), and K
i
2
N
(
0
).As in [28], under the assumptions of stabilizability, for any x
0
∈
R
n,
(56)
lim
⁡
N
→
∞
x
0
′
L
i
1
N
0
x
0
=
lim
⁡
N
→
∞
min
⁡
J
1
N
u
N
*
,
v
=
min
⁡
J
1
∞
u
*
,
v
=
x
0
′
L
i
1
x
0
,
lim
⁡
N
→
∞
x
0
′
L
i
2
N
(
0
)
x
0
=
lim
⁡
N
→
∞
min
⁡
J
2
N
(
u
,
v
N
*
)
=
min
⁡
J
2
∞
(
u
,
v
*
)
=
x
0
′
L
i
2
x
0
,
lim
⁡
N
→
∞
K
i
1
N
(
0
)
=
K
i
1
,
lim
⁡
N
→
∞
K
i
2
N
(
0
)
=
K
i
2
.
Therefore,
(57)
lim
⁡
N
→
∞
L
i
1
N
0
,
L
i
2
N
0
;
K
i
1
N
0
,
K
i
2
N
0
=
L
i
1
,
L
i
2
;
K
i
1
,
K
i
2
,
where (
L
i
1
,
L
i
2
;
K
i
1
,
K
i
2
) are the solutions of (40)–(43).


### 4.2. A Simulation Example

To verify the efficiency of the above iterative algorithm, we consider the following 2-D example. In the system (30), we set θ
k
=
i
∈
X
=
{
1,2
}, R
i
τ
=
S
i
τ
=
1
(
τ
=
1,2
),
(58)
A
1
=
0.65
0
0
0.9
,
A
¯
1
=
0.45
0
0
0.55
,
B
1
=
0.6
0.55
,
B
¯
1
=
0.45
0.85
,
C
1
=
0.75
0.55
,
C
¯
1
=
0.5
0.85
,
Q
1
1
=
0.55
0
0
0.65
,
Q
1
2
=
0.75
0
0
0.25
,
A
2
=
0.75
0
0
0.7
,
A
¯
2
=
0.35
0
0
0.45
,
B
2
=
0.5
0.45
,
B
¯
2
=
0.55
0.85
,
C
2
=
0.65
0.55
,
C
¯
2
=
0.4
0.85
,
Q
2
1
=
0.35
0
0
0.45
,
Q
2
2
=
0.55
0
0
0.35
.
For convenience, let p
11
=
0.4, p
22
=
0.5, and N
=
50. When θ
k
=
1, by applying the above iterative algorithm, we obtain the solutions of the four coupled equations (51)–(54) as follows:
(59)
L
1
1
N
0
=
L
1
1
1,1
L
1
1
1,2
L
1
1
2,1
L
1
1
2,2
=
0.4023
-
0.0588
-
0.0588
0.6820
,
L
1
2
N
0
=
L
1
2
1,1
L
1
2
1,2
L
1
2
2,1
L
1
2
2,2
=
0.7111
-
0.0331
-
0.0331
0.1487
,
K
1
1
N
(
0
)
=
K
1
1
(
1,1
)
K
1
1
(
1,2
)
=
-
0.1245
-
0.0053
,
K
1
2
N
0
=
K
1
2
1,1
K
1
2
1,2
=
-
0.0390
-
0.1739
.
(
L
1
1
N
(
0
)
,
L
1
2
N
(
0
)
;
K
1
1
N
(
0
)
,
K
1
2
N
(
0
)
) are also the solutions of (40)–(43) according to (57). By the solutions, it shows that L
1
1
>
0 and L
1
2
>
0. The evolution of L
1
1
N
k
,
L
1
2
N
k
;
K
1
1
N
k
,
K
1
2
N
k is exhibited in Figures 1 and 2. And the figures clearly illustrate the convergence and speediness of the backward iterations. When θ
k
=
2, it is easy to get L
2
1
N
0
,
L
2
2
N
0
;
K
2
1
N
0
,
K
2
2
N
0 that are also the solutions of (40)–(43). And L
2
1
>
0 and L
2
2
>
0. Because it is the same as the above process θ
k
=
1, we do not introduce it again due to space limitations.Figure 1
Evolution ofL
1
1
N
(
k
) and L
1
2
N
(
k
).Figure 2
Evolution ofK
1
1
N
(
k
) and K
1
2
N
(
k
).


## 4.1. An Iterative Algorithm

In this section, an iterative algorithm is proposed to solve the four coupled GAREs (40)–(43). Infinite horizon Riccati equations are hard to be solved; hence the particular problems can be solved via finite horizon equations. N represents the finite number of iterations in the following equations:
(51)
L
i
1
N
k
=
p
i
i
A
i
+
B
i
K
i
1
N
k
′
L
i
1
N
k
+
1
×
A
i
+
B
i
K
i
1
N
k
+
p
i
i
A
¯
i
+
B
¯
i
K
i
1
N
k
′
×
L
i
1
N
k
+
1
A
¯
i
+
B
¯
i
K
i
1
N
k
+
Q
i
1
′
Q
i
1
+
K
i
1
N
k
′
R
i
1
K
i
1
N
k
-
K
i
3
N
k
′
H
i
1
L
i
1
N
k
+
1
-
1
K
i
3
N
(
k
)
,
L
i
1
N
(
k
+
1
)
=
0
,
H
i
1
L
i
1
N
k
+
1
>
0
,
(52)
K
i
1
N
(
k
)
=
-
H
i
2
L
i
2
N
k
+
1
-
1
K
i
4
N
(
k
)
,
(53)
L
i
2
N
k
=
p
i
i
A
i
+
C
i
K
i
2
N
k
′
L
i
2
N
(
k
+
1
)
×
A
i
+
C
i
K
i
2
N
k
+
p
i
i
A
¯
i
+
C
¯
i
K
i
2
N
k
′
×
L
i
2
N
k
+
1
A
¯
i
+
C
¯
i
K
i
2
N
k
+
Q
i
2
′
Q
i
2
+
K
i
2
N
k
′
S
i
2
K
i
2
N
k
-
K
i
4
N
k
′
H
i
2
L
i
2
N
k
+
1
-
1
K
i
4
N
k
,
L
i
2
N
k
+
1
=
0
,
H
i
2
L
i
2
N
k
+
1
>
0
,
(54)
K
i
2
N
(
k
)
=
-
H
i
1
L
i
1
N
k
+
1
-
1
K
i
3
N
(
k
)
,
where
(55)
H
i
1
L
i
1
N
k
+
1
=
S
i
1
+
p
i
i
C
i
′
L
i
1
N
k
+
1
C
i
vvvvvvvvvvv
+
C
¯
i
′
L
i
1
N
k
+
1
C
¯
i
,
H
i
2
(
L
i
2
N
(
k
+
1
)
)
=
R
i
2
+
p
i
i
B
i
′
L
i
2
N
k
+
1
B
i
vvvvvvvvvvv
+
B
¯
i
′
L
i
2
N
k
+
1
B
¯
i
,
K
i
3
N
k
=
p
i
i
C
i
′
L
i
1
N
k
+
1
A
i
+
B
i
K
i
1
N
k
+
1
vvvvv
+
C
¯
i
′
L
i
1
N
k
+
1
A
¯
i
+
B
¯
i
K
i
1
N
k
+
1
,
K
i
4
N
k
=
p
i
i
B
i
′
L
i
2
N
k
+
1
A
i
+
C
i
K
i
2
N
k
+
1
vvvvv
+
B
¯
i
′
L
i
2
N
k
+
1
A
¯
i
+
C
¯
i
K
i
2
N
k
+
1
.
An iterative process for solving (40)–(43) based on the above recursions is presented as follows.(a)
Given appropriate natural numberN and the initial conditions L
i
1
N
(
N
+
1
)
=
0, L
i
2
N
(
N
+
1
)
=
0, K
i
1
N
(
N
+
1
)
=
0, and K
i
2
N
(
N
+
1
)
=
0.
(b)
Through the numerical values ofL
i
1
N
(
N
+
1
), L
i
2
N
(
N
+
1
), K
i
1
N
(
N
+
1
), and K
i
2
N
(
N
+
1
), we have H
i
1
(
L
i
1
N
(
N
+
1
)
), H
i
2
(
L
i
2
N
(
N
+
1
)
), K
i
3
N
(
N
), and K
i
4
N
(
N
) according to (55).
(c)
K
i
1
N
(
N
) and K
i
2
N
(
N
) can be, respectively, computed by (52) and (54). Then, L
i
1
N
(
N
) and L
i
2
N
(
N
) can also be, respectively, obtained by (51) and (53).
(d)
LetL
i
1
N
(
N
+
1
)
=
L
i
1
N
(
N
), L
i
2
N
(
N
+
1
)
=
L
i
2
N
(
N
), K
i
1
N
(
N
+
1
)
=
K
i
1
N
(
N
), and K
i
2
N
(
N
+
1
)
=
K
i
2
N
(
N
).
(e)
Then,N
:
=
N
-
1. Repeat steps (b)–(d) until the number of iterations is N
+
1. We can finally obtain the numerical values of L
i
1
N
(
0
), L
i
2
N
(
0
), K
i
1
N
(
0
), and K
i
2
N
(
0
).As in [28], under the assumptions of stabilizability, for any x
0
∈
R
n,
(56)
lim
⁡
N
→
∞
x
0
′
L
i
1
N
0
x
0
=
lim
⁡
N
→
∞
min
⁡
J
1
N
u
N
*
,
v
=
min
⁡
J
1
∞
u
*
,
v
=
x
0
′
L
i
1
x
0
,
lim
⁡
N
→
∞
x
0
′
L
i
2
N
(
0
)
x
0
=
lim
⁡
N
→
∞
min
⁡
J
2
N
(
u
,
v
N
*
)
=
min
⁡
J
2
∞
(
u
,
v
*
)
=
x
0
′
L
i
2
x
0
,
lim
⁡
N
→
∞
K
i
1
N
(
0
)
=
K
i
1
,
lim
⁡
N
→
∞
K
i
2
N
(
0
)
=
K
i
2
.
Therefore,
(57)
lim
⁡
N
→
∞
L
i
1
N
0
,
L
i
2
N
0
;
K
i
1
N
0
,
K
i
2
N
0
=
L
i
1
,
L
i
2
;
K
i
1
,
K
i
2
,
where (
L
i
1
,
L
i
2
;
K
i
1
,
K
i
2
) are the solutions of (40)–(43).


## 4.2. A Simulation Example

To verify the efficiency of the above iterative algorithm, we consider the following 2-D example. In the system (30), we set θ
k
=
i
∈
X
=
{
1,2
}, R
i
τ
=
S
i
τ
=
1
(
τ
=
1,2
),
(58)
A
1
=
0.65
0
0
0.9
,
A
¯
1
=
0.45
0
0
0.55
,
B
1
=
0.6
0.55
,
B
¯
1
=
0.45
0.85
,
C
1
=
0.75
0.55
,
C
¯
1
=
0.5
0.85
,
Q
1
1
=
0.55
0
0
0.65
,
Q
1
2
=
0.75
0
0
0.25
,
A
2
=
0.75
0
0
0.7
,
A
¯
2
=
0.35
0
0
0.45
,
B
2
=
0.5
0.45
,
B
¯
2
=
0.55
0.85
,
C
2
=
0.65
0.55
,
C
¯
2
=
0.4
0.85
,
Q
2
1
=
0.35
0
0
0.45
,
Q
2
2
=
0.55
0
0
0.35
.
For convenience, let p
11
=
0.4, p
22
=
0.5, and N
=
50. When θ
k
=
1, by applying the above iterative algorithm, we obtain the solutions of the four coupled equations (51)–(54) as follows:
(59)
L
1
1
N
0
=
L
1
1
1,1
L
1
1
1,2
L
1
1
2,1
L
1
1
2,2
=
0.4023
-
0.0588
-
0.0588
0.6820
,
L
1
2
N
0
=
L
1
2
1,1
L
1
2
1,2
L
1
2
2,1
L
1
2
2,2
=
0.7111
-
0.0331
-
0.0331
0.1487
,
K
1
1
N
(
0
)
=
K
1
1
(
1,1
)
K
1
1
(
1,2
)
=
-
0.1245
-
0.0053
,
K
1
2
N
0
=
K
1
2
1,1
K
1
2
1,2
=
-
0.0390
-
0.1739
.
(
L
1
1
N
(
0
)
,
L
1
2
N
(
0
)
;
K
1
1
N
(
0
)
,
K
1
2
N
(
0
)
) are also the solutions of (40)–(43) according to (57). By the solutions, it shows that L
1
1
>
0 and L
1
2
>
0. The evolution of L
1
1
N
k
,
L
1
2
N
k
;
K
1
1
N
k
,
K
1
2
N
k is exhibited in Figures 1 and 2. And the figures clearly illustrate the convergence and speediness of the backward iterations. When θ
k
=
2, it is easy to get L
2
1
N
0
,
L
2
2
N
0
;
K
2
1
N
0
,
K
2
2
N
0 that are also the solutions of (40)–(43). And L
2
1
>
0 and L
2
2
>
0. Because it is the same as the above process θ
k
=
1, we do not introduce it again due to space limitations.Figure 1
Evolution ofL
1
1
N
(
k
) and L
1
2
N
(
k
).Figure 2
Evolution ofK
1
1
N
(
k
) and K
1
2
N
(
k
).


## 5. Conclusions

In this paper we have discussed theT
n-stability for the discrete-time MJSLS with a finite number of jump times and its infinite horizon LQ differential games. Based on the relations between the Lyapunov equation and the stability of discrete-time MJSLS, we have obtained some useful theorems on finding the solutions of the LQ differential games. Moreover, an iterative algorithm has been presented for the solvability of the four coupled equations. Finally, a numerical example is offered to demonstrate the efficiency of the algorithm. Exact observability and W-observability for discrete-time MJSLS are investigated by [29, 30]. On the basis of exact observability and W-observability, infinite horizon stochastic differential games should be discussed and we will do further research in the future.



---

*Source: 265621-2014-11-23.xml*