# Design for Nonlinear Networked Control Systems with Time Delay Governed by Markov Chain with Partly Unknown Transition Probabilities

Authors: Yan-Feng Wang; Zu-Xin Li; Hui-Ying Chen; Li-Di Quan; Xiao-Rui Guo
Journal: Mathematical Problems in Engineering
 (2015)

---

## Abstract

The problem of state feedback control for a class of nonlinear networked control systems with time delay is discussed in this paper. The time delay is modeled as a finite state Markov chain of which transition probabilities are partly unknown. The closed-loop system model is obtained by means of state augmentation. A sufficient condition is given which guarantees the stochastic stability of the closed-loop system in the form of linear matrix inequalities and the maximum bound of the nonlinearity is also obtained. Finally, a simulation example is used to show the validity of the proposed method.


---

## Body

## 1. Introduction

Networked control systems (NCSs) are feedback control systems with control loops closed via digital communication channels [1]. As is well known, NCSs are widely used in many fields such as remote surgery, mobile sensor networks, and unmanned aerial vehicles especially in a number of emerging engineering applications such as arrays of microactuators and even social-economical systems [2–4]. Compared with the point-to-point wiring, the use of communication channels can reduce the costs of cables and simplify the installation and maintenance of the system. However, the insertion of the network will make system analysis and synthesis more challenging [5–8]. Network-induced delays can degrade the performance of control systems designed without considering them and even destabilize the system [9–11].Up to now, the effect of time delay on the stability and performance of NCSs has received great attention [12, 13]. The time delay may be constant, time-varying, and, in most cases, random [14–16]. For the random cases, the time delays are commonly modeled as white sequences taking on values of zero and one with certain probability [17, 18] or as finite state Markov chains [19–21]. In the real networks, the current time delays are usually related with the previous delays, so the Markov chain model is proper compared to the white sequence model to describe the time delay.In [22, 23], the random delays are modeled as Markov chains such that the closed-loop system is a jump linear system. In [24–26], the random time delays are modeled as Markov chains, and output feedback controller design methods for NCSs with random network-induced delays are proposed. In [27], the mixed H
2/H
∞ control issue of NCSs with random time delays has been investigated based on Markov jump linear systems method. In the above literature [22–27], all parts of the elements in the transition probabilities matrix of the time delays are assumed to be completely known. However, in almost all kinds of networks, all parts of the elements in the desired transition probabilities matrix are hard or costly to obtain. In [28–30], the stability issue of discrete-time networked control systems with random Markov delays and uncertain transition probabilities is studied. And the controlled plants in [22–30] are assumed to be linear. However, almost all the practical controlled plants are nonlinear; it is significant and necessary to further study nonlinear NCSs with partly unknown transition probabilities. To the best of the authors’ knowledge, up to now, very limited efforts have been devoted to studying the system with uncertain transition probability matrices for nonlinear NCSs, which motivates our investigation.This paper studies the state feedback problem of a class of nonlinear NCSs. The time delay is modeled as a finite state Markov chain of which transition probabilities are partly unknown. The closed-loop system model is obtained by means of state augmentation and the mode-dependent state feedback controller is designed which guarantees the stochastic stability of the closed-loop system. The state feedback controller is designed which guarantees the stability of the resulting closed-loop systems. The maximum bound of nonlinear disturbance is also obtained.Notations. The superscript T stands for the transpose of a matrix. R
n and R
n
×
m denote the n-dimensional Euclidean space and the set of all n
×
m real matrices, respectively. d
i
a
g
{
⋯
} denotes the block diagonal matrix. I and 0 stand for unit matrix and zero matrix with appropriate dimensions, respectively. ∗ denotes the entry of matrices implied by symmetry.


## 2. Problem Formulation

Consider the NCS setup in Figure1, in which the controlled plant is a nonlinear time-invariant discrete-time system and τ
k
>
0 is the bounded and random time delay from the senor to the controller which takes value in Λ
=
{
1,2
,
…
,
d
} and the transition probability matrix of τ
k is Π
=
[
π
i
j
].Figure 1
NCSs with random delay.That is,τ
k jump from mode i to j with probability π
i
j which is defined by π
i
j
=
P
r
(
τ
k
+
1
=
j
∣
τ
k
=
i
), where π
i
j
≥
0, ∀
i
,
j
∈
Λ, ∑
j
=
1
d
π
i
j
=
1. Likewise, the transition probability matrix is defined by(1)
Π
=
π
11
π
12
⋯
π
1
d
π
21
π
22
⋯
π
2
d
⋮
⋮
⋱
⋮
π
d
1
π
d
2
⋯
π
d
d
.The set Λ contains d modes of τ
k, and the transition probabilities of the jumping process in this paper are considered to be partly accessed; that is, some elements in matrix Π are unknown. For example, for the time delay τ
k with 4 modes, the transition probabilities matrix Π may be as follows:(2)
Π
=
?
π
12
?
?
?
?
π
23
?
π
31
?
π
33
?
?
?
?
π
44
,where “?” represents the inaccessible elements. For notational clarity, ∀
i
∈
Λ, we denote Λ
=
Λ
k
i
+
Λ
u
k
i by (3)
Λ
k
i
=
j
:
π
i
j
i
s
k
n
o
w
n
,
Λ
u
k
i
=
j
:
π
i
j
i
s
u
n
k
n
o
w
n
.Moreover, if Λ
k
i
≠
∅, it is further described as Λ
k
i
=
{
k
1
i
,
k
2
i
…
k
μ
i
}, 1
≤
μ
≤
d, where k
μ
i represents the μ
t
h known element with the index k
μ
i in the i
t
h row of the matrix Π. And Λ
u
k
i is described as Λ
u
k
i
=
{
k
-
1
i
,
k
-
2
i
⋯
k
-
d
-
μ
i
}, where k
-
d
-
μ
i represents the (
d
-
μ
)
t
h unknown element with the index (
d
-
μ
)
t
h in the i
t
h row of the matrix Π.Assume that the model of the plant is a nonlinear discrete-time system as follows:(4)
x
k
+
1
=
A
p
x
k
+
B
p
u
k
+
f
k
,
x
k
,where x
(
k
)
∈
R
n is state vector, u
(
k
)
∈
R
m is the control input, A
p
∈
R
n
×
n and B
p
∈
R
n
×
m are all real constant matrices, and f
(
k
,
x
(
k
)
) is nonlinear disturbance, satisfying the following Lipschitz condition [31]:(5)
f
T
k
,
x
k
f
k
,
x
k
≤
α
2
x
T
k
H
T
H
x
k
,where H is a real constant matrix and α
≥
0 is the bound of the nonlinear disturbance.It is noticed that when the controller is designed, the exactτ
k is known. Hence, the controller gain can be designed depending on τ
k; that is, (6)
u
k
=
K
τ
k
x
k
-
τ
k
,
x
t
=
η
t
,
t
∈
-
d
,
…
,
0
.Consequently, the closed-loop system from (4) and (6) can be expressed as(7)
x
k
+
1
=
A
p
x
k
+
B
p
K
τ
k
x
k
-
τ
k
+
f
k
,
x
k
.At sampling time k, if we augment the state variable as (8)
X
k
=
x
k
T
x
k
-
1
T
⋯
x
k
-
d
T
T
,the closed-loop system (7) can be written as(9)
X
k
+
1
=
A
+
B
K
τ
k
E
τ
k
X
k
+
D
F
k
,
X
k
,
X
0
=
η
0
T
η
-
1
T
⋯
η
-
d
T
T
,where(10)
A
=
A
p
0
⋯
0
0
I
0
⋯
0
0
0
I
⋯
0
0
⋮
⋮
⋱
0
0
0
0
⋯
I
0
∈
R
n
1
+
d
×
n
1
+
d
,
B
=
B
p
0
0
⋮
0
∈
R
n
1
+
d
×
m
,
D
=
I
0
0
⋮
0
∈
R
n
1
+
d
×
n
,
G
=
I
0
0
⋯
0
∈
R
n
×
n
1
+
d
,
F
k
,
X
k
=
f
k
,
G
X
k
,
E
τ
k
=
0
0
I
⋯
0
∈
R
n
×
n
1
+
dhas all elements being zeros except for the (
1
+
τ
k
)
t
h block being identity. It can be seen that the closed-loop system (9) is a jump linear system with d different modes.To proceed, we will need the following lemmas.Lemma 1 (see [2]).
LetT
1, T
2
∈
R
n
×
n be symmetric matrices. The conditions on x
T
T
1
x
<
0, ∀
x
≠
0 such that x
T
T
2
x
≤
0 hold if there exists a scalar ε
≥
0 such that T
1
-
ε
T
2
<
0.Lemma 2 (see [32]).
For given scalarsλ
i
≥
0 and matrices P
i
≥
0, ∑
i
=
1
N
λ
i
P
i
≤
∑
i
=
1
N
λ
i
∑
i
=
1
N
P
i always holds.Lemma 3 (see [18]).
The matrixE
(
i
)
T is of full-array rank; then, there exist two orthogonal matrices U
i
∈
R
n
(
1
+
d
)
×
n
(
1
+
d
) and V
i
∈
R
g
×
g, such that E
(
i
)
T
=
U
i
T
Σ
i
0
V
i
T, where Σ
i
=
d
i
a
g
(
σ
1
,
σ
2
,
…
,
σ
g
), where σ
i
(
i
=
1,2
,
…
,
g
) are nonzero singular values of E
i
T. If matrix F
(
i
)
∈
R
n
(
1
+
d
)
×
n
(
1
+
d
) has the following structure,(11)
F
i
=
U
i
T
F
i
1
0
0
F
i
2
U
i
,there exists a nonsingular matrix X
i
∈
R
n
×
n such that F
(
i
)
E
(
i
)
T
=
E
(
i
)
T
X
i, where F
i
1
∈
R
g
×
g
>
0 and F
i
2
∈
R
n
(
1
+
d
-
g
)
×
n
(
1
+
d
-
g
)
>
0.Throughout this paper, we use the following definition.Definition 4 (see [20]).
The system (9) is stochastically stable if, for every finite X
(
0
) and initial mode τ
0
∈
Λ, there exists a finite matrix W
>
0 such that the following holds:(12)
E
∑
k
=
0
∞
X
k
2
∣
X
0
,
τ
0
<
X
0
T
W
X
0
.
Our objective for this paper is to find the controller gainK
(
τ
k
) such that the closed-loop system (7) is stochastically stable. In the following, if we know that τ
k
=
i, K
(
τ
k
) and E
(
τ
k
) for this paper are denoted by K
(
i
) and E
(
i
), respectively.


## 3. Controller Design

Theorem 55.
Consider system (9) with partly unknown transition probabilities (3). If there exist matrices F
(
i
)
>
0, Y
(
i
) and a scalar ε
>
0 such that(13)
-
F
i
0
Ω
13
Ω
14
F
i
G
T
H
T
∗
-
I
Ω
23
Ω
24
0
∗
∗
Ω
33
0
0
∗
∗
∗
Ω
44
0
∗
∗
∗
∗
-
ε
I
<
0
,where(14)
Ω
13
=
A
F
i
+
B
Y
i
E
i
T
×
π
i
k
1
i
I
π
i
k
2
i
I
⋯
π
i
k
μ
i
I
,
Ω
14
=
A
F
i
+
B
Y
i
E
i
T
×
λ
i
I
λ
i
I
⋯
λ
i
I
,
Ω
23
=
π
i
k
1
i
D
π
i
k
2
i
D
⋯
π
i
k
μ
i
D
,
Ω
24
=
λ
i
D
λ
i
D
⋯
λ
i
D
,
Ω
33
=
-
d
i
a
g
F
k
1
i
F
k
2
i
⋯
F
k
μ
i
,
Ω
44
=
-
d
i
a
g
F
k
-
1
i
F
k
-
2
i
⋯
F
k
-
d
-
μ
i
,
F
i
=
U
i
T
F
i
1
0
0
F
i
2
U
i
,
λ
i
=
1
-
∑
j
∈
Λ
k
i
π
i
j
,
ε
=
1
α
2
,hold for all i
∈
Λ; there exists a mode-dependent controller of the form (6) such that the resulting system (9) is stochastically stable. Furthermore, if the LMI (13) has a solution, an admissible controller is given by(15)
K
i
=
Y
i
V
i
Σ
i
-
1
F
i
1
-
1
Σ
i
V
i
T
.Proof.
For the closed-loop system (9), consider the quadratic function which is given by V
X
k
,
k
=
X
T
(
k
)
P
(
τ
k
)
X
(
k
). We have (16)
E
Δ
V
X
k
,
k
=
E
X
T
k
+
1
P
τ
k
+
1
X
k
+
1
∣
X
k
,
τ
k
=
i
-
X
T
k
P
τ
k
X
k
=
δ
T
L
τ
k
δ
,where(17)
L
τ
k
=
L
11
A
+
B
K
i
E
i
T
∑
j
=
1
d
π
i
j
P
j
D
∗
D
T
∑
j
=
1
d
π
i
j
P
j
D
,
L
11
=
A
+
B
K
i
E
i
T
×
∑
j
=
1
d
π
i
j
P
j
A
+
B
K
i
E
i
-
P
i
,
δ
=
X
k
T
F
k
,
X
k
T
T
.If L
(
τ
k
)
<
0, it can be proved that the closed-loop system (9) is stochastically stable which is similar to that of [18] and is omitted here.
From (5), one can get(18)
δ
T
-
α
2
G
T
H
T
H
G
0
∗
I
δ
≤
0
.According to Lemma 1, if there exists a scalar ε
≥
0 such that (19)
L
11
+
ε
α
2
G
T
H
T
H
G
A
+
B
K
i
E
i
T
∑
j
=
1
d
π
i
j
P
j
D
∗
D
T
∑
j
=
1
d
π
i
j
P
j
D
-
ε
I
<
0
,the inequality (16) subject to (18) holds. Assume that ε
≠
0; inequality (19) can be written as (20)
L
~
11
+
α
2
G
T
H
T
H
G
A
+
B
K
i
E
i
T
∑
j
=
1
d
π
i
j
P
~
j
D
∗
D
T
∑
j
=
1
d
π
i
j
P
~
j
D
-
I
<
0
,where (21)
L
~
11
=
A
+
B
K
i
E
i
T
×
∑
j
=
1
d
π
i
j
P
~
j
A
+
B
K
i
E
i
-
P
~
i
,
P
~
j
=
P
j
ε
,
P
~
i
=
P
i
ε
.According to Lemma 2, one can get (22)
∑
j
=
1
d
π
i
j
P
~
j
≤
P
~
Λ
k
i
+
λ
i
P
~
Λ
u
k
i
,where(23)
P
~
Λ
k
i
=
∑
j
∈
Λ
k
i
π
i
j
P
~
j
,
P
~
Λ
u
k
i
=
∑
j
∈
Λ
u
k
i
P
~
j
,
λ
i
=
1
-
∑
j
∈
Λ
k
i
π
i
j
.By Schur complement, one can obtain(24)
-
P
~
i
0
Ω
-
13
Ω
-
14
G
T
H
T
0
-
I
Ω
23
Ω
24
0
∗
∗
Ω
-
33
0
0
∗
∗
∗
Ω
-
44
0
∗
∗
∗
∗
-
ε
I
<
0
,where(25)
Ω
-
13
=
A
+
B
K
i
E
i
T
×
π
i
k
1
i
I
π
i
k
2
i
I
⋯
π
i
k
μ
i
I
,
Ω
-
14
=
A
+
B
K
i
E
i
T
λ
i
I
λ
i
I
⋯
λ
i
I
,
Ω
-
33
=
-
d
i
a
g
P
-
1
k
1
i
P
-
1
k
2
i
⋯
P
-
1
k
μ
i
,
Ω
-
44
=
-
d
i
a
g
P
-
1
k
-
1
i
P
-
1
k
-
2
i
⋯
P
-
1
k
-
d
-
μ
i
.Performing a congruence transformation to (24) by d
i
a
g
P
-
1
i
I
I
I
I and letting P
~
-
1
i = F
i, d
i
a
g
P
-
1
(
k
1
i
)
P
-
1
(
k
2
i
)
⋯
P
-
1
(
k
μ
i
) = d
i
a
g
F
k
1
i
F
k
2
i
⋯
F
k
μ
i, d
i
a
g
P
-
1
(
k
-
1
i
)
P
-
1
(
k
-
2
i
)
⋯
P
-
1
(
k
-
d
-
μ
i
) = d
i
a
g
F
k
-
1
i
F
k
-
2
i
⋯
F
k
-
d
-
μ
i, we can get(26)
-
F
i
0
Ω
~
13
Ω
~
14
F
i
G
T
H
T
∗
-
I
Ω
23
Ω
24
0
∗
∗
Ω
33
0
0
∗
∗
∗
Ω
44
0
∗
∗
∗
∗
-
ε
I
<
0
,where(27)
Ω
~
13
=
F
i
A
+
B
K
i
E
i
T
×
π
i
k
1
i
I
π
i
k
2
i
I
⋯
π
i
k
μ
i
I
,
Ω
~
14
=
F
i
A
+
B
K
i
E
i
T
λ
i
I
λ
i
I
⋯
λ
i
I
.For the matrix E
(
i
)
T of full-array rank, there always exist two orthogonal matrices U
i
∈
R
n
(
1
+
d
)
×
n
(
1
+
d
) and V
i
∈
R
g
×
g such that E
i
T
=
U
i
T
Σ
i
0
V
i
T, where Σ
i
=
d
i
a
g
(
σ
1
,
σ
2
,
…
,
σ
g
) and  σ
i
(
i
=
1,2
,
…
,
g
) are nonzero singular values of E
i
T. Assume that the matrix F
(
i
) has the following structure:(28)
F
i
=
U
i
T
F
i
1
0
0
F
i
2
U
i
;according to Lemma 3, there exists matrix X
i
∈
R
g
×
g such that F
(
i
)
E
(
i
)
T
=
E
(
i
)
T
X
i; setting X
i
K
(
i
)
T
=
Y
(
i
)
T, we can obtain (13) from (26). Since F
(
i
)
E
(
i
)
T
=
E
(
i
)
T
X
i, we can get (29)
U
i
T
F
i
1
0
0
F
i
2
Σ
i
0
V
i
T
=
U
i
T
Σ
i
0
V
i
T
X
i
;that is, X
i
=
V
i
Σ
i
-
1
F
i
1
Σ
i
V
i
T, which implies that K
(
i
)
=
Y
(
i
)
V
i
Σ
i
-
1
F
i
1
-
1
Σ
i
V
i
T, which completes the proof.Remark 6.
When all the elements in the transition probabilities matrix of the time delays are known, thenλ
i
=
0, which is a special case.Remark 7.
We introduce some structure restrictions on the matrix variablesF
(
i
) which enable us to present the existence conditions of the controller in terms of LMIs. Nevertheless, such restriction will introduce some extents of conservatism into the controller design.
The maximum bound of nonlinear disturbanceα
m
a
x
=
1
/
ε can be found by solving the following optimization problem:(30)
m
i
n
ε
s
.
t
.
13
,
F
i
>
0
,
ε
>
0
.


## 4. Numerical Examples

In this section, one numerical example will be given to show the validity and potential of our developed theoretical results. Consider the nonlinear time-invariant discrete-time plant with the following data:(31)
A
p
=
0.2
0.5
-
0.67
0.6
,
B
p
=
0.1
0.2
,
f
k
,
x
k
=
3
sin
⁡
x
1
k
0.7
x
2
k
cos
⁡
x
2
k
.Assume that the time delayτ
k from the sensor to the controller takes values from Λ
=
{
1,2
,
3,4
}. Consider the following transition probabilities matrix:(32)
Π
=
0.4
0.2
0.3
0.1
?
0.1
?
0.7
0.5
0.1
0.3
0.1
?
0.6
0.2
?
.Our purpose is to design a mode-dependent controller of the form (6) such that the closed-loop system (9) is stochastically stable with the partly unknown transition probabilities. Let H in (5) be unit matrix; by solving (13) in Theorem 5, the controller gain under the transition probabilities matrix Π is solved as follows:(33)
K
1
=
-
0.2030
0.0564
,
K
2
=
-
0.0834
-
0.0229
,
K
3
=
0.0086
-
0.0301
,
K
4
=
0.0022
-
0.0015
.The maximum bound of nonlinear disturbanceα
m
a
x under the transition probabilities matrix Π is 0.1876.Figure2 shows one simulation run of the Markov jump delays under the transition probabilities matrix Π. Figure 3 shows the state response of the closed-loop system using the obtained controller under the transition probabilities matrix Π when x
(
0
)
=
2
-
1
T.Figure 2
Step of delayτ
k.Figure 3
States of the closed-loop system.It can be seen that the closed-loop system with the partly unknown transition probabilities under the designed controller is stochastically stable.


## 5. Conclusions

In this paper, the state feedback stabilization problem has been considered for a kind of nonlinear networked control systems with random communication delays. A Markov chain is employed to describe the random communication time delay of which transition probabilities are partly unknown. The closed-loop system is established through the state augmentation technique and the state feedback controller is designed which guarantees the stability of the resulting closed-loop systems. It is shown that the controller design problem under consideration is solvable if a set of LMIs is feasible. Simulation results show the validity of the proposed method.



---

*Source: 604246-2015-09-16.xml*