# Novel Global Exponential Stability Criterion for Recurrent Neural Networks with Time-Varying Delay

Authors: Wenguang Luo; Xiuling Wang; Yonghua Liu; Hongli Lan
Journal: Abstract and Applied Analysis
 (2013)

---

## Abstract

The problem of global exponential stability for recurrent neural networks with time-varying delay is investigated. By dividing the time delay interval [0,τ(t)] into K+1 dynamical subintervals, a new Lyapunov-Krasovskii functional is introduced; then, a novel linear-matrix-inequality (LMI-) based delay-dependent exponential stability criterion is derived, which is less conservative than some previous literatures (Zhang et al., 2005; He et al., 2006; and Wu et al., 2008). An illustrate example is finally provided to show the effectiveness and the advantage of the proposed result.


---

## Body

## 1. Introduction

In the past decades, recurrent neural networks (RNNs) have been extensively investigated because of their applications, such as combinatorial optimization [1, 2], associative memories [3–5], signal processing [6], image processing [7], pattern recognition [8, 9], and so forth. Some of these applications often require that equilibrium points of the designed networks be stable. Meanwhile, in the hard implementation of RNNs, time delay commonly occurrs due to the finite switching speed of amplifiers or finite speed of signal processing, and its existence is always an origin of oscillation, divergence, and instability in neural networks. Therefore, the stability of RNNs with time delay has received much attention, and a large amount of results have been proposed to ensure the asymptotic or exponential stability of delayed neural networks [10–21].So far, there is a main means handling the stability of delayed neural networks: free-weighting matrix approach [22–26]. Recently, a novel method was proposed for Hopfield neural networks with constant delay in [27], which brings more free-weighting matrices by dividing equally the constant time delay interval [0,τ] into m subintervals. Further more, by dividing the time delay interval [0,τ(t)] into K+1 dynamical subintervals, Zhang et al. [28] generalize this method to study global asymptotic stability of RNNs with time-varying delay. This method mainly utilizes the information in the time delay interval [0,τ(t)], which brings more freedom degrees and can reduce conservativeness.Motivated by above-mentioned discussions, in this paper, we consider the global exponential stability of RNNs with time-varying delay. By dividing the time delay interval[0,τ(t)] into K+1 dynamical subintervals, we construct a new Lyapunov-Krasovskii functional (LKF) and derive a novel sufficient condition, which is presented in term of linear matrix inequality (LMI). The obtained stability result is less conservative than some existing results [22, 23, 29]. Finally, an illustrating example is given to verify the effectiveness and the advantage of the proposed result.The rest of this paper is organized as follows. In Section2, the problem of exponential stability analysis for RNNs with time-varying delay is formulated. Section 3 presents our main results. An illustrating example is provided in Section 4. The conclusion is stated in Section 5.Throughout this paper,C=[Cij]n×n denotes an n×n real matrix. CT,  ∥C∥,  λm(C), and  λM(C) represent the transpose, the Euclidean norm, the minimum eigenvalue, and the maximum eigenvalue of matrix C, respectively. C>0(C<0) denotes that C is a positive (negative) definite matrix. I denotes an identify matrix with compatible dimensions, and * denotes the symmetric terms in a symmetric matrix.


## 2. Problem Formulation

Consider the following RNNs with time-varying delay:(1)z˙(t)=-Dz(t)+Af(z(t))+Bf(z(t-τ(t)))+J,z(t)=ψ(t)∀t∈[-τ,0],
where z(·)=[z1(·),z2(·),…,zn(·)]T is the state vector, f(z(·))=[f1(z1(·)),f2(z2(·)),…,fn(zn(·))]T denotes the neuron activation function, and J is a bias value vector. D=diag(di) is diagonal matrix with di>0, i=1,2,…,n. A and B are connection weight matrix and the delay connection weight matrix, respectively. The initial condition ψ(t) is a continuous and differentiable vector-valued function, where t∈[-τ,0]. The time delay τ(t) is a differentiable function that satisfies: 0≤τ(t)≤τ,  τ˙(t)≤μ, where τ>0 and μ≥0.To obtain the proposal result, we assume that eachfi is bounded and satisfies
(2)li-≤fi(u)-fi(v)u-v≤li+,
where ∀u,v∈R,  u≠v. li- and li+ are some constants, i=1,2,…,n.Letgi(zi(t))=fi(zi(t))-li-zi(t),  li=li+-li-, system (1) is equivalent to the following form:
(3)z˙(t)=-(D-AL0)z(t)+Ag(z(t))+Bg(z(t-τ(t)))+BL0z(t-τ(t))+J,
where L0=diag(l1-,l2-,…,ln-),  L1=diag(l1+,l2+,…,ln+),  L=diag(l1,l2,…,ln)=L1-L0.Noting assumption (2), we have
(4)0≤gi(u)-gi(v)u-v≤li∀u,v∈R,u≠v,li=li+-li-(i=1,2,…,n).
Assuming system (3) has an equilibrium point z*=(z1*,z2*,…,zn*)T. Then, let x=(x1(t),x2(t),…,xn(t))T, and we define xi(t)=zi(t)-zi*, system (3) is transformed into the following form:
(5)x˙(t)=-(D-AL0)x(t)+Ag-(x(t))+Bg-(x(t-τ(t)))+BL0x(t-τ(t)),
where g-i(xi(t))=gi(xi(t)+zi*)-gi(zi*) with g-i(0)=0,  g-i(xi(t-τ(t)))=gi(xi(t-τ(t))+zi*)-gi(zi*).In the derivation of the main results, we need the following lemmas and definitions.Definition 1 (global exponential stability).
System (5) is said to be globally exponentially stable with convergence rate k, if there exist constants k>0 and M≥1, such that
(6)∥x(t)∥≤Mϕe-kt,∀t≥0,
where ϕ=sup-τ≤θ≤0∥x(θ)∥.Lemma 2.
Letx(t)∈Rn be a vector-valued function with the first-order continuous-derivative entries. Then, the following integral inequality holds for matrix X=XT>0 and any matrices M1,  M2, and two scalar functions h1(t) and h2(t), where h2(t)≥h1(t)≥0(7)-∫t-h2(t)t-h1(t)x˙T(s)Xx˙(s)ds≤ζT(t)[M1T+M1-M1T+M2*-M2T-M2]ζ(t)+(h1(t)-h2(t))ζT(t)HTR-1Hζ(t),
where H=[M1,M2]∈Rn×2n and ζ(t)=[xT(t-h1(t))xT(t-h2(t))]T.Proof.
This proof can be completed in a manner similar to [30].Lemma 3 (see [31]).
For any two vectorsa,  b∈Rn, any matrix A, any positive definite symmetric matrix B with the same dimensions, and any two positive constants m, n, the following inequality holds:
(8)-maTBa+2naTAb≤n2bTAT(mB)-1Ab.


## 3. Main Results

In this section, we will consider the delay interval[0,τ(t)], which is divided into K+1 dynamical subintervals, namely, [0,ρ1τ(t)],…,[ρKτ(t),τ(t)], where ρ1<⋯<ρK. This is to say, there is a parameter sequence (ρ1,…,ρK), which satisfies the following conditions:
(9)0<ρ1τ(t)<ρ2τ(t)<⋯<ρKτ(t)<τ(t),0≤ρiτ˙(t)≤ρiμ,
where ρi∈(0,1),i=1,2,…,K,K is positive integer.Utilizing the useful information ofK+1 dynamical subintervals, a novel LKF is constructed, and then a newly LMI-based delay-dependent sufficient condition can be proposed to guarantee the global exponential stability of RNNs with time-varying delay.Theorem 4.
The equilibrium point of system (5) with μ<1 is globally exponentially stable with convergence rate k>0, if there exists parameter ρi satisfying 0<ρ1<⋯<ρK<1, some positive definite symmetric matrices P,  R1,  R2,  R3,  Qi,  Z, some positive definite diagonal matrices Λ,  X1,  X2,  Y1 and Y2, and any matrices Mj, where i=1,2,…,K,  j=1,2,…,2K+4, and K is a positive integer, such that the following LMI has feasible solution:
(10)[ΣW0000*-Zρ1W100**-Zρ2-ρ1⋯0***⋯WK****-Z1-ρK]<0,
where Wi=τe-kτHiT,Hi=[M2i+1M2i+2]∈Rn×2n,i=0,1,2,…,K.(11)
Σ
=
[
Σ
1,1
Σ
1,2
0
0
0
0
Σ
1
,
K
+
2
0
Σ
1
,
K
+
4
Σ
1
,
K
+
5
*
Σ
2,2
Σ
2,3
0
0
0
0
0
0
0
*
*
Σ
3,3
Σ
3,4
0
0
0
0
0
0
*
*
*
⋯
0
0
0
0
0
0
*
*
*
*
Σ
K
,
K
Σ
K
,
K
+
1
0
0
0
0
*
*
*
*
*
Σ
K
+
1
,
K
+
1
Σ
K
+
1
,
K
+
2
0
0
0
*
*
*
*
*
*
Σ
K
+
2
,
K
+
2
Σ
K
+
2
,
K
+
3
Σ
K
+
2
,
K
+
4
Σ
K
+
2
,
K
+
5
*
*
*
*
*
*
*
Σ
K
+
3
,
K
+
3
0
0
*
*
*
*
*
*
*
*
Σ
K
+
4
,
K
+
4
Σ
K
+
4
,
K
+
5
*
*
*
*
*
*
*
*
*
Σ
K
+
5
,
K
+
5
]
,
(12)
Σ
1,1
=
2
k
P
-
P
(
D
-
A
L
0
)
-
(
D
-
A
L
0
)
T
P
+
R
1
+
R
3
+
τ
2
(
D
-
A
L
0
)
T
Z
(
D
-
A
L
0
)
+
∑
i
=
1
K
Q
i
+
L
X
1
L
+
τ
e
-
2
k
τ
(
M
1
T
+
M
1
)
,
Σ
1,2
=
τ
e
-
2
k
τ
(
-
M
1
T
+
M
2
)
,
Σ
1
,
K
+
2
=
P
B
L
0
-
τ
2
(
D
-
A
L
0
)
Z
B
L
0
,
Σ
1
,
K
+
4
=
P
A
+
2
k
Λ
-
(
D
-
A
L
0
)
T
Λ
-
τ
2
(
D
-
A
L
0
)
T
Z
A
+
L
X
2
,
Σ
1
,
K
+
5
=
P
B
-
τ
2
(
D
-
A
L
0
)
T
Z
B
,
Σ
2,2
=
-
e
-
2
k
ρ
1
τ
Q
1
(
1
-
ρ
1
μ
)
+
τ
e
-
2
k
τ
(
-
M
2
T
-
M
2
)
+
τ
e
-
2
k
τ
(
M
3
T
+
M
3
)
,
Σ
2,3
=
τ
e
-
2
k
τ
(
-
M
3
T
+
M
4
)
,
Σ
3,3
=
-
e
-
2
k
ρ
2
τ
Q
2
(
1
-
ρ
2
μ
)
+
τ
e
-
2
k
τ
(
-
M
4
T
-
M
4
)
+
τ
e
-
2
k
τ
(
M
5
T
+
M
5
)
,
Σ
3,4
=
τ
e
-
2
k
τ
(
-
M
5
T
+
M
6
)
,
…
,
Σ
K
+
1
,
K
+
1
=
-
e
-
2
k
ρ
K
τ
Q
K
(
1
-
ρ
K
μ
)
+
τ
e
-
2
k
τ
(
-
M
2
K
T
-
M
2
K
)
+
τ
e
-
2
k
τ
(
M
2
K
+
1
T
+
M
2
K
+
1
)
,
Σ
K
+
1
,
K
+
2
=
τ
e
-
2
k
τ
(
-
M
2
K
+
1
T
+
M
2
K
+
2
)
,
Σ
K
+
2
,
K
+
2
=
-
e
-
2
k
τ
R
1
(
1
-
μ
)
+
τ
2
L
0
B
T
Z
B
L
0
+
L
Y
1
L
+
τ
e
-
2
k
τ
(
-
M
2
K
+
2
T
-
M
2
K
+
2
)
+
τ
e
-
2
k
τ
(
M
2
K
+
3
T
+
M
2
K
+
3
)
Σ
K
+
2
,
K
+
3
=
τ
e
-
2
k
τ
(
-
M
2
K
+
3
T
+
M
2
K
+
4
)
,
Σ
K
+
2
,
K
+
4
=
L
0
B
T
Λ
+
τ
2
L
0
B
T
Z
A
,
Σ
K
+
2
,
K
+
5
=
τ
2
L
0
B
T
Z
B
+
L
Y
2
,
Σ
K
+
3
,
K
+
3
=
e
-
2
k
τ
R
3
+
τ
e
-
2
k
τ
(
-
M
2
K
+
4
T
-
M
2
K
+
4
)
,
Σ
K
+
4
,
K
+
4
=
Λ
A
+
A
T
Λ
+
R
2
+
τ
2
A
T
Z
A
-
X
1
-
2
X
2
,
Σ
K
+
4
,
K
+
5
=
Λ
B
+
τ
2
A
T
Z
B
,
Σ
K
+
2
,
K
+
5
=
-
e
-
2
k
τ
R
2
(
1
-
μ
)
+
τ
2
B
T
Z
B
-
Y
1
-
2
Y
2
,
L
0
=
diag
(
l
i
-
)
,
L
1
=
diag
(
l
i
+
)
,
L
0
=
diag
(
l
i
)
,
D
=
diag
(
d
i
)
,
A
=
[
a
i
j
]
n
×
n
,
B
=
[
b
i
j
]
n
×
n
.Proof.
Construct the following Lyapunov-Krasovskii functional candidate:(13)V(x(t))=V1(x(t))+V2(x(t))+V3(x(t))+V4(x(t))+V5(x(t)),
where
(14)V1(x(t))=e2ktxT(t)Px(t),V2(x(t))=2e2kt∑i=1nλi∫0xi(t)g-i(s)ds,V3(x(t))=∫t-τ(t)te2ksxT(s)R1x(s)ds+∫t-τ(t)te2ksg-T(s)R2g-(s)ds+∫t-τte2ksxT(s)R3x(s)ds,V4(x(t))=∑i=1K∫t-ρiτ(t)te2ksxT(s)Qix(s)ds,V5(x(t))=τ∫-τ0∫t+θte2ksx˙T(s)Zx˙(s)ds,
where P=PT>0, Λ=diag(λi)>0, R1=R1T>0, R2=R2T>0, R3=R3T>0, Qi=QiT>0, Z=ZT>0, and i=0,1,2,…,K.
Let the parametersρ0=0,  ρK+1=1; calculating the time derivatives Vi(i=1,2,3,4,5) along the trajectories of system (5) yields
(15)V˙1(x(t))=2ke2ktxT(t)Px(t)+2e2ktxT(t)Px˙(t),V˙2(x(t))=4ke2kt∑i=1nλi∫0xi(t)g-i(s)ds+2e2ktg-T(x(t))Λx˙(t),V˙3(x(t))=e2kt[R3g-(x(t-τ))xT(t)R1x(t)-e-2kτ(t)xT(t-τ(t))R1x(t-τ(t))×(1-τ˙(t))+g-T(x(t))R2g-(x(t))-e-2kτ(t)g-T(x(t-τ(t)))R2g-(x(t-τ(t)))×(1-τ˙(t))+xT(t)R3x(t)-e-2kτg-T(x(t-τ))R3g-(x(t-τ))],V˙4(x(t))=∑i=1Ke2kt[xT(t)Qix(t)-e-2kρiτ(t)xT(t-ρiτ(t))×Qix(t-ρiτ(t))(1-ρiτ˙(t))e-2kρiτ(t)],V˙5(x(t))=τ2e2ktx˙T(t)Zx˙(t)-τ∫t-τte2ksx˙T(s)Zx˙(s)ds.
It is clear that the following inequality is true:
(16)x˙T(t)Zx˙(t)≤[-(D-AL0)x(t)+Ag-(x(t))+Bg-(x(t-τ(t)))+Bx(t-τ(t))]T×Z[-(D-AL0)x(t)+Ag-(x(t))+Bg-(x(t-τ(t)))+Bx(t-τ(t))],
According to (4), for some diagonal matrices X1>0,  X2>0,  Y1>0,  Y2>0, we have
(17)g-T(x(t))X1g-(x(t))≤xT(t)LX1Lx(t),g-T(x(t))X2g-(x(t))≤xT(t)LX1g-(x(t)),g-T(x(t-τ(t)))Y1g-(x(t-τ(t)))≤xT(t-τ(t))LY1Lx(t-τ(t)),g-T(x(t-τ(t)))Y2g-(x(t-τ(t)))≤xT(t-τ(t))LY2g-(x(t-τ(t))).
Using Lemma 2, we have
(18)-∫t-ρi+1τ(t)t-ρiτ(t)x˙T(s)Zx˙(s)ds≤ζiT(t)[M2i+1T+M2i+1-M2i+1T+M2i+2*-M2i+2T-M2i+2]ζi(t)+(ρi+1-ρi)τ(t)ζiT(t)HiTZ-1Hiζi(t)-∫t-τt-τ(t)x˙T(s)Zx˙(s)ds≤ζK+1T(t)[M2K+3T+M2K+3-M2K+3T+M2K+4*-M2K+4T-M2K+4]ζK+1(t),
where Hi=[M2i+1M2i+2]∈Rn×2n, ζi(t)=[xT(t-ρiτ(t))xT(t-ρi+1τ(t))]T, ζK+1(t)=[xT(t-τ(t))xT(t-τ)]T,i=0,1,…,K.
From (15)–(17), and using (18), we finally get
(19)V˙(x(t))≤e2ktΩ,
where Ω=ζT(t)Σζ(t)+Ω0+Ω1+⋯+ΩK, Σ is defined in (11). 							(20)Ωi=(ρi+1-ρi)τ2e-2kτζiT(t)HiTZ-1Hiζi(t),i=0,1,…,K,ζj(t)=[xT(t-ρjτ(t))xT(t-ρj+1τ(t))]T,j=0,1,…,K+1.ζ(t)=[(x(t-τ(t)))xT(t),xT(t-ρ1τ(t)),…,xT(t-ρKτ(t)),xT(t-τ(t)),xT(t-τ),g-T(x(t)),g-T(x(t-τ(t)))]T.
Obviously, if Ω<0, it implies V˙(x(t))<0 for any ζ(t)≠0.
And(21)V(x(0))=xT(0)Px(0)+2∑i=1nλi∫0xi(0)g-i(s)ds+∫-τ(0)0e2ksxT(s)R1x(s)ds+∫-τ(0)0e2ksg-T(x(s))R2g-(x(s))ds+∫-τ0e2ksxT(s)R3x(s)ds+∑i=1K∫-ρiτ(0)0e2ksxT(s)Qix(s)ds+τ∫-τ0∫θ0e2ksx˙T(s)Zx˙(s)ds.
On the other hand, the following inequality holds:
(22)x˙T(s)x˙(s)≤[(x(t-τ(s)))-(D-AL0)x(s)+Ag-(x(s))+Bg-(x(t-τ(s)))+Bx(t-τ(s))]T×[(x(t-τ(s)))-(D-AL0)x(s)+Ag-(x(s))+Bg-(x(t-τ(s)))+Bx(t-τ(s))].
Combining Lemma 3, we have
(23)x˙T(s)x˙(s)≤4[λM((D-AL0)T(D-AL0))+λM(L2)λM(ATA)+λM(L2)λM(BTB)+λM(L0BTBL0)((D-AL0)T(D-AL0))]∥ϕ∥.
Thus,
(24)V(x(0))=λM(P)∥ϕ∥2+(λM(L2)+λM(ΛTΛ))∥ϕ∥2+λM(R1)τ∥ϕ∥2+λM(R2)λM(L2)τ∥ϕ∥2+λM(R3)τ∥ϕ∥2+∑i=1KλM(Qi)ρiτ∥ϕ∥2+2τ3λM(Z)[λM((D-AL0)T(D-AL0))+λM(L2)λM(ATA)+λM(L2)λM(BTB)+λM(L0BTBL0)((D-AL0)T(D-AL0))]∥ϕ∥,
where
(25)Δ=λM(P)+(λM(L2)+λM(ΛTΛ))+λM(R1)τ+λM(R2)λM(L2)τ+λM(R3)τ+∑i=1KλM(Qi)ρiτ∥ϕ∥2+2τ3λM(Z)[λM((D-AL0)T(D-AL0))+λM(L2)λM(ATA)+λM(L2)λM(BTB)+λM(L0BTBL0)((D-AL0)T(D-AL0))].
On the other hand, we have
(26)V(x(t))≥e2ktλm(P)∥ϕ∥.
Therefore,
(27)∥x(t)∥≤e-ktΔλm(P)∥ϕ∥.
Thus, according to Definition 1, we can conclude that the equilibrium point x* of system (5) is globally exponentially stable. This completes the proof.Remark 5.
Differential from the results in [22, 23, 29], we divide the time delay interval [0,τ(t)] into K+1 dynamical subintervals, and a novel Lyapunov-Krasovskii functional is introduced. This brings more degrees of freedom to ensure the global exponential stability. Therefore, Theorem 4 is less conservative than some previous results.Remark 6.
In Theorem4, by setting R1=R2=Qi=0(i=1,2,…,K), similar to the proof of Theorem 4, we can derive a criterion to guarantee the global exponential stability of RNNs with time-varying delay when τ˙(t) is unknown or τ(t) is not differentiable.


## 4. Illustrating Example

In this section, an illustrating example is given to verify the effectiveness and advantage of the criteria proposed in this paper.Example 7.
Consider system (5) with the following parameters:
(28)D=[2003.5],A=[-10.50.5-1],B=[-0.50.50.50.5],L=[1000].At first, we suppose that the time delay interval[0,τ(t)] is divided into 2 (K=1) subintervals, and ρ=0.1. While the upper bound τ=1, the exponential convergence rates for various μ obtained from Theorem 4 and those in [22, 23, 29] are listed in Table 1. In addition, while the exponential convergence rate of k=0.8, the upper bounds of τ for various μ from Theorem 4 and those in [22, 23, 29] are listed in Table 2.Table 1
Allowable exponential convergence ratek for variable μ and τ=1.
μ
0
0.5
0.9
Unsure
[22]
0.25
Fail
Fail
Fail
[23]
1.15
0.7538
0.6106
0.3391
[29]
1.15
0.8643
0.8344
0.8169
Theorem4 (ρ=1)
1.15
0.9105
0.9105
0.8608Table 2
Allowable upper bound ofτ for variable μ and k=0.8.
μ
0.5
0.8
Unsure
[22]
Fail
Fail
Fail
[23]
1.2606
0.9442
0.8310
[29]
1.2787
1.0819
1.0366
Theorem4 (ρ=1)
1.3022
1.1646
1.0817Thus, from Tables1 and 2, we can say that, the result in this paper is much effective and less conservative than those in [22, 23, 29]. Figure 1 shows the state response of Example 7 with constant delay τ=1, when the initial value is [0.8,-0.6]T.Figure 1
State response curves withτ=1 and μ=0 for Example 7.


## 5. Conclusion

In this paper, we consider the global exponential stability for RNNs with time-varying delay. By dividing the time delay interval[0,τ(t)] into K+1 dynamical subintervals, a novel Lyapunov-Krasovskii functional is introduced. A less conservative LMI-based delay-dependent stability criterion is derived based on Lyapunov stability theory. Furthermore, an illustrating example is given to show the effectiveness of the proposed result.



---

*Source: 540951-2013-02-28.xml*