# Delay-Dependent Stability Analysis for Recurrent Neural Networks with Time-Varying Delays

Authors: Shu Lv; Junkang Tian; Shouming Zhong
Journal: Mathematical Problems in Engineering
 (2012)

---

## Abstract

This paper concerns the problem of delay-dependent stability criteria for recurrent neural networks with time varying delays. By taking more information of states and activation functions as augmented vectors, a new class of the Lyapunov functional is proposed. Then, some less conservative stability criteria are obtained in terms of linear matrix inequalities (LMIs). Finally, two numerical examples are given to illustrate the effectiveness of the proposed method.


---

## Body

## 1. Introduction

In the past few decades, the stability analysis for recurrent neural networks has been extensively investigated because of their successful applications in various scientific fields, such as pattern recognition, image processing, associative memories, and fixed-point computations. It is well known that time delay is frequently encountered in neural networks, and it is often a major cause of instability and oscillation. Thus, much more attention has been paid to recurrent delayed neural networks. Many interesting stability conditions, including delay-independent results [1, 2] and delay-dependent results [3–41], have been obtained for neural networks with time delays. Generally speaking, the delay-dependent stability criteria are less conservative than delay-independent ones when the size of time delay is small. For the delay-dependent case, some criteria have been derived by using Lyapunov-Krasovskii functional (LKF). It is well known that the construction of an appropriate LKF is crucial for obtaining less conservative stability conditions. Thus, some new methods have been developed for reducing conservatism, such as free-weighting matrix method [4–8], augmented LKF [9], discretized LKF [10], delay-partitioning method [12–18], and delay-slope-dependent method [19]. Some less conservative stability criteria were proposed in [6] by considering some useful terms which have been usually neglected in the previous literature and using the free-weighting matrices method. Recently, a novel method was proposed for Hopfield neural networks in [12], which divides the constant time delay interval [0,h] into subintervals with the same size. This method utilizes more information about the delay interval [0,h] to reduce the conservativeness. Very recently, by proposing the idea of dividing the delay interval with the weighted parameters, the weighting-delay-based stability criteria for neural networks with time-varying delay were investigated in [15]. The delay-partitioning method proved to be less conservative than most of the previous results, and the conservatism can be notably reduced by thinning the delay partitioning. However, the above methods suffer two common shortcomings. First, many matrix variables are introduced in the obtained results, which brings a large computational burden. Second, the information of neuron activation function is not adequately considered, which may lead to much conservatism.In this paper, the problem of delay-dependent stability analysis for neural networks with time-varying delays is investigated. Different from the previous methods of [4–8, 12–18], no delay-partitioning methods or free-weighting matrix methods are utilized. Instead, by taking more information of states and activation functions as augmented vectors, an augmented Lyapunov-Krasovskii functional is proposed. Then, inspired by the results of [19, 42], a less conservative condition is derived to guarantee the asymptotical stablity of the considered systems. Finally, two numerical examples are given to indicate significant improvements over some existing results.


## 2. Problem Formulation

Consider the following neural networks with time-varying delay:(2.1)x˙(t)=-Cx(t)+Ag(x(t))+Bg(x(t-τ(t)))+μ,
where x(t)=[x1(t),x2(t),…,xn(t)]T∈ℛn is the neuron state vector, g(x(·))=[g1(x1(·)),g2(x2(·)),…,gn(xn(·))]T∈ℛn denotes the neuron activation function, and μ=(μ1,μ2,…,μn)T∈ℛn is a constant input vector. A,B∈ℛn×n are the connection weight matrix and the delayed connection weight matrix, respectively. C=diag(C1,C2,…,Cn) with Ci>0,i=1,2,…,n. τ(t) is a time-varying continuous function that satisfies 0⩽τ(t)⩽h, τ˙(t)⩽u, where h and u are constants. In addition, it is assumed that each neuron activation function in (2.1), gi(·),i=1,2,…,n, is bounded and satisfies the following condition:
(2.2)ki-⩽gi(x)-gi(y)x-y⩽ki+,∀x,y∈R,x≠y,i=1,2,…,n,
where ki-,ki+,i=1,2,…,n are constants.Assuming thatx*=[x1*,x2*,…,xn*]T is the equilibrium point of (2.1) whose uniqueness has been given in [29] and using the transformation z(·)=x(·)-x*, system (2.1) can be converted to the following system:
(2.3)z˙(t)=-Cz(t)+Af(z(t))+Bf(z(t-τ(t))),
where z(t)=[z1(t),z2(t),…,zn(t)]T, f(z(·))=[f1(z1(·)),f2(z2(·)),…,fn(zn(·))]T and fi(zi(·))=gi(zi(·)+xi*)-gi(xi*),i=1,2,…,n. According to the inequality (2.2), one can obtain that
(2.4)ki-⩽fi(zi(t))zi(t)⩽ki+fi(0)=0,i=1,2,…,n.
Thus, under this assumption, the following inequality holds for any diagonal matrix Q4>0,
(2.5)zT(t)KQ4Kz(t)-fT(z(t))Q4f(z(t))⩾0,
where K=diag(k1,k2,…,kn), ki=max(|ki-|,|ki+|).Lemma 2.1 (see[43]).
For any constant matrixZ∈ℛn×n,Z=ZT>0, scalars h2>h1>0, such that the following integrations are well defined, then
(2.6)-(h2-h1)∫t-h2t-h1‍xT(s)Zx(s)ds⩽-∫t-h2t-h1‍xT(s)dsZ∫t-h2t-h1‍x(s)ds.


## 3. Main Results

In this section, a new Lyapunov functional is constructed and a less conservative delay-dependent stability criterion is obtained.Theorem 3.1.
For given scalarsh⩾0,u, diagonal matrices K1=diag(k1-,k2-,…,kn-), K2=diag(k1+,k2+,…,kn+), the system (2.3) is globally asymptotically stable if there exist symmetric positive matrices P=[Pij]4×4,Q=[Qij]2×2,X=[Xij]2×2,Y=[Yij]2×2,Qi(i=1,2,3), positive diagonal matrices T1,T2,T3,Q4,Δ=diag(δ1,δ2,…,δn),Λ=diag(λ1,λ2,…,λn), and any matrices Si(i=1,2,…,5) with appropriate dimensions, such that the following LMIs hold:
(3.1)[Y11Y12S1S2*Y22S3S4**Y11Y12***Y22]>0,(3.2)[Q1S5*Q1]>0,(3.3)[E𝒜TR*-R]<0,
where
(3.4)E=[E11E12E13E14E15-P13E17E18E19E1,10E1,11*E22E230E25000000**E3300E36-P22-P22-P23-P23-P24***E44E450E47E48E49E4,10E4,11****E550BTP12BTP12BTP13BTP13BTP14*****E66-P23T-P23T-P33-P33-P34******E77E78E7,9E7,10-P44*******E88E89E8,10-P44********-Y22-S40*********-Y220**********-Q3],𝒜=[-C00AB000000],R=h2Q1+h44Q2,E11=-P11C-CP11+P12+P12T+h(P14+P14T)+Q11+2K1ΛC1-2K2ΔC+X11+h2Y11-Q1-h2Q2+KQ4K-2K2T1K1+h44Q3,E12=Q1-S5,E13=-P12+S5,E14=P11A+P13+Q12-CΛ-K1ΛA+K2ΔA+CΔ+X12+h2Y12+T1(K1+K2),E15=P11B-K1ΛB+K2ΔB,E17=-CP12+P22+hP24T-P14+hQ2,E18=-CP12+P22+hP24T-P14+hQ2,E19=-CP13+P23+hP34T,E1,10=-CP13+P23+hP34T,E1,11=-CP14+P24+hP44,E22=-(1-u)X11-2Q1+S5+S5T-(1-u)KQ4K-2K2T2K1,E23=Q1-S5,E25=-(1-u)X12+T2(K1+K2),E33=-Q11-Q1-2K2T3K1,E36=-Q12+T3(K1+K2),E44=Q22+ΛA+ATΛ-ΔA-ATΔ+X22+h2Y22-Q4-2T1,E45=ΛB-ΔB,E47=ATP12+P23T,E48=ATP12+P23T,E49=ATP13+P33,E4,10=ATP13+P33,E4,11=ATP14+P34,E55=-(1-u)X22+(1-u)Q4-2T2,E66=-Q22-2T3,E77=-P24-P24T-Y11-Q2,E78=-P24-P24T-S1-Q2,E79=-P34T-Y12,E7,10=-P34T-S2,E88=-P24-P24T-Y11-Q2,E89=-P34T-S3T,E8,10=-P34T-Y12.Proof.
Construct a new Lyapunov functional as follow:(3.5)V(zt)=∑i=18‍Vi(zt),
where
(3.6)V1(zt)=[z(t)∫t-ht‍z(s)ds∫t-ht‍f(z(s))ds∫-h0‍∫t+θt‍z(s)dsdθ]T[P11P12P13P14*P22P23P24**P33P34***P44][z(t)∫t-ht‍z(s)ds∫t-ht‍f(z(s))ds∫-h0‍∫t+θt‍z(s)dsdθ],V2(zt)=∫t-ht‍[z(s)f(z(s))]T[Q11Q12*Q22][z(s)f(z(s))]ds,V3(zt)=∫t-τ(t)t[z(s)f(z(s))]T[X11X12*X22][z(s)f(z(s))]ds‍,V4(zt)=2∑i=1n‍{∫0zi(t)‍λi(fi(s)-ki-s)ds+∫0zi(t)‍δi(ki+s-fi(s))ds},V5(zt)=h∫-h0‍∫t+θt‍[z(s)f(z(s))]T[Y11Y12*Y22][z(s)f(z(s))]dsdθ,V6(zt)=h∫-h0‍∫t+θt‍z˙T(s)Q1z˙(s)dsdθ,V7(zt)=h22∫-h0‍∫θ0‍∫t+λt‍z˙T(s)Q2z˙(s)dsdλdθ,V8(zt)=h22∫-h0‍∫θ0‍∫t+λt‍zT(s)Q3z(s)dsdλdθ,V9(zt)=∫t-τ(t)t‍[zT(s)KQ4Kz(s)-fT(z(s))Q4f(z(s))]ds.Remark 3.2.
Since the terms2∑i=1nδi∫0zi(t)‍(ki+s-fi(s))ds in V4(zt) and ∫t-τ(t)t‍[zT(s)KQ4Kz(s)-fT(z(s))Q4f(z(s))]ds are taken into account, it is clear that the Lyapunov functional candidate in this paper is more general than that in [5, 6, 8, 9]. So the stability criteria in this paper may be more applicable.The time derivative ofV(zt) along the trajectory of system (2.4) is given by:
(3.7)V˙(zt)=∑i=18‍V˙i(zt),
where
(3.8)V˙1(zt)=2[z(t)∫t-ht‍z(s)ds∫t-ht‍f(z(s))ds∫-h0‍∫t+θt‍z(s)dsdθ]T[P11P12P13P14*P22P23P24**P33P34***P44][z˙(t)z(t)-z(t-h)f(z(t))-f(z(t-h))hz(t)-∫t-ht‍z(s)ds],(3.9)V˙2(zt)=[z(t)f(z(t))]T[Q11Q12*Q22][z(t)f(z(t))]-[z(t-h)f(z(t-h))]T[Q11Q12*Q22][z(t-h)f(z(t-h))],(3.10)V˙3(zt)⩽[z(t)f(z(t))]T[X11X12*X22][z(t)f(z(t))]-(1-u)[z(t-τ(t))f(z(t-τ(t)))]T[X11X12*X22][z(t-τ(t))f(z(t-τ(t)))],(3.11)V˙4(zt)=2[f(z(t))-K1z(t)]TΛz˙(t)+2[K2z(t)-f(z(t))]TΔz˙(t),z(t)=2[f(z(t))-K1z(t)]TΛ[-Cz(t)+Af(z(t))+Bf(z(t-τ(t)))]z(t)+2[K2z(t)-f(z(t))]TΔ[-Cz(t)+Af(z(t))+Bf(z(t-τ(t)))].
By the use of Lemma 2.1 and Theorem 3.1 in [42], one can obtain
(3.12)V˙5(zt)=h2[z(t)f(z(t))]T[Y11Y12*Y22][z(t)f(z(t))]-h∫t-ht[z(s)f(z(s))]T‍[Y11Y12*Y22][z(s)f(z(s))]ds=h2[z(t)f(z(t))]T[Y11Y12*Y22][z(t)f(z(t))]-h∫t-τ(t)t‍[z(s)f(z(s))]T[Y11Y12*Y22][z(s)f(z(s))]ds-h∫t-ht-τ(t)‍[z(s)f(z(s))]T[Y11Y12*Y22][z(s)f(z(s))]ds⩽h2[z(t)f(z(t))]T[Y11Y12*Y22][z(t)f(z(t))]-hτ(t)∫t-τ(t)t‍[z(s)f(z(s))]T×ds[Y11Y12*Y22]∫t-τ(t)t‍[z(s)f(z(s))]ds-hh-τ(t)∫t-ht-τ(t)‍[z(s)f(z(s))]Tds[Y11Y12*Y22]∫t-ht-τ(t)‍[z(s)f(z(s))]ds(3.13)⩽h2[z(t)f(z(t))]T[Y11Y12*Y22][z(t)f(z(t))]-[∫t-τ(t)t‍z(s)ds∫t-τ(t)t‍f(z(s))ds∫t-ht-τ(t)‍z(s)ds∫t-ht-τ(t)‍f(z(s))ds]T×[Y11Y12S1S2*Y22S3S4**Y11Y12***Y22][∫t-τ(t)t‍z(s)ds∫t-τ(t)t‍f(z(s))ds∫t-ht-τ(t)‍z(s)ds∫t-ht-τ(t)‍f(z(s))ds],
where [Y11Y12S1S2*Y22S3S4**Y11Y12***Y22]>0 should be satisfied. Similar to (3.12), one can obtain
(3.14)V˙6(zt)⩽h2z˙T(t)Q1z˙(t)-[∫t-τ(t)t‍z˙(s)ds∫t-ht-τ(t)‍z˙(s)ds]T[Q1S5*Q1][∫t-τ(t)t‍z˙(s)ds∫t-ht-τ(t)‍z˙(s)ds],
where [Q1S5*Q1]>0 should be satisfied.Consider the following:(3.15)V˙7(zt)=h44z˙T(t)Q2z˙(t)-h22∫-h0‍∫t+θt‍z˙T(s)Q2z˙(s)dsdθ⩽h44z˙T(t)Q2z˙(t)-(∫-h0‍∫t+θt‍z˙(s)dsdθ)TQ2(∫-h0‍∫t+θt‍z˙(s)dsdθ),=h44z˙T(t)Q2z˙(t)-(hz(t)-∫t-ht‍z(s)ds)TQ2(hz(t)-∫t-ht‍z(s)ds),(3.16)V˙8(zt)=h44zT(t)Q3z(t)-h22∫-h0‍∫t+θt‍zT(s)Q3z(s)dsdθ⩽h44zT(t)Q3z(t)-(∫-h0‍∫t+θt‍z(s)dsdθ)TQ3(∫-h0‍∫t+θt‍z(s)dsdθ)(3.17)V˙9(zt)⩽zT(t)KQ4Kz(t)-fT(z(t))Q4f(z(t))-(1-u)zT(t-τ(t))KQ4Kz(t-τ(t))+(1-u)fT(z(t-τ(t)))Q4f(z(t-τ(t))).
Furthermore, there exist positive diagonal matrices T1,T2,T3, such that the following inequalities hold based on (2.4):
(3.18)-2fT(z(t))T1f(z(t))+2zT(t)T1(K1+K2)f(z(t))-2zT(t)K2T1K1z(t)⩾0,(3.19)-2fT(z(t-τ(t)))T2f(z(t-τ(t)))+2zT(t-τ(t))T2(K1+K2)f(z(t-τ(t)))-2zT(t-τ(t))K2T2K1z(t-τ(t))⩾0,(3.20)-2fT(z(t-h))T3f(z(t-h))+2zT(t-h)T3(K1+K2)f(z(t-h)),-2zT(t-h)K2T3K1z(t-h)⩾0.
From (3.8)–, one can obtain that
(3.21)V˙(zt)⩽ζT(t)(E+𝒜TR𝒜)ζ(t),
where
(3.22)ζT(t)=[∫-h0‍∫t+θt‍zT(t)zT(t-τ(t))zT(t-h)fT(z(t))f(zT(t-τ(t)))f(zT(t-h))(s)∫t-τ(t)t‍zT(s)ds∫t-ht-τ(t)‍zT(s)ds∫t-τ(t)t‍fT(z(s))ds∫t-ht-τ(t)‍fT(z(s))ds(s)∫-h0‍∫t+θt‍zT(s)dsdθ].IfE+𝒜TR𝒜<0, then there exists a scalar ε>0, such that
(3.23)V˙(zt)⩽-εζT(t)ζ(t)⩽-εzT(t)z(t)<0,∀z(t)≠0.
Thus, according to [44], system (2.1) is globally asymptotically stable. By Schur complement, E+𝒜TR𝒜<0 is equivalent to (3.3), this completes the proof.Remark 3.3.
By taking the states∫t-τ(t)t‍fT(z(s))ds,∫t-ht-τ(t)‍fT(z(s))ds, as augmented variables, the stability condition in Theorem 3.1 utilizes more information about f(z(t)) on state variables, which may lead to less conservative results.Remark 3.4.
Recently, the reciprocally convex optimization technique [42] is used to reduce the conservatism of stability criteria for systems with time-varying delays. Motivated by this work, the proposed method of [42] was utilized in (3.12) and (3.14), which have potential to yield less conservative conditions. However, an augmented vector with ∫t-τ(t)t‍zT(s)ds,∫t-ht-τ(t)‍zT(s)ds,∫t-τ(t)t‍fT(z(s))ds,∫t-ht-τ(t)‍fT(z(s))ds,∫-h0‍∫t+θt‍zT(s)dsdθ was used, which is different from the method of [42].
In many cases,u is unknown. Considering this situation, a rate-independent corollary for the delay τ(t) satisfying 0≤τ(t)≤h is derived by setting X=0,Q4=0 in the proof of Theorem 3.1.Corollary 3.5.
For given scalarh⩾0, diagonal matrices K1=diag(k1-,k2-,…,kn-), K2=diag(k1+,k2+,…,kn+), the system (2.3) is globally asymptotically stable if there exist symmetric positive matrices P=[Pij]4×4,Q=[Qij]2×2,Y=[Yij]2×2,Qi(i=1,2,3), positive diagonal matrices T1,T2,T3,Δ=diag(δ1,δ2,…,δn),Λ=diag(λ1,λ2,…,λn), and any matrices Si(i=1,2,…,5) with appropriate dimensions, such that (3.1), (3.2) and the following LMI hold:
(3.24)[E-𝒜TR*-R]<0,
where
(3.25)E-=(E-11E12E13E-14E15-P13E17E18E19E1,10E1,11*E-22E230E-25000000**E3300E36-P22-P22-P23-P23-P24***E-44E450E47E48E49E4,10E4,11****-2T20BTP12BTP12BTP13BTP13BTP14*****E66-P23T-P23T-P33-P33-P34******E77E78E7,9E7,10-P44*******E88E89E8,10-P44********-Y22-S40*********-Y220**********-Q3),E-11=-P11C-CP11+P12+P12T+h(P14+P14T)+Q11+2K1ΛC4C-2K2ΔC+h2Y11-Q1-h2Q2+KQ4K-2K2T1K1+h44Q3,E-14=P11A+P13+Q12-CΛ-K1ΛA+K2ΔA+CΔ+h2Y12+T1(K1+K2),E-22=-2Q1+S5+S5T-2K2T2K1,E-25=T2(K1+K2),E-44=Q22+ΛA+ATΛ-ΔA-ATΔ+h2Y22-Q4-2T1.
The other Eij is defined in Theorem 3.1.


## 4. Numerical Examples

In this section, two numerical examples are given to demonstrate the effectiveness of the proposed method.Example 4.1.
Consider the system (2.3) with the following parameters:
(4.1)C=[1.276900000.623100000.923000000.4480],A=[-0.03730.4852-0.33510.2336-1.60330.5988-0.32241.23520.3394-0.0860-0.3824-0.5785-0.13110.3253-0.9534-0.5015],B=[0.8674-1.2405-0.53250.02200.0474-0.91640.03600.98161.84952.6117-0.37880.8428-2.04130.51791.1734-0.2775],K1=diag{0,0,0,0},K2=diag{0.1137,0.1279,0.7994,0.2368},f1(s)=0.05685(|s+1|-|s-1|),f2(s)=0.06395(|s+1|-|s-1|),f3(s)=0.3997(|s+1|-|s-1|),f4(s)=0.1184(|s+1|-|s-1|).
The upper bounds of h for different u are derived by Theorem 3.1 in our paper and the results in [13–16] are listed in Table 1. According to Table 1, this example shows that the stability condition in this paper gives much less conservative results than those in the literature. For h=2.7098, the global asymptotic stability with the initial state(-0.2,0.3,-0.4,0.2)T is shown in Figure 1.Table 1
Allowable upper bound ofh for different u.
Method
u
=
0.5
u
=
0.9
[3, 4]
2.1502
1.3164
[5]
2.2245
1.5847
[6]
2.5376
2.0853
[11]
2.2261
1.6035
[15]
2.5915
2.1306
[16] (m=2)
2.6438
2.1349
[17] (m=2)
2.4530
1.8593
Theorem3.1
2.7098
2.2055Figure 1
The dynamical behavior ofz(t).Example 4.2.
Consider the system (2.3) with the following parameters:
(4.2)C=diag(2,2),A=[11-1-1],B=[0.88111],(4.3)K1diag{0,0},K2=diag{0.4,0.8}.
Our purpose is to estimate the allowable upper bounds delayh under different u such that the system (2.3) is globally asymptotically stable. According to the Table 2, this example is given to indicate significant improvements over some existing results.Table 2
Allowable upper bound ofh for different u.
Method
u
=
0.8
u
=
0.9
Unknownu
[3, 4]
1.2281
0.8636
0.8298
[5]
1.6831
1.1493
1.0880
[6]
2.3534
1.6050
1.5103
[11]
1.5948
1.1323
1.0743
[15]
2.5406
1.7273
—
[16] (m=2)
2.2495
1.5966
1.4902
[17] (m=2)
2.1150
1.4286
1.3126
Our results
2.3731
1.6219
1.5103


## 5. Conclusions

In this paper, a new Lyapunov functional was proposed to investigate the stability of neural networks with time-varying delays. Some improved generalized delay-dependent stability criteria have been established. The obtained criteria are less conservative because a convex optimization approach is considered. Finally, two numerical examples have shown that these new stability criteria are less conservative than some existing ones in the literature.



---

*Source: 910140-2012-10-15.xml*