# Input-to-State Stability of Stochastic Memristive Neural Networks with Time-Varying Delay

Authors: Xu Y. Lou; Qian Ye
Journal: Mathematical Problems in Engineering
 (2015)

---

## Abstract

This paper is concerned with the input-to-state stability problem of a class
of memristive neural networks. We consider the neural networks that take into account
both the stochastic effects and time-varying delay, and introduce the notions of meansquare
exponential input-to-state stability. Using the stochastic analysis theory and Itô
formula for stochastic differential equations, we establish sufficient conditions for both
mean-square exponential input-to-state stability and mean-square exponential stability.
Numerical simulations are also provided to demonstrate the theoretical results.


---

## Body

## 1. Introduction

In the recent years, the memristor-based or memristive neural network models have been extensively investigated and successfully applied to function approximation [1], associative memory [2], chaos synchronization [3], and image processing [4]. As is well known, due to the existence of time delays, the stability issue in delayed memristive neural networks becomes one of the most important problems and there have been many stability results on delayed memristive neural networks reported in the literature; for instance, see [5–9] and references therein. The authors in [5] studied the global exponential stability for a class of memristor-based recurrent neural networks with time-varying delays. Exponential stability was addressed in [6] for a class of stochastic memristor-based recurrent neural networks with time-varying delays. The authors in [7] considered the exponential dissipativity of memristor-based recurrent neural networks with time-varying delays. The recent work of [8] analyzed the robust stability for uncertain memristive neural networks with norm-bounded parameter uncertainty.Despite the rich achievements, most of the above results mainly focused on deterministic memristive neural network models rather than stochastic memristive neural network models. However, while modeling neural networks, noise is unavoidable and should be taken into consideration. In real neural networks, as pointed out in [10], the synaptic transmission can be viewed as a noisy process introduced by random fluctuations from the release of neurotransmitters and other probabilistic causes [11]. Therefore, it is of practical importance to analyze the dynamics of the stochastic memristive neural networks (SMNNs). However, to the best of our knowledge, few researches focus on the stability of stochastic memristor-based neural networks.The input-to-state stability (ISS) [12] is an important property in nonlinear systems as it provides an effective way to tackle the stabilization of nonlinear systems or the problem of robust/adaptive nonlinear control in the presence of various uncertainties arising from control engineering applications. As we know, the states in stochastic neural networks may remain bounded instead of converging to an equilibrium point as time goes to infinity. The ISS analysis opens a new path for application of dynamic neural networks to nonlinear control. For instance, when we design a nonlinear controller based on an identified model built by using this kind of neural network from experimental data, the identification error remains bounded even in the presence of model mismatching [13]. Therefore, it is significant to study the input-to-state stability of neural networks [11, 13]. Note that the ISS property of neural networks investigated in [13] was analyzed in deterministic case and without considering delays. Moreover, only asymptotical ISS property was considered in that work rather than exponential ISS property. The recent work of [11] presented sufficient conditions for mean-square exponential ISS of stochastic delayed neural networks, but the synaptic weights are constants. However, memristor-based synapses are shown to be essential for performing useful computation and adaptation in large scale artificial neural networks [14]. Moreover, memristive neural networks have been used in image denoising, edge extraction, and Chinese character recognition and have shown that the memristor synaptic neural network has more bionic feasibility, more integrated and more easy to replace the template [4]. Therefore, it is vital to deeply explore the ISS property of delayed SMNNs and this is also a difficult task due to the presence of the memristor-based synapses.In this paper, we attempt to construct SMNNs with time-varying delay and focus on the mean-square exponential input-to-state stability (eISS). The proposed method is applicable to deal with the synchronization problem of SMNNs. We present sufficient conditions for mean-square eISS and the mean-square exponential stability based on the stochastic analysis theory and Itô formula. In addition, the exploited conditions do not require assuming the boundedness, differentiability, and monotonicity of the activation functions in SMNNs.The paper is organized as follows. In Section2, we introduce the model of SMNNs and some preliminaries. In Section 3, some sufficient conditions are derived to guarantee the mean-square eISS and mean-square exponential stability of the proposed model. Section 4 gives an example to illustrate the effectiveness of our results. Finally, conclusions are drawn in Section 5.Notations. Throughout the paper, we write Rn for n-dimensional Euclidean space. Given a vector x∈Rn, x=(∑i=1n‍xi2)1/2 denotes the Euclidean norm. l∞ denotes the class of essentially bounded functions u from [0,∞) to Rn with u∞=esssupt≥0u(t)<∞. co⁡[β_,β¯] denotes closure of the convex hull of [β_,β¯]. Let C([-τ,0];Rn) denote the family of continuous functions ϕ from [-τ,0] to Rn with the uniform norm ϕ=sup⁡-τ≤θ≤0ϕ(θ). Let LF02([-τ,0];Rn) denote the family of all F0 measurable, C([-τ,0];Rn)-valued stochastic variables ϕ={ϕ(s):-τ≤s≤0} such that ∫-τ0‍E[ϕs2]ds<∞, where E[·] stands for the correspondent expectation operator with respect to the given probability measure P.


## 2. Model Description and Preliminaries

Consider an SMNN described by the following differential equations:(1)dxit=-dixi(t)+∑j=1n‍aij(xi(t))fj(xj(t))+∑j=1n‍bij(xi(t))gj(xj(t-τ(t)))+ui(t)·dt+∑j=1n‍σij(t,xj(t),xj(t-τ(t)))dωj(t),i=1,2,…,n,
where n denotes the number of the neurons in the network, xi(t) is the state of the ith neuron at time t,fj(xj(t)) and gj(xj(t)) represent the neuron activation functions of the jth neuron at time t, and  ui(t)∈l∞ is the external constant input of the ith neuron at time t. τ(t) corresponds to the transmission delay which is supposed to be differential and satisfies 0<τ(t)≤τ, τ˙(t)≤ρ<1, where τ and ρ are positive constants. ωj is a standard Brownian motion defined on the complete probability space (Ω,F,P) with a natural filtration {Ft}t≥0. σij:R×R×R→R is a Borel measurable function. D=diag⁡(d1,…,dn)>0 is a self-feedback connection matrix. A(x(t))=(aij(xi(t)))n×n and B(x(t))=(bij(xi(t)))n×n are memristive connection weights, which represent the neuron interconnection matrix and the delayed neuron interconnection matrix, respectively. In artificial neural networks, aij(xi) and bij(xi) are state-dependent bounded functions for the existence of the memristors working as synaptic weights. The connection weights change according to the state of each subsystem.The memristor-based synaptic weightsaij(xi) and bij(xi) will change, as pinched hysteresis loops change [15]. According to the feature of the memristor, let(2a)aij(xi)=aij′,|xi(t)|≤Ti,aij′′,|xi(t)|>Ti,(2b)bij(xi)=bij′,|xi(t)|≤Ti,bij′′,|xi(t)|>Ti,where Ti>0 and aij′,aij′′,bij′,bij′′, i,j=1,2,…,n, are known constants relating to memristance.Leta_ij=min⁡{aij′,aij′′}, a¯ij=max⁡{aij′,aij′′}, b_ij=min⁡{bij′,bij′′}, and b¯ij=max⁡{bij′,bij′′} for i,j=1,2,…,n. Using the set-valued analysis [16], it follows from (1) that
(3)dxit∈-dixi(t)+∑j=1n‍co⁡[a_ij,a¯ij]fj(xj(t))+∑j=1n‍co⁡[b_ij,b¯ij]gj(xj(t-τ(t)))+ui(t)·dt+∑j=1n‍σij(t,xj(t),xj(t-τ(t)))dωj(t).Then by Filippov’s Theorem [17], there exist a^ij∈co⁡[a_ij,a¯ij], b^ij∈co⁡[b_ij,b¯ij], such that solutions to the differential inclusion (3) are the trajectories of the following system
(4)dxit=-dixi(t)+∑j=1n‍a^ijfj(xj(t))+∑j=1n‍b^ijgj(xj(t-τ(t)))+uit∑j=1n‍·dt+∑j=1n‍σij(t,xj(t),xj(t-τ(t)))dωj(t),
for i=1,2,…,n.The initial conditions associated with system (1) are given by
(5)xi(t)=ϕi(t),-τ≤t≤0,i=1,2,…,n,
where ϕi(·) is bounded and continuous on [-τ,0].In order to obtain the main theorem, we require the following assumptions.Assumption 1.
For alli=1,2,…,n, the neuron activation functions fi and gi satisfy the following Lipschitz condition:
(6)fi(x)-fi(y)≤αix-y,gi(x)-gi(y)≤βix-y,
where x,y∈R,αi,βi are positive Lipschitz constants.Assumption 2.
For alli,j=1,2,…,n, there exist nonnegative constants μij and νij such that
(7)σijt,x,y-σijt,x^,y^2≤μijx-x^2+νijy-y^2,
for all x,x^,y,y^∈R, i,j=1,2,…,n.Assumption 3.
Considerfj(0)=gj(0)=0, σij(0,0,0)=0, i,j=1,2,…,n.Define(8)Fxit,xit-τt=-dixi(t)+∑j=1n‍a^ijfj(xj(t))+∑j=1n‍b^ijgj(xj(t-τ(t)))+ui(t),
for all i=1,2,…,n. Under Assumptions 1 and 2 and by using the inequality (a1+⋯+an)2≤n(a12+⋯+an2), we have
(9)Fxit,xit-τt-Fyi,yit-τt2≤3di2xit-yit2+3∑j=1n‍a^ij2αj2xjt-yjt2+3∑j=1n‍b^ij2βj2xjt-τt-yjt-τt2,σijt,xit,xit-τt-σijt,yit,yit-τt2≤μijxit-yit2+νijxjt-τt-yjt-τt2.
Therefore, F(xi(t),xi(t-τ(t))) and σij(t,xi(t),xi(t-τ(t))) satisfy the Lipschitz condition. Moreover, it follows from Assumptions 1–3 that
(10)Fxit,xit-τt2≤4di2xit2+4∑j=1n‍a^ij2αj2xjt2+4∑j=1n‍b^ij2βj2xjt-τt2+4uit,σijt,xit,xit-τt2≤Kij(xit2+xjt-τt2),
where Kij=max⁡{μij,νij}. Therefore, F(xi(t),xi(t-τ(t))) and σij(t,xi(t),xi(t-τ(t))) satisfy the linear growth condition. It follows by Theorem 3.1 in the book [18] that there exists a unique solution of system (4) or (1). Let x(t;ϕ) denote the solution from the initial data x(s)=ϕ(s) on -τ≤s≤0 in LF02([-τ,0];Rn). It is clear that, under Assumption 3, system (1) with zero inputs admits a trivial solution x(t;0)=0.Definition 4.
The trivial solution of system (1) is said to be mean-square exponentially input-to-state stable if, for every ϕ∈LF02([-τ,0];Rn) and u(t)∈l∞, there exist scalars α>0,β>0, and γ>0 such that the following inequality holds:
(11)E[xt;ϕ2]≤αe-βtE[ϕ2]+γu∞2.Remark 5.
It follows from the definition that if the system is mean-square input-to-state stable, then the states remain bounded when the inputs of the system are bounded. In addition, the definition is more general than mean-square asymptotic stability or mean-square exponential stability, which is explicit by takingu=0.


## 3. Main Results

In this section, the mean-square eISS and exponential stability of the trivial solution of SMNN (1) are addressed.Theorem 6.
Under Assumptions1–3, the trivial solution of system (1) is mean-squre exponentially input-to-state stable, if there exist positive scalars λ,pj,qj(j=1,2,…,n) and constants εk>0,k=1,2 such that the following conditions hold:
(12)2pjdj≥λpj+∑l=1n‍plμlj+pj+qj+pj∑l=1n‍a^jlαl1ε1+∑l=1n‍pla^ljαjε1+pj∑l=1n‍b^jlβl1ε2,(1-ρ)qj≥eλτ∑l=1n‍plb^ljβjε2+eλτ∑l=1n‍plνlj.Proof.
For the sake of simplicity, denoteσ(t)=(σij(t))n×n, where σij(t)=σij(t,xj(t),xj(t-τ)). Let us construct the following Lyapunov-Krasovskii functional:
(13)V(t,x(t))=eλt∑i=1n‍pixi2(t)+∫t-τ(t)t‍eλs∑i=1n‍qixi2(s)ds.
According to the Itô formula, it follows
(14)dVt,xt=LVt,xtdt+Vxt,xtσtdωt,
where
(15)Vxt,xt=∂Vt,xt∂x1,…,∂Vt,xt∂xn,
and L is the weak infinitesimal operator such that
(16)LV(t,x(t))=λeλt∑i=1n‍pixi2(t)+2eλt∑i=1n‍pixi(t)·-dixi(t)+∑j=1n‍a^ijfj(xj(t))mmmm+∑j=1n‍b^ijgjxjt-τt+ui(t)+eλt∑i=1n‍pi∑j=1n‍σij2t,xjt,xjt-τt+eλt∑i=1n‍qixi2(t)-(1-τ˙(t))eλt-τt·∑i=1n‍qixi2(t-τ(t)).
Based on Assumptions 1–3, we have
(17)LVt,xt≤λeλt∑i=1n‍pixi2(t)-2eλt∑i=1n‍pidixi2(t)+2eλt∑i=1n‍pixi(t)∑j=1n‍a^ijfj(xj(t))+2eλt∑i=1n‍pixi(t)∑j=1n‍b^ijgj(xj(t-τ(t)))+2eλt∑i=1n‍pixi(t)ui(t)+eλt∑i=1n‍pi∑j=1n‍μijxj2(t)+eλt∑i=1n‍pi∑j=1n‍νijxj2(t-τ(t))+eλt∑i=1n‍qixi2(t)-1-ρeλt-τ∑i=1n‍qixi2t-τt≤eλt∑i=1n‍λpi-2pidi+∑j=1n‍pjμji+qixi2(t)+2eλt∑i=1n‍pixit∑j=1n‍a^ijαjxjt+2eλt∑i=1n‍pixit∑j=1n‍b^ijβjxjt-τt+2eλt∑i=1n‍pixi(t)ui(t)+eλt∑i=1n‍-(1-ρ)e-λτqi+∑j=1n‍pjνjixi2(t-τ(t)).
Using the well-known Young’s inequality 2ab≤a2/ε+εb2, it follows that
(18)2eλt∑i=1n‍pixit∑j=1n‍a^ijαjxjt≤eλt∑i=1n‍∑j=1n‍pia^ijαj1ε1xit2+ε1xjt2,2eλt∑i=1n‍pixit∑j=1n‍b^ijβjxjt-τt≤eλt∑i=1n‍∑j=1n‍pib^ijβj1ε2xit2+ε2xjt-τt2,2eλt∑i=1n‍pixi(t)ui(t)≤eλt∑i=1n‍pixit2+uit2,
where εi, i=1,2, are any positive constants.
Substituting (18) into (17) yields
(19)LVt,xt≤eλt∑i=1n‍λpi-2pidi+∑j=1n‍pjμji+qixi2(t)+eλt∑i=1n‍∑j=1n‍pia^ijαj1ε1xit2+ε1xjt2+eλt∑i=1n‍∑j=1n‍pib^ijβj1ε2xit2+ε2xjt-τt2+eλt∑i=1n‍pixit2+uit2+eλt∑i=1n‍-(1-ρ)e-λτqi+∑j=1n‍pjνjixi2(t-τ(t)).
Consequently, we can derive(20)LVt,xt≤-eλtmin⁡1≤j≤n2pjdj-λpj-∑l=1n‍plμlj-qjmmmmmmmm-pj∑l=1n‍a^jlαl1ε1-∑l=1n‍pla^ljαjε1mmmmmmmm-pj∑l=1n‍b^jlβl1ε2-pj∑i=1n‍xi2(t)-eλtmin⁡1≤j≤n1-ρqj-eλτ∑l=1n‍plb^ljβjε2-eλτ∑l=1n‍plνlj·∑i=1n‍xi2t-τt+eλtmax⁡1≤j≤npjuit∞2.
In order to further derive the eISS in mean-square sense, similar to [11], define ρk:=inf⁡{s≥0:|x(s)|≥k} as a stopping time. Applying the Dynkin formula, we obtain
(21)EVt∧ρk,xt∧ρk-EV0,x0=E∫0t∧ρk‍LVx,xsds;
that is,
(22)EVt∧ρk,xt∧ρk=E[V(0,x(0))]+E∫0t∧ρk‍LV(x,x(s))ds.
Using the monotone convergence theorem and conditions (12), when k goes to infinity one can easily derive
(23)EVt,xt≤EV0,x0+u∞2max⁡1≤j≤npj∫0t‍eλsds=EV0,x0+1λu∞2max⁡1≤j≤npjeλt-1=∑i=1n‍piExi20+∫-τ00‍eλsE∑i=1n‍qixi2sds+1λu∞2max⁡1≤j≤npjeλt-1≤max⁡1≤j≤npj∑i=1n‍E[xi2(0)]+max⁡1≤j≤nqj∫-τ00‍eλs∑i=1n‍Exi2sds+1λu∞2max⁡1≤j≤npjeλt-1≤max⁡1≤j≤npj+τmax⁡1≤j≤nqjEϕ2+1λu∞2max⁡1≤j≤npjeλt.
On the other hand, it follows from (13) that
(24)E[V(t,x(t))]≥Eeλt∑i=1n‍pixi2t≥eλtmin⁡1≤i≤npiExt2.
Combining (23) and (24) yields
(25)Ext2≤αe-λtEϕ2+γu∞2,
where α=(max⁡1≤i≤npi+τmax⁡1≤i≤nqi)/min⁡1≤i≤npi, γ=(max⁡1≤i≤npi)/(λmin⁡1≤i≤npi). Therefore, the inequality in Definition 4 holds, which verifies that the trivial solution of system (1) is mean-square exponentially input-to-state stable. The proof of Theorem 6 is completed.Remark 7.
Since there are several free parameters related to the sufficient conditions of Theorem6, by selecting proper parameters one has more choices to sufficiently satisfy the conditions for a given neural network. Theorem 6 seems to be a natural extension of the main scheme proposed in [11] for stochastic delayed neural networks, but the ISS analysis is more challenging and the conditions are different due to the presence of the memristor-based synaptic weights. Moreover, the criterion in Theorem 6 is less conservative due to the introduction of extra positive constants ε1,ε2.Remark 8.
With the progress in the experimental realization of memristive devices, the implementation of artificial neural networks with memristive synapses is feasible. For instance, a simple winner-take-all architecture in neural systems is implemented in [19] based on spike-timing-dependent plasticity (STDP) using the device integration in memristor-MOS technology.
Whenu(t)≡0, Theorem 6 reduces to mean-square exponential stability of the SMNNs. This conclusion is stated as follows.Corollary 9.
Under Assumptions1–3, the trivial solution of system (1) with u(t)≡0 is mean-square exponentially stable, if there exist positive scalars λ,pj,qj(j=1,2,…,n) and constants εk>0, k=1,2, such that the following conditions hold:
(26)2pjdj≥λpj+∑l=1n‍plμlj+pj+qj+pj∑l=1n‍a^jlαl1ε1+∑l=1n‍pla^ljαjε1+pj∑l=1n‍b^jlβl1ε2,(1-ρ)qj≥eλτ∑l=1n‍plb^ljβjε2+eλτ∑l=1n‍plνlj.


## 4. Example

In this section, a numerical example is employed to illustrate our results. Simulation results show that the derived conditions are valid.Example 1.
Consider a two-dimensional SMNN with time-varying delay:(27)dx1tdx1t=x1(t)x2(t)-Dx1(t)x2(t)+A(x)f(x1(t))f(x2(t))+B(x)g(x1(t-τ(t)))g(x2(t-τ(t)))+u1(t)u2(t)·dt+σt,x,xt-τdω1tdω2t,
where f(s)=g(s)=0.5tanh(s), u1(t)=u2(t)=1.8cos⁡(5t), D=diag⁡(3,4), τ(t)=0.3sin(t)+0.2, and
(28)σ(t,x,x(t-τ))=0.3x1t0.4x1t-τt0.2x2t+x2t-τt0.5x2t-τt,Ax=a11x1a12x1a21x2a22x2,B(x)=b11(x1)b12(x1)b21(x2)b22(x2)
with
(29)a11(x1)=0.5,|x1(t)|<1,-0.5,|x1(t)|≥1,a12(x1)=-0.5,|x1(t)|<1,0.5,|x1(t)|≥1,a21(x2)=0.4,|x2(t)|<1,-0.4,|x2(t)|≥1,a22(x2)=-0.4,|x2(t)|<1,0.4,|x2(t)|≥1,b11(x1)=-0.2,|x1(t)|<1,0.2,|x1(t)|≥1,b12(x1)=0.2,|x1(t)|<1,-0.2,|x1(t)|≥1,b21(x2)=0.3,|x2(t)|<1,-0.3,|x2(t)|≥1,b22(x2)=-0.3,|x2(t)|<1,0.3,|x2(t)|≥1.Obviously, the activation functionsf(·) and g(·) satisfy Assumptions 3 and 1 with α1=α2=β1=β2=0.5. In addition, σ(t,x,x(t-τ)) satisfies Assumption 2 with μ11=0.09, μ12=0, μ21=0.08, μ22=0, ν11=0, ν12=0.16, ν21=0.08, and ν22=0.25. According to the definition of τ(t), we have τ=0.5, ρ=0.3. Take λ=0.2, p1=0.6, p2=0.4, q1=1.0, q2=1.2, and ε1=ε2=1. Then, it is easy to check conditions in (12) as follows:(30)M1=1.144,M2=1.01,N1=0.532,N2=0.4908,
where
(31)Mj=2pjdj-λpj-∑l=12‍plμlj-pj-qj-pj∑l=12‍a^jlαl-∑l=12‍pla^ljαj-pj∑l=12‍b^jlβl,Nj=(1-ρ)qj-eλτ∑l=12‍plb^ljβj-eλτ∑l=12‍plνlj.Therefore, all conditions in Theorem6 are satisfied, which implies that the trivial solution of system (27) is mean-square exponentially input-to-state stable. Figure 1 shows the time responses of the state x with 20 different initial values (where x(s)=ϕ(0),s∈[-τ,0)). When the inputs u1(t)=u2(t)=0, by Corollary 9, the trivial solution of system (27) is mean-square exponentially stable, which is verified in Figure 2.Figure 1
Transient response of variablesx(t) in system (27).Figure 2
Transient response of variablesx(t) with zero inputs in system (27).


## 5.  Conclusions

In this paper, the input-to-state stability for a class of stochastic memristive recurrent neural networks with time-varying delay has been addressed. By utilizing the Lyapunov functional method and combining the stochastic analysis theory, a sufficient criterion for exponential input-to-state stability of the memristive recurrent neural networks has been obtained, which has never been discussed in the field of stochastic memristive neural networks. Finally, simulation results have been proposed to illustrate the effectiveness of the proposed results. Further investigations will be aimed at studying the input-to-output stability of stochastic memristive neural networks and the learning mechanism of artificial neural networks with memristor-based synaptic weights.



---

*Source: 140857-2015-02-23.xml*