# Analysis and Design of Associative Memories for Memristive Neural Networks with Deviating Argument

Authors: Jin-E Zhang
Journal: Mathematical Problems in Engineering
 (2017)

---

## Abstract

We investigate associative memories for memristive neural networks with deviating argument. Firstly, the existence and uniqueness of the solution for memristive neural networks with deviating argument are discussed. Next, some sufficient conditions for this class of neural networks to possess invariant manifolds are obtained. In addition, a global exponential stability criterion is presented. Then, analysis and design of autoassociative memories and heteroassociative memories for memristive neural networks with deviating argument are formulated, respectively. Finally, several numerical examples are given to demonstrate the effectiveness of the obtained results.


---

## Body

## 1. Introduction

In recent decades, analysis and design of neurodynamic systems have received much attention [1–16]. Specifically, neurodynamics of associative memories is a hot research issue [9–16]. Associative memories refer to brain-inspired computing designed to store a set of prototype patterns such that the stored patterns can be retrieved with the recalling probes containing sufficient information about the contents of patterns. In associative memories, for any given probe (e.g., noisy or corrupted version of a prototype pattern), the retrieval dynamics can converge to an ideal equilibrium point representing the prototype pattern. At present, system analysts have two main theories for explaining strange dynamical behaviors in recalling processes: autoassociative memories and heteroassociative memories [12, 13]. Two types of methods are usually used for solving problems in regard to analysis and synthesis of associative memories. The first is when neurodynamic systems are multistable, and then system states may converge to locally stable equilibrium points, and these equilibrium points are encoded as the memorized patterns. The second is when neurodynamic systems are globally monostable, and then the memorized patterns are associated with the external input patterns.As we know, the human brain is believed to be the most powerful information processor because of its structure of synapses. Actually, there is always a high demand for a single device to emulate artificial synaptic functions. Using a memristor, essential synaptic plasticity and learning behaviors can be mimicked. By integrating memristors into a large-scale neuromorphic circuit, memristive neural networks based neuromorphic circuits demonstrate spike-timing-dependent plasticity (a form of the Hebbian learning rule) [17–21]. From the viewpoint of theory and experiment, a memristive neural network model is a state-dependent switching system. On the other hand, this new dynamical system shows many of the characteristic features of magnetic tunnel junction [19, 20]. For these reasons, system analysis and integration for memristive neural networks will be extremely tricky. For the moment, there are two major methods for qualitative analysis of memristive neural networks. One is the differential inclusions approach [17–20], and the other is the fuzzy uncertainty approach [21]. For the differential inclusions approach, the core idea is to integrate a switched network cluster. For the fuzzy uncertainty approach, the central idea is to divide multijump network flows into a continuous subsystem and bounded subsystem. The two kinds of analysis frameworks are just the type and level of points, not the merits of good points of difference. From the perspective of cybernetics, many reported analytical skills are merely a breakthrough of system theory more than substance. How to develop more effective analysis methods for memristive neural networks is worth studying.For the past few years, hybrid dynamic systems have remained one of the most active fields of research in the control community [22–30]. For instance, to describe the stationary distribution of temperature along the length of a wire that is bended, the nonlinear dynamic model with deviating argument is often used. The right-hand side in nonlinear systems with deviating argument features a combination of continuous and discrete systems. Thus, nonlinear systems with deviating argument unify the advanced and retarded systems. Because of this property, related works on the control strategies for nonlinear systems with deviating argument are relatively rare. Viewed from systems involving the interplay, differential equations and difference equations are included in the nonlinear systems with deviating argument [28–30]. However, it is still obvious that many basic issues on nonlinear systems with deviating argument remain to be addressed, such as nonlinear dynamics, systems design, and analysis.Inspired by the above discussions, the goal of this paper is to formulate the analysis and design of associative memories for a class of memristive neural networks with deviating argument. On the whole, the highlights of this paper can be outlined as follows:(1)
Sufficient conditions are derived to ascertain the existence, uniqueness, and global exponential stability of the solution for memristive neural networks with deviating argument.(2)
The synthetic mechanism of the analysis and design of associative memories for a general class of memristive neural networks with deviating argument is revealed.(3)
A uniform associative memories model, which unites autoassociative memories and heteroassociative memories, is proposed for memristive neural networks with deviating argument.In addition, it should be noted that when special and strict conditions are exerted, associative memories’ performance of neural networks is usually limited. Therefore, the methods applicable to conventional stability analysis of neural networks cannot be directly employed to investigate the associative memories for memristive neural networks. Moreover, the study of dynamics for memristive neural networks with deviating argument is not an easy work, since the conditions for dynamics of neural networks cannot be simply utilized to analyze hybrid dynamic systems with deviating argument. In this paper, according to the fuzzy uncertainty approach, in combination with the theories of set-valued maps and differential inclusions, analysis and design of associative memories for memristive neural networks with deviating argument are described in detail.The rest of the paper is arranged as follows. Design problem and preliminaries are given in Section2. Main results are stated in Section 3. In Section 4, several numerical examples are presented. Finally, concluding remarks are given in Section 5.


## 2. Design Problem and Preliminaries

### 2.1. Problem Description

LetΩ=β=(β1,β2,…,βn)T∣βi={-p,p},i=1,2,…,n be the set of n-dimensional bipolar vectors, where p is a positive constant and T denotes the transpose of a vector or matrix. The problem considered is described as follows.Givenl(l≤2n) pairs of bipolar vectors (u(1),q(1)),(u(2),q(2)),…,(u(l),q(l)), where u(r),q(r)∈Ω, for r=1,2,…,l, design an associative memory model which satisfies the notion that the output of the model will converge to the corresponding pattern q(r) when u(r) is fed to the model input as a memory pattern.Definition 1.
The neural network is said to be an autoassociative memory ifu(r)=q(r) and a heteroassociative memory if u(r)≠q(r), for r=1,2,…,l.In this paper, letN be the natural number set; the norm of vector x=(x1,x2,…,xn)T is defined as x=∑i=1n|xi|. Denote Rn as the n-dimensional Euclidean space. Fix two real number sequences ηk, ϑk, k∈N, satisfying ηk<ηk+1, ηk≤ϑk≤ηk+1, and limk→+∞ηk=+∞.


### 2.2. Model

Consider the following neural network model:(1)x˙t=-Axt+Bxtfxt+Cxtfxγt+Eur,wt=fxt,r=1,2,…,l,where x(t)=(x1t,x2t,…,xn(t))T∈Rn is the neuron state, A=diag(a1,a2,…,an), which is a constant diagonal matrix with ai>0, i∈1,2,…,n, indicates the self-inhibition matrix, B(x(t))=(bij(xj(t)))n×n and C(x(t))=(cij(xj(t)))n×n are connection weight matrices at time t and γ(t), respectively, bij(xj(t)) and cij(xj(t)) are defined by(2)bijxjt=b^ij,xjt>Tj,bˇij,xjt<Tj,cijxjt=c^ij,xjt>Tj,cˇij,xjt<Tj,for i,j=1,2,…,n, and bij(±Tj)=b^ij or bˇij, cij(±Tj)=c^ij or cˇij, where Tj>0 represents the switching jump, and b^ij,bˇij,c^ij, and cˇij are all constants. f(x(·))=(f1(x1(·),f2(x2(·),…,fn(xn(·))T is the activation function. The deviating function γ(t)=ϑk, when t∈[ηk,ηk+1) for any k∈N. E stands for a transform matrix of order n satisfying Eu(r)=q(r) for r=1,2,…,l. u(r) and q(r) indicate the external input pattern and the corresponding memorized pattern, respectively. w(t)=(w1(t),w2(t),…,wn(t))T is the output of (1) corresponding to u(r).Neural network model (1) is called an associative memory if the output w(t) converges to q(r).Obviously, neural network model (1) is of hybrid type: switching and mixed. When |xj(t)|>Tj, bij(xj(t))=b^ij, cij(xj(t))=c^ij. When |xj(t)|<Tj, bij(xj(t))=bˇij, cij(xj(t))=cˇij. From this perspective, (1) is a switching system. On the other hand, for any fixed interval [ηk,ηk+1), when γ(t)=ϑk<t<ηk+1, neural network model (1) is a retarded system. When ηk≤t<γ(t)=ϑk, neural network model (1) is an advanced system. Then. from this point of view, (1) is a mixed system.For neural network model (1), the conventional definition of solution for differential equations cannot apply here. To tackle this problem, the solution concept for differential equations with deviating argument is introduced [24–30]. According to this theory, a solution x(t)=(x1(t),x2(t),…,xn(t))T of (1) is a consecutive function such that (1) x˙(t) exists on [0,+∞), except at the points ηk, k∈N, where a one-sided derivative exists, and (2) x(t) satisfies (1) within each interval ηk,ηk+1,k∈N.In this paper, the activation functions are selected as(3)fiχ=∑l=12-1lp2dχ+zli-χ-zli,i=1,2,…,n,where p>0 and d>0 are used to adjust the slope and corner point of activation functions and z1(i)=Ti, z2(i)=Ti+d.It is easy to see thatfi(·) satisfies fi(±Ti)=0 and(4)fil1-fil2≤αl1-l2,i=1,2,…,n,for any l1,l2∈(-∞,+∞), where α=p/d.For technical convenience, denoteb¯ij=max{|b^ij|,|bˇij|}, bij0=1/2(b^ij+bˇij), bij+=1/2|b^ij+bˇij|, bij-=1/2|b^ij-bˇij|, c¯ij=max{|c^ij|,|cˇij|}, cij0=1/2(c^ij+cˇij), cij+=1/2|c^ij+cˇij|, cij-=1/2|c^ij-cˇij|, and Si=max{|b^iip-aid|,|bˇiip-aid|}, for i,j=1,2,…,n. Let Sb=max1≤i≤n(ai+∑j=1nα(bji++bji-)), Sb-=min1≤i≤n(ai-∑j=1n(bji++bji-)α), and Sc=max1≤i≤n(∑j=1nα(cji++cji-)). co¯{M^,Mˇ} denotes the convex closure of a set constituted by real numbers M^ and Mˇ.


### 2.3. Autoassociative Memories

From Definition1, matrix E is selected as an identity matrix when designing autoassociative memories (i.e., Eu(r)=u(r)=q(r) for r=1,2,…,l). For the convenience of analysis and formulation in autoassociative memories, neural network model (1) can be rewritten in component form:(5)x˙it=-aixit+∑j=1nbijxjtfjxjt+∑j=1ncijxjtfjxjγt+ui,wit=fixit,i=1,2,…,n.


### 2.4. Properties

For neural network model (1), we consider the following set-valued maps:(6)Kbijxjt=b^ij,xjt>Tj,co¯b^ij,bˇ,xjt=Tj,bˇij,xjt<Tj,(7)Kcijxjt=c^ij,xjt>Tj,co¯c^ij,cˇ,xjt=Tj,cˇij,xjt<Tj,for i,j=1,2,…,n.Based on the theory of differential inclusions [17–20], from (5), we can get(8)x˙it∈-aixit+∑j=1nKbijxjtfjxjt+∑j=1nKcijxjtfjxjγt+ui,or, equivalently, there exist bij∗xjt∈K[bij(xj(t))] and cij∗(xj(t))∈K[cij(xj(t))] such that(9)x˙it=-aixit+∑j=1nbij∗xjtfjxjt+∑j=1ncij∗xjtfjxjγt+ui,i=1,2,…,n,for almost all t∈[0,+∞).Employing the fuzzy uncertainty approach [21], neural network model (5) can be expressed as follows:(10)x˙it=-aixit+∑j=1nbij0fjxjt+∑j=1ncij0fjxjγt+ui+∑j=1nδijbxjtbij-fjxjt+∑j=1nδijcxjt×cij-fjxjγt,where(11)δijbxjt=sgn⁡b^ij-bˇij,xjt>Tj,-sgn⁡b^ij-bˇij,xjt<Tj,δijcxjt=sgn⁡c^ij-cˇij,xjt>Tj,-sgn⁡c^ij-cˇij,xjt<Tj,and sgn⁡(·) is the sign function.Definition 2.
A constant vectorx∗=(x1∗,x2∗,…,xn∗)T is said to be an equilibrium point of neural network model (5) if and only if(12)0∈-aixi∗+∑j=1nKbijxj∗fjxj∗+∑j=1nKcijxj∗fjxj∗+ui,i=1,2,…,n,or, equivalently, there exist b~ij(xj∗)∈K[bij(xj∗)] and c~ij(xj∗)∈K[cij(xj∗)] such that(13)-aixi∗+∑j=1nb~ijxj∗fjxj∗+∑j=1nc~ijxj∗fjxj∗+ui=0,for i=1,2,…,n.Definition 3.
The equilibrium pointx∗=(x1∗,x2∗,…,xn∗)T of neural network model (5) is globally exponentially stable if there exist constants M>0 and ϵ>0 such that(14)xt-x∗≤Mx0-x∗exp⁡-ϵt,for any t≥0, where x(t) is the state vector of neural network model (5) with initial condition x0.Lemma 4.
For neural network model (5), one has(15)Kbijxjfjxj-Kbijyjfjyj≤b¯ijαxj-yj,Kcijxjfjxj-Kcijyjfjyj≤c¯ijαxj-yj,for i,j=1,2,…,n, xj,yj∈(-∞,+∞), where K[bij(·)] and K[cij(·)] are defined as those in (6) and (7), respectively.Lemma4 is a basic conclusion. To get more specific, please see [Neural Networks, vol. 51, pp. 1–8, 2014], or [Information Sciences, vol. 279, pp. 358–373, 2014], or others.In the following, we end this section with some basic assumptions:(A1)
There exists a positive constantη satisfying(16)ηk+1-ηk<η,k∈N.(A2)
ηSb+2Scexp⁡Sbη<1.(A3)
η[Sc+Sb1+Scηexp⁡Sbη]<1.


## 2.1. Problem Description

LetΩ=β=(β1,β2,…,βn)T∣βi={-p,p},i=1,2,…,n be the set of n-dimensional bipolar vectors, where p is a positive constant and T denotes the transpose of a vector or matrix. The problem considered is described as follows.Givenl(l≤2n) pairs of bipolar vectors (u(1),q(1)),(u(2),q(2)),…,(u(l),q(l)), where u(r),q(r)∈Ω, for r=1,2,…,l, design an associative memory model which satisfies the notion that the output of the model will converge to the corresponding pattern q(r) when u(r) is fed to the model input as a memory pattern.Definition 1.
The neural network is said to be an autoassociative memory ifu(r)=q(r) and a heteroassociative memory if u(r)≠q(r), for r=1,2,…,l.In this paper, letN be the natural number set; the norm of vector x=(x1,x2,…,xn)T is defined as x=∑i=1n|xi|. Denote Rn as the n-dimensional Euclidean space. Fix two real number sequences ηk, ϑk, k∈N, satisfying ηk<ηk+1, ηk≤ϑk≤ηk+1, and limk→+∞ηk=+∞.


## 2.2. Model

Consider the following neural network model:(1)x˙t=-Axt+Bxtfxt+Cxtfxγt+Eur,wt=fxt,r=1,2,…,l,where x(t)=(x1t,x2t,…,xn(t))T∈Rn is the neuron state, A=diag(a1,a2,…,an), which is a constant diagonal matrix with ai>0, i∈1,2,…,n, indicates the self-inhibition matrix, B(x(t))=(bij(xj(t)))n×n and C(x(t))=(cij(xj(t)))n×n are connection weight matrices at time t and γ(t), respectively, bij(xj(t)) and cij(xj(t)) are defined by(2)bijxjt=b^ij,xjt>Tj,bˇij,xjt<Tj,cijxjt=c^ij,xjt>Tj,cˇij,xjt<Tj,for i,j=1,2,…,n, and bij(±Tj)=b^ij or bˇij, cij(±Tj)=c^ij or cˇij, where Tj>0 represents the switching jump, and b^ij,bˇij,c^ij, and cˇij are all constants. f(x(·))=(f1(x1(·),f2(x2(·),…,fn(xn(·))T is the activation function. The deviating function γ(t)=ϑk, when t∈[ηk,ηk+1) for any k∈N. E stands for a transform matrix of order n satisfying Eu(r)=q(r) for r=1,2,…,l. u(r) and q(r) indicate the external input pattern and the corresponding memorized pattern, respectively. w(t)=(w1(t),w2(t),…,wn(t))T is the output of (1) corresponding to u(r).Neural network model (1) is called an associative memory if the output w(t) converges to q(r).Obviously, neural network model (1) is of hybrid type: switching and mixed. When |xj(t)|>Tj, bij(xj(t))=b^ij, cij(xj(t))=c^ij. When |xj(t)|<Tj, bij(xj(t))=bˇij, cij(xj(t))=cˇij. From this perspective, (1) is a switching system. On the other hand, for any fixed interval [ηk,ηk+1), when γ(t)=ϑk<t<ηk+1, neural network model (1) is a retarded system. When ηk≤t<γ(t)=ϑk, neural network model (1) is an advanced system. Then. from this point of view, (1) is a mixed system.For neural network model (1), the conventional definition of solution for differential equations cannot apply here. To tackle this problem, the solution concept for differential equations with deviating argument is introduced [24–30]. According to this theory, a solution x(t)=(x1(t),x2(t),…,xn(t))T of (1) is a consecutive function such that (1) x˙(t) exists on [0,+∞), except at the points ηk, k∈N, where a one-sided derivative exists, and (2) x(t) satisfies (1) within each interval ηk,ηk+1,k∈N.In this paper, the activation functions are selected as(3)fiχ=∑l=12-1lp2dχ+zli-χ-zli,i=1,2,…,n,where p>0 and d>0 are used to adjust the slope and corner point of activation functions and z1(i)=Ti, z2(i)=Ti+d.It is easy to see thatfi(·) satisfies fi(±Ti)=0 and(4)fil1-fil2≤αl1-l2,i=1,2,…,n,for any l1,l2∈(-∞,+∞), where α=p/d.For technical convenience, denoteb¯ij=max{|b^ij|,|bˇij|}, bij0=1/2(b^ij+bˇij), bij+=1/2|b^ij+bˇij|, bij-=1/2|b^ij-bˇij|, c¯ij=max{|c^ij|,|cˇij|}, cij0=1/2(c^ij+cˇij), cij+=1/2|c^ij+cˇij|, cij-=1/2|c^ij-cˇij|, and Si=max{|b^iip-aid|,|bˇiip-aid|}, for i,j=1,2,…,n. Let Sb=max1≤i≤n(ai+∑j=1nα(bji++bji-)), Sb-=min1≤i≤n(ai-∑j=1n(bji++bji-)α), and Sc=max1≤i≤n(∑j=1nα(cji++cji-)). co¯{M^,Mˇ} denotes the convex closure of a set constituted by real numbers M^ and Mˇ.


## 2.3. Autoassociative Memories

From Definition1, matrix E is selected as an identity matrix when designing autoassociative memories (i.e., Eu(r)=u(r)=q(r) for r=1,2,…,l). For the convenience of analysis and formulation in autoassociative memories, neural network model (1) can be rewritten in component form:(5)x˙it=-aixit+∑j=1nbijxjtfjxjt+∑j=1ncijxjtfjxjγt+ui,wit=fixit,i=1,2,…,n.


## 2.4. Properties

For neural network model (1), we consider the following set-valued maps:(6)Kbijxjt=b^ij,xjt>Tj,co¯b^ij,bˇ,xjt=Tj,bˇij,xjt<Tj,(7)Kcijxjt=c^ij,xjt>Tj,co¯c^ij,cˇ,xjt=Tj,cˇij,xjt<Tj,for i,j=1,2,…,n.Based on the theory of differential inclusions [17–20], from (5), we can get(8)x˙it∈-aixit+∑j=1nKbijxjtfjxjt+∑j=1nKcijxjtfjxjγt+ui,or, equivalently, there exist bij∗xjt∈K[bij(xj(t))] and cij∗(xj(t))∈K[cij(xj(t))] such that(9)x˙it=-aixit+∑j=1nbij∗xjtfjxjt+∑j=1ncij∗xjtfjxjγt+ui,i=1,2,…,n,for almost all t∈[0,+∞).Employing the fuzzy uncertainty approach [21], neural network model (5) can be expressed as follows:(10)x˙it=-aixit+∑j=1nbij0fjxjt+∑j=1ncij0fjxjγt+ui+∑j=1nδijbxjtbij-fjxjt+∑j=1nδijcxjt×cij-fjxjγt,where(11)δijbxjt=sgn⁡b^ij-bˇij,xjt>Tj,-sgn⁡b^ij-bˇij,xjt<Tj,δijcxjt=sgn⁡c^ij-cˇij,xjt>Tj,-sgn⁡c^ij-cˇij,xjt<Tj,and sgn⁡(·) is the sign function.Definition 2.
A constant vectorx∗=(x1∗,x2∗,…,xn∗)T is said to be an equilibrium point of neural network model (5) if and only if(12)0∈-aixi∗+∑j=1nKbijxj∗fjxj∗+∑j=1nKcijxj∗fjxj∗+ui,i=1,2,…,n,or, equivalently, there exist b~ij(xj∗)∈K[bij(xj∗)] and c~ij(xj∗)∈K[cij(xj∗)] such that(13)-aixi∗+∑j=1nb~ijxj∗fjxj∗+∑j=1nc~ijxj∗fjxj∗+ui=0,for i=1,2,…,n.Definition 3.
The equilibrium pointx∗=(x1∗,x2∗,…,xn∗)T of neural network model (5) is globally exponentially stable if there exist constants M>0 and ϵ>0 such that(14)xt-x∗≤Mx0-x∗exp⁡-ϵt,for any t≥0, where x(t) is the state vector of neural network model (5) with initial condition x0.Lemma 4.
For neural network model (5), one has(15)Kbijxjfjxj-Kbijyjfjyj≤b¯ijαxj-yj,Kcijxjfjxj-Kcijyjfjyj≤c¯ijαxj-yj,for i,j=1,2,…,n, xj,yj∈(-∞,+∞), where K[bij(·)] and K[cij(·)] are defined as those in (6) and (7), respectively.Lemma4 is a basic conclusion. To get more specific, please see [Neural Networks, vol. 51, pp. 1–8, 2014], or [Information Sciences, vol. 279, pp. 358–373, 2014], or others.In the following, we end this section with some basic assumptions:(A1)
There exists a positive constantη satisfying(16)ηk+1-ηk<η,k∈N.(A2)
ηSb+2Scexp⁡Sbη<1.(A3)
η[Sc+Sb1+Scηexp⁡Sbη]<1.


## 3. Main Results

In this section, we first present the conditions ensuring the existence and uniqueness of solution for neural network model (5) and then analyze its global exponential stability; finally, we discuss how to design the procedures of autoassociative memories and heteroassociative memories.


### 3.1. Existence and Uniqueness of Solution

Theorem 5.
Let (A1) and (A2) hold. Then, there exists a unique solutionx(t)=x(t,t0,x0),t≥t0, of (5) for every (t0,x0)∈(R+,Rn).Proof.
The proof of the assertion is divided into two steps.
Firstly, we discuss the existence of solution.
For a fixedk∈N, assume that ηk≤t0≤ϑk=γ(t)<t<ηk+1 and the other case ηk≤t0<t<ϑk=γ(t)<ηk+1 can be discussed in a similar way.
Letv(t)=x(t,t0,x0), and consider the following equivalent equation:(17)vit=xi0+∫t0t-aivis+∑j=1nbij0fjvjs+∑j=1ncij0×fjvjϑk+ui+∑j=1nδijbvjtbij-fjvjs+∑j=1nδijcvjtcij-fjvjϑkds,i=1,2,…,n.
Construct the following sequencevim(t), m∈N, i=1,2,…,n:(18)vim+1t=xi0+∫t0t-aivims+∑j=1nbij0fjvjms+∑j=1ncij0×fjvjmϑk+ui+∑j=1nδijbvjmtbij-fjvjms+∑j=1nδijcvjmtcij-fjvjmϑkds,i=1,2,…,n,and vi0(t)=xi0.
Define a normv(t)M=max[t0,t]v(t), and we can get(19)vm+1t-vmtM≤Sb+Scηmς,where ς=η[∑i=1nai|xi0|+∑i=1n∑j=1n(bij++bij-+cij++cij-)p+∑i=1n|ui|].
From (A2), we have(Sb+Sc)η<1, and hence(20)limm→+∞⁡vm+1t-vmtM=0.Therefore, there exists a solution v(t)=x(t,t0,x0) on the interval [t0,ϑk] for (17). From (4), the solution can be consecutive to ηk+1. Utilizing a similar method, x(t) can be consecutive to ϑk+1 and then to ηk+2. Applying the mathematical induction, the proof of existence of solution for (5) is completed.
Next, we analyze the uniqueness of solution.
Fixk∈N and choose t0∈[ηk,ηk+1], x1∈Rn, x2∈Rn. Denote x1(t)=x(t,t0,x1) and x2(t)=x(t,t0,x2) as two solutions of (5) with different initial conditions (t0,x1) and (t0,x2), respectively.
Then, we get(21)x1t-x2t≤x1-x2+Scηx1ϑk-x2ϑk+∫t0tSbx1s-x2sds.
Based on the Gronwall-Bellman inequality,(22)x1t-x2t≤x1-x2+Scηx1ϑk-x2ϑk×exp⁡Sbη,and, particularly,(23)x1ϑk-x2ϑk≤x1-x2+Scηx1ϑk-x2ϑk×exp⁡Sbη.
Hence,(24)x1t-x2t≤exp⁡Sbη1-Scηexp⁡Sbηx1-x2.
Suppose that there exists somet~∈[ηk,ηk+1] satisfying x1(t~)=x2(t~); then,(25)x1-x2≤Scηx1ϑk-x2ϑk+∫t0t~Sbx1s-x2sds.
From (A2), we get(26)0<ηexp⁡SbηSb+Sc<1-ηScexp⁡Sbη;that is,(27)0<ηexp⁡SbηSb+Sc1-ηScexp⁡Sbη<1.
Substituting (24) into (25), from (27), we obtain(28)x1-x2≤Scηexp⁡Sbη1-Scηexp⁡Sbηx1-x2+Sbηexp⁡Sbη1-Scηexp⁡Sbηx1-x2≤ηexp⁡SbηSb+Sc1-ηScexp⁡Sbηx1-x2<x1-x2.This poses a contradiction. Therefore, the uniqueness of solution is true. Taken together, the proof of the existence and uniqueness of solution for (5) is completed.


### 3.2. Invariant Manifolds

Lemma 6.
The solutionx(t) of neural network model (5) will fall into ∏i=1n[d+Ti,+∞) ultimately if, for i=1,2,…,n, the following conditions hold:(29)ui>Si+aiTi+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp.Proof.
We distinguish four cases to prove the lemma.(1)
If, for somet,xi(t)∈[Ti,d+Ti), i∈{1,2,…,n}, then(30)x˙it=-aixit+∑j=1nbijxjtfjxjt+∑j=1ncijxjt×fjxjγt+ui≥-aixit+biixitpdxit-Ti-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui≥-aixit-Ti+biixitpdxit-Ti-aiTi-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui≥-ai+biixitpdxit-Ti-aiTi-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui≥--ai+biixitpdd-aiTi-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui≥-biixitp-aid-aiTi-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui≥-Si-aiTi-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui>0.That is to say, x˙i(t)>0, when xi(t)∈[Ti,d+Ti).(2)
If, for somet, xi(t)∈(-Ti,Ti), i∈{1,2,…,n}, then(31)x˙it≥-aixit+biixitfixit-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui≥-aiTi-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui>Si>0.That is to say, x˙i(t)>0, when xi(t)∈(-Ti,Ti).(3)
If, for somet, xi(t)∈(-Ti-d,-Ti), i∈{1,2,…,n}, then(32)x˙it≥-aixit+biixitfixit-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui=-aixit+Ti+biixitpdxit+Ti-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui+aiTi=-ai+biixitpdxit+Ti-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui+aiTi≥-biixitp-aid+aiTi-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui>2aiTi>0.That is to say, x˙i(t)>0, when xi(t)∈(-Ti-di,-Ti).(4)
If, for somet, xi(t)∈(-∞,-Ti-di), i∈{1,2,…,n}, then(33)x˙it≥-aixit+biixitfixit-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui≥aiTi+d-biixitp-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui≥-biixitp-aid+aiTi-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui>2aiTi>0.That is to say, x˙i(t)>0, when xi(t)∈(-∞,-Ti-di).
From the above analysis, we can get that any componentxi(t)(i=1,2,…,n) of the solution x(t) of neural network model (5) will fall into [d+Ti,+∞) ultimately. Hence, the proof is completed.Lemma 7.
The solutionx(t) of neural network model (5) will fall into ∏i=1n(-∞,-d-Ti] ultimately if, for i=1,2,…,n, the following conditions hold:(34)ui<-Si-aiTi-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp.Proof.
We distinguish four cases to prove the lemma.(1)
If, for somet, xi(t)∈[d+Ti,+∞), i∈{1,2,…,n}, then(35)x˙it=-aixit+∑j=1nbijxjtfjxjt+∑j=1ncijxjtfjxjγt+ui≤-aixit+biixitfixit+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui≤-aid+Ti+biixitp+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui≤aid-biixitp-aiTi+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui≤Si-aiTi+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui<-2aiTi<0.That is to say, x˙i(t)<0, when xi(t)∈[d+Ti,+∞).(2)
If, for somet, xi(t)∈(Ti,d+Ti), i∈{1,2,…,n}, then(36)x˙it=-aixit+∑j=1nbijxjtfjxjt+∑j=1ncijxjtfjxjγt+ui≤-aixit-Ti+biixitpdxit-Ti-aiTi+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui≤-ai+biixitpdxit-Ti-aiTi+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui≤-ai+biixitpdd-aiTi+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui≤biixitp-aid-aiTi+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui≤Si-aiTi+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui<-2aiTi<0.That is to say, x˙i(t)<0, when xi(t)∈(Ti,d+Ti).(3)
If, for somet, xi(t)∈(-Ti,Ti), i∈{1,2,…,n}, then(37)x˙it≤-aixit+biixitfixit+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui≤aiTi+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui<-Si<0.That is to say, x˙i(t)<0, when xi(t)∈(-Ti,Ti).(4)
If, for somet, xi(t)∈(-Ti-d,-Ti), i∈{1,2,…,n}, then(38)x˙it≤-aixit+biixitfixit+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui=-aixit+Ti+biixitpdxit+Ti+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui+aiTi=-ai+biixitpdxit+Ti+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui+aiTi≤biixitp-aid+aiTi+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui<0.That is to say, x˙i(t)<0, when xi(t)∈(-Ti-di,-Ti).
From the above analysis, we can get that any componentxi(t)(i=1,2,…,n) of the solution x(t) of neural network model (5) will fall into (-∞,-d-Ti] ultimately. Hence, the proof is completed.


### 3.3. Global Exponential Stability Analysis

Denote(39)Δ1=i∣i∈1,2,…,n,ui>Si+aiTi+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp,Δ2=i∣i∈1,2,…,n,ui>Si+aiTi+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp.In order to guarantee the existence of equilibrium point for neural network model (5), the following assumption is needed:(A4)
For anyi∈{1,2,…,n}, i∈Δ1∪Δ2.Theorem 8.
Let (A4) hold. Then, neural network model (5) has at least one equilibrium point x∗∈Θ, where(40)Θ=x∈Rn∣xi≤-Ti-dorxi≥d+Ti,i∈1,2,…,n.Proof.
Define a mapping(41)Fx=F1x,F2x,…,FnxT,where(42)Fix=1ai∑j=1nKbijxjfjxj+∑j=1nKcijxjfjxj+ui,i=1,2,…,n.It is obvious that F(x) is equicontinuous.
Wheni∈Δ1, there exists a small enough positive number ε1 satisfying(43)ui>Si+aiTi+∑j=1,j≠ib¯ijp+∑j=1c¯ijp+aiε1,and when i∈Δ2, there exists a small enough positive number ε2 satisfying(44)ui<-Si-aiTi-∑j=1,j≠ib¯ijp-∑j=1c¯ijp-aiε2.
Take0<ε<min⁡(ε1,ε2) and denote(45)Θ¯=x∈Rn∣xi∈d+Ti+ε,d+Tiε,i∈Δ1,orxi∈-d+Tiε,-d-Ti-ε,i∈Δ2,which is a bounded and closed set.
Whenxi∈[d+Ti+ε,(d+Ti)/ε], i∈Δ1,(46)Fix=1ai∑j=1nKbijxjfjxj+∑j=1nKcijxjfjxj+ui≥1aiKbiixip-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui>1aiKbiixip-aid+Si+aiTi+aid+aiε≥Ti+d+ε.
Similarly, we can obtain(47)Fix<-Ti-d-εfor xi∈[-(d+Ti)/ε,-d-Ti-ε], i∈Δ2.
From the above discussion, we can getF(x)∈Θ for x∈Θ. Based on the generalized Brouwer’s fixed point theorem, there exists at least one equilibrium x∗∈Θ. This completes the proof.Recalling (10), for i=1,2,…,n, then(48)x˙it∈-aixit+∑j=1nbij0fjxjt+∑j=1ncij0fjxjγt+ui+∑j=1nKδijbxjtbij-fjxjt+∑j=1nKδijcxjtcij-fjxjγt.Letyi(t)=xi(t)-xi∗, i=1,2,…,n,(49)y˙it∈-aiyit+∑j=1nbij0gjyjt+∑j=1ncij0gjyjγt+∑j=1nAijbyjt+∑j=1nAijcyjγt,i=1,2,…,n,where gj(yj(t))=fj(yj(t)+xj∗)-fj(xj∗), Aijb(yj(t))=bij-(K[δijb(yj(t)+xj∗)]fj(yj(t)+xj∗)-K[δijb(xj∗)]fj(xj∗)), or Aijc(yj(γ(t)))=cij-(K[δijc(yj(γ(t))+xj∗)]fj(yj(γ(t))+xj∗)-K[δijc(xj∗)]fj(xj∗)), or there exists φijb(yj(t))∈Aijb(yj(t)),φijc(yj(t))∈Aijc(yj(t)) for i,j=1,2,…,n, such that(50)y˙it=-aiyit+∑j=1nbij0gjyjt+∑j=1ncij0gjyjγt+∑j=1nφijbyjt+∑j=1nφijcyjγt,i=1,2,…,n.According to Lemma4,(51)Aijbyjt≤bij-αyjt,Aijcyjγt≤cij-αyjγt,for i,j=1,2,…,n.Lemma 9.
Let (A1)–(A3) hold andy(t)=(y1(t),y2(t),…,yn(t))T be a solution of (49) or (50). Then,(52)yγt≤μyt,for t≥0, where μ=(1-η[Sc+Sb1+Scηexp⁡Sbη])-1.Proof.
For anyt>0, there exists a unique k∈N, such that t∈[ηk,ηk+1); then,(53)yit=yiϑk+∫ϑkt-aiyis+∑j=1nbij0gjyjs+∑j=1ncij0gjyjϑk+∑j=1nφijbyjs+∑j=1nφijcyjϑkds,i=1,2,…,n.
Taking the absolute value on both sides of (53),(54)yit≤yiϑk+∫ϑktaiyis+∑j=1nbij+gjyjs+∑j=1ncij+gjyjϑk+∑j=1nφijbyjs+∑j=1nφijcyjϑkds,i=1,2,…,n,and then(55)∑i=1nyit≤∑i=1nyiϑk+∫ϑkt∑i=1naiyis+∑i=1n∑j=1nbij+gjyjs+∑i=1n∑j=1ncij+gjyjϑk+∑i=1n∑j=1nφijbyjs+∑i=1n∑j=1nφijcyjϑkds.
Applying Lemma4,(56)∑i=1nyit≤∑i=1nyiϑk+∫ϑkt∑i=1naiyis+∑i=1n∑j=1nbij+αyjs+∑i=1n∑j=1ncij+αyjϑk+∑i=1n∑j=1nbij-αyjs+∑i=1n∑j=1ncij-αyjϑkds=∑i=1nyiϑk+∫ϑkt∑i=1naiyis+∑i=1n∑j=1nbji+αyis+∑i=1n∑j=1ncji+αyiϑk+∑i=1n∑j=1nbji-αyis+∑i=1n∑j=1ncji-αyiϑkds;that is(57)yt≤∑i=1nyiϑk+∫ϑkt∑i=1nai+∑j=1nαbji++bji-yis+∑i=1n∑j=1nαcji++cji-yiϑkds;then(58)yt≤yϑk+∫ϑktSbys+Scyϑkds≤1+Scηyϑk+∫ϑktSbysds.
According to the Gronwall-Bellman inequality,(59)yt≤1+Scηyϑkexp⁡∫ϑktSbds≤1+Scηexp⁡Sbηyϑk.
Exchanging the location ofyi(t) and yi(ϑk) in (53), we get(60)yϑk≤yt+∫ϑktSbys+Scyϑkds≤yt+∫ϑktSb1+Scηexp⁡Sbηyϑk+Scyϑkds≤yt+ηSc+Sb1+Scηyϑk,and then(61)yϑk≤μytholds for t∈[ηk,ηk+1), so y(γ(t))≤μy(t), where μ=(1-η[Sc+Sb1+Scηexp⁡Sbη])-1. This completes the proof.Remark 10.
Neural network model (49) is also of a hybrid type. The difficulty in investigating this class of neural networks lies in the existence of a switching jump and deviating argument. For the switching jump, we introduce the differential inclusions and fuzzy uncertainty approach to compensate state-jump uncertainty. For the deviating argument, by the aid of effective computational analysis, we estimate the norm of deviating state y(γ(t)) by the corresponding state y(t).In the following, the criterion to guarantee the global exponential stability of neural network model (5) based on the Lyapunov method is established.Theorem 11.
Let (A1)–(A4) hold. The origin of neural network model (49) is globally exponentially stable, which implies that the equilibrium point x∗=(x1∗,x2∗,…,xn∗)T of neural network model (5) is globally exponentially stable, if the following condition is satisfied:(62)μSc-Sb-<0.Proof.
Denote(63)zit=exp⁡λtyit,i=1,2,…,n,where λ=-1/2(μSc-Sb-)>0 is a positive constant satisfying(64)λ+μSc-Sb-=12μSc-Sb-<0.Then, for t≠ηk, k∈N, calculate the derivative of zi(t) along the trajectory of (49) or (50):(65)z˙it=λexp⁡λtyit+exp⁡λtsgnyity˙it=λexp⁡λtyit+exp⁡λtsgnyit-aiyit+∑j=1nbij0gjyjt+∑j=1ncij0gjyjγt+∑j=1nφijbyjt+∑j=1nφijcyjγt≤λexp⁡λtyit+exp⁡λt-aiyit+∑j=1nbij+gjyjt+∑j=1ncij+gjyjγt+∑j=1nφijbyjt+∑j=1nφijcyjγt,i=1,2,…,n.
Applying Lemma4,(66)z˙it≤exp⁡λtλ-aiyit+∑j=1nbij+αyjt+∑j=1ncij+αyjγt+∑j=1nbij-αyjt+∑j=1ncij-αyjγt,i=1,2,…,n.
Let(67)Vt=∑i=1nzit;then, for t≠ηk, k∈N,(68)V˙t=∑i=1nz˙it≤exp⁡λt∑i=1nλ-aiyit+∑i=1n∑j=1nbij+αyjt+∑i=1n∑j=1ncij+αyjγt+∑i=1n∑j=1nbij-αyjt+∑i=1n∑j=1ncij-αyjγt=exp⁡λt∑i=1nλ-aiyit+∑i=1n∑j=1nbji+αyit+∑i=1n∑j=1ncji+αyiγt+∑i=1n∑j=1nbji-αyit+∑i=1n∑j=1ncji-αyiγt=exp⁡λt∑i=1nλ-ai+∑j=1nαbji++bji-yit+∑i=1n∑j=1nαcji++cji-yiγt≤exp⁡λtλ-Sb-yt+Scyγt.
Based on Lemma9,(69)V˙t≤exp⁡λtλ-Sb-+μScyt.From (64),(70)λ-Sb-+μSc<0;then,(71)V˙t≤exp⁡λtλ-Sb-+μScyt≤0.
It is easy to see that(72)Vt=exp⁡λt∑i=1nyit=exp⁡λtyt≤V0=y0;hence(73)yt≤y0exp⁡-λt;that is(74)xt-x∗≤x0-x∗exp⁡-λt,for t≥0. This completes the proof.


### 3.4. Design Procedure of Autoassociative Memories

Givenl(l≤2n) vectors u(1),u(2),…,u(l) to be memorized, where u(k)∈Ω, for k=1,2,…,l, the design procedure of autoassociative memories is stated as follows:(1)
Select matrixE as an identity matrix.(2)
According to (29) and (34), choose appropriate Si, ai, Ti, b¯ij, and c¯ij.(3)
Based onSi=max⁡(|b^iip-aid|,|bˇiip-aid|), b¯ij=max⁡(|b^ij|,|bˇij|), and c¯ij=max⁡(|c^ij|,|cˇij|), the values of b^ij, bˇij, c^ij, and cˇij can be selected.(4)
CalculateSb, Sb-, and Sc. Adjust the values of b^ij, bˇij, c^ij, and cˇij to make sure that Sc-Sb-<0.(5)
Solving the following inequalities:ηSb+2Scexp⁡Sbη<1, η(Sc+Sb1+Scηexp⁡Sbη)<1, we can get 0<η<η¯.(6)
Take a properη∈(0,η¯) such that μSc-Sb-<0.The outputw(t) will converge to the related memory pattern when matrices A, B, C, and E and scalar η are chosen as above.


### 3.5. Design Procedure of Heteroassociative Memories

Givenl(l≤2n) vectors to be memorized as q(1), q(2),…,q(l), which correspond to the external input patterns u(1), u(2),…,u(l), where q(k)∈Ω, u(k)∈Ω, k=1,2,…,l, set EU=Q, where U=[u(1), u(2),…,u(l)], Q=[q(1), q(2),…,q(l)]. It is clear that heteroassociative memories can be treated as autoassociative memories when transform matrix E is obtained. Matrices A, B, and C and scalar η can be selected as the method used in autoassociative memories. The transform matrix E is obtained by the following steps:(1)
Whenl=n, that is, Q and U are square matrices, utilizing Matlab Toolbox, we can get inverse matrix of U as U-1, and E can be obtained as E=QU-1.(2)
Whenl<n, add n-l proper column vectors to U and Q, which constructs new matrices U¯ and Q¯, respectively, such that the ranks of matrices U¯ and Q¯ are full. And the transform matrix E can be obtained by E=Q¯U¯-1.(3)
Whenn<l, add l-n proper row vectors to U and Q, which constructs new matrices U~ and Q~, such that the ranks of matrices U~ and Q~ are full. And the transform matrix E can be obtained by E=Q~U~-1. Note in particular, in this circumstance, that the dimensionality of each input and output pattern increases from n to l. And the front n values of each external input pattern carry out associative memories.The outputw(t) will converge to the related memory pattern when matrices A, B, C, and E and scalar η are chosen as above.


## 3.1. Existence and Uniqueness of Solution

Theorem 5.
Let (A1) and (A2) hold. Then, there exists a unique solutionx(t)=x(t,t0,x0),t≥t0, of (5) for every (t0,x0)∈(R+,Rn).Proof.
The proof of the assertion is divided into two steps.
Firstly, we discuss the existence of solution.
For a fixedk∈N, assume that ηk≤t0≤ϑk=γ(t)<t<ηk+1 and the other case ηk≤t0<t<ϑk=γ(t)<ηk+1 can be discussed in a similar way.
Letv(t)=x(t,t0,x0), and consider the following equivalent equation:(17)vit=xi0+∫t0t-aivis+∑j=1nbij0fjvjs+∑j=1ncij0×fjvjϑk+ui+∑j=1nδijbvjtbij-fjvjs+∑j=1nδijcvjtcij-fjvjϑkds,i=1,2,…,n.
Construct the following sequencevim(t), m∈N, i=1,2,…,n:(18)vim+1t=xi0+∫t0t-aivims+∑j=1nbij0fjvjms+∑j=1ncij0×fjvjmϑk+ui+∑j=1nδijbvjmtbij-fjvjms+∑j=1nδijcvjmtcij-fjvjmϑkds,i=1,2,…,n,and vi0(t)=xi0.
Define a normv(t)M=max[t0,t]v(t), and we can get(19)vm+1t-vmtM≤Sb+Scηmς,where ς=η[∑i=1nai|xi0|+∑i=1n∑j=1n(bij++bij-+cij++cij-)p+∑i=1n|ui|].
From (A2), we have(Sb+Sc)η<1, and hence(20)limm→+∞⁡vm+1t-vmtM=0.Therefore, there exists a solution v(t)=x(t,t0,x0) on the interval [t0,ϑk] for (17). From (4), the solution can be consecutive to ηk+1. Utilizing a similar method, x(t) can be consecutive to ϑk+1 and then to ηk+2. Applying the mathematical induction, the proof of existence of solution for (5) is completed.
Next, we analyze the uniqueness of solution.
Fixk∈N and choose t0∈[ηk,ηk+1], x1∈Rn, x2∈Rn. Denote x1(t)=x(t,t0,x1) and x2(t)=x(t,t0,x2) as two solutions of (5) with different initial conditions (t0,x1) and (t0,x2), respectively.
Then, we get(21)x1t-x2t≤x1-x2+Scηx1ϑk-x2ϑk+∫t0tSbx1s-x2sds.
Based on the Gronwall-Bellman inequality,(22)x1t-x2t≤x1-x2+Scηx1ϑk-x2ϑk×exp⁡Sbη,and, particularly,(23)x1ϑk-x2ϑk≤x1-x2+Scηx1ϑk-x2ϑk×exp⁡Sbη.
Hence,(24)x1t-x2t≤exp⁡Sbη1-Scηexp⁡Sbηx1-x2.
Suppose that there exists somet~∈[ηk,ηk+1] satisfying x1(t~)=x2(t~); then,(25)x1-x2≤Scηx1ϑk-x2ϑk+∫t0t~Sbx1s-x2sds.
From (A2), we get(26)0<ηexp⁡SbηSb+Sc<1-ηScexp⁡Sbη;that is,(27)0<ηexp⁡SbηSb+Sc1-ηScexp⁡Sbη<1.
Substituting (24) into (25), from (27), we obtain(28)x1-x2≤Scηexp⁡Sbη1-Scηexp⁡Sbηx1-x2+Sbηexp⁡Sbη1-Scηexp⁡Sbηx1-x2≤ηexp⁡SbηSb+Sc1-ηScexp⁡Sbηx1-x2<x1-x2.This poses a contradiction. Therefore, the uniqueness of solution is true. Taken together, the proof of the existence and uniqueness of solution for (5) is completed.


## 3.2. Invariant Manifolds

Lemma 6.
The solutionx(t) of neural network model (5) will fall into ∏i=1n[d+Ti,+∞) ultimately if, for i=1,2,…,n, the following conditions hold:(29)ui>Si+aiTi+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp.Proof.
We distinguish four cases to prove the lemma.(1)
If, for somet,xi(t)∈[Ti,d+Ti), i∈{1,2,…,n}, then(30)x˙it=-aixit+∑j=1nbijxjtfjxjt+∑j=1ncijxjt×fjxjγt+ui≥-aixit+biixitpdxit-Ti-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui≥-aixit-Ti+biixitpdxit-Ti-aiTi-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui≥-ai+biixitpdxit-Ti-aiTi-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui≥--ai+biixitpdd-aiTi-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui≥-biixitp-aid-aiTi-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui≥-Si-aiTi-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui>0.That is to say, x˙i(t)>0, when xi(t)∈[Ti,d+Ti).(2)
If, for somet, xi(t)∈(-Ti,Ti), i∈{1,2,…,n}, then(31)x˙it≥-aixit+biixitfixit-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui≥-aiTi-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui>Si>0.That is to say, x˙i(t)>0, when xi(t)∈(-Ti,Ti).(3)
If, for somet, xi(t)∈(-Ti-d,-Ti), i∈{1,2,…,n}, then(32)x˙it≥-aixit+biixitfixit-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui=-aixit+Ti+biixitpdxit+Ti-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui+aiTi=-ai+biixitpdxit+Ti-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui+aiTi≥-biixitp-aid+aiTi-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui>2aiTi>0.That is to say, x˙i(t)>0, when xi(t)∈(-Ti-di,-Ti).(4)
If, for somet, xi(t)∈(-∞,-Ti-di), i∈{1,2,…,n}, then(33)x˙it≥-aixit+biixitfixit-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui≥aiTi+d-biixitp-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui≥-biixitp-aid+aiTi-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui>2aiTi>0.That is to say, x˙i(t)>0, when xi(t)∈(-∞,-Ti-di).
From the above analysis, we can get that any componentxi(t)(i=1,2,…,n) of the solution x(t) of neural network model (5) will fall into [d+Ti,+∞) ultimately. Hence, the proof is completed.Lemma 7.
The solutionx(t) of neural network model (5) will fall into ∏i=1n(-∞,-d-Ti] ultimately if, for i=1,2,…,n, the following conditions hold:(34)ui<-Si-aiTi-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp.Proof.
We distinguish four cases to prove the lemma.(1)
If, for somet, xi(t)∈[d+Ti,+∞), i∈{1,2,…,n}, then(35)x˙it=-aixit+∑j=1nbijxjtfjxjt+∑j=1ncijxjtfjxjγt+ui≤-aixit+biixitfixit+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui≤-aid+Ti+biixitp+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui≤aid-biixitp-aiTi+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui≤Si-aiTi+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui<-2aiTi<0.That is to say, x˙i(t)<0, when xi(t)∈[d+Ti,+∞).(2)
If, for somet, xi(t)∈(Ti,d+Ti), i∈{1,2,…,n}, then(36)x˙it=-aixit+∑j=1nbijxjtfjxjt+∑j=1ncijxjtfjxjγt+ui≤-aixit-Ti+biixitpdxit-Ti-aiTi+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui≤-ai+biixitpdxit-Ti-aiTi+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui≤-ai+biixitpdd-aiTi+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui≤biixitp-aid-aiTi+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui≤Si-aiTi+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui<-2aiTi<0.That is to say, x˙i(t)<0, when xi(t)∈(Ti,d+Ti).(3)
If, for somet, xi(t)∈(-Ti,Ti), i∈{1,2,…,n}, then(37)x˙it≤-aixit+biixitfixit+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui≤aiTi+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui<-Si<0.That is to say, x˙i(t)<0, when xi(t)∈(-Ti,Ti).(4)
If, for somet, xi(t)∈(-Ti-d,-Ti), i∈{1,2,…,n}, then(38)x˙it≤-aixit+biixitfixit+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui=-aixit+Ti+biixitpdxit+Ti+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui+aiTi=-ai+biixitpdxit+Ti+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui+aiTi≤biixitp-aid+aiTi+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp+ui<0.That is to say, x˙i(t)<0, when xi(t)∈(-Ti-di,-Ti).
From the above analysis, we can get that any componentxi(t)(i=1,2,…,n) of the solution x(t) of neural network model (5) will fall into (-∞,-d-Ti] ultimately. Hence, the proof is completed.


## 3.3. Global Exponential Stability Analysis

Denote(39)Δ1=i∣i∈1,2,…,n,ui>Si+aiTi+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp,Δ2=i∣i∈1,2,…,n,ui>Si+aiTi+∑j=1,j≠inb¯ijp+∑j=1nc¯ijp.In order to guarantee the existence of equilibrium point for neural network model (5), the following assumption is needed:(A4)
For anyi∈{1,2,…,n}, i∈Δ1∪Δ2.Theorem 8.
Let (A4) hold. Then, neural network model (5) has at least one equilibrium point x∗∈Θ, where(40)Θ=x∈Rn∣xi≤-Ti-dorxi≥d+Ti,i∈1,2,…,n.Proof.
Define a mapping(41)Fx=F1x,F2x,…,FnxT,where(42)Fix=1ai∑j=1nKbijxjfjxj+∑j=1nKcijxjfjxj+ui,i=1,2,…,n.It is obvious that F(x) is equicontinuous.
Wheni∈Δ1, there exists a small enough positive number ε1 satisfying(43)ui>Si+aiTi+∑j=1,j≠ib¯ijp+∑j=1c¯ijp+aiε1,and when i∈Δ2, there exists a small enough positive number ε2 satisfying(44)ui<-Si-aiTi-∑j=1,j≠ib¯ijp-∑j=1c¯ijp-aiε2.
Take0<ε<min⁡(ε1,ε2) and denote(45)Θ¯=x∈Rn∣xi∈d+Ti+ε,d+Tiε,i∈Δ1,orxi∈-d+Tiε,-d-Ti-ε,i∈Δ2,which is a bounded and closed set.
Whenxi∈[d+Ti+ε,(d+Ti)/ε], i∈Δ1,(46)Fix=1ai∑j=1nKbijxjfjxj+∑j=1nKcijxjfjxj+ui≥1aiKbiixip-∑j=1,j≠inb¯ijp-∑j=1nc¯ijp+ui>1aiKbiixip-aid+Si+aiTi+aid+aiε≥Ti+d+ε.
Similarly, we can obtain(47)Fix<-Ti-d-εfor xi∈[-(d+Ti)/ε,-d-Ti-ε], i∈Δ2.
From the above discussion, we can getF(x)∈Θ for x∈Θ. Based on the generalized Brouwer’s fixed point theorem, there exists at least one equilibrium x∗∈Θ. This completes the proof.Recalling (10), for i=1,2,…,n, then(48)x˙it∈-aixit+∑j=1nbij0fjxjt+∑j=1ncij0fjxjγt+ui+∑j=1nKδijbxjtbij-fjxjt+∑j=1nKδijcxjtcij-fjxjγt.Letyi(t)=xi(t)-xi∗, i=1,2,…,n,(49)y˙it∈-aiyit+∑j=1nbij0gjyjt+∑j=1ncij0gjyjγt+∑j=1nAijbyjt+∑j=1nAijcyjγt,i=1,2,…,n,where gj(yj(t))=fj(yj(t)+xj∗)-fj(xj∗), Aijb(yj(t))=bij-(K[δijb(yj(t)+xj∗)]fj(yj(t)+xj∗)-K[δijb(xj∗)]fj(xj∗)), or Aijc(yj(γ(t)))=cij-(K[δijc(yj(γ(t))+xj∗)]fj(yj(γ(t))+xj∗)-K[δijc(xj∗)]fj(xj∗)), or there exists φijb(yj(t))∈Aijb(yj(t)),φijc(yj(t))∈Aijc(yj(t)) for i,j=1,2,…,n, such that(50)y˙it=-aiyit+∑j=1nbij0gjyjt+∑j=1ncij0gjyjγt+∑j=1nφijbyjt+∑j=1nφijcyjγt,i=1,2,…,n.According to Lemma4,(51)Aijbyjt≤bij-αyjt,Aijcyjγt≤cij-αyjγt,for i,j=1,2,…,n.Lemma 9.
Let (A1)–(A3) hold andy(t)=(y1(t),y2(t),…,yn(t))T be a solution of (49) or (50). Then,(52)yγt≤μyt,for t≥0, where μ=(1-η[Sc+Sb1+Scηexp⁡Sbη])-1.Proof.
For anyt>0, there exists a unique k∈N, such that t∈[ηk,ηk+1); then,(53)yit=yiϑk+∫ϑkt-aiyis+∑j=1nbij0gjyjs+∑j=1ncij0gjyjϑk+∑j=1nφijbyjs+∑j=1nφijcyjϑkds,i=1,2,…,n.
Taking the absolute value on both sides of (53),(54)yit≤yiϑk+∫ϑktaiyis+∑j=1nbij+gjyjs+∑j=1ncij+gjyjϑk+∑j=1nφijbyjs+∑j=1nφijcyjϑkds,i=1,2,…,n,and then(55)∑i=1nyit≤∑i=1nyiϑk+∫ϑkt∑i=1naiyis+∑i=1n∑j=1nbij+gjyjs+∑i=1n∑j=1ncij+gjyjϑk+∑i=1n∑j=1nφijbyjs+∑i=1n∑j=1nφijcyjϑkds.
Applying Lemma4,(56)∑i=1nyit≤∑i=1nyiϑk+∫ϑkt∑i=1naiyis+∑i=1n∑j=1nbij+αyjs+∑i=1n∑j=1ncij+αyjϑk+∑i=1n∑j=1nbij-αyjs+∑i=1n∑j=1ncij-αyjϑkds=∑i=1nyiϑk+∫ϑkt∑i=1naiyis+∑i=1n∑j=1nbji+αyis+∑i=1n∑j=1ncji+αyiϑk+∑i=1n∑j=1nbji-αyis+∑i=1n∑j=1ncji-αyiϑkds;that is(57)yt≤∑i=1nyiϑk+∫ϑkt∑i=1nai+∑j=1nαbji++bji-yis+∑i=1n∑j=1nαcji++cji-yiϑkds;then(58)yt≤yϑk+∫ϑktSbys+Scyϑkds≤1+Scηyϑk+∫ϑktSbysds.
According to the Gronwall-Bellman inequality,(59)yt≤1+Scηyϑkexp⁡∫ϑktSbds≤1+Scηexp⁡Sbηyϑk.
Exchanging the location ofyi(t) and yi(ϑk) in (53), we get(60)yϑk≤yt+∫ϑktSbys+Scyϑkds≤yt+∫ϑktSb1+Scηexp⁡Sbηyϑk+Scyϑkds≤yt+ηSc+Sb1+Scηyϑk,and then(61)yϑk≤μytholds for t∈[ηk,ηk+1), so y(γ(t))≤μy(t), where μ=(1-η[Sc+Sb1+Scηexp⁡Sbη])-1. This completes the proof.Remark 10.
Neural network model (49) is also of a hybrid type. The difficulty in investigating this class of neural networks lies in the existence of a switching jump and deviating argument. For the switching jump, we introduce the differential inclusions and fuzzy uncertainty approach to compensate state-jump uncertainty. For the deviating argument, by the aid of effective computational analysis, we estimate the norm of deviating state y(γ(t)) by the corresponding state y(t).In the following, the criterion to guarantee the global exponential stability of neural network model (5) based on the Lyapunov method is established.Theorem 11.
Let (A1)–(A4) hold. The origin of neural network model (49) is globally exponentially stable, which implies that the equilibrium point x∗=(x1∗,x2∗,…,xn∗)T of neural network model (5) is globally exponentially stable, if the following condition is satisfied:(62)μSc-Sb-<0.Proof.
Denote(63)zit=exp⁡λtyit,i=1,2,…,n,where λ=-1/2(μSc-Sb-)>0 is a positive constant satisfying(64)λ+μSc-Sb-=12μSc-Sb-<0.Then, for t≠ηk, k∈N, calculate the derivative of zi(t) along the trajectory of (49) or (50):(65)z˙it=λexp⁡λtyit+exp⁡λtsgnyity˙it=λexp⁡λtyit+exp⁡λtsgnyit-aiyit+∑j=1nbij0gjyjt+∑j=1ncij0gjyjγt+∑j=1nφijbyjt+∑j=1nφijcyjγt≤λexp⁡λtyit+exp⁡λt-aiyit+∑j=1nbij+gjyjt+∑j=1ncij+gjyjγt+∑j=1nφijbyjt+∑j=1nφijcyjγt,i=1,2,…,n.
Applying Lemma4,(66)z˙it≤exp⁡λtλ-aiyit+∑j=1nbij+αyjt+∑j=1ncij+αyjγt+∑j=1nbij-αyjt+∑j=1ncij-αyjγt,i=1,2,…,n.
Let(67)Vt=∑i=1nzit;then, for t≠ηk, k∈N,(68)V˙t=∑i=1nz˙it≤exp⁡λt∑i=1nλ-aiyit+∑i=1n∑j=1nbij+αyjt+∑i=1n∑j=1ncij+αyjγt+∑i=1n∑j=1nbij-αyjt+∑i=1n∑j=1ncij-αyjγt=exp⁡λt∑i=1nλ-aiyit+∑i=1n∑j=1nbji+αyit+∑i=1n∑j=1ncji+αyiγt+∑i=1n∑j=1nbji-αyit+∑i=1n∑j=1ncji-αyiγt=exp⁡λt∑i=1nλ-ai+∑j=1nαbji++bji-yit+∑i=1n∑j=1nαcji++cji-yiγt≤exp⁡λtλ-Sb-yt+Scyγt.
Based on Lemma9,(69)V˙t≤exp⁡λtλ-Sb-+μScyt.From (64),(70)λ-Sb-+μSc<0;then,(71)V˙t≤exp⁡λtλ-Sb-+μScyt≤0.
It is easy to see that(72)Vt=exp⁡λt∑i=1nyit=exp⁡λtyt≤V0=y0;hence(73)yt≤y0exp⁡-λt;that is(74)xt-x∗≤x0-x∗exp⁡-λt,for t≥0. This completes the proof.


## 3.4. Design Procedure of Autoassociative Memories

Givenl(l≤2n) vectors u(1),u(2),…,u(l) to be memorized, where u(k)∈Ω, for k=1,2,…,l, the design procedure of autoassociative memories is stated as follows:(1)
Select matrixE as an identity matrix.(2)
According to (29) and (34), choose appropriate Si, ai, Ti, b¯ij, and c¯ij.(3)
Based onSi=max⁡(|b^iip-aid|,|bˇiip-aid|), b¯ij=max⁡(|b^ij|,|bˇij|), and c¯ij=max⁡(|c^ij|,|cˇij|), the values of b^ij, bˇij, c^ij, and cˇij can be selected.(4)
CalculateSb, Sb-, and Sc. Adjust the values of b^ij, bˇij, c^ij, and cˇij to make sure that Sc-Sb-<0.(5)
Solving the following inequalities:ηSb+2Scexp⁡Sbη<1, η(Sc+Sb1+Scηexp⁡Sbη)<1, we can get 0<η<η¯.(6)
Take a properη∈(0,η¯) such that μSc-Sb-<0.The outputw(t) will converge to the related memory pattern when matrices A, B, C, and E and scalar η are chosen as above.


## 3.5. Design Procedure of Heteroassociative Memories

Givenl(l≤2n) vectors to be memorized as q(1), q(2),…,q(l), which correspond to the external input patterns u(1), u(2),…,u(l), where q(k)∈Ω, u(k)∈Ω, k=1,2,…,l, set EU=Q, where U=[u(1), u(2),…,u(l)], Q=[q(1), q(2),…,q(l)]. It is clear that heteroassociative memories can be treated as autoassociative memories when transform matrix E is obtained. Matrices A, B, and C and scalar η can be selected as the method used in autoassociative memories. The transform matrix E is obtained by the following steps:(1)
Whenl=n, that is, Q and U are square matrices, utilizing Matlab Toolbox, we can get inverse matrix of U as U-1, and E can be obtained as E=QU-1.(2)
Whenl<n, add n-l proper column vectors to U and Q, which constructs new matrices U¯ and Q¯, respectively, such that the ranks of matrices U¯ and Q¯ are full. And the transform matrix E can be obtained by E=Q¯U¯-1.(3)
Whenn<l, add l-n proper row vectors to U and Q, which constructs new matrices U~ and Q~, such that the ranks of matrices U~ and Q~ are full. And the transform matrix E can be obtained by E=Q~U~-1. Note in particular, in this circumstance, that the dimensionality of each input and output pattern increases from n to l. And the front n values of each external input pattern carry out associative memories.The outputw(t) will converge to the related memory pattern when matrices A, B, C, and E and scalar η are chosen as above.


## 4. Experimental Verification

In this section, in order to verify the effectiveness of the proposed results, several numerical examples are provided.Example 12.
The first example is state response of autoassociative memory.
Consider neural network model (5) with n=12, where p=d=1, η=2/9, Ti=0.1, ai=0.4, and(75)b^ij=c^ij=0.012,i=j,0.01,i≠j,bˇij=cˇij=0.008,i=j,0.006,i≠j,with the activation functions(76)fixi=-12xi+0.1-xi-0.1+12xi+1.1-xi-1.1for i,j=1,2,…,12. Denote u=q=(1,1,1,1,-1,-1,1,-1,-1,1,1,1)T as the external input vector and the memorized vector, respectively.
It follows from Theorem11 that neural network model (5) has a unique equilibrium point, which is globally exponentially stable. The simulation result is shown in Figure 1.
It can be calculated easily that the equilibrium point in Figure1 is(77)x∗=2.71,2.71,2.71,2.71,-2.31,-2.31,2.71,-2.31,-2.31,2.71,2.71,2.71T.Clearly, the output pattern is(78)w∗=1,1,1,1,-1,-1,1,-1,-1,1,1,1T,which is equivalent to the memorized pattern q. Hence, neural network model (5) can be implemented effectively as an autoassociative memory.Figure 1
State response in neural network model (5) with external input u.Remark 13.
Because the equilibrium point is globally exponentially stable, the initial values can be selected randomly, and there is no influence on the performance of associative memories.Example 14.
The second example includes influences ofai, d, and p on the location of equilibrium points.The influences are divided into two cases to discuss.Case 1.
Letp=1 be a constant and denote u=q=(1,1,1,1,-1,-1,1,-1,-1,1,1,1)T as the external input vector and the memorized vector, respectively. Consider ai and d as variables in neural network model (5), taking η=1/9, and the rest of the parameters are the same as those in Example 12. The results are shown in Table 1, from which we can see that the smaller the value of ai, the bigger the value range d can take for the sake of memorized pattern, and vice versa.Table 1
The influences ofai and d on the location of equilibrium points when p=1 is a constant.
d
1.2
1.4
1.6
2.0
2.4
3.0
a
i
0.3
v
v
v
v
v
—
0.4
v
v
v
v
—
—
0.5
v
v
v
—
—
—
0.6
v
v
—
—
—
—
0.7
v
—
—
—
—
—
“v” (“—”) denotes the equilibrium points located (not located) in the saturation regions.Case 2.
Letd=0.25 be a constant and denote u=q=(p,p,p,p,-p,-p,p,-p,-p,p,p,p)T as the external input vector and the memorized vector, respectively, where p>0. Consider ai and p (or α=p/d) as variables in neural network model (5), taking η=1/9, and the rest of the parameters are the same as those in Example 12. The results are shown in Table 2, from which we can see that α (or p) should increase when ai increases for the sake of memorized pattern, and vice versa.Table 2
The influences ofai and α=p/d on the location of equilibrium points when d=0.25 is a constant.
α
=
p
d
1
2
2
3
3
4
1
1.2
1.5
a
i
0.40
—
v
v
v
v
v
0.45
—
—
v
v
v
v
0.60
—
—
—
v
v
v
0.70
—
—
—
—
v
v
0.90
—
—
—
—
—
v
“v” (“—”) denotes the equilibrium points located (not located) in the saturation region.Remark 15.
From Tables1 and 2, it can be concluded that the associative memories performance becomes better with the increase of α=p/d when ai is fixed, but not vice versa.Example 16.
The third example includes the influences ofai and perturbed u on the recall success rate of memorized patterns.
Letu=(1,1,1,1,-1,-1,1,-1,-1,1,1,1)T,η=1/9, and consider ai (i=1,2,…,12) as variables in neural network model (5). The other parameters are the same as those in Example 12. In this example, the influences of ai and perturbed u are analyzed on the recall success rate of memorized vectors. Experimental results are illustrated in Table 3, which makes it clear that the robustness of neural network model (5) for perturbed input u will become stronger as ai is small enough.Table 3
The influences ofai and perturbed u on the recall success rate.
a
i
0.3
0.4
0.5
0.6
0.7
0.9u
100
%
100
%
100
%
100%
100%
0.84u
100
%
100
%
100
%
100%
66.67%
0.63u
100
%
100
%
100
%
66.67%
0
0.52u
100
%
100
%
66.67
%
0
0.41u
100
%
66.67
%
0
0.25u
66.67
%
0
0.24u
0Remark 17.
Table3 illustrates that the robustness of associative memories on perturbed u becomes stronger with the decrease of ai.Example 18.
The fourth example includes the application of autoassociative memory without input perturbation.
According to neural network model (5), design an autoassociative memory to memorize pattern “T” which is represented by a 4×3-pixel image (black pixel =1; white pixel =-1) and the corresponding input pattern is u=(1,1,1,-1,1,-1,-1,1,-1,-1,1,-1)T. The parameters are the same as those in Example 12.
Simulation results with three random initial values are depicted in Figure2, where the dark grey block indicates that the value of the related component of output w(t) is in (0,1) and the light grey block indicates that the value of the related component of output w(t) is in (-1,0). Obviously, the output pattern w(t) converges to the memorized pattern correctly.Figure 2
Typical evolutions of output pattern with input vectoru=q under three random initial values.Example 19.
The fifth example includes the application of autoassociative memory with input perturbation.
The aim of this experiment is to discuss the influence of input pattern perturbation on the output pattern. The parameters are the same as those in Example12. Choose pattern “T” to be memorized, and the perturbed vector is selected as u=k∗q, where q=(1,1,1,-1,1,-1,-1,1,-1,-1,1,-1)T is the corresponding memorized vector and k is a perturbation coefficient.
In what follows, the perturbed vectoru is imposed on neural network model (5) as input vector with k=0.52 and k=0.41, respectively. Simulation results are shown in Figures 3 and 4. In Figure 3, the output pattern is exactly the same as the memorized pattern, and in Figure 4 the output pattern is different from the memorized pattern although they can be distinguished from the outlines.Figure 3
Typical evolutions of output pattern with input vectoru=0.52∗q under three random initial values.Figure 4
Typical evolutions of output pattern with input vectoru=0.41∗q under three random initial values.Remark 20.
From the analysis above, we can see that the output pattern will converge to the memorized pattern when the perturbation coefficient is not small enough. In fact, neural network model (5) is implemented well as an associative memory if the coefficient satisfying k≥0.52 and the other parameters are the same as those in Example 19.Example 21.
The sixth example includes the application of heteroassociative memory.
Consider neural network model (1), and the parameters are the same as those in Example 12 except the transform matrix E, which will be given later. The aim of this experiment is to design a heteroassociative memory to memorize pattern “U” when the input pattern is “C.” It is clear that the input vector is u=(1,1,1,1,-1,-1,1,-1,-1,1,1,1)T and the corresponding output vector is q=(1,-1,1,1,-1,1,1,-1,1,1,1,1,)T.According to the heteroassociative memories design procedure, the matricesU¯ and Q¯ are constructed as(79)U¯=100000000000110000000000101000000000100100000000-100010000000-100001000000100000100000-100000010000-100000001000100000000100100000000010100000000001,Q¯=100000000000-110000000000101000000000100100000000-100010000000100001000000100000100000-100000010000100000001000100000000100100000000010100000000001,respectively.Using the Matlab Toolbox, we can have(80)E=Q¯U¯-1=100000000000-210000000000001000000000000100000000000010000000200001000000000000100000000000010000200000001000000000000100000000000010000000000001.The simulation results are depicted in Figure5, which shows that the output pattern “U” is memorized when the input pattern is “C” and the transform matrix E is selected as above. This indicates that neural network model (1) can perform well as a heteroassociative memory under a proper transform matrix E, which can be easily obtained by using the Matlab Toolbox.Figure 5
Typical evolutions of pattern “U” with input pattern “C” under three random initial values.Remark 22.
Simulation results in Figure5 show that the associative memories model presented in this paper can be implemented well as autoassociative memories and heteroassociative memories. This model possesses the resistance capacity against external input perturbations and can be regarded as an extension and complement to the relevant results [10, 11, 16].


## 5. Concluding Remarks

Analysis and design of associative memories for recurrent neural networks have drawn considerable attention, whereas analysis and design of associative memories for memristive neural networks with deviating argument have not been well investigated. This paper presents a general class of memristive neural networks with deviating argument and discusses the analysis and design of autoassociative memories and heteroassociative memories. Meanwhile, the robustness of this class of neural networks is demonstrated when external input patterns are seriously perturbed. To the best of the author’s knowledge, this is the first theoretical result on associative memories for memristive neural networks with deviating argument. Future works may aim at (1) studying associative memories based on multistability of memristive neural networks with deviating argument; (2) analyzing associative memories based on discrete-time memristive neural networks with deviating argument; (3) exploring high-capacity associative memories for memristive neural networks with deviating argument.



---

*Source: 1057909-2017-08-09.xml*