# Global Convergence for Cohen-Grossberg Neural Networks with Discontinuous Activation Functions

Authors: Yanyan Wang; Jianping Zhou
Journal: Abstract and Applied Analysis
 (2012)

---

## Abstract

Cohen-Grossberg neural networks with discontinuous activation functions is considered. Using the property ofM-matrix and a generalized Lyapunov-like approach, the uniqueness is proved for state solutions and corresponding output solutions, and equilibrium point and corresponding output equilibrium point of considered neural networks. Meanwhile, global exponential stability of equilibrium point is obtained. Furthermore, by contraction mapping principle, the uniqueness and globally exponential stability of limit cycle are given. Finally, an example is given to illustrate the effectiveness of the obtained results.


---

## Body

## 1. Introduction

Recently, different types of neural networks with or without time delays have been widely investigated due to their wide applicability [1–32]. Obviously, considerable research interests are focused on the studies of Cohen-Grossberg neural networks (CGNNs) with their various generalizations due to their potential applications in classification, associative memory, and parallel computation and their ability to solve optimization problems. This class of neural networks is proposed by Cohen and Grossberg [1] in 1983, and can be modeled as
(1.1)dui(t)dt=-ai(ui(t))[bi(ui(t))-∑j=1n‍wijfj(uj(t))-Ii],i=1,2,…,n,
where n≥2 is the number of neurons in the network, ui denotes the state variable associated with the ith neuron, ai represents an amplification function, and bi is an appropriately behaved function. wij represents the connection strengths between neurons, and if the output from neuron j excites (resp., inhibits) neuron i, then wij≥0 (resp., wij≤0). The activation function fj shows how neurons respond to each other. CGNNs include a lot of famous ecological systems and neural networks as special cases such as the Lotka-Volterra system, the Gilpia-Analg competition system, the Eingen-Schuster system, and the Hopfield neural networks [1–3], where the Hopfield neural networks can be described as follows:
(1.2)dui(t)dt=-bi(ui(t))+∑j=1n‍wijfj(uj(t))+Ii,i=1,2,…,n.For CGNNs, dynamics behavior have been studied in literature; we refer to [4–10, 27–29] and the references cited therein. In [4], by using the concept of Lyapunov diagonally stable (LDS) and linear matrix inequality approach, some criteria were given to ensure global stability and global exponential stability. Yuan and Cao in [5] considered global asymptotic stability of delayed Cohen-Grossberg neural networks via nonsmooth analysis. Robust exponentially stability of delayed Cohen-Grossberg neural networks is discussed in [10]. In [27], the authors studied the stochastic stability of a class of Cohen-Grossberg neural networks, in which the interconnections and delays are time varying.In the above papers, a common feature is that the activation functions are assumed to be continuous and even Lipschitz continuous. However, in [11], Forti and Nistri pointed out that neural networks modeled by differential equations with discontinuous right-hand side are important and do frequently arise in practice. In order to model discrete-time cellular neural networks, a conceptually analogous model based on hard comparators was used [12]. The class of neural networks introduced in [13] to deal with linear and nonlinear programming problems can be considered as another important example. Those networks make use of constraint neurons with a diode-like input-output activations. Once again, in order to ensure satisfaction of the constraints, the diodes are required to have a very high slope in the conducting region; that is, they should approximate the discontinuous characteristic of an ideal diode [14]. When treating with dynamical systems with high-slope nonlinear elements, a system of differential equations with discontinuous right-hand side is often used, rather than the model with high but finite slope [15]. The reason of analyzing the ideal discontinuous case is that such analysis is able to reveal crucial features of the dynamics, such as the possibility that trajectories be confined for some time intervals on discontinuity surfaces. Another interesting phenomenon which is peculiar to discontinuous systems is the possibility that trajectories converge toward an equilibrium point in finite time [16, 17], which is of special interest for designing real-time neural optimization solvers.In [11], Forti and Nistri discussed the global convergence of neural networks with discontinuous neuron activations by means of the concepts and results of differential equations with discontinuous right-hand side introduced by Filippov [21]. In [18], they extended the results in [11] under the assumption that the interconnection matrix is an M-matrix or H-matrix. In [19], without assuming the boundedness and the continuity of the neuron activations, the authors presented sufficient conditions for the global stability of neural networks with time delay based on linear matrix inequality. Also, in [20], they present some sufficient conditions for the global stability and exponential stability of a class of the CGNNs by using the LDS, and provided an estimate of the convergence rate. In [24–26], the authors discussed the stability or multistability of the neural networks with discontinuous activation functions. However, [11, 24–26] have shown that convergence of the state does not imply convergence of the outputs. In addition, in the practical applications, the result of the neural computation is usually the steady-state neuron output, rather than the asymptotic value of the state. Hence, in this paper, we will study global convergence of CGNNs with discontinuous activation functions, where the interconnection matrix is assumed to be an M-matrix or H-matrix. Firstly, using the property of M-matrix and a generalized Lyapunov-like approach, we prove the uniqueness of state solutions and corresponding output solutions, and equilibrium point and corresponding output equilibrium point for the considered neural networks. Then, global exponential stability of unique equilibrium point is discussed and exponential convergence rate is estimated. Also, by contraction mapping principle, the globally exponential stability of limit cycle is given. Finally, we use a numerical example to illustrate the effectiveness of the theoretical results. The rest of the paper is organized as follows. In Section 2, model description and preliminaries are presented. The main results are stated in Section 3. In Section 4, an example is given to show the validity of the obtained results. Finally, in Section 5, the conclusions are drawn.Notations. Throughout the paper, the transpose of and inverse of any square matrix A are expressed as AT and A-1, respectively. For α=(α1,…,αn)T∈ℝn,α>0 denotes αi>0 for i=1,2,…,n. For x,y∈ℝn,   〈x,y〉=∑i=1nxiyi denotes the scalar product of x,y.


## 2. Model Description and Preliminaries

In this paper, we consider the CGNNs (1.1) with discontinuous right-hand side. The compact form of model (1.1) is expressed as follows:
(2.1)du(t)dt=-A(u(t))[Bu(t)-Wf(u(t))-I],
where  u(t)=(u1(t),u2(t),…,un(t))T∈ℝn,A(u(t))=diag(a1(u1(t)),a2(u2(t)),…,an(un(t))),B=diag(b1,b2,…,bn),W=(wij)n×n,I=(I1,I2,…,In)T∈ℝn, and f(u(t))=(f1(u1(t)),…,fn(un(t)))T.Throughout this paper, we make the following assumptions.(A1)
The functionai(r) is continuous, 0<aˇi≤ai(r)≤a^i for all r∈ℝ, where aˇi and a^i are positive constants, i=1,2,…,n.(A2)
The matrixW=(wij)n×n is nonsingular, that is, det W≠0.Moreover,f=(f1,…,fn) is supposed to belong to the following class of discontinuous functions.Definition 2.1 (see [18] (Function Class ℱD)).
f
(
x
)
∈
ℱ
D if and only if for i=1,2,…,n, the following conditions hold:(i)
fi is bounded on ℝ;(ii)
fi is piecewise continuous on ℝ; namely, fi is continuous on ℝ except a countable set of points of discontinuity pki, where there exist finite right and left limits fi(pki+) and fi(pki-), respectively; moreover, fi has finite discontinuous points in any compact interval of ℝ;(iii)
fi is nondecreasing on ℝ.
Denote the set of discontinuous points offi,i=1,2,…,n, by
(2.2)Si={pki∈ℝ:fi(pki+)>fi(pki-)}.
Sometimes,f=(f1,…,fn) is supposed to belong to the next class of discontinuous functions, which is included in ℱD.Definition 2.2 (see [18] (Function Class ℱDL)).
f
(
x
)
∈
ℱ
D
L if and only if f(x)∈ℱD and for i=1,2,…,n,fi is locally Lipschitz with Lipschitz constant li(xi)≥0 for all xi∈ℝ∖Si. Furthermore, we have li(xi)≥Li<+∞ for all xi∈ℝ∖Si.
For model (1.1) or model (2.1) with discontinuous right-hand side, a solution of Cauchy problem need to be explained. In this paper, solutions in the sense of Filippov [21] are considered whose definition will be given next.
LetK[f(u)]=(K[f1(u1),K[f2(u2)],…,K[fn(un)])T, where K[fi(ui)]=[fi(ui-),fi(ui+)].Definition 2.3.
A functionu(t),t∈[t1,t2], where t1<t2≤+∞ is a solution (in the sense of Filippov) of (2.1) in the interval [t1,t2], with initial condition u(t1)=u0∈ℝn, if u(t) is absolutely continuous on [t1,t2] and u(t1)=u0, and for almost all (a.a.) t∈[t1,t2] we have
(2.3)du(t)dt∈-A(u(t))[Bu(t)-WK[f(u(t))]-I].
Let u(t),t∈[t1,t2], be a solution of model (2.1). For a.a. t∈[t1,t2], one can obtain
(2.4)du(t)dt=-A(u(t))[Bu(t)-Wγ(t)-I],
where
(2.5)γ(t)=W-1(A-1(u)u˙(t)+Bu(t)-I)∈K[f(u(t))]
is the output solution of model (2.1) corresponding to u(t). And γ(t) is a bounded measurable function [11], which is uniquely defined by the state solution u(t) for a.a. t∈[t1,t2].Definition 2.4 (equilibrium point).
u
*
∈
ℝ
n is an equilibrium point of model (2.1) if and only if the following algebraic inclusion is satisfied:
(2.6)0∈A(u*)(-Bu*+WK[f(u*)]+I).Definition 2.5 (output equilibrium point).
Letu* be an equilibrium point of model (1.1);
(2.7)γ*=W-1(Bu*-I)∈K[f(u*)]
is the output equilibrium point of model (2.1) corresponding to u*.In this paper, we also need the following definitions and lemma.Definition 2.6 (see [18]).
LetQ∈ℝn×n be a square matrix. Matrix Q is said to be an M-matrix if and only if Qij≤0 for each i≠j, and all successive principal minors of Q are positive.Definition 2.7 (see [18]).
LetQ∈ℝn×n be a square matrix. Matrix Q is said to be an H-matrix if and only if the comparison matrix of Q, which is defined by
(2.8)[ℳ(Q)]ij={|Qii|,i=j,-|Qij|,i≠j,
is an M-matrix.Lemma 2.8 (see [18]).
Suppose thatQ is an M-matrix. Then, there exists a vector ξ>0 such that QTξ>0.All results of this paper are under one of the following assumptions:(a)
-W is an M-matrix;(b)
-W is an H-matrix such that Wii<0.(a) and (b) can be applied to cooperative neural networks [22] and cooperative-competitive neural networks, respectively.From [23], the result that -W is LDS under (a) or (b) can be obtained; hence, all results in [20] hold. So, for any u0∈ℝn, model (2.1) has a bounded absolutely continuous solution u(t) for t≥0 which satisfies u(0)=u0. Meanwhile, there exists an equilibrium point u*∈ℝn of model (2.1).If-W is an M-matrix, then, there exists ξ=(ξ1,…,ξn)T>0 such that
(2.9)(-W)Tξ=β>0.
If -W is an H-matrix, then, there exists ξ=(ξ1,…,ξn)T>0 such that
(2.10)[ℳ(-W)]Tξ=β>0.
Using the positive vector ξ, we define the distance in ℝn as follows: for any x,y∈ℝn, define
(2.11)∥x-y∥ξ=∑i=1n‍ξi|xi-yi|.Definition 2.9.
The equilibrium pointu* of (2.1) is said to be globally exponentially stable, if there exist constants α>0 and M>0 such that for any solution u(t) of model (2.1), we have
(2.12)∥u(t)-u*∥ξ≤M∥u0-u*∥ξexp{-αt}.
Also, we can consider the CGNNs with periodic input:
(2.13)du(t)dt=-A(u(t))[Bu(t)-Wf(u(t))-I(t)],
where I(t)=(I1(t),I2(t),…,In(t))T is periodic input vectors with period ω.Definition 2.10.
A periodic orbitu*(t) of Cohen-Grossberg networks is said to be globally exponentially stable, if there exist constants α>0 and M>0 such that such that for any solution u(t) of model (2.13), we have
(2.14)∥u(t)-u*(t)∥≤M∥u0-u0*∥ξexp{-αt}.


## 3. Main Results

In this section, we shall establish some sufficient conditions to ensure the uniqueness of solutions, equilibrium point, output equilibrium point, and limit cycle as well as the global exponential stability of the state solutions.Because Filippov solution includes set-valued function, in the general case, for a given initial condition, a discontinuous differential equation has multiple solutions starting at it [16]. Next, it will be shown that the uniqueness of solutions of model (2.1) can be obtained under the assumptions (A1) and (A2).Theorem 3.1.
Under the assumptions (A1) and (A2), iff∈ℱD and -W is an M-matrix or -W is an H-matrix such that Wii<0, then, for any u0 there is a unique solution u(t) of model (2.1) with initial condition u(0)=u0, which is defined and bounded for all t≥0. Meanwhile, the corresponding output solution γ(t) of model (2.1) is uniquely defined and bounded for a.a. t≥0.Proof.
We only need to prove the uniqueness. Letu(t) and u~(t),t≥0 are two solutions of model (2.1) with the initial condition u(0)=u~(0)=u0.
Define(3.1)V(u-u~)=∑i=1n‍ξi|∫u~iui‍dsai(s)|.
Computing the time derivative of V along the solutions of (2.1) gives
(3.2)dV(u-u~)dt=∑i=1n‍ξisgn(ui(t)-u~i(t))(-bi(ui(t)-u~i(t))+∑j=1n‍wij(γj(t)-γ~j(t))),
where
(3.3)sgn(s)={1,s>0,0,s=0,-1,s<0.
From f∈ℱD and γj(t)∈K[fj(uj(t))],γ~j(t)∈K[fj(u~j(t))], one can have
(3.4)sgn(uj(t)-u~j(t))(γj(t)-γ~j(t))=|γj(t)-γ~j(t)|.
Hence
(3.5)dV(u-u~)dt=-∑i=1n‍ξibi|ui(t)-u~i(t)|+∑i=1n‍ξiwii|γi(t)-γ~i(t)|+∑i=1n‍ξisgn(ui(t)-u~i(t))∑j=1,j≠in‍wij(γj(t)-γ~j(t))≤-∑i=1n‍ξibi|ui(t)-u~i(t)|-∑i=1n‍ξi|wii||γi(t)-γ~i(t)|+∑i=1n‍ξi∑j=1,j≠in‍|wij||γj(t)-γ~j(t)|=-∑i=1n‍ξibi|ui(t)-u~i(t)|-∑i=1n‍ξi[|wii||γi(t)-γ~i(t)|+∑j=1,j≠in‍(-|wij|)|γj(t)-γ~j(t)|]=-∑i=1n‍ξibi|ui(t)-u~i(t)|-∑i=1n‍ξi∑j=1n‍[ℳ(-W)]ij|γj(t)-γ~j(t)|=-∑i=1n‍ξibi|ui(t)-u~i(t)|-〈ξ,ℳ(-W)v(t)〉=-∑i=1n‍ξibi|ui(t)-u~i(t)|-〈[ℳ(-W)]Tξ,v(t)〉=-∑i=1n‍ξibi|ui(t)-u~i(t)|-〈β,v(t)〉≤0,
where v(t)=(|γ1(t)-γ~1(t)|,…,|γn(t)-γ~n(t)|)T. Integrating (3.1) between 0 and t0>0, we have
(3.6)V(u(t0)-u~(t0))≤V(u(0)-u~(0))=V(u0-u0)=0,
and hence, u(t0)=u~(t0) for any t0>0; that is, the solution of model (2.1) with initial condition u(0)=u0 is unique.
From (2.5), the output solution γ(t) corresponding to u(t) is uniquely defined and bounded for a.a. t≥0. The proof of Theorem 3.1 is completed.Remark 3.2.
Under the assumptions (A1) and (A2), iff∈ℱD and -W is an M-matrix or -W is an H-matrix such that Wii<0, then, for any I∈ℝn, model (2.1) has a unique equilibrium point and a unique corresponding output equilibrium point. Because from the assumptions, we have -W is LDS, hence, from Theorem 6 in [20], model (2.1) has a unique equilibrium point. By Definition 2.5, it is easily obtained that corresponding output equilibrium point is unique.
Next, global exponential stability of the equilibrium point of model (2.1) and the uniqueness and global exponential stability of limit cycle of model (2.13) are addressed. The results are given in following theorems.Theorem 3.3.
Under the assumptions(A1) and (A2), if f∈ℱD and -W is an M-matrix or -W is an H-matrix such that Wii<0, then, for any I∈ℝn, model (2.1) has a unique equilibrium point which is globally exponentially stable.Proof.
Letu(t),t≥0, be the solution of model (2.1) such that u(0)=u0, and for a.a. t≥0, let γ(t) be the corresponding output solution. For equilibrium point u*, γ* is corresponding output equilibrium point.
Sincebi>0, we can choose a small ε>0 such that
(3.7)bi-εaˇi>0.
Define
(3.8)V~(u(t)-u*)=eεt∑i=1n‍ξi|∫ui*ui(t)‍dsai(s)|.
Computing the time derivative of V~ along the solutions of (2.1), it follows that
(3.9)dV~(u(t)-u*)dt≤eεt[-∑i=1n‍ξi(bi-εaˇi)|ui(t)-ui*|-〈β,v~(t)〉]≤0,
where v~(t)=(|γ1(t)-γ1*|,…,|γn(t)-γn*|)T.
Hence,(3.10)V~(u(t)-u*)≤V~(u0-u*)≤1aˇ∥u0-u*∥ξ,
where a˘=min{aˇ1,aˇ2,…,aˇn}.
On the other hand,(3.11)V~(u(t)-u*)≥eεt1a^∥u(t)-u*∥ξ,
where a^=max{a^1,a^2,…,a^n}.
So, the following inequality holds:(3.12)∥u(t)-u*∥ξ≤a^aˇ∥u0-u*∥ξe-εt,
that is, u* is globally exponentially stable.Remark 3.4.
Sincebi-ε/aˇi>0, the exponential convergence rate ε can be estimated by means of the maximal allowable value by virtue of inequality ε<aˇibi,i=1,2,…,n. From this, one can see that amplification functions have key effect on the convergence rate of the stability for the considered model.
Next, the uniqueness and the exponentially stability of limit cycle for model (2.13) is given.Theorem 3.5.
Under the assumptions (A1) and (A2), iff∈ℱD and -W is an M-matrix or -W is an H-matrix such that Wii<0, then model (2.13) has a unique globally exponentially stable limit cycle.Proof.
Letu(t), u~(t) are two solutions of model (2.13), such that u(0)=u0,u~(0)=u~0 respectively.
Define(3.13)V-(u(t)-u~(t))=eεt∑i=1n‍ξi|∫u~i(t)ui(t)‍dsai(s)|.
Similar to the proof of Theorem 3.3, the following inequality holds:
(3.14)∥u(t)-u~(t)∥ξ≤a^aˇ∥u0-u~0∥ξe-εt,
Define u(t)(θ)=u(t+θ). Define a mapping L:Rn→Rn by L(u0)=u0(ω), then Lk(u0)=u0(kω). We can choose a positive integer k, such that for a positive constant ρ<1,
(3.15)a^aˇexp{-εkω}≤ρ<1.
And, from (3.14), we have
(3.16)∥Lk(u0)-Lk(u~0)∥ξ≤a^aˇ∥u0-u~0∥ξexp{-ε(kω)}≤ρ∥u0-u~0∥ξ.
By contraction mapping principle, there exists a unique fixed point u0* such that Lk(u0*)=u0*. In addition, Lk(L(u0*))=L(Lk(u0*))=L(u0*); that is, L(u0*) is also a fixed point of Lk. By the uniqueness of the fixed point of the mapping Lk,L(u0*)=u0*; that is, u0*(ω)=u0*. Let u*(t) be a state of model (1.1) with initial condition u0*; we obtain for all i∈{1,2,…,n},
(3.17)dui*(t)dt=-ai(ui*(t))[ui*(t)-∑j=1n‍wijfj(uj*(t))-Ii(t)].
Then, for all i∈{1,2,…,n},
(3.18)dui*(t+ω)dt=-ai(ui*(t+ω))[ui*(t+ω)-∑j=1n‍wijfj(uj*(t+ω))-Ii(t+ω)]=-ai(ui*(t+ω))[ui*(t+ω)-∑j=1n‍wijfj(uj*(t+ω))-Ii(t)],
That is, u*(t+ω)T is also a state of the model (2.13) with initial condition u0*(ω); here, u0*(ω)=u0*; hence, for all t≥0, from Theorem 3.1,
(3.19)u*(t+ω)=u*(t).
Hence, u*(t) is an isolated periodic orbit of model (2.13) with period ω, that is, a limit cycle of model (2.13). From (3.14), we can obtain that it is globally exponentially stable. The proof of Theorem 3.5 is completed.Remark 3.6.
Similar to those that are given in [18], global convergence of the output solutions in finite time also can be discussed, which can be embodied in the following example, and the detailed results are omitted.


## 4.     Illustrative Example

In this section, we shall give an example to illustrate the effectiveness of our results.Example 4.1.
Consider the following CGNN model:(4.1)du1(t)dt=(2+0.4cos(u1(t)))[-u1(t)-4sgn(u1(t))-2sgn(u2(t))+I1(t)],du2(t)dt=(2+0.4cos(u2(t)))[-u2(t)+3sgn(u1(t))-2sgn(u2(t))+I2(t)],
where
(4.2)sgn(s)={1,s>0,undefined,s=0,-1,s<0.
Obviously, -W is an H-matrix with wii<0 and
(4.3)ξ=(154),β=(1414).
Also, the subsets ΠC, ΠD, and ΠCD in this example are the same as those in example 1 in [18] which are depicted in detail in Figure 3 in [18].Firstly, we chooseI=(0,0)T∈ΠD,u0=(6,-6)T. The equilibrium point u* of model (4.1) is (0,0)T, and the corresponding output equilibrium point γ*=(0,0)T. Global convergence of u(t) and γ(t) in finite time can be obtained. Figure 1 depicts the behavior of state solution u(t) and output solution γ(t) with I=(0,0)T,u0=(6,-6)T.Transient behavior of(u1,u2) and (γ1,γ2) for I=(0,0)T,u0=(6,-6)T.
(a)
(b)
(c)
(d)Secondly, we chooseI=(4,-6)T∈ΠC,u0=(-6,6)T. Model (4.1) has a unique equilibrium point u*=(2,-1)T and a unique output equilibrium point γ*=(1,-1)T. Behavior of state solution and output solution is depicted in Figure 2.Transient behavior of(u1,u2) and (γ1,γ2) for I=(0,0)T,u0=(6,-6)T.
(a)
(b)
(c)
(d)Transient behavior of(u1,u2) and (γ1,γ2) for I=(0,0)T,u0=(6,-6)T.
(a)
(b)
(c)
(d)Then, we chooseI=(0,5)T∈ΠCD,u0=(4,-2)T.u*=(0,1.5)T and γ*=(-0.5,1)T are equilibrium point and output equilibrium point of model (4.1), respectively. Simulation results with I=(0,5)T,u0=(4,-2)T about global convergence in finite time of the state solution u(t) and corresponding output solution γ(t) are depicted in Figure 3.


## 5.   Conclusions

In this paper, by using the property ofM-matrix and a generalized Lyapunov-like approach, global convergence of CGNNs possessing discontinuous activation functions is investigated under the condition that neuron interconnection matrix belongs to the class of M-matrices or H-matrices. The uniqueness is proved for equilibrium point and corresponding output equilibrium point of considered neural networks. It is also proved that for considered model, the solution starting at a given initial condition is unique. Meanwhile, global exponential stability of equilibrium point is obtained for any input. Furthermore, by contraction mapping principle, the uniqueness and the globally exponential stability of limit cycle are given.



---

*Source: 109319-2012-11-20.xml*