# Convergence and Stability of the Split-Stepθ-Milstein Method for Stochastic Delay Hopfield Neural Networks

Authors: Qian Guo; Wenwen Xie; Taketomo Mitsui
Journal: Abstract and Applied Analysis
 (2013)

---

## Abstract

A new splitting method designed for the numerical solutions of stochastic delay Hopfield neural networks is introduced and analysed. Under Lipschitz and linear growth conditions, this split-stepθ-Milstein method is proved to have a strong convergence of order 1 in mean-square sense, which is higher than that of existing split-step θ-method. Further, mean-square stability of the proposed method is investigated. Numerical experiments and comparisons with existing methods illustrate the computational efficiency of our method.


---

## Body

## 1. Introduction

Hopfield neural networks, which originated with Hopfield in the 1980s [1], have been successfully applied in many areas such as combinatorial optimization [2, 3], signal processing [4], and pattern recognition [5, 6]. In the last decade, neural networks in the presence of signal transmission delay and stochastic perturbations, also named as stochastic delay Hopfield neural networks (SDHNNs), have gained considerable research interest (see, e.g., [7–9] and the references therein). It is noticed that, so far, most works on SDHNNs focus mainly on the stability analysis of the analytical solutions, including mean-square exponential stability [7], global asymptotic stability [9], and so forth. Not only simulation is an important tool to explore interesting dynamics of kinds of Hopfield neural networks (HNNs) (see, e.g., [10] and the references therein), but also parameter estimation in dynamical systems based on HNNs (see, e.g., [11]) needs to solve HNNs numerically. Moreover, because most of SDHNNs do not have explicit solutions, the numerical analysis of SDHNNs recently stirred some initial research attention. For example, Li et al. [12] investigated the exponential stability of the Euler method and the semi-implicit Euler method for SDHNNs. Rathinasamy [13] introduced a split-step θ-method (SST) for SDHNNs and analysed the mean-square stability of this method, and the SST is only given for the commensurable delay case. To the best of our current knowledge, the authors mainly discussed the stability of numerical solutions for stochastic Hopfield neural networks with discrete time delays but skipped the details of convergence analysis.The split-step Euler method for stochastic differential equations (SDEs) was proposed by Higham et al. [14], further, the splitting Euler-type algorithms have been derived for stochastic delay differential equations (SDDEs) [15, 16]. In this paper, we will present a splitting method with higher order convergence for SDHNNs. To be specific, we will go into detail about the convergence analysis and comparing the stability with split-step θ-method given in [13].The rest of this paper is organized as follows. In Section2, we recall the stochastic delay neural networks model and present a split-step θ-Milstein method. In Section 3, we derive the convergence results of the split-step θ-Milstein method for the model. In Section 4, the numerical stability analysis is performed. In Section 5, some numerical examples are given to confirm the theory. In the last Section, we draw some conclusions.


## 2. Model and the Split-Stepθ-Milstein Method

### 2.1. Model

Consider the stochastic delay Hopfield neural networks of the form(1)dx(t)=[-Cx(t)+Af(x(t))+Bg(z(t))]dt+Φ(x(t))dW(t),
where  x(t)=[x1(t),…,xn(t)]T∈ℝn is the state vector associated with the n neurons,  z(t)=[x1(t-τ1),…,xn(t-τn)]T, the diagonal matrix C=diag(c1,c2,…,cn) has positive entries, and ci represents the rate at which the ith unit will reset its potential to the resting state in isolation when discounted from the network and the external stochastic perturbation. The matrices A=(aij)n×n and B=(bij)n×n are the connection weight matrix and the discretely delayed connection weight matrix, respectively. Furthermore, the vector functions  f(x(t))=[f1(x1(t)),…,fn(xn(t))]T and  g(z(t))=[g1(x1(t-τ1)),…,gn(xn(t-τn))]T denote the neuron activation functions with the conditions fi(0)=0,gi(0)=0 for all positive τj.On the initial segment[-τ,0] the state vector satisfies  x(t)=ψ(t), where ψ(t)=[ψ1(t),…,ψn(t)]T is a given function in C([-τ,0],ℝn) and τ stands for max1≤i≤n{τi}.Moreover,Φ(x(t))=diag(ϕ1(x1(t)),…,ϕn(xn(t))) is a diagonal matrix with ϕi(0)=0 and W(t)=[W1(t),…,Wn(t)]T∈ℝn is an n-dimensional Wiener process defined on the complete probability space (Ω,ℱ,{ℱt}t≥0,ℙ) with a filtration {ℱt}t≥0 satisfying the usual conditions (i.e., it is increasing and right continuous while ℱ0 contains all ℙ-null sets).Letfi and gi be functions in C2(D;ℝ)⋂ℒ2([0,T];ℝ) and ϕi be in C1(D;ℝ)∩ℒ2([0,T];ℝ). Here Cl(D;ℝ) denotes the family of continuously l-times differentiable real-valued function defined on D, while ℒl([0,T];ℝ) denotes the family of all real-valued measurable {ℱt}-adapted stochastic processes {f(t)}t∈[0,T] such that ∫0T‍|f(t)|ldt<+∞.


### 2.2. Numerical Scheme

We define the mesh with a uniform step-sizeh(0<h<1) on the interval [0,T]; that is, tk=k·h(k=0,1,…,K) and T=Kh.LetΔWik=Wi(tk+1)-Wi(tk) denote the increment of the Wiener process. The split-step θ-Milstein (SSTM) scheme for the solution of SDEs (1) is given by
(2)Yk=Xk+[-(1-θ)CXk-θCYk+Af(Xk)+Bg(Zk)]h,Xk+1=Yk+Φ(Yk)ΔWk+12Φ^(Yk)[ΔWk∘ΔWk-1h],
where the merging parameter θ satisfies 0≤θ≤1, Xk=[X1k,…,Xnk]T is an approximation to  x(tk), and for 1≤qj∈ℤ+(3)Zjk={ψj(tk-τj),tk-τj≤0,Xjk-qj,0<tk-τj∈[tk-qj,tk-qj+1).
Moreover, we adopt the symbols Φ^(Yk) = diag(ϕ1′(Y1k)ϕ1(Y1k),…,ϕn′(Ynk)ϕn(Ynk)) and ΔWk = [ΔW1k,…,ΔWnk]T, the Hadamard product ΔWk∘ΔWk means [(ΔW1k)2,…,(ΔWnk)2]T, and 1=[1,…,1]T∈ℝn. When tk≤0, we define Xk=ψ(tk).Then scheme (2) can be written in equivalent form as(4a)Yk=Xk-(I+θCh)-1ChXk+(I+θCh)-1h[Af(Xk)+Bg(Zk)],(4b)Xk+1=Yk+Φ(Yk)ΔWk+12Φ^(Yk)[ΔWk∘ΔWk-1h].Substituting (4a) into (4b), we have a stochastic explicit single-step method with an increment function Λ(ξ,η,h,ΔWk)∈ℝn; that is,
(5)Xk+1=Xk+Λ(Xk,Zk,h,ΔWk).


## 2.1. Model

Consider the stochastic delay Hopfield neural networks of the form(1)dx(t)=[-Cx(t)+Af(x(t))+Bg(z(t))]dt+Φ(x(t))dW(t),
where  x(t)=[x1(t),…,xn(t)]T∈ℝn is the state vector associated with the n neurons,  z(t)=[x1(t-τ1),…,xn(t-τn)]T, the diagonal matrix C=diag(c1,c2,…,cn) has positive entries, and ci represents the rate at which the ith unit will reset its potential to the resting state in isolation when discounted from the network and the external stochastic perturbation. The matrices A=(aij)n×n and B=(bij)n×n are the connection weight matrix and the discretely delayed connection weight matrix, respectively. Furthermore, the vector functions  f(x(t))=[f1(x1(t)),…,fn(xn(t))]T and  g(z(t))=[g1(x1(t-τ1)),…,gn(xn(t-τn))]T denote the neuron activation functions with the conditions fi(0)=0,gi(0)=0 for all positive τj.On the initial segment[-τ,0] the state vector satisfies  x(t)=ψ(t), where ψ(t)=[ψ1(t),…,ψn(t)]T is a given function in C([-τ,0],ℝn) and τ stands for max1≤i≤n{τi}.Moreover,Φ(x(t))=diag(ϕ1(x1(t)),…,ϕn(xn(t))) is a diagonal matrix with ϕi(0)=0 and W(t)=[W1(t),…,Wn(t)]T∈ℝn is an n-dimensional Wiener process defined on the complete probability space (Ω,ℱ,{ℱt}t≥0,ℙ) with a filtration {ℱt}t≥0 satisfying the usual conditions (i.e., it is increasing and right continuous while ℱ0 contains all ℙ-null sets).Letfi and gi be functions in C2(D;ℝ)⋂ℒ2([0,T];ℝ) and ϕi be in C1(D;ℝ)∩ℒ2([0,T];ℝ). Here Cl(D;ℝ) denotes the family of continuously l-times differentiable real-valued function defined on D, while ℒl([0,T];ℝ) denotes the family of all real-valued measurable {ℱt}-adapted stochastic processes {f(t)}t∈[0,T] such that ∫0T‍|f(t)|ldt<+∞.


## 2.2. Numerical Scheme

We define the mesh with a uniform step-sizeh(0<h<1) on the interval [0,T]; that is, tk=k·h(k=0,1,…,K) and T=Kh.LetΔWik=Wi(tk+1)-Wi(tk) denote the increment of the Wiener process. The split-step θ-Milstein (SSTM) scheme for the solution of SDEs (1) is given by
(2)Yk=Xk+[-(1-θ)CXk-θCYk+Af(Xk)+Bg(Zk)]h,Xk+1=Yk+Φ(Yk)ΔWk+12Φ^(Yk)[ΔWk∘ΔWk-1h],
where the merging parameter θ satisfies 0≤θ≤1, Xk=[X1k,…,Xnk]T is an approximation to  x(tk), and for 1≤qj∈ℤ+(3)Zjk={ψj(tk-τj),tk-τj≤0,Xjk-qj,0<tk-τj∈[tk-qj,tk-qj+1).
Moreover, we adopt the symbols Φ^(Yk) = diag(ϕ1′(Y1k)ϕ1(Y1k),…,ϕn′(Ynk)ϕn(Ynk)) and ΔWk = [ΔW1k,…,ΔWnk]T, the Hadamard product ΔWk∘ΔWk means [(ΔW1k)2,…,(ΔWnk)2]T, and 1=[1,…,1]T∈ℝn. When tk≤0, we define Xk=ψ(tk).Then scheme (2) can be written in equivalent form as(4a)Yk=Xk-(I+θCh)-1ChXk+(I+θCh)-1h[Af(Xk)+Bg(Zk)],(4b)Xk+1=Yk+Φ(Yk)ΔWk+12Φ^(Yk)[ΔWk∘ΔWk-1h].Substituting (4a) into (4b), we have a stochastic explicit single-step method with an increment function Λ(ξ,η,h,ΔWk)∈ℝn; that is,
(5)Xk+1=Xk+Λ(Xk,Zk,h,ΔWk).


## 3. Order and Convergence Results for SSTM

In this section we consider the global error of SSTM (2) as applied to SDHNNs (1) with initial condition. In what follows, ∥·∥ denotes Euclidean norm in ℝn.For convergence purpose we make the following standard assumptions.Assumption 1.
Assume thatf, g, Φ, and Φ^ satisfy the Lipschitz condition
(6)|fi(a)-fi(b)|≤αi|a-b|,|gi(a)-gi(b)|≤βi|a-b|,(a,b∈ℝandt∈[0,T])|ϕi(a)-ϕi(b)|≤γi|a-b|,|ϕiϕi′(a)-ϕiϕi′(b)|≤κi|a-b|,
for every i and the linear growth condition
(7)∥f(x)∥2∨∥g(x)∥2∨∥Φ(x)·1∥2∨∥Φ^(x)·1∥2≤L~(1+∥x∥2),∀x∈ℝn,
where L~ is a positive constant and ∨ is the maximal operator. We also define Li as Li=max{αi,βi,γi,κi}  (i=1,2,…,n).We also need the following assumption on the initial condition.Assumption 2.
Assume that the initial functionψ(t) is Lipschitz continuous from [-τ,0] to ℝn, that is, there is a positive constant Lψ satisfying
(8)∥ψ(t)-ψ(s)∥≤Lψ(t-s)if-τ≤s<t<0.Now we give the definition of local and global errors.Definition 1.
Letx(t) denote the exact solution of (1). The local approximate solution  x~(tk+1) starting from  x(tk) by SSTM (2) given by
(9)x~(tk+1):=x(tk)+Λ(x(tk),Z~(tk),h,ΔWk),
where Z~(tk) denotes the evaluation of (3) using the exact solution, yields the difference
(10)δk+1:=x(tk+1)-x~(tk+1).
Then the local error of SSTM is defined by ∥δk+1∥, whereas its global error means ∥ϵk∥ where ϵk:=x(tk)-Xk.Definition 2.
If the global error satisfies(11)E(∥ϵk∥)2≤Γh2p∀h∈(0,h0)
with positive constants h0 and Γ and a finite p, then we say that the order of mean-square convergence accuracy of the method is p. Here E is the expectation with respect to ℙ.We then give the following lemmas that are useful in deriving the convergence results.Lemma 3 (see also [17]).
Let the linear growth condition (7) hold, and the initial function ψ(t) is assumed to be ℱ0-measurable and right continuous. And one puts Eψ:=E(sup-τ≤t≤0∥ψ(t)∥2)<∞. For any given positive T, there exist positive numbers Γψ and Γ2 such that the solution of (1) satisfies
(12)E(sup-τ≤s≤T∥x(s)∥2)≤Γψ,
where the constant Γψ is independent of step-size h but dependent on T. Moreover, for any 0≤s<t≤T,  t-s<1, the estimation
(13)E(∥x(t)-x(s)∥)2≤Γ2(t-s)
holds.The Jensen inequality derives(14)E(sup-τ≤s≤T∥x(s)∥)≤Γψ
from (12).Lemma 4.
Fors∈[tk,tk+h], one has
(15)E(∥z(s)-Z~(tk)∥2)≤Γτh.
Here the constant Γτ is independent of step-size h.Proof.
Iftk-τj≤0 and s-τj≤0, under Assumption 2 we have
(16)E[xj(s-τj)-Z~j(tk)]2=E[ψj(s-τj)-ψj(tk-τj)]2≤Lψ(s-tk)2≤Lψh2.
If  tk-τj≤0 and s-τj>0, with (13) we obtain
(17)E[xj(s-τj)-Z~j(tk)]2=E[xj(s-τj)-ψj(tk-τj)]2≤2E[xj(s-τj)-xj(0)]2+2E[ψj(0)-ψj(tk-τj)]2≤2(Γ2+Lψh)h.
If  tk-τj>0 and s-τj>0, we assume tk-τj∈[tk-qj,tk-qj+1) without loss of generality. Hence,
(18)E[xj(s-τj)-Z~j(tk)]2=E[xj(s-τj)-xj(tk-qj)]2≤2E[xj(s-τj)-xj(tk-τj)]2+2E[xj(tk-qj)-xj(tk-τj)]2≤4Γ2h
by using inequality (13).Lemma 5.
Letx(t) denote the exact solution of (1). One assumes conditions (6) and (7). Then for the local intermediate value  y(tk):=x(tk)-(I+θCh)-1Chx(tk)+(I+θCh)-1h[Af(x(tk))+Bg(Z~(tk))], one has the estimation
(19)E(∥x(tk)-y(tk)∥2)≤Γ3h2.Proof.
The difference between theith components of  x(tk) and  y(tk) leads to
(20)|xi(tk)-yi(tk)|2=h2(1+ciθh)2[cixi(tk)-∑j=1n‍aijfj(xj(tk))-∑j=1n‍bijgj(Z~j(tk))]2,
whose expectation, together with ci>0, fi(0)=gi(0)=0, the Lipschitz condition (6), and the estimation (12), gives
(21)E(∥x(tk)-y(tk)∥2)≤∑i=1n‍3h2(1+ciθh)2×[ci2E|xi(tk)|2+n∑j=1n‍aij2αj2E|xj(tk)|2+n∑j=1n‍bij2βj2E|Z~j(tk)|2]≤3Γψ∑i=1n‍(ci2+n∑j=1n‍aij2αj2+n∑j=1n‍bij2βj2)h2.Now we discuss local error estimates.Theorem 6.
When one assumes Assumptions1 and 2 and the conditions of Lemma 3, there exist positive constants Γ0 and Γ1, such that
(22)max0≤k≤K-1∥E(δk+1)∥≤Γ0h2,(23)max0≤k≤K-1E(∥δk+1∥2)≤Γ1h3
as h↓0.Proof.
The Itô integral form of theith component of (1) on [tk,t] implies
(24)xi(t)-xi(tk)=∫tkt‍[-cixi(s)+∑j=1n‍aijfj(xj(s))+∑j=1n‍bijgj(xj(s-τj))]ds+∫tkt‍ϕi(xi(s))dWi(s).
By utilizing the previous identity, the ith component of the difference δk+1 introduced in Definition 1 can be calculated as
(25)δik+1=xi(tk+1)-xi(tk)+cih1+ciθhxi(tk)-h1+ciθh[∑j=1n‍aijfj(xj(tk))+∑j=1n‍bijgj(Z~j(tk))]-ϕi(yi(tk))ΔWik-ϕi′(yi(tk))×ϕi(yi(tk))(ΔWik)2-h2=∫tktk+1‍ci[xi(tk)-xi(s)]ds-ci2h2θ1+ciθhxi(tk)+∫tktk+1‍∑j=1n‍aij[fj(xj(s))-fj(xj(tk))]+∑j=1n‍bij[gj(xj(s-τj))-gj(Z~j(tk))]ds+ciθh21+ciθh[∑j=1n‍aijfj(xj(tk))+∑j=1n‍bijgj(Z~j(tk))]+∫tktk+1‍ϕi(xi(s))dWi(s)-∫tktk+1‍ϕi(yi(tk))dWi(s)-∫tktk+1‍∫tks‍ϕi′(yi(tk))ϕi(yi(tk))dWi(r)dWi(s),
where ΔWik=∫tktk+1‍dWi(s) and ((ΔWik)2-h)/2=∫tktk+1‍∫tks‍dWi(r)dWi(s).
Taking expectations of both sides of (25),
(26)E(δik+1)=E(∫tktk+1‍ci∫tks‍-μi(xi(r))drds)-ci2h2θ1+ciθhE(xi(tk))+E(∫tktk+1‍∑j=1n‍aij∫tks‍fj′(xj(r))μj(xj(r))+12fj′′(xj(r))ϕj2(xj(r))drds∑j=1n‍)+E(∫tktk+1‍∑j=1n‍bij∫tk-qjs-τj‍gj′(xj(r))μj(xj(r))+12gj′′(xj(r))ϕj2(xj(r))drds∑j=1n‍)+ciθh21+ciθh[∑j=1n‍aijE(fj(xj(tk)))+∑j=1n‍bijE(gj(Z~j(tk)))]
by (24) and Itô formula, where μi(xi(r))  := -cixi(r)+∑j=1n‍aijfj(xj(r))+∑j=1n‍bijgj(xj(r-τj)). Under conditions of this theorem, we have |E(δik+1)|≤Γ~0h2 by (14), Jensen inequality |E(X)|≤E(|X|), triangle inequality, and properties of definite integral. Then we have max0≤k≤K-1∥E(δk+1)∥≤Γ0h2 from the relation between |E(δik+1)| and ∥E(δk+1)∥.
Now we prove (23). By Itô formula,
(27)∫tktk+1‍ϕi(xi(s))dWi(s)-∫tktk+1‍ϕi(yi(tk))dWi(s)-∫tktk+1‍∫tks‍ϕi′(yi(tk))ϕi(yi(tk))dWi(r)dWi(s)=∫tktk+1‍[ϕi(xi(tk))-ϕi(yi(tk))]dWi(s)+∫tktk+1‍∫tks‍ϕi′(xi(r))μi(xi(r))+12ϕi′′(xi(r))ϕi(xi(r))2drdWi(s)+∫tktk+1‍∫tks‍[ϕi′(xi(r))ϕi(xi(r))-ϕi′(yi(tk))×ϕi(yi(tk))ϕi′(xi(r))]dWi(r)dWi(s).
From (25) and (27), we have
(28)E(|δik+1|2)≤9{cih∫tktk+1‍E[xi(tk)-xi(s)]2ds︸Cauchy-Schwartzinequality+n∑j=1n‍aij2αj2h∫tktk+1‍E[xj(s)-xj(tk)]2ds︸Cauchy-SchwartzandHo¨lderinequality+n∑j=1n‍bij2βj2h∫tktk+1‍E[xj(s-τj)-Z~j(tk)]2ds︸Cauchy-SchwartzandHo¨lderinequality+ci4h4θ2(1+ciθh)2E|xi(tk)|2+nci2h4θ2(1+ciθh)2[∑j=1n‍aij2E|fj(xj(tk))|2+∑j=1n‍bij2E|gj(Z~j(tk))|2]+∫tktk+1‍γj2E[xi(tk)-yi(tk)]2ds︸ItôIsometry+∫tktk+1‍(s-tk)∫tks‍E|ϕi′(xi(r))μi(xi(r))+(1/2)ϕi′′(xi(r))ϕi(xi(r))2|2drds︸Cauchy-SchwartzinequalityandItôIsometry+∫tktk+1‍∫tks‍E[ϕi′(xi(r))ϕi(xi(r))-ϕi′(yi(tk))ϕi(yi(tk))]2drds︸ItôIsometry}≤9{∫tktk+1‍∫tks‍Γ5hdrds︸TriangleInequality,(6),(13),(19)h(ci+n∑j=1n‍aij2αj2)∫tktk+1‍Γ2h︸(13)ds+h(n∑j=1n‍bij2βj2)∫tktk+1‍Γτh︸(15)ds+h4ci2θ2(1+ciθh)2Γψ[ci2+n∑j=1n‍aij2αj2+n∑j=1n‍bij2βj2]︸(6)and(12)+∫tktk+1‍γj2Γ3h2︸(19)ds+∫tktk+1‍Γ4(s-tk)2︸momentboundedds+∫tktk+1‍∫tks‍Γ5hdrds︸TriangleInequality,(6),(13),(19)}≤Γ~1h3.
Finally, it is easy to prove E(∥δk+1∥2)≤Γ1h3.Thanks to Theorem 1 in [18], we can conclude that
(29)E(∥ϵk∥2)≤Γh2∀h∈(0,1),
that is, the mean-square order of global error of the SSTM is 1.


## 4. Stability of SSTM

We are concerned with the stability of SSTM solution. Since (1) has an equilibrium solution  x(t)≡0, we will discuss whether the SSTM solution Xk with a positive step-size can attain a similar stability when k goes to infinity. First we give a sufficient condition for the exponential stability in mean-square sense of the equilibrium solution. The references [13, 19] give the condition as
(30)∑j=1n‍|aij|αj+∑j=1n‍|bij|βj≤αi∑j=1n‍|aji|+βi∑j=1n‍|bji|,-2ci+∑j=1n‍|aij|αj+∑j=1n‍|bij|βj+αi∑j=1n‍|aji|+βi∑j=1n‍|bji|+γi2<0
for every i(i=1,…,n).Definition 7.
A numerical method is said to be mean-square stable (MS-stable) if there exists anh0>0 such that any application of the method to problem (1) generates numerical approximations Xik, which satisfies
(31)limk→∞E(|Xik|2)=0,i=1,2,…,n,
for all h∈(0,h0).Theorem 8.
Assume (6), (30), and (1-θ)cih≤1 are satisfied; then the SSTM (2) are mean-square stable if h∈(0,h0), where
(32)h0={min1≤i≤n{1,hi}forθ=1,min1≤i≤n{1,1(1-θ)ci,hi}forθ∈[0,1).
Here hi is the smallest positive root of the cubic equation with respect to z given by
(33)𝒜iz3+ℬiz2+𝒞iz+𝒟i=0,
where the coefficients mean
(34)𝒜i=κi22(-(1-θ)ci+∑j=1n‍|aij|αj+∑j=1n‍|bij|βj)2,ℬi=γi2(-(1-θ)ci+∑j=1n‍|aij|αj+∑j=1n‍|bij|βj)2+κi2(∑j=1n‍|aij|αj+∑j=1n‍|bij|βj)-κi2(1-θ)ci,𝒞i=(-(1-θ)ci+∑j=1n‍|aij|αj+∑j=1n‍|bij|βj)2+2γi2(-(1-θ)ci+∑j=1n‍|aij|αj+∑j=1n‍|bij|βj)-ci2θ2+κi22,𝒟i=2(-ci+∑j=1n‍|aij|αj+∑j=1n‍|bij|βj)+γi2.
for i=1,2,…,n.Proof.
Squaring on both sides of (4b) and (4a), we have
(35)|Xik+1|2=(Yik)2+[ϕi(Yik)ΔWik]2+[ϕi′(Yik)ϕi(Yik)]2[(ΔWik)2-h2]2+[(ϕi(Yik))ΔWik+Yik]×ϕi′(Yik)ϕi(Yik)[(ΔWik)2-h]+2Yikϕi(Yik)ΔWik,(36)(1+θcih)2|Yik|2=[(1-(1-θ)cih)Xik]2+h2[∑j=1n‍aijfj(Xjk)]2+h2[∑j=1n‍bijgj(Zjk)]2+∑j=1n‍{[aijfj(Xjk)+bijgj(Zjk)]2h(1-(1-θ)cih)Xik×[aijfj(Xjk)+bijgj(Zjk)]}+2h2[∑j=1n‍aijfj(Xjk)][∑j=1n‍bijgj(Zjk)].
Taking expectations of both sides of (35), we can get
(37)E(|Xik+1|2)=E(|Yik|2)+hE(|ϕi(Yik)|2)+h22E(|ϕi′(Yik)ϕi(Yik)|2)≤(1+γi2h+h22κi2)E(|Yik|2).
Together with (36), we have
(38)E(|Xik+1|2)≤(1+γi2h+h22κi2)E(|Yik|2)≤1+γi2h+(h2/2)κi2(1+θcih)2×{(∑j=1n‍|bij|βj)[1-(1-θ)cih]2E(|Xik|2)+h2(∑r=1n‍|air|αr)(∑j=1n‍|aij|αjE(|Xjk|2))+h2(∑r=1n‍|bir|βr)(∑j=1n‍|bij|βjE(|Zjk|2))+∑j=1n‍h|1-(1-θ)cih|×[|aij|αj[E(|Xik|2)+E(|Xjk|2)]+|bij|βj[E(|Xik|2)+E(|Zjk|2)]]+2h2(∑j=1n‍|aij|αj)×(∑j=1n‍|bij|βj)[E(|Xjk|2)+E(|Zjk|2)]}.
Thus, we attain
(39)E(|Xik+1|2)≤[P(h)+∑j=1n‍Qj(h)+∑j=1n‍Rj(h)]×max1≤j≤n{E(|Xik|2),E(|Xjk|2),E(|Zjk|2)},
where
(40)P(h)=1+γi2h+(h2κi2/2)(1+θcih)2×{[1-(1-θ)cih]2+h∑j=1n‍|1-(1-θ)cih|×(|aij|αj+|bij|βj)∑j=1n‍[1-(1-θ)cih]2},(41)Qj(h)=1+γi2h+(h2κi2/2)(1+θcih)2×{h2|aij|αj∑j=1n‍(|air|αr+|bir|βr)+h|1-(1-θ)cih|(|aij|αj)∑j=1n‍},(42)Rj(h)=1+γi2h+(h2κi2/2)(1+θcih)2×{h2(|aij|αj+|bij|βj)(∑r=1n‍|bir|βr)+h|1-(1-θ)cih|(|bij|βj)(∑r=1n‍|bir|βr)}.
Note that the assumption of Theorem implies the nonnegativity of 1-(1-θ)cih.
Obviously, whenk→+∞, E(|Xik|2)→0 if the inequality P(h)+∑j=1nQj(h)+∑j=1nRj(h)<1 holds, which is equivalent to the inequality 𝒜ih3+ℬih2+𝒞ih+𝒟i<0.
Furthermore, it is easy to prove𝒜i>0 and 𝒟i<0 by virtue of (30). By Vieta's formulas, the product of three roots of (33) satisfies z1z2z3=-(𝒟i/𝒜i)>0. This means that (33) has at least one positive root. Therefore, let hi denote the smallest positive root of the equation. Moreover, we note that at the origin the right-hand side polynomial of (33) is negative. This completes the proof.


## 5. Numerical Results

Now, we apply the introduced SSTM method to two test cases of SDHNNs in order to compare their performance with the split-stepθ-method in [13], which has strong convergence order 0.5.The mean-square errorɛ of numerical approximations at time T versus the step-size is depicted in log-log diagrams, where ɛ:=(1/2000)∑r=12000∥XωrK-x(T)∥2. Here  x(T) stands for the value of explicit solution of (1) at time T=3 and XωrK is its numerical approximation along the rth sample path {ωr:r=1,2,…,2000}. We compute the numerical solution using the split-step θ-Milstein method (2) with step-size h=2-12, and we will call this the “exact solution.”Example 9.
Consider the following two-dimensional stochastic delay Hopfield neural networks of the form(43)d(x1(t)x2(t))=-C(x1(t)x2(t))dt+A(f(x1(t))f(x2(t)))dt+B(g(x1(t-τ1))g(x2(t-τ2)))dt+ΦdW(t)
on t⩾0 with the initial condition x1(t)=t+1,t∈[-1,0] and x2(t)=t+1,t∈[-2,0].Case 1.
Letf(x)=sin(x),g(x)=arctan(x),
(44)C=(200020),A=(4-563),B=(-6431),Φ=(x1(t)00-5x2(t)).Case 2.
Letf(x)=x,g(x)=tanh(x),
(45)C=(100010),A=(0000),B=(5-115),Φ=(1.5x1(t)00-1.5x2(t)).In Figure1, SSTM is applied with 7 different step-sizes: hm=2m-12 for m=1,2,…,7. Two pairs of time delays (τ1,τ2) are set to (1,2) and (1.13,2.31). The first pair has common factor hm; however, the second pair is incommensurable by hm. The computation errors ɛ versus step-sizes h are plotted on a log-log scale and the reference lines of slope 1 are added. It illustrates that SSTM has raised the strong order of the split-step θ-method at least to 1 for SDHNNs [13].Errors versus step-sizes for the SDHNNs (43).
(a)
Case1 with τ1=1,τ2=2
(b)
Case2 with τ1=1,τ2=2
(c)
Case1 with τ1=1.13,τ2=2.31
(d)
Case2 with τ1=1.13,τ2=2.31Next, Table1 shows a comparison of stability intervals between the SST and the SSTM for (43). Two sets of the interval in the Table are calculated through Theorem 8 in this paper and Theorem 5.1 in [13]. It is easy to see that the stability intervals of the two methods are similar.Table 1
Calculated intervals of step-size for mean-square stability of the numerical schemes.
Numerical scheme
Case1
Case2
Split-stepθ-Milstein method (θ=0.0)
(0.0000, 0.0500]
(0.0000, 0.1000]
Split-stepθ-Milstein method (θ=0.2)
(0.0000, 0.0625]
(0.0000, 0.1250]
Split-stepθ-Milstein method (θ=0.5)
(0.0000, 0.1000]
(0.0000, 0.2000]
Split-stepθ-Milstein method (θ=0.8)
(0.0000, 0.0650)
(0.0000, 0.5000]
Split-stepθ-Milstein method (θ=1.0)
(0.0000, 0.0500)
(0.0000, 0.3534)
Split-stepθ-method (θ=0.0)
(0.0000, 0.0500]
(0.0000, 0.1000]
Split-stepθ-method (θ=0.2)
(0.0000, 0.0625]
(0.0000, 0.1250]
Split-stepθ-method (θ=0.5)
(0.0000, 0.1000]
(0.0000, 0.2000]
Split-stepθ-method (θ=0.8)
(0.0000, 0.0689)
(0.0000, 0.5000]
Split-stepθ-method (θ=1.0)
(0.0000, 0.0540)
(0.0000, 0.5792)We know that Theorem 5.1 in [13] and Theorem 8 in this paper only give sufficient conditions of mean-square stability. Therefore the stability intervals given by these theorems are only subsets of real ones. To confirm the situation, we calculated the sample moments of the approximate solution and plotted them along the time t. Here the sample moment ηi means (1/2000)∑r=12000∥Xωri∥2 for the numerical solution Xωri approximating  x(ti) along the rth sample path. Figures 2, 3, 4, 5, and 6 depict the results by SST and SSTM in the log-scaled vertical axis. All the figures can give a rough estimate of the stability interval in each case.Mean-square stability comparison of SST and SSTM for the SDHNNs (43), Case 1, θ=0.
(a)
SST
(b)
SSTMMean-square stability comparison of SST and SSTM for the SDHNNs (43), Case 2, θ=0.
(a)
SST
(b)
SSTMMean-square stability comparison of SST and SSTM for the SDHNNs (43), Case 1, θ=0.2.
(a)
SST
(b)
SSTMMean-square stability comparison of SST and SSTM for the SDHNNs (43), Case 2, θ=0.2.
(a)
SST
(b)
SSTMMean-square stability comparison of SST and SSTM for the SDHNNs (43).
(a)
Case1
(b)
Case2


## 6. Concluding Remarks

We introduce the split-stepθ-Milstein method (SSTM), which exhibits higher strong convergence rate than the split-step θ-method (SST, see [13]) for a stochastic delay Hopfield neural networks, and the scheme proposed in this paper can deal with incommensurable time delays which were not considered in [13]. We give the proof of convergence results, which has generally been omitted in the previous works on the same subject. By comparing the stability intervals of step size for the SST and SSTM for a test example, we find they exhibit similar mean-square stability.In this paper, we have found a delay-independent sufficient condition for mean-square stability of split-stepθ-Milstein method applied to nonlinear stochastic delay Hopfield neural networks. Further, Figure 6 suggests that the value of h0, the right end-point of the stability interval, given by Theorem 5.1 in [13] and Theorem 8 in this paper is much smaller than the true value when θ is close to unity. In this case, we need other techniques for stability analysis in this kind of stochastic delay differential system. To the best of our knowledge, the works in [20, 21] put forward good attempts. On the other hand, with respect to stochastic delay differential equations, some other types of stability have been successfully discussed for the Euler-type scheme, for example, mean-square exponential stability [12], delay-dependent stability [22], delay-dependent exponential stability [23], and almost sure exponential stability [24]. To Milstein-type scheme, in view of more sophisticated derivations, these issues would be challenging for future research.



---

*Source: 169214-2013-04-07.xml*