# Robust Linear Neural Network for Constrained Quadratic Optimization

Authors: Zixin Liu; Yuanan Liu; Lianglin Xiong
Journal: Discrete Dynamics in Nature and Society
 (2017)

---

## Abstract

Based on the feature of projection operator under box constraint, by using convex analysis method, this paper proposed three robust linear systems to solve a class of quadratic optimization problems. Utilizing linear matrix inequality (LMI) technique, eigenvalue perturbation theory, Lyapunov-Razumikhin method, and LaSalle’s invariance principle, some stable criteria for the related models are also established. Compared with previous criteria derived in the literature cited herein, the stable criteria established in this paper are less conservative and more practicable. Finally, a numerical simulation example and an application example in compressed sensing problem are also given to illustrate the validity of the criteria established in this paper.


---

## Body

## 1. Introduction

Quadratic optimization problem is a simple but very important and basic problem in convex optimization theory. It is widely applied in many scientific and engineering applications, such as regression analysis, data fusion, system identification, filter design, and compressed sensing [1–5]. Among these applications, the real-time solutions of such quadratic optimization problems are often required. In order to solve quadratic optimization problem, many different algorithms are provided in previous reference, such as proximal point algorithm (PPA), extended PPA, and splitting methods [6–8]. However, in many practical optimization problems, the numbers of decision variables and constraints are usually very large. When a large-scale quadratic optimization problem has to be performed in practical problem, computation complexity becomes more challenging. For such applications, classical optimization techniques may not be competent due to the problem dimension and stringent requirement on computational time [9]. One promising method for solving these problems is to employ artificial recurrent neural networks, since neural network has parallel computing capacity [10]. Mathematically, the optimization problem to be solved is mapped into a dynamical system so that its state output can give the optimal solution and the optimal solution is then obtained by tracking the state trajectory of the designed dynamical system based on the numerical ordinary differential equation technique [11]. From the pioneering work of McCulloch and Pittes, numerous neural network models have been developed. Compared with conventional numerical optimization algorithm, neural network has a low model complexity and parallel computing capacity, it is more suitable for engineering applications, and it has a weaker global convergence condition.In the recent decades, all kinds of different neural network models were established to solve variant constrained optimization problems. These optimization problems include game theory, linear programming problems, linear complement problems, projection equation problems, variational inequality problems, nonlinear optimization problems, general convex optimization problems, nonconvex optimization problems, and nonsmooth optimization problems.Neural network for solving linear programming problem perhaps stemmed back from Pyne’s work [12] and Tank and Hopfield’s work [13]. Their seminal work has inspired other researchers to develop recurrent neural networks for nonlinear optimization. Zhang and Constantinides derived a Lagrangian neural network for solving nonlinear convex optimization problems with linear equality constraints in [14]. In [15], Zhang researched the exponential stability of quadratic optimization problem on neural network and established a discrete-time neural network model to solve quadratic optimization problem with convex constraint only. Tan et al. studied the global exponential stability of discrete-time neural network for constrained quadratic optimization problem in [16]. In order to solve more general optimization problem, Yashtini and Malek investigated a discrete-time neural network model for solving nonlinear convex problems with hybrid constraints in [17]. Bouzerdoum and Pattison presented a neural network for solving quadratic convex optimization problems with bounded variables in [18]. Xia proposed primal-dual neural networks for solving linear and quadratic programming problems in [19], researched a dual neural network for solving strictly convex quadratic programming problems in [20], and proposed a Bi-projection neural network for solving constrained quadratic optimization problems in [21]. To solve quadratic minimax optimization problems, Liu and Wang [22] proposed a projection neural network (PNN) for constrained quadratic minimax optimization problem. To solve nonsmooth optimization problems, Liu and Wang [23] proposed a one-layer PNN for solving a class of pseudoconvex and nonsmooth nonlinear optimization problems.It is worth pointing out that most of above established neural network models are nonlinear forms. And the stability criteria derived in the literature are based on Lyapunov stable theory. However, when the constrained conditions of quadratic optimization are box constrained, nonlinear projection operator satisfies section constraint; in this case, nonlinear projection operator can be expressed by a linear form with uncertain term. This means that nonlinear projection neural network can be rewritten as a robust linear system. Thus, utilizing eigenvalue perturbation theory, we can give out some new system stable criteria. This idea inspires this work. In this paper, using convex analysis tool, we first establish some new robust linear neural network for box constrained quadratic optimization; then, by using LMI technique and eigenvalue perturbation theory, some exponentially stable criteria are established. When time delays are considered, by using Lyapunov-Razumikhin method and LaSalle’s invariance principle, we further derive some asymptotical stability criteria for the established time-delayed linear robust neural network. To illustrate the efficiencies and validity of the derived stability criteria in this paper, a numerical example and an application example in compressed sensing problem are also given.The remainder of this paper is organized as follows. In Section2, a constrained quadratic optimization problem and related neural network models are described. In Section 3, the global stability and convergence of the proposed neural network are analyzed. In Section 4, a simulation numerical example and an application to compressed sensing are given. Finally, conclusions are drawn in Section 5.


## 2. Problem and Neural Network Model

Consider the following constrained quadratic optimization problem:(1)min⁡Ex=min⁡12xTMx+bTx∣x∈Ω,where x=(x1,x2,…,xn)T∈Rn, M is a symmetric nonnegative definite matrix, b is a vector, and the superscript T denotes the transpose operator. Ω is defined by Ω={x∈Rn∣xi∈[ci,di],i=1,2,…,n}, where ci, di are constants such that ci≤di,i=1,2,…,n. It is seen that (1) can contain the constrained least square problem as a special case. Many solution methods for solving (1) were presented, including neural networks and numerical optimization algorithms. They all have a solution space being greater than the solution space of (1). As a result, when n is large, these solution methods will have a slow convergence rate. However, fast computation of such large optimization problems is often required in practice. Due to low model complexity and parallel computing capacity, neural networks algorithm becomes a popular method to solve problem (1). In what follows, by using projection theory and equivalent transform, some neural network models for solving problem (1) will be first introduced.


### 2.1. Neural Network without Time Delays

As is well known, without considering time delays, the optimal solutionx∗ for problem (1) is equivalent to the equilibrium point of the following project neural network:(2)dxtdt=-xt+PΩxt-ρMxt-ρb,t>t0xt0=x0,t≤t0,where ρ>0 is an arbitrary constant and PΩ(·) is a projection operator defined by(3)PΩx=argminν∈Ω⁡x-ν2,where ·2 denotes l2 norm of Rn. Denote PΩ(x)=(PΩ(x1),PΩ(x2),PΩ(xn))T; since Ω={x∈Rn∣xi∈[ci,di],i=1,2,…,n}, from [2], the display expression of PΩ(xi) is as follows:(4)PΩxi=ci,xi<cixi,ci≤xi≤didi,xi>di.Set u(t)=x(t)-ρMx(t)-ρb and substitute u(t) into (2); it yields that(5)dutdt=I-ρMdxtdt=-I-ρMxt+I-ρMPΩut=-ut+I-ρMPΩut-ρb.Denote u∗ as an equilibrium point of system (5), and let y(t)=u(t)-u∗; system (5) can be rewritten in the following form:(6)dytdt=-yt+I-ρMPΩyt+u∗-PΩu∗=-yt+I-ρMP¯Ωyt,where P¯Ω(y(t))=PΩ(y(t)+u∗)-PΩ(u∗). Notice the expression of PΩ(·) in (4); one can obtain that(7)0≤PΩu-PΩνu-ν≤1,∀u≠ν∈R.Denote y(t)=(y1(t),y2(t),…,yn(t))T; since P¯Ω(y(t))=PΩ(y(t)+u∗)-PΩ(u∗), it follows that(8)0≤P¯Ωyityit≤1,∀u≠ν∈R.Set(9)λiyit=P¯Ωyityit,Δt=diag⁡λ1y1t,λ2y2t,…,λnynt;it follows that 0≤λi(yi(t))≤1, Δ(t)TΔ(t)≤I.(10)P¯Ωyit=λiyit·1+1-λiyit·0yit.And system (6) can be rewritten in the following linear robust neural network model form:(11)dytdt=-I+I-ρMΔtyt.


### 2.2. Neural Network with Time Delays

#### 2.2.1. Time-Delayed Neural Network Type I

When the time delay is taken into account, a delayed projection neural network, which can be regarded as an improvement form for model (2), can be suggested for solving (1) as follows:(12)dxtdt=-xt+PΩxt-τ-ρMxt-τ-ρb,t>t0,xt0=x0,t≤t0,where time delay τ>0 is a constant. Set u(t-τ)=x(t-τ)-ρMx(t-τ)-ρb, and substitute u(t-τ) into (12); it yields that(13)dutdt=I-ρMdxtdt=-I-ρMxt+I-ρMPΩut-τ=-ut+I-ρMPΩut-τ-ρb.Denote u∗ as an equilibrium point of system (13), and let y(t)=u(t)-u∗; system (13) can be rewritten in the following form:(14)dytdt=-yt+I-ρMPΩyt-τ+u∗-PΩu∗=-yt+I-ρMP^Ωyt-τ,where P^Ω(y(t-τ))=PΩ(y(t-τ)+u∗)-PΩ(u∗). Notice the expression of PΩ(·) in (4); one can obtain that(15)0≤PΩu-PΩνu-ν≤1,∀u≠ν∈R.Denote y(t-τ)=(y1(t-τ),y2(t-τ),…,yn(t-τ))T; since P^Ω(y(t-τ))=PΩ(y(t-τ)+u∗)-PΩ(u∗), it follows that(16)0≤P^Ωyit-τyit-τ≤1,∀u≠ν∈R.Set(17)λiyit-τ=P^Ωyit-τyit-τ,Δ^t=diag⁡λ1y1t-τ,λ2y2t-τ,…,λnynt-τ;it follows that 0≤λi(yi(t-τ))≤1, Δ^(t)TΔ^(t)≤I.(18)P¯Ωyit-τ=λiyit-τ·1+1-λiyit-τ·0yit-τ.And system (14) can be rewritten in the following time-delayed linear robust neural network form:(19)dytdt=-yt+I-ρMΔ^tyt-τ.


#### 2.2.2. Time-Delayed Neural Network Type II

Another time-delayed neural network, which can be regarded as another improvement form for model (2), can be suggested for solving (1) as follows:(20)dxtdt=PΩxt-τ-ρMxt-τ-ρb-xt-τ,t>t0,xt0=x0,t≤t0,where time delay τ>0 is a constant. Set u(t-τ)=x(t-τ)-ρMx(t-τ)-ρb, and substitute u(t-τ) into (20); similar to the technique used above, system (20) can be transformed into the following equivalent time-delayed linear robust neural network form:(21)dytdt=-I+I-ρMΔ^tyt-τ.


## 2.1. Neural Network without Time Delays

As is well known, without considering time delays, the optimal solutionx∗ for problem (1) is equivalent to the equilibrium point of the following project neural network:(2)dxtdt=-xt+PΩxt-ρMxt-ρb,t>t0xt0=x0,t≤t0,where ρ>0 is an arbitrary constant and PΩ(·) is a projection operator defined by(3)PΩx=argminν∈Ω⁡x-ν2,where ·2 denotes l2 norm of Rn. Denote PΩ(x)=(PΩ(x1),PΩ(x2),PΩ(xn))T; since Ω={x∈Rn∣xi∈[ci,di],i=1,2,…,n}, from [2], the display expression of PΩ(xi) is as follows:(4)PΩxi=ci,xi<cixi,ci≤xi≤didi,xi>di.Set u(t)=x(t)-ρMx(t)-ρb and substitute u(t) into (2); it yields that(5)dutdt=I-ρMdxtdt=-I-ρMxt+I-ρMPΩut=-ut+I-ρMPΩut-ρb.Denote u∗ as an equilibrium point of system (5), and let y(t)=u(t)-u∗; system (5) can be rewritten in the following form:(6)dytdt=-yt+I-ρMPΩyt+u∗-PΩu∗=-yt+I-ρMP¯Ωyt,where P¯Ω(y(t))=PΩ(y(t)+u∗)-PΩ(u∗). Notice the expression of PΩ(·) in (4); one can obtain that(7)0≤PΩu-PΩνu-ν≤1,∀u≠ν∈R.Denote y(t)=(y1(t),y2(t),…,yn(t))T; since P¯Ω(y(t))=PΩ(y(t)+u∗)-PΩ(u∗), it follows that(8)0≤P¯Ωyityit≤1,∀u≠ν∈R.Set(9)λiyit=P¯Ωyityit,Δt=diag⁡λ1y1t,λ2y2t,…,λnynt;it follows that 0≤λi(yi(t))≤1, Δ(t)TΔ(t)≤I.(10)P¯Ωyit=λiyit·1+1-λiyit·0yit.And system (6) can be rewritten in the following linear robust neural network model form:(11)dytdt=-I+I-ρMΔtyt.


## 2.2. Neural Network with Time Delays

### 2.2.1. Time-Delayed Neural Network Type I

When the time delay is taken into account, a delayed projection neural network, which can be regarded as an improvement form for model (2), can be suggested for solving (1) as follows:(12)dxtdt=-xt+PΩxt-τ-ρMxt-τ-ρb,t>t0,xt0=x0,t≤t0,where time delay τ>0 is a constant. Set u(t-τ)=x(t-τ)-ρMx(t-τ)-ρb, and substitute u(t-τ) into (12); it yields that(13)dutdt=I-ρMdxtdt=-I-ρMxt+I-ρMPΩut-τ=-ut+I-ρMPΩut-τ-ρb.Denote u∗ as an equilibrium point of system (13), and let y(t)=u(t)-u∗; system (13) can be rewritten in the following form:(14)dytdt=-yt+I-ρMPΩyt-τ+u∗-PΩu∗=-yt+I-ρMP^Ωyt-τ,where P^Ω(y(t-τ))=PΩ(y(t-τ)+u∗)-PΩ(u∗). Notice the expression of PΩ(·) in (4); one can obtain that(15)0≤PΩu-PΩνu-ν≤1,∀u≠ν∈R.Denote y(t-τ)=(y1(t-τ),y2(t-τ),…,yn(t-τ))T; since P^Ω(y(t-τ))=PΩ(y(t-τ)+u∗)-PΩ(u∗), it follows that(16)0≤P^Ωyit-τyit-τ≤1,∀u≠ν∈R.Set(17)λiyit-τ=P^Ωyit-τyit-τ,Δ^t=diag⁡λ1y1t-τ,λ2y2t-τ,…,λnynt-τ;it follows that 0≤λi(yi(t-τ))≤1, Δ^(t)TΔ^(t)≤I.(18)P¯Ωyit-τ=λiyit-τ·1+1-λiyit-τ·0yit-τ.And system (14) can be rewritten in the following time-delayed linear robust neural network form:(19)dytdt=-yt+I-ρMΔ^tyt-τ.


### 2.2.2. Time-Delayed Neural Network Type II

Another time-delayed neural network, which can be regarded as another improvement form for model (2), can be suggested for solving (1) as follows:(20)dxtdt=PΩxt-τ-ρMxt-τ-ρb-xt-τ,t>t0,xt0=x0,t≤t0,where time delay τ>0 is a constant. Set u(t-τ)=x(t-τ)-ρMx(t-τ)-ρb, and substitute u(t-τ) into (20); similar to the technique used above, system (20) can be transformed into the following equivalent time-delayed linear robust neural network form:(21)dytdt=-I+I-ρMΔ^tyt-τ.


## 2.2.1. Time-Delayed Neural Network Type I

When the time delay is taken into account, a delayed projection neural network, which can be regarded as an improvement form for model (2), can be suggested for solving (1) as follows:(12)dxtdt=-xt+PΩxt-τ-ρMxt-τ-ρb,t>t0,xt0=x0,t≤t0,where time delay τ>0 is a constant. Set u(t-τ)=x(t-τ)-ρMx(t-τ)-ρb, and substitute u(t-τ) into (12); it yields that(13)dutdt=I-ρMdxtdt=-I-ρMxt+I-ρMPΩut-τ=-ut+I-ρMPΩut-τ-ρb.Denote u∗ as an equilibrium point of system (13), and let y(t)=u(t)-u∗; system (13) can be rewritten in the following form:(14)dytdt=-yt+I-ρMPΩyt-τ+u∗-PΩu∗=-yt+I-ρMP^Ωyt-τ,where P^Ω(y(t-τ))=PΩ(y(t-τ)+u∗)-PΩ(u∗). Notice the expression of PΩ(·) in (4); one can obtain that(15)0≤PΩu-PΩνu-ν≤1,∀u≠ν∈R.Denote y(t-τ)=(y1(t-τ),y2(t-τ),…,yn(t-τ))T; since P^Ω(y(t-τ))=PΩ(y(t-τ)+u∗)-PΩ(u∗), it follows that(16)0≤P^Ωyit-τyit-τ≤1,∀u≠ν∈R.Set(17)λiyit-τ=P^Ωyit-τyit-τ,Δ^t=diag⁡λ1y1t-τ,λ2y2t-τ,…,λnynt-τ;it follows that 0≤λi(yi(t-τ))≤1, Δ^(t)TΔ^(t)≤I.(18)P¯Ωyit-τ=λiyit-τ·1+1-λiyit-τ·0yit-τ.And system (14) can be rewritten in the following time-delayed linear robust neural network form:(19)dytdt=-yt+I-ρMΔ^tyt-τ.


## 2.2.2. Time-Delayed Neural Network Type II

Another time-delayed neural network, which can be regarded as another improvement form for model (2), can be suggested for solving (1) as follows:(20)dxtdt=PΩxt-τ-ρMxt-τ-ρb-xt-τ,t>t0,xt0=x0,t≤t0,where time delay τ>0 is a constant. Set u(t-τ)=x(t-τ)-ρMx(t-τ)-ρb, and substitute u(t-τ) into (20); similar to the technique used above, system (20) can be transformed into the following equivalent time-delayed linear robust neural network form:(21)dytdt=-I+I-ρMΔ^tyt-τ.


## 3. Stability Analysis

Since systems (11), (19), and (21) are all linear differential equations with uncertain term, the stability criteria for these systems can be derived by using eigenvalue perturbation theory, Lyapunov-Razumikhin method, and LMI technique. To derive the stability criteria, we first introduce the following lemma.Lemma 1 (see [24]).
Suppose thatΔ(t)=[I-F(t)J]-1F(t), where J is a constant matrix satisfying I-JJT>0, and F(t) is an uncertain matrix satisfying FT(t)F(t)≤I, MT=M, S, and N of appropriate dimensions; the inequality(22)M+SΔtN+NTΔTtST<0holds if and only if, for some δ>0,(23)δMSδNTST-IJTδNJ-I<0.By using Lemma1, the following stability criterion can be derived.Theorem 2.
The equilibrium pointy∗ of system (2) is exponentially stable if there exists a positive constant δ such that the following linear matrix inequality holds:(24)-2δII-ρMδII-ρM-I0δI0-I<0,and the exponential convergence rate is mini{μi}, where μi is the eigenvalue of matrix B=(A+AH)/2, A=-I+(I-ρM)Δ(t).Proof.
From Section2.1, it follows that the equilibrium point’s exponential stability of system (2) is equivalent to the exponential stability of the trivial solution of system (11). Notice that system (11) is a linear system structure; thus the exponential stability of the trivial solution of system (11) is equivalent to Re(λ)<0, where λ is an arbitrary eigenvalue of matrix -I+(I-ρM)Δ(t).
SetB=(b)n×n=(A+AH)/2, A=-I+(I-ρM)Δ(t). Denote {μ1,μ2,…,μn} as the eigenvalues of matrix B satisfying μ1≥μ2≥⋯≥μn and {λ1,λ2,…,λn} as the eigenvalues of matrix A satisfying λ1≥λ2≥⋯≥λn. Let X≠0 be an identity eigenvector belonging to eigenvalue λi; namely,(25)AX=λiX,X2=XHX=1.It follows that(26)X,AX=X,λiX=λiX,X=λi,XHAX=λi,XHAHX=λ¯i.Thus(27)Reλi=X,A+AHX2=X,BX.Since B is a normal matrix, there exists a unitary matrix U such that(28)UHBU=diag⁡μ1,μ2,…,μn≡D;this means that(29)Re⁡λi=X,BX=X,UDUHX=XHUDUHX.Set Y=UHX; one can obtain that(30)YHY=∑i=1nyi2=XHUUHX=XHX=1;thus(31)Re⁡λi=YHDY=∑i=1nμiyi2.Notice that μ1≥μ2≥⋯≥μn; it yields(32)μn=∑i=1nμnyi2≤∑i=1nμiyi2≤∑i=1nμ1yi2=μ1,which means that μn≤Re(λi)≤μ1. Obviously, if μi<0(i=1,2,…,n) it means that Re(λi)<0. Because μi is eigenvalue of matrix B and matrix B is a symmetric real matrix, this implies that if matrix B<0, then Re(λi)<0. On the other hand, B<0 is equivalent to(33)-2I+I-ρMΔtI+IΔtI-ρMT<0.Since ΔT(t)Δ(t)≤I, by Lemma 1, if there is a positive constant δ such that(34)-2δII-ρMδII-ρM-I0δI0-I<0,then Re(λi)<0, i=1,2,…,n, which means that the trivial solution is exponentially stable; this completes the proof.Remark 3.
Obviously, the stability condition established in Theorem2 is a LMI form. Using Matlab LMI tool box, it can be easily solved. However, when a large-scale quadratic optimization problem has to be performed, the computation complexity becomes challenging. In order to overcome this flaw, by using eigenvalue perturbation theory, another more simple and practical result can be derived as follows. Before continuing, the following lemma is needed.Lemma 4 (see [25]).
LetA=PΛP-1∈Cn×n, Λ=diag⁡λ1,λ2,…,λn; δ∈Cn×n; μ1,μ2,…,μn denotes the eigenvalues of matrix A+δ; then for arbitrary μj, there exists an eigenvalue λi such that(35)λi-μj≤P-1δP∞,where P-1δP∞≡maxi∑j=1npij and P-1δP≡(pij)n×n.Theorem 5.
The equilibrium pointy∗ of system (2) is exponentially stable if I-ρM∞<1, and the exponential convergence rate is mini{μi}.Proof.
From the proof of Theorem2, it follows that the equilibrium point’s exponential stability of system (2) is equivalent to Re(λ)<0, where λ is an arbitrary eigenvalue of matrix -I+(I-ρM)Δ(t). Let μi be an arbitrary eigenvalue of matrix -I+(I-ρM)Δ(t); by Lemma 4, it yields that(36)-1-μi≤I-ρMΔt∞≤I-ρM∞·Δt∞.By the definition of Δ(t), we have(37)Δt∞=diag⁡λ1y1t,λ2y2t,…,λnynt∞=maxi⁡∑j=1nΔtij=maxi⁡λiyit≤1.From (36) and (37), it yields that(38)-1-I-ρM∞≤μi≤I-ρM∞-1,i=1,2,…,n.
Obviously,I-ρM∞<1 means that arbitrary eigenvalue λ of matrix -I+(I-ρM)Δ(t) is negative; this means that the equilibrium point of system (2) is exponentially stable, which completes the proof.Remark 6.
Usually, matrix infinite norm is larger than matrix 2-norm; thus the result established in Theorem5 can be rewritten by matrix 2-norm form further. And notice the special property of identity matrix I; by using improved eigenvalue perturbation theory, another exponentially stable criterion can be derived as follows. Before continuing, the following lemma is needed.Lemma 7 (see [26]).
LetA∈Cn×n be normal matrix, λ1,λ2,…,λn are eigenvalues of matrix A,δ∈Cn×n is an arbitrary matrix, and μ denotes the eigenvalue of matrix A+δ; then there exists an eigenvalue λi of matrix A such that(39)μ-λi≤δ2,where ·2 denotes matrix 2-norm.Theorem 8.
The equilibrium pointy∗ of system (2) is exponentially stable if I-ρM2<1, and the exponential convergence rate is mini{μi}.Proof.
From the proof of Theorem2, it follows that the equilibrium point’s exponential stability of system (2) is equivalent to Re(λ)<0, where λ is an arbitrary eigenvalue of matrix -I+(I-ρM)Δ(t). Let μi be an arbitrary eigenvalue of matrix -I+(I-ρM)Δ(t); since -I is a normal matrix and all of the eigenvalues of matrix -I are −1, by Lemma 7, it yields that(40)μi+1≤I-ρMΔt2≤I-ρM2·Δt2.By the definition of Δ(t), we have(41)Δt2=diag⁡λ1y1t,λ2y2t,…,λnynt2=max⁡rΔtTΔt=maxi⁡λiyit≤1.From (40) and (41), it yields that(42)-1-I-ρM2≤μi≤I-ρM2-1,i=1,2,…,n.
Obviously,I-ρM2<1 means that arbitrary eigenvalue λ of matrix -I+(I-ρM)Δ(t) is negative; this means that the equilibrium point of system (2) is exponentially stabile, which completes the proof.Remark 9.
As is well known, Lyapunov-Razumikhin method is a powerful stability analysis tool for linear time-delayed system. When time delays are considered, by using Lyapunov-Razumikhin method, a similar asymptotically stable result with Theorem8 for systems (12) and (19) can be derived as follows. Before continuing, the following Razumikhin condition is needed.Lemma 10 (see [27] (Razumikhin condition)).
The equilibrium pointy∗ of system (12) is asymptotically stable if there exists a positive definite Lyapunov function V(t,y(t)) satisfying(43)Vt+s,yt+s≤Vt,yt,s∈-τ,0⟹ddtVt,yt<0.By using Razumikhin condition, the following asymptotically stable criterion for systems (12) and (19) can be obtained as follows.Theorem 11.
The equilibrium pointy∗ of system (12) is asymptotically stable if I-ρM2<1.Proof.
Construct a positive definite Lyapunov functionV(t,y(t))=y(t)Ty(t); it follows that(44)Vt+s,yt+s≤Vt,yt⟺yt+s2≤yt2.When V(t+s,y(t+s))≤V(t,y(t)),s∈[-τ,0], it yields y(t+s)2≤y(t)2, and the derivatives of Lyapunov function V(t,y(t)) along the trajectory of system (19) satisfy(45)dVt,ytdt=2yTt-yt+I-ρMΔ^tyt-τ=2yTtI-ρMΔ^tyt-τ-2yt22≤2yTtI-ρMΔ^tyt-τ2-2yt22≤2yTt2·I-ρM2Δ^t2·yt-τ2-2yt22.By the definition of Δ^(t), we have(46)Δ^t2=diag⁡λ1y1t-τ,…,λnynt-τ2=max⁡rΔ^TtΔ^t=maxi⁡λiyit-τ≤1.From (45), (46), and Lemma 10, we have(47)dVt,ytdt≤2yTt2·I-ρM2·yt-τ2-2yt22≤2yTt2·I-ρM2·yt2-2yt22=-21-I-ρM2yt22.
Obviously, from Razumikhin condition, ifI-ρM2<1, then the equilibrium point y∗ of system (12) is asymptotically stable, which completes the proof.Remark 12.
Similar to the proof of Theorem11, by using Razumikhin condition, a delay-dependent stable result for systems (20) and (21) can be derived as follows.Theorem 13.
The equilibrium pointy∗ of system (21) is asymptotically stable if I-ρM2+τ[1+2I-ρM2+I-ρM22]<1.Proof.
Construct a positive definite Lyapunov functionV(t,y(t))=y(t)Ty(t); it follows that(48)Vt+s,yt+s≤Vt,yt⟺yt+s2≤yt2.When V(t+s,y(t+s))≤V(t,y(t)),s∈[-τ,0], it yields y(t+s)2≤y(t)2, and the derivatives of Lyapunov function V(t,y(t)) along the trajectory of system (21) satisfy(49)dVt,ytdt=2yTt-I+I-ρMΔ^tyt-τ=2yTt-I+I-ρMΔ^t×yt--I+I-ρMΔ^t∫t-τtys-τds≤-2yt22+2yTtI-ρMΔ^tyt2+2yTt-I+I-ρMΔ^t2∫t-τtys-τds2≤-2yt22+2I-ρM2Δ^t2yt22+2τ-I+I-ρMΔ^t22yt22.Since Δ^(t)2<1, from (49), we have(50)dVt,ytdt≤-2yt22+2I-ρM2yt22+2τI-2I-ρMΔ^t+I-ρMΔ^tI-ρMΔ^t2yt22≤-2yt22+2I-ρM2yt22+2τ1+2I-ρM2+I-ρM22yt22.
Obviously, from Razumikhin condition, ifI-ρM2+τ[1+2I-ρM2+I-ρM22]<1, then the equilibrium point y∗ of system (21) is asymptotically stable, which completes the proof.Remark 14.
Ifτ=0, system (21) degenerates into system (11); Theorem 13 becomes the same as in Theorem 8; thus the criterion derived in Theorem 8 can be regarded as a special case of Theorem 13.Theorem 15.
If the symbol< in Theorems 2, 5, 8, and 11 becomes ≤, then, for any initial value x0∈Ω, the solutions of systems (2) and (12) converge to a related equilibrium point y∗. In particular, these systems are asymptotically stable when their equilibrium set contains exactly one point.Proof.
SinceΩ is a bounded and closed convex set, as is well known, the state vectors of systems (2) and (12) are bounded, and Ω is an invariance set. Utilizing LaSalle’s invariance principle (see [28]), similar to the proofs in [20–23], one can easily obtain the results described in Theorem 15; this complete the proof.Remark 16.
Theorems2, 5, 8, and 11 show that if the inequality signs are strict, then, for any initial x0 whether it is in feasible set Ω or not, the state vectors are exponentially or asymptotically convergent. When the inequality signs are not strict, Theorem 15 shows that the sufficient condition ensuring system being asymptotically stable is x0∈Ω. On the other hand, notice that if ρ is sufficiently small, nonstrict inequality sign in Theorems 2, 5, 8, and 11 can be guaranteed; this means that, for arbitrary initial value x0∈Ω with appropriate dimension, systems (2) and (12) are asymptotically stable.


## 4. Numerical Simulation

### 4.1. Numerical Example

Example 1.
Consider the quadratic optimization problem (1) with(51)M=0.1800.6480.2880.6482.8800.7200.2880.7200.720,b=0.4,0.2,0.3T,and c=-d=[20,-20,-20]T. For quadratic optimization (1), the author in [29] constructed a discrete-time neural network to solve this problem and gave out a globally convergent criterion. In order to further illustrate the globally exponential convergence of the neural networks constructed in [29], Tan et al. gave out some new improvement stable criteria in [16]. In [30], the authors constructed a continuous neural network to solve problem (1) and derived a globally exponential convergent criterion. In order to reduce the conservation of the criterion established in [30], literature [31] further gave out an improved stable result as I-ρMm2<1. If we construct model (11) to solve problem (1), by direct computation, it follows that I-ρM2=0.9984<1; from Theorem 8, it yields that the state vector of system (11) is exponentially convergent to the optimization value of problem (1). Simulation result is illustrated in Figure 1 with initial value [2,-1,1.5]T, from which one can see that the state vector of system (11) is convergent to the equilibrium point exponentially. Obviously, I-ρM2=0.9984<I-ρM∞=1.0696<I-ρMm2=1.5338; this means that the criterion established in Theorem 8 is less conservative than some earlier results.Figure 1
State vector of system (2) with parameters given in Example 1.If we adopt model (19) to solve problem (1), by direct computing, it follows that I-ρM2=0.9984<1; from Theorem 11, it means that the state vector of system (19) is exponentially stable. Simulation result is illustrated in Figure 2 with τ=4. If we use model (21) to solve problem (1), from the result established in Theorem 13, one can see that the state vector stable condition is stringently required for time delay. For example, in system (21), let τ=0.01; simulation result in Figure 3 shows that the state vector of system (21) is divergent. This means that even small time delay can cause the state vector of system (21) to be instable. Hence, when time delays must be considered, model (19) has more superiority than model (21).Figure 2
State vector of system (12) with τ=4.Figure 3
State vector of system (21) with τ=0.01.


### 4.2. Application to Compressed Sensing

The sparsest solution of an undermined linear system of equations can be found by solving the so-calledl0-norm minimization problem; that is,(52)minX∈Rn⁡X0,s.t.AX=b,where A∈Rk×n, k<n, b∈Rk, and X0 denotes the number of nonzero components in X. It is well known that problem (52) is NP-hard. For this problem, an approximation model named “Basic Pursuit Problem” was proposed as(53)minX∈Rn⁡X1,s.t.AX=b,since the convex envelope of X0 is X1, where X1=∑ixi is the “l1”-norm of X. Further, if b is contaminated by small dense noise, then problem (53) can be modified as the following form:(54)minX∈Rn⁡X1,s.t.AX-b2<ε,where ε is nonnegative parameter and ·2 is the Euclidean norm of X. Recently, convex analysis shows that (54) is equivalent to the following Lagrangian version:(55)minX∈Rn⁡12AX-b2+τX1,where τ>0 is a penalty parameter. Obviously, optimization problem (55) is convex but not differentiable. In order to solve problem (55), Liu and Hu [32] introduced a transform ui=max⁡{xi,0}, vi=max⁡{-xi,0}, and X=U-V. It follows that X1=1nTU+1nTV, where 1n=(1,1,…,1)T is the vector consisting of n ones. Therefore, problem (55) can be rewritten as follows:(56)minU,V∈Rn12b-AU-V22+τ1nTU+τ1nTV,s.t.U>0V>0.Namely,(57)minZ≥0⁡FZ=12ZTBZ+cTZ,where(58)Z=UV,c=τ12nT+-b¯b¯,b¯=ATb,B=ATA-ATA-ATAATA.Obviously, problem (57) is a typical quadratic optimization problem. From (1) and (2), the optimal solutions of (57) are equivalent to the equilibrium points of the following projective neural network:(59)dZdt=-Zt+PΩZt-ρBZt-ρc,Zt0=Z0∈Ω,where Ω is the positive orthant, ρ>0, and PΩ(zi)=max⁡{zi,0}. By direct computing, it follows that(60)0≤PΩzi-PΩzjzi-zj≤1,zi≠zj.From the analysis in Section2, system (59) can be rewritten in the following equivalent linear robust neural network form:(61)dytdt=-I+I-ρBΔtyt.Let(62)A=358415296574347216896574,ρ=0.0008;direct computing shows that I-ρB2=1; from Theorem 15, if we adopt initial x0 in feasible set Ω, then the state vector of system (61) converges to optimizing solution of problem (55).Remark 17.
In addition to the application to compressed sensing, projection neural networks can also be applied to the motion generation and control of redundant robot manipulators. In [33], based on control perspective and projection neural networks, the authors researched distributed task allocation problem of multiple robots. Utilizing projection neural networks, literature [34] researches manipulability optimization problem of redundant manipulators. These new applications in robotics extend new application fields for projection neural network. How to use the technique derived in this paper to robotics field and solve related optimization problem will be a meaningful work, and this is also our future work direction.


## 4.1. Numerical Example

Example 1.
Consider the quadratic optimization problem (1) with(51)M=0.1800.6480.2880.6482.8800.7200.2880.7200.720,b=0.4,0.2,0.3T,and c=-d=[20,-20,-20]T. For quadratic optimization (1), the author in [29] constructed a discrete-time neural network to solve this problem and gave out a globally convergent criterion. In order to further illustrate the globally exponential convergence of the neural networks constructed in [29], Tan et al. gave out some new improvement stable criteria in [16]. In [30], the authors constructed a continuous neural network to solve problem (1) and derived a globally exponential convergent criterion. In order to reduce the conservation of the criterion established in [30], literature [31] further gave out an improved stable result as I-ρMm2<1. If we construct model (11) to solve problem (1), by direct computation, it follows that I-ρM2=0.9984<1; from Theorem 8, it yields that the state vector of system (11) is exponentially convergent to the optimization value of problem (1). Simulation result is illustrated in Figure 1 with initial value [2,-1,1.5]T, from which one can see that the state vector of system (11) is convergent to the equilibrium point exponentially. Obviously, I-ρM2=0.9984<I-ρM∞=1.0696<I-ρMm2=1.5338; this means that the criterion established in Theorem 8 is less conservative than some earlier results.Figure 1
State vector of system (2) with parameters given in Example 1.If we adopt model (19) to solve problem (1), by direct computing, it follows that I-ρM2=0.9984<1; from Theorem 11, it means that the state vector of system (19) is exponentially stable. Simulation result is illustrated in Figure 2 with τ=4. If we use model (21) to solve problem (1), from the result established in Theorem 13, one can see that the state vector stable condition is stringently required for time delay. For example, in system (21), let τ=0.01; simulation result in Figure 3 shows that the state vector of system (21) is divergent. This means that even small time delay can cause the state vector of system (21) to be instable. Hence, when time delays must be considered, model (19) has more superiority than model (21).Figure 2
State vector of system (12) with τ=4.Figure 3
State vector of system (21) with τ=0.01.


## 4.2. Application to Compressed Sensing

The sparsest solution of an undermined linear system of equations can be found by solving the so-calledl0-norm minimization problem; that is,(52)minX∈Rn⁡X0,s.t.AX=b,where A∈Rk×n, k<n, b∈Rk, and X0 denotes the number of nonzero components in X. It is well known that problem (52) is NP-hard. For this problem, an approximation model named “Basic Pursuit Problem” was proposed as(53)minX∈Rn⁡X1,s.t.AX=b,since the convex envelope of X0 is X1, where X1=∑ixi is the “l1”-norm of X. Further, if b is contaminated by small dense noise, then problem (53) can be modified as the following form:(54)minX∈Rn⁡X1,s.t.AX-b2<ε,where ε is nonnegative parameter and ·2 is the Euclidean norm of X. Recently, convex analysis shows that (54) is equivalent to the following Lagrangian version:(55)minX∈Rn⁡12AX-b2+τX1,where τ>0 is a penalty parameter. Obviously, optimization problem (55) is convex but not differentiable. In order to solve problem (55), Liu and Hu [32] introduced a transform ui=max⁡{xi,0}, vi=max⁡{-xi,0}, and X=U-V. It follows that X1=1nTU+1nTV, where 1n=(1,1,…,1)T is the vector consisting of n ones. Therefore, problem (55) can be rewritten as follows:(56)minU,V∈Rn12b-AU-V22+τ1nTU+τ1nTV,s.t.U>0V>0.Namely,(57)minZ≥0⁡FZ=12ZTBZ+cTZ,where(58)Z=UV,c=τ12nT+-b¯b¯,b¯=ATb,B=ATA-ATA-ATAATA.Obviously, problem (57) is a typical quadratic optimization problem. From (1) and (2), the optimal solutions of (57) are equivalent to the equilibrium points of the following projective neural network:(59)dZdt=-Zt+PΩZt-ρBZt-ρc,Zt0=Z0∈Ω,where Ω is the positive orthant, ρ>0, and PΩ(zi)=max⁡{zi,0}. By direct computing, it follows that(60)0≤PΩzi-PΩzjzi-zj≤1,zi≠zj.From the analysis in Section2, system (59) can be rewritten in the following equivalent linear robust neural network form:(61)dytdt=-I+I-ρBΔtyt.Let(62)A=358415296574347216896574,ρ=0.0008;direct computing shows that I-ρB2=1; from Theorem 15, if we adopt initial x0 in feasible set Ω, then the state vector of system (61) converges to optimizing solution of problem (55).Remark 17.
In addition to the application to compressed sensing, projection neural networks can also be applied to the motion generation and control of redundant robot manipulators. In [33], based on control perspective and projection neural networks, the authors researched distributed task allocation problem of multiple robots. Utilizing projection neural networks, literature [34] researches manipulability optimization problem of redundant manipulators. These new applications in robotics extend new application fields for projection neural network. How to use the technique derived in this paper to robotics field and solve related optimization problem will be a meaningful work, and this is also our future work direction.


## 5. Conclusions

This paper proposed three linear robust systems to solve a class of quadratic optimization problems. Utilizing LMI technique, eigenvalue perturbation theory, Lyapunov-Razumikhin method, and LaSalle’s invariance principle, some stable criteria for related models are also established. Compared with previous criteria derived in the literature cited herein, the stable criteria established in this paper are less conservative and more practicable. Meanwhile, simulation results show that time-delay linear robust system type II is more sensitive to time delay than type I. This means that, in practical engineering problems, when time delay is needed to consider, model (19) has more superiority than model (21). Simulation example and application example in compressed sensing show that the results derived in this paper are valid.



---

*Source: 5073640-2017-08-28.xml*