# A Nonpenalty Neurodynamic Model for Complex-Variable Optimization

Authors: Bao Liu; Xuehui Mei; Haijun Jiang; Lijun Wu
Journal: Discrete Dynamics in Nature and Society
 (2021)

---

## Abstract

In this paper, a complex-variable neural network model is obtained for solving complex-variable optimization problems described by differential inclusion. Based on the nonpenalty idea, the constructed algorithm does not need to design penalty parameters, that is, it is easier to be designed in practical applications. And some theorems for the convergence of the proposed model are given under suitable conditions. Finally, two numerical examples are shown to illustrate the correctness and effectiveness of the proposed optimization model.


---

## Body

## 1. Introduction

In practical applications, many engineering problems such as machine learning, signal processing, and resource allocation [1–4] can be transformed into the optimization problems, and a lot of algorithms have been developed in recent years [5–7]. Since many engineering systems can be described by complex signals and complex-variable optimization problems widely exist in fields of medical imaging, adaptive filters, and remote sensing [8–10], the study of complex-variable optimization has also attracted more and more scholars’ attention.In this paper, we consider the following complex-variable optimization:(1)minimizegω,subject toHω≤0,Bω=b,where ω∈ℂn is the decision variable, g:ℂn⟶ℝ is the convex objective function, Hω=h1ω,h2ω,…,hlωT:ℂn⟶ℝl, hi⋅ is convex, hω=maxhiω:i=1,…,l, B∈ℂm×n is a full row-rank matrix, and b∈ℂm. Ω1=ω∈ℂn|hω≤0, Ω2=ω∈ℂn|Bω=b, and Ω=Ω1∩Ω2 is the feasible region of optimization (1). Ω∗ means the optimal solution set for (1). Ω0,Ω¯, ∂Ω, and Ω′ represent the interior, the closure, the boundary, and the complement of the set Ω.The method of neurodynamic optimization is to establish the corresponding relationship between the solution of the optimization problem and the state of the neurodynamic system and to obtain the optimal solution by tracking the state output of the neurodynamic system. Because of its parallel operation and fast convergence speed, the neural dynamic optimization method is superior to the traditional optimization algorithm in solving optimization problems. Therefore, various neural networks have been proposed solving optimization problems [11–18]. In [11], the authors proposed a Hopfield-type neural network solving linear programming. In [12], Kennedy and Chua designed a model of neural networks based on the gradient to solve the constrained nonlinear programming problem with equality constraints. In [13], Liu et al. proposed a recursive neural network model with convergence in finite time to solve linear programming problems. In [14], Forti et al. promoted the optimal problem into the neural network model described by differential inclusion for solving the nonsmooth optimization. Furthermore, in [15, 16], Bian et al. extended the problem to the one with equality and inequality constraints. In [17, 18], based on the nonpenalty idea, Hosseini proposed the neural network models upon differential inclusions solving the nonsmooth optimization problems. However, all these works focused on real-variable optimization problems.Because the properties of the complex-variable function and real-variable function are different, complex-variable optimization problems cannot be solved by traditional optimization methods. Most scholars usually decompose complex-variable optimizations into real-variable problems. The main drawback of this approach is that it breaks the two-dimensional structure of the data. In order to overcome the above difficulties, more and more scholars begin to pay attention to the neural dynamic optimization methods for complex-variable optimization.With the improvement of complex-variable dynamic system theories including asymptotic stability [19, 20], exponential stability [21, 22], robust stability [23, 24], multistability [25, 26], finite-time stability [27], periodicity [28], synchronization [29], and dissipation [30], complex-variable optimization problems were also gradually studied. Zhang et al. [31] proposed a complex-variable model to solve the convex problem with equality constraints. In [32], the authors designed a three-layer projection neural network model to solve the complex-variable convex problems. In [33], a complex-variable projection model was proposed to solve an optimization with equality and inequality constraints. In fact, the semidefinite condition of global convergence of the neural networks in [31, 32] is not valid for the nonsmooth problems, and it is difficult to choose the penalty parameters of the neural networks designed based on the algorithm in [33].In order to overcome the above difficulties, this paper designs a complex-variable model using a neural network described by differential inclusion for solving the optimization problem. Compared with the models in the existing literature [31–33], the model proposed in this paper has the following characteristics. Firstly, the nonsmooth optimization problem can be solved because the model adopts differential inclusion description. Moreover, the model does not need to select penalty parameters in practical applications because the proposed model is based on the idea of nonpenalty design and does not contain any penalty parameters. Finally, the model structure we designed is simpler and more convenient to be applied in practical problems.The sections of this article are as follows. In Section2, we present some preliminaries and lemmas. In Section 3, a model is proposed for solving complex-variable convex problems, and the state of the model is proved to converge under some conditions. In Section 4, some typical examples are given to illustrate the effectiveness of the proposed model. Finally, a brief conclusion is given.


## 2. Preliminaries

In this paper, notationI means the identity matrix. i=−1 is the imaginary unit. The superscripts ⋅T,⋅H, and ⋅¯ represent the transpose, conjugate transpose, and complex conjugate, respectively. ⋅ means the absolute value of a real number or the model of a complex number. ⋅1 and ⋅ denote 1-norm and 2-norm. The real and complex field are represented by ℝ and ℂ, respectively. The real and imaginary parts of a complex matrix or vector are represented by Re⋅ and Im⋅.Definition 1 (see [34]).
LetX and Z be normed spaces. If, for any ε>0, there exists θ>0 such that ℋx+θB0,1⊆ℋx+εB0,1, the set-valued map ℋ:Q⊆X⟶Z is upper semicontinuous at point x.Definition 2 (see [35]).
(Letg be Lipschitz near a point z in norm space X. The directional derivative of g in direction h in the Clarke sense is the following limit:(2)g0z,h=limsupv⟶z,t↓0gv+th−gvt.
A set is the subdifferential ofg at a point z in the Clarke sense if(3)∂gz=η∈X:g0z,h≥η,l,eachh∈X.
Denote(4)g′z,l=lims↓0gz+sl−gzs,and g is regular in the Clarke sense if g0z,h=g′z,h.Lemma 1 (see [36]).
Suppose thatvt is absolutely continuous on each compact interval and gv is regular in the Clarke sense; then,(5)dgvtdt=ξ,v˙t,anyξ∈∂gvt.Definition 3 (see [35]).
LetS⊂ℂn be a convex set. For any two points z1,z2∈S and any t∈0,1, gz is a strictly convex function on S if gtz1+1−tz2<tgz1+1−tgz2. For the rest of the calculation, the concept of relaxed CR calculus [37, 38] will be introduced. First, the auxiliary function bμ,ν is denoted by(6)bμ,ν≔gμ+iν=gω,in which ω=μ+iν. Note that μ=ω+ω¯/2 and ν=−iω+ω¯/2, and we have(7)gω=gω,ω¯=bμ,ν.
Suppose∂b/∂μ and ∂b/∂ν exist. The following formula is the R-derivative of gω,ω¯:(8)∂g∂ω¯=12∂b∂μ+i∂b∂ν.Definition 4 (see [37, 38]).
If∂b/∂μ and ∂b/∂ν exist, the complex gradient of gω is expressed as(9)∇g=2∂g∂ω¯=∂b∂μ+i∂b∂ν.
If the complex gradient of convex functiongω does not exist, the generalized complex gradient or subdifferential of gω is defined as follows.Definition 5.
The generalized subdifferential ofgω at point ω is expressed as(10)∂gω=ς=ςμ+iςν:ςμ,ςν∈∂bμ,ν.Lemma 2.
Suppose thatωt is absolutely continuous on each compact interval and gω is a convex function; then,(11)dgωtdt=ReςHω.t,anyς∈∂gωt,for a.e. t∈0,+∞.Proof.
From (10), one obtains(12)dgωtdt=dbμt,νtdt.
By Lemma1, one has(13)dbμt,νtdt=ςμT,ςνT=ςμTμ˙t+ςνTν˙t,∀ςμ,ςν∈∂bμt,νt.
Denoteς=ςμ+iςν. According to Definition 5, ς∈∂gωt. Therefore,(14)dgωtdt=ςμTμ˙t+ςνTν˙t=ReςHω.t.Lemma 3 (see [35]).
Ifgω is convex, for any ω1,ω2∈ℂn and ς∈∂gω1, we have(15)gω2−gω1≥ReςHω2−ω1.Lemma 4 (see [35]).
Letgω=maxgiω,i=1,…,m and giω,i=1,…,m be a finite collection of convex functions; we have(16)∂gω=co¯∂giω,i∈Iω,where Iω=i:giω=gω,i=1,…,m.Definition 7 (see [34]).
An absolutely continuous functionvt is a solution of ω˙t∈ℋt,ωt on interval t0,t1 if v˙t∈ℋt,vt, for almost all t∈t0,t1, where ℋ:0,∞×ℂn⇉ℂn is a set-valued map.
Before moving on, we define the following functions:(17)χy≔0,ify<0,0,1,ify=0,1,ify>0,ΞBz−b≔ξReBz−b+iξReBz−b,where(18)ξy≔ξ1y1,ξ2y2,…,ξmymT,ξiyi=1,ifyi>0,−1,1,ifyi=0,−1,ifyi<0.


## 3. Neural Network Model

In this section, the following model is proposed to solve complex-variable problem (1):(19)dωtdt∈I−Pχhωt∂gωt−∂hωt−∂gωt−BHΞBωt−b,in which P=BHBBH−1B, and I is the identity matrix.Remark 1.
In [32], with common projection methods, the authors proposed a n+m+p-neurons and three-layer model. In [33], the authors proposed a single-layer model which needs to select the penalty parameters. In our paper, based on the nonpenalty idea, we propose a single-layer and n-neurons model which avoids selection of penalty parameters. Thus, the proposed model is of lower dimensions and easier to apply in practical problems.Lemma 5 (see [17]).
χhω∂gωt−∂hωt is upper semicontinuous. From Lemma 5, the following theorem can be obtained by the result of Theorem 3 in [17].Theorem 1.
For any initial valueωt0∈ℂn, the neural network (19) has a local solution. A vector function ω⋅ is a state of model (19) if ω⋅ is its solution. There exist measurable functions τt∈∂gωt, θhωt∈χhωt, ρt∈∂hωt, and ζt∈ΞBωt−b satisfying ω.t=I−Pθgωtτt−ρt−τt−BHζt.Definition 8.
ω∗ is the equilibrium point for model (19) if there exist τ∗∈∂gω∗, θhω∗∈χhω∗, ρ∗∈∂hω∗, and ζ∗∈ΞBω∗−b satisfying I−Pθhω∗τ∗−ρ∗−τ∗−BHζ∗=0.Theorem 2.
ω∗ is the optimal solution of the complex-variable optimization (1) if and only if it is the equilibrium point of model (19).Proof.
Firstly, let us prove the necessity. Letω∗ be the optimal solution of the complex-variable optimization (19); based on the Lagrange multiplier theorem in [32], we have(20)0∈∂gω∗+λ1∂hω∗+BHλ2,hω∗≤0,λ1≥0,λ1hω∗=0,Bω=b.
We consider three cases, respectively:(1)
Ifhω∗=0, one has χhω∗=0,1 and(21)0∈I−P−∂gω∗−λ1∂hω∗.Multiplying both sides of (21) by 1/1+λ1, we obtain(22)0∈I−P−11+λ1∂gω∗−λ11+λ1∂hω∗.Takingφ=λ1/1+λ1 and 1−φ=1/1+λ1, 0≤φ≤1, and(23)0∈I−P−1−φ∂gω∗−φ∂hω∗∈I−Pχhω∗∂gω∗−∂hω∗−∂gω∗.That is, there existτ∗∈∂gω∗, θhω∗∈χhω∗, and ρ∗∈∂hω∗ such that(24)0=I−Pθhω∗τ∗−ρ∗−τ∗.In combination with0∈ΞBω∗−b, ω∗ is the equilibrium point of model (19).(2)
Ifhω∗<0, we have χhω∗=0 and λ1=0. Then,(25)0∈∂gω∗+BHλ2.
Multiplying both sides of (25) by I−P, one has 0∈I−P∂gω∗.
Fromθhω∗=0 and 0∈ΞBω∗−b, we obtain ω∗ is the equilibrium point for (19).
Secondly, we prove the sufficiency. Letω∗ be the equilibrium point of model (19). There exist τ∗∈∂gω∗, θhω∗∈χhω∗, ρ∗∈∂hω∗, and ζ∗∈ΞBω∗−b satisfying(26)I−Pθhω∗τ∗−ρ∗−τ∗−BHζ∗=0.
Becausegω is convex, for τ∗∈∂gω∗ and ω∈Ω1∩Ω2, one gets(27)gω−gω∗≥Reτ∗Hω−ω∗.
According to (26), we obtain ω−ω∗HI−Pθhω∗τ∗−ρ∗−τ∗−ω−ω∗BHζ∗=0 for any ω∈Ω1∩Ω2. It is clear that ω−ω∗HP=ω−ω∗HBH=0. Hence, one obtains ω−ω∗Hθhω∗τ∗−ρ∗−τ∗=0 which means Reω−ω∗H1−θhω∗τ∗=−Reω−ω∗Hθhω∗ρ∗. According to the convexity of gω, we have 0≥hω−hω∗≥Reρ∗Hω−ω∗. Thus, Reω−ω∗Hτ∗≥0. From (27), we get(28)gω≥gω∗,∀ω∈Ω1∩Ω2.
Therefore,ω∗ is the optimal solution for complex-variable problem (1).


## 4. Convergence Analysis

The convergence and stability of the neural network (19) will be analyzed and proved in this section. Firstly, it is proved that the state of the neural network (19) will reach Ω2=ω∈ℂn|Aω=b in finite time even if the initial point is chosen from outside of feasible region Ω. Secondly, we prove that the equilibrium point of neural network (19) converges to the optimal solution of complex-variable optimization problem (1).Theorem 3.
For the neural network (19), given an arbitrary initial point ω0∈ℂn, there exist T>0 and t≥T such that the state ωt∈Ω2.Proof.
From Theorem1, there exist three measurable functions τt∈∂gωt, ρt∈∂hωt, and ζt∈ΞBωt−b, satisfying(29)ω.t=I−Pθgωtτt−ρt−τt−BHζt,ω0=ω0.
Now, we will prove thatωt will reach Ω2=ω∈ℂn|Bω=b in finite time by contradiction. That is, there exists T>0 such that ωt∉Ω2 for t∈0,T. The upper bound of T, t∈0,T, will be evaluated for Aωt−b≠0.
DefineV1ω=Bω−b1; then,(30)dV1ωtdt=BHζt,ω.t=ζtHBI−Pθgωtτt−ρt−τt−ζtHBBHζt=−ζtHBBHζt≤−λminζt2,where λmin represents the minimum eigenvalue of BBH. BBH is a full row-rank matrix because B is a full row-rank matrix, that is, λmin>0. From ζt∈ΞBωt−b, we obtain ζt22≥1. Hence, one has(31)dV1ωtdt≤−λmin.
Integrating both sides of (31) from 0 to T, we obtain(32)0≤BωT−b1≤Bω0−b1−λminT.
Therefore,T≤Bω0−b1/λmin. Hence, an upper bound for Aωt−b≠0 is T0=Bω0−b1/λmin. So, we have that ωt will reach Ω2 in finite time.
Next, it will be proved that the stateωt will stay Ω2 once ωt reaches Ω2. Otherwise, there exists an interval t1,t2 such that ωt∉Ω2 for any t∈t1,t2. Note that (31) still holds for t∈t1,t2; thus,(33)0≤Bωt2−b1≤Bωt1−b1−λmint2−t1=−λmint2−t1<0,which leads to a contradiction. So, the assumption does not support.
For the convergence of system (19), we define a function Vω=1/2ω−ω∗2, and one has(34)dVωtdt=Reω−ω∗Hω.t,based on Lemma 2 if ωt is a solution of (19). Lemma 6 supposed that ω∗∈Ω∗ and ωt is a solution trajectory of the neural network (19); then, dVωt/dt≤0 for t>0.Proof.
By Theorem1, we assume that ωt∈Ω2 for any t>0. That is, for any t>0, Bωt−b=0. There exist measurable functions τt∈∂gωt, θhωt∈χhωt, ρt∈∂hωt, and ζt∈ΞBωt−b such that(35)ω.t=I−Pθhωtτt−ρt−τt−BHζt.
BecauseBI−P=0, one has(36)Bω.t=BI−Pθhωtτt−ρt−τt−BBHζt=−BBHζt=0.
Then, we haveζt=0 from BBH which is full rank. Formulation (35) can be written as(37)ω.t=I−Pθhωtτt−ρt−τt,Bωt−b=0.
Hence,(38)dVωtdt=Reω−ω∗Hω.t=Reω−ω∗HI−Pθhωtτt−ρt−τt=Reω−ω∗Hθhωtτt−ρt−τt,where ω−ω∗HP=0.
We discuss three cases of stateωt:Case I:ωt∈Ω10. This implies that θhωt=0, that is,(39)dVωtdt=−Reω−ω∗Hτt≤hω∗−hω≤0.Case II:ωt∈Ω1′. It follows that θhωt=1. We have(40)dVωtdt=−Reω−ω∗Hρt≤gω∗−gω≤0.Case III:ωt∈∂Ω1. We have(41)dVωtdt=−1−θhωtReω−ω∗Hτt−θhωtReω−ω∗Hρt≤1−θhωtgω∗−gω+θhωthω∗−hω≤0.
Above all, we draw the conclusion.Remark 2.
According to Lemma6 and Theorem 1, it is obvious that the solution of model (19) is bounded and global for any initial value ω0∈ℂn.Lemma 6.
Suppose thatΛ∈ℂn is a compact set, ω∗∈Ω∗,Λ∩Ω∗=∅, and hωt is strictly convex on Ω1. There exist ε>0 and T>0 satisfying(42)dVωtdt<−ε,for any t>T.Proof.
From Theorem3, there exists T>0 such that Bωt−b=0 when t>T. One has(43)ω.t=I−Pθhωtτt−ρt−τt,Bωt−b=0,where the measurable functions τt∈∂gωt, ρt∈∂hωt, and θhωt∈χhωt.
By (38), we obtain(44)dVωtdt=Reω−ω∗Hθhωtτt−ρt−τt=ℱωt,ω∗,where(45)ℱω,ω∗=Re−τHω−ω∗,ω∈Ω10,Re−1−θhωηH−θhωρHω−ω∗,ω∈∂Ω1,Re−ρHω−ω∗,ω∈Ω1′.
Firstly, ifω∈Λ∩Ω1, it is true that there exists ε1>0 such that ℱω,ω∗<−ε1. From the convexity of gω, we obtain(46)Re−ηHω−ω∗≤gω−gω∗<0,for anyω∈Λ∩Ω1.
By the compactness ofΛ∩Ω1 and the continuity of gω−gω∗, there exists ε1>0 satisfying(47)ℱω,ω∗=Re−ηHω−ω∗≤gω−gω∗<−ε1,for anyω∈Λ∩Ω1.
Secondly, ifω∈Λ∩Ω1′¯, there exists ε2>0 such that ℱω,ω∗<−ε2. Suppose the result is not true, for n>0, there exist ρn∈∂hωn and ωn∈Λ∩Ω1′¯ such that(48)−ρnHωn−ω∗≤1n.
Without loss of generality, letlimn⟶∞ωn=ω¯∈Λ∩Ω1′¯. By the upper semicontinuity of ∂hω, there exists ρ¯∈∂hω¯ such that limn⟶∞ρn=ρ¯∈Λ∩Ω1′¯. Let n⟶∞; from (48), we obtain(49)−ρ¯Hω¯−ω∗=0.
By (17), when ω¯∈Ω1′ and hω¯>hω∗, we have(50)ℱω¯,ω∗=Re−ρ¯Hω¯,ω∗≤hω∗−hω¯<0.
Otherwise,ω¯∈∂Ω1; since hωt is strictly convex on Ω1, we have(51)ℱω¯,ω∗=Re−ρ¯Hω¯,ω∗<hω∗−hω¯=0.
This leads to a contradiction.
Consequently, chooseε=minε1,ε2; when t>T, one obtains(52)dVωtdt=ℱωt,ω∗<−ε.Theorem 4.
Suppose thatΩ∗ is bounded and the conditions of Lemma 6 hold; then, the state ωt of model (19) converges to Ω∗.Proof.
By means of reduction to absurdity, suppose that the result is not true. We getlimt⟶∞Vωt,ω∗>0, for any ω∗∈Ω∗. From Lemma 6, limt⟶∞Vωt,ω∗ exists. Then, there exists r>0 such that(53)limt⟶∞Vωt,ω∗≥r,for any ω∗∈Ω∗. Otherwise, there exists ωn∗∈Ω∗ for n>0, and it will be(54)limt⟶∞Vωt,ωn∗<1n.
ByΩ∗ compactness, there exists a sequence ωnk∗⊂ωn∗ satisfying ωnk∗⟶ω¯∈Ω∗. Therefore,(55)ωt−ω¯≤ωt−ωnk∗+ωnk∗−ω¯.
Taking the limit of (55) as t⟶∞, we obtain(56)limn⟶∞ωt−ω¯≤ωnk∗−ω¯+1nk.
Taking the limit of (56) as t⟶∞, we get limn⟶∞ωt−ω¯=0, that is, ωt⟶ω¯∈Ω∗. This contradicts between the above conclusion and the hypothesis. Thus, we obtain the existence of r.
We can assume there exists constantα>0 such that ωt≤α. Define the set(57)D=∪ω∗∈Ω∗ω∈ℂn:ω−ω∗≤r,and the set Λ=D′∩ω∈ℂn:ω≤α.
Thus,Λ is bounded and Λ∩Ω∗=∅. According to Lemma 6, there exist T>0 and ε>0 such that(58)dVωtdt<−ε.
Integrating (58) from T to t,(59)Vωt,ω∗<Vωt,ω∗−εt.
This meanslimt⟶∞Vωt,ω∗=−∞. This contradicts the existence of limt⟶∞Vωt,ω∗.Remark 3.
From Theorem4, the solution of neural network (19) is actually convergent to its equilibrium point.


## 5. Simulations

In this section, we illustrate the effectiveness of the neural network (19) in solving the complex-valued nonlinear and nonsmooth convex optimization problem by two examples.Example 1.
Consider the following problem:(60)minimizegω=ReqHω+ω,subject toω≤3,ω=ω1,ω2∈ℂ2,ω1+ω2=1,q=3+2i,1+i.
Here, we design ten arbitrary initial states in our model (19). Figures 1 and 2 show that the state of the neural network (19) will converge to the optimal solution ω∗=1.5042+0.4916i,−0.5022−0.4936iT, and the minimum is gω∗=1.1909.Figure 1
The state variablesωt with ten arbitrary initial states for Example 1.
(a)(b)Figure 2
gωt with ten arbitrary initial states for Example 1.Example 2.
Consider the quadratic optimization with ten arbitrary initial states:(61)minimizegω=12ω2,subject toBω=b,b=1−i−1+i,B=6−4−6−464.
From Figure3, we can find the convergent behaviors of the state variables ωt with the arbitrary initial states for Example 2. The optimal solution of Example 2 is ω∗=0.0497−0.0503i,−0.0993+0.1001i,−0.0497+0.0503iT. Simulation results show the minimum is gω∗=0.0299.Figure 3
The state variablesωt with ten arbitrary initial states for Example 2.
(a)(b)


## 6. Conclusion

This paper aims to solve a complex-variable nonlinear nonsmooth convex optimization problem. We present a nonpenalty neurodynamic model by differential inclusion to obtain the optimal solution of the problem. Different from the existing algorithms, the proposed neurodynamic model need not choose penalty parameters, which makes it easier to be designed in practical applications. And some theorems are given in order to ensure convergence. Finally, numerical examples are shown to illustrate the effectiveness of the proposed neural networks. In further studies, we will seek a new neurodynamic model with simpler structure to solve complex-variable nonlinear nonsmooth convex optimization based on the nonpenalty approach.



---

*Source: 6632257-2021-02-16.xml*