# A New Conjugate Gradient Projection Method for Convex Constrained Nonlinear Equations

Authors: Pengjie Liu; Jinbao Jian; Xianzhen Jiang
Journal: Complexity
 (2020)

---

## Abstract

The conjugate gradient projection method is one of the most effective methods for solving large-scale monotone nonlinear equations with convex constraints. In this paper, a new conjugate parameter is designed to generate the search direction, and an adaptive line search strategy is improved to yield the step size, and then, a new conjugate gradient projection method is proposed for large-scale monotone nonlinear equations with convex constraints. Under mild conditions, the proposed method is proved to be globally convergent. A large number of numerical experiments for the presented method and its comparisons are executed, which indicates that the presented method is very promising. Finally, the proposed method is applied to deal with the recovery of sparse signals.


---

## Body

## 1. Introduction

Solving a system of nonlinear equations can be transformed as an optimization problem, which is widely applied in many fields of sciences and engineering, for instance, the economic equilibrium problem [1], the neural networks problem [2], the financial problem [3, 4], the chemical equilibrium system [5], and the compressed sensing problem [6, 7].In this paper, the following system of constrained monotone nonlinear equations is considered:(1)Fx=0,s.t.x∈C,where C is a nonempty closed convex set of ℝn and F:ℝn⟶ℝn is a monotone mapping, namely,(2)Fx−FyTx−y≥0,∀x,y∈ℝn.It follows from (2) that the solution set of system (1) is convex [8]. Throughout this paper, the symbol ⋅ stands for the Euclidean norm. Denote Fk≔Fxk for xk∈ℝn, PCx=argminy−xy∈C,∀x,y∈ℝn.Many algorithms have been proposed to deal with (1) during the past few decades (see, e.g., [9–16]), such as the projected Newton method [9], the projected quasi-Newton method [10–13], the Levenberg–Marquardt method [14], the trust region method [15], and the Lagrangian global method [16]. As we know, these methods converge rapidly if the sufficiently good initial points are chosen. However, they are not well-suited for solving large-scale constrained nonlinear equations due to the computation of the Jacobian matrix or its approximation at each iteration. Therefore, in the past few years, the projected derivative-free method (PDFM) has become more and more popular, i.e., the spectral gradient projection method [17–19], the multivariate spectral gradient-type projection method [20, 21], and the conjugate gradient projection method (CGPM) [22–31], and more other PDFM can be seen in references [32, 33].In this work, we concentrate on studying CGPM for large-scale nonlinear equations with convex constraints. We aim to establish a more efficient CGPM by improving the search direction and the line search rule and use the proposed CGPM to deal with the problem of sparse signals recovery.The contributions of this article are listed as follows:(i)
To guarantee the search direction satisfying the sufficient descent condition and trust region property independent of any line searches, a new conjugate parameter is proposed;(ii)
Based on classic line searches for nonlinear equations, an adaptive line search is improved to seek suitable step size easily;(iii)
Under general assumptions, the convergence analysis of the proposed algorithm is proved;(iv)
The reported numerical experiments show that our method is promising for solving large-scale nonlinear constrained equations and handling the problem of recovering sparse signals.The remainder of this paper is organized as follows. In Section2, a new search direction and an adaptive line search are proposed, and the corresponding algorithm is given. The global convergence is studied in Section 3. In Section 4, the numerical experiments for the proposed algorithm and its comparisons are performed, and the corresponding results are reported. Application of the proposed algorithm in compressed sensing is introduced in Section 5. In Section 6, a conclusion for this work is given.


## 2. A New CGPM Algorithm

The CGPM has been attracting extensive attention, since it not only inherits the advantages of the conjugate gradient method (CGM) with a simple algorithm structure, rapid convergence, and low storage requirements but also uses no any jacobian information of the equation in practice. To the best of our knowledge, the computation cost of CGPM mainly exists in the process of generating the search direction and computing the step size. Therefore, in the following part, we design our search direction by the CGM and give an improved inexact line search to yield the step size.


### 2.1. The New Search Direction Yielded by CGM

It is well-known that the search direction of the classical CGM is generated by(3)dk=−Fk,k=0,−Fk+βkdk−1,k≥1,which is decided by the conjugate parameter βk. Usually, a different βk leads to a different search direction.In the recent years, many scholars have made efforts to extend the CGM to solve the large-scale nonlinear monotone equations system. For example, based on the hybrid conjugate parameterβkJHJ in [34], Sun and Liu [28] proposed a modified conjugate parameter as follows:(4)βkN=Fk2−max0,Fk/Fk−1FkTFk−1maxμdk−1Fk,dk−1Tyk−1,μ>1,where yk−1=Fk−Fk−1 and extended it to solve problem (1).Recently, Tsegay et al. [35] gave a new CGM with sufficient descent property, that is,(5)βkSDCG=Fk2υFk2+dk−12,υ>12.Inspired by [28, 35], we propose a new hybrid formula as follows:(6)βkLJJ=Fk2−max0,Fk/Fk−1FkTFk−1νFk2+dk−12,where ν>1/2. Clearly, the parameter βkLJJ satisfies(7)0≤βkLJJ≤Fk22νFkdk−1=Fk2νdk−1.It is interesting that the search direction withβkLJJ have better theoretical properties, that is, it satisfies the sufficient descent condition and trust region property, automatically.


### 2.2. An Improved Adaptive Line Search Strategy

For an efficient CGPM, choosing an inexpensive line search is a key technique. To this end, many researchers try to exploit an inexact line search strategy to obtain step size with minimal cost. Zhang and Zhou [18] adopt an Armijo-type line search procedure, that is, the step size αk=maxβρii=0,1,2,⋯, such that(8)−Fxk+αkdkTdk≥σαkdk2,where β is an initial guess for the step size, ρ∈0,1, and σ is a positive constant. Li and Li [31] obtained the step size αk=maxβρii=0,1,2,⋯ by the following line search:(9)−Fxk+αkdkTdk≥σαkFxk+αkdkdk2,which was originally proposed by Solodov and Svaiter [36]. Guo and Wan [26] proposed an adaptive line search, i.e., the step size αk=maxβρii=0,1,2,⋯ satisfied the following inequality:(10)−Fxk+αkdkTdk≥σαkdk2min1,Fxk+αkdk.Obviously, whenxk is far from the solutions of problem (1) and the Fxk+αkdk is too large, it follows from (9) that the step size αk becomes small, which increases the computation cost. A similar case can appear for the line search (8) when xk is close to the solution set of problem (1) and dk is too large. However, it is worth noting that the line search (10) can overcome the previously mentioned weaknesses and take advantage of line searches (8) and (9).Inspired by [26], we introduce another new adaptive line search strategy with a disturbance factor, that is, taking αk=maxβρii=0,1,2,⋯, such that(11)−Fxk+αkdkTdk≥σαkγkdk2,where γk=Fxk+αkdk/maxFxk+αkdk,μ, β is an initial guess for the step size, σ,ρ∈0,1, and μ≥1. Here, μ is a disturbance factor, which can adjust the size of the right side of the line search (11) and further reduce the computation cost.Remark 1.
In fact, for a givenμ, Fxk+αkdk is too large if xk is far away from the solution set, namely, Fxk+αkdk/maxFxk+αkdk,μ=1, and then, the new line search (11) is similar to (8) in performance. Otherwise, when xk is close to the solution set, Fxk+αkdk approaches 0 and so γk approaches Fxk+αkdk, and then, the new line search (11) comes back to (9).


### 2.3. Algorithm

Based on the new search direction withβkLJJ (6) and adaptive line search (11), we describe our algorithm in detail for solving problem (1) as follows (Algorithm 1).Algorithm 1: LJJ CGPM.
Step 0: givenx0∈C, the constants ε>0,σ,ρ∈0,1,μ≥1,ν>1/2 and an initial stepsize β. Set k≔0Step 1: if a given terminate criterion is satisfied, e.g.,Fxk≤ε, terminate. Otherwise, go to Step 2Step 2: compute search directiondk by dk=−Fk≔Fxkk=0−Fk+βkLJJdk−1k≥1 and yield the conjugate parameter βkLJJ by (6)Step 3: compute the step sizeαk using the adaptive line search strategy (11) and set zk=xk+αkdkStep 4: computexk+1 by xk+1=PCxk−ξkFzk,whereξk=FzkTxk−zk/Fzk2Setk≔k+1, and return to Step 1


## 2.1. The New Search Direction Yielded by CGM

It is well-known that the search direction of the classical CGM is generated by(3)dk=−Fk,k=0,−Fk+βkdk−1,k≥1,which is decided by the conjugate parameter βk. Usually, a different βk leads to a different search direction.In the recent years, many scholars have made efforts to extend the CGM to solve the large-scale nonlinear monotone equations system. For example, based on the hybrid conjugate parameterβkJHJ in [34], Sun and Liu [28] proposed a modified conjugate parameter as follows:(4)βkN=Fk2−max0,Fk/Fk−1FkTFk−1maxμdk−1Fk,dk−1Tyk−1,μ>1,where yk−1=Fk−Fk−1 and extended it to solve problem (1).Recently, Tsegay et al. [35] gave a new CGM with sufficient descent property, that is,(5)βkSDCG=Fk2υFk2+dk−12,υ>12.Inspired by [28, 35], we propose a new hybrid formula as follows:(6)βkLJJ=Fk2−max0,Fk/Fk−1FkTFk−1νFk2+dk−12,where ν>1/2. Clearly, the parameter βkLJJ satisfies(7)0≤βkLJJ≤Fk22νFkdk−1=Fk2νdk−1.It is interesting that the search direction withβkLJJ have better theoretical properties, that is, it satisfies the sufficient descent condition and trust region property, automatically.


## 2.2. An Improved Adaptive Line Search Strategy

For an efficient CGPM, choosing an inexpensive line search is a key technique. To this end, many researchers try to exploit an inexact line search strategy to obtain step size with minimal cost. Zhang and Zhou [18] adopt an Armijo-type line search procedure, that is, the step size αk=maxβρii=0,1,2,⋯, such that(8)−Fxk+αkdkTdk≥σαkdk2,where β is an initial guess for the step size, ρ∈0,1, and σ is a positive constant. Li and Li [31] obtained the step size αk=maxβρii=0,1,2,⋯ by the following line search:(9)−Fxk+αkdkTdk≥σαkFxk+αkdkdk2,which was originally proposed by Solodov and Svaiter [36]. Guo and Wan [26] proposed an adaptive line search, i.e., the step size αk=maxβρii=0,1,2,⋯ satisfied the following inequality:(10)−Fxk+αkdkTdk≥σαkdk2min1,Fxk+αkdk.Obviously, whenxk is far from the solutions of problem (1) and the Fxk+αkdk is too large, it follows from (9) that the step size αk becomes small, which increases the computation cost. A similar case can appear for the line search (8) when xk is close to the solution set of problem (1) and dk is too large. However, it is worth noting that the line search (10) can overcome the previously mentioned weaknesses and take advantage of line searches (8) and (9).Inspired by [26], we introduce another new adaptive line search strategy with a disturbance factor, that is, taking αk=maxβρii=0,1,2,⋯, such that(11)−Fxk+αkdkTdk≥σαkγkdk2,where γk=Fxk+αkdk/maxFxk+αkdk,μ, β is an initial guess for the step size, σ,ρ∈0,1, and μ≥1. Here, μ is a disturbance factor, which can adjust the size of the right side of the line search (11) and further reduce the computation cost.Remark 1.
In fact, for a givenμ, Fxk+αkdk is too large if xk is far away from the solution set, namely, Fxk+αkdk/maxFxk+αkdk,μ=1, and then, the new line search (11) is similar to (8) in performance. Otherwise, when xk is close to the solution set, Fxk+αkdk approaches 0 and so γk approaches Fxk+αkdk, and then, the new line search (11) comes back to (9).


## 2.3. Algorithm

Based on the new search direction withβkLJJ (6) and adaptive line search (11), we describe our algorithm in detail for solving problem (1) as follows (Algorithm 1).Algorithm 1: LJJ CGPM.
Step 0: givenx0∈C, the constants ε>0,σ,ρ∈0,1,μ≥1,ν>1/2 and an initial stepsize β. Set k≔0Step 1: if a given terminate criterion is satisfied, e.g.,Fxk≤ε, terminate. Otherwise, go to Step 2Step 2: compute search directiondk by dk=−Fk≔Fxkk=0−Fk+βkLJJdk−1k≥1 and yield the conjugate parameter βkLJJ by (6)Step 3: compute the step sizeαk using the adaptive line search strategy (11) and set zk=xk+αkdkStep 4: computexk+1 by xk+1=PCxk−ξkFzk,whereξk=FzkTxk−zk/Fzk2Setk≔k+1, and return to Step 1


## 3. Convergence Property

In order to obtain some important properties and convergence property of Algorithm1, the following basic assumptions are necessary.Assumption H:(H1) The solution set of system (1), denoted by S, is nonempty, and the mapping F is monotone on ℝn.(H2) The mappingF is L-Lipschitz continuous on ℝn, i.e., there exists a constant L>0 such that(12)Fx−Fy≤Lx−y,∀x,y∈ℝn.The well-known nonexpansive property of the projection operator [37] is reviewed in the following lemma.Lemma 1 (see [37]).
LetΩ⊆ℝn be a nonempty closed convex set. Then,(13)PΩx−PΩy≤x−y,∀x,y∈ℝn.Therefore, the projection operatorP is L-Lipschitz continuous on ℝn.The following lemma shows that the search direction yielded by equation in step 2 in Algorithm1 satisfies the sufficient descent condition and possesses some important properties.Lemma 2.
Suppose that Assumption H holds, then the search direction generated by equation in step 2 in Algorithm1 satisfies the sufficient descent condition,(14)FkTdk≤−τFk2,and τFk≤dk≤τ^Fk, for some positive constants τ and τ^.Proof.
Fork=0, it is easy to know that F0Td0=−F02 and Lemma 2 holds. To proceed, we consider the case k≥1. If FkTdk−1=0, it follows from equation in step 2 in Algorithm 1 that FkTdk=−Fk2. Otherwise, multiplying both sides of equation in step 2 in Algorithm 1 by FkT, from (6) and (7), we have(15)FkTdk=−Fk2+βkLJJFkTdk−1≤−Fk2+βkLJJFkdk−1≤−Fk2+Fk2νdk−1Fkdk−1,=−1−12νFk2≕−τFk2,which shows that the sufficient descent property (14) holds by taking τ≔1−1/2ν. Again, according to(16)FkdkcosFk,dk=FkTdk,the following relation holds:(17)−Fkdk≤FkTdk≤−τFk2,and then, dk≥τFk.
On the other hand, it follows from (7) and equation in step 2 in Algorithm 1 that(18)dk≤Fk+βkLJJdk−1≤Fk+Fk2νdk−1dk−1=1+12νFk≕τ^Fk,and the proof is completed.
The next lemma not only indicates that the line search strategy (11) is well-defined but also provides a lower bound for step size αk.Lemma 3.
(i)
Let the sequencesdk and xk be generated by Algorithm 1; then, there exists a step size satisfying the line search (11)(ii)
Suppose that Assumption H holds; then, the step size yielded by Algorithm1 satisfies(19)αk≥minβ,τρL+σFk2dk2.Proof.
(i)
Suppose that for any nonnegative integeri, (11) does not hold at the k0-th iterate,then(20)−Fxk0+βρidk0Tdk0<σβρidk02Fxk0+βρidk0maxFxk0+βρidk0,μ.From the continuity ofF and ρ∈0,1, let i⟶∞, and it is clear that(21)−Fxk0Tdk0≤0,which contradicts (14). The proof is completed.(ii)
For the second part, it is clear that ifαk=β, then (19) holds. If αk≠β, αk is computed by the backtracking process in Algorithm 1. Let α^k≔ρ−1αk, and then, α^k does not satisfy (11), namely,(22)−Fz^kTdk<σα^kdk2Fz^kmaxFz^k,μ≤σα^kdk2,where z^k=xk+α^kdk. It follows from (12), (14), and (22) that(23)τFk2≤−FkTdk=Fz^k−FkTdk−Fz^kTdk<α^kL+σdk2=ρ−1αkL+σdk2.
Then,(24)αk>τρL+σFk2dk2,which completes the proof.
The following lemma is necessary to analyze the global convergence of Algorithm1.Lemma 4.
Suppose that Assumptions H holds, letxk and zk be generated by Algorithm 1, and let x∗ be any given solution for system (1), i.e., x∗∈S. Then, the sequence xk−x∗ is convergent, and sequences xk and zk are both bounded. Furthermore, it holds that(25)limk⟶∞αkdk=0.Proof.
In view of the definition ofzk and (11), we know that(26)FzkTxk−zk=−αkFzkTdk≥σαk2dk2γk=σxk−zk2γk.
On the other hand, taking Assumption (H1) andx∗∈S into consideration, we have(27)FzkTxk−x∗=FzkTxk−zk+FzkTzk−x∗≥FzkTxk−zk+Fx∗Tzk−x∗=FzkTxk−zk.
According to equation in step 4 in Algorithm1, Assumption (H1), Lemma 1, and (26) and (27), it follows that(28)xk+1−x∗2=PCxk−ξkFzk−PCx∗2≤xk−ξkFzk−x∗2=xk−x∗2−2ξkFzkTxk−x∗+ξk2Fzk2≤xk−x∗2−2ξkFzkTxk−zk+ξk2Fzk2=xk−x∗2−FzkTxk−zk2Fzk2≤xk−x∗2−σ2xk−zk4Fzk2γk2,which shows that the inequalities 0≤xk+1−x∗≤xk−x∗ hold, that is, the sequence xk−x∗ is monotone nonincreasing and bounded below. Hence, xk−x∗ is convergent. Furthermore, the boundedness of xk is obtained.
By Lemma2, it holds that dk is bounded and so is zk. Without the loss of generality, there exists a constant M¯>0 such that Fzk≤M¯,∀k≥0. If maxFzk,μ=Fzk, then maxFzk,μ≤M¯; otherwise, maxFzk,μ=μ. Hence, the following relation holds:(29)γkFzk=1maxFzk,μ≥1maxM¯,μ≔A.
This together with (28) implies that(30)A2σ2∑k=0∞xk−zk4≤∑k=0∞xk−x∗2−xk+1−x∗2=x0−x∗2−limk⟶∞xk+1−x∗2<∞.
Thus, this further implieslimk⟶∞αkdk=limk⟶∞xk−zk=0, and the proof is completed.
Next, based on Assumption H and Lemmas1–4, the global convergence of the proposed algorithm is established.Theorem 1.
Suppose that Assumption H holds and the sequencesxk be generated by the Algorithm 1; then,(31)liminfk⟶∞Fk=0.Furthermore, the whole sequencexk converges to a solution of system (1).Proof.
First, by contradiction, suppose that relation (31) is not true; then, there exists a constant ε>0 such that(32)Fk≥ε,∀k≥0.
Again, the following inequality comes directly from Lemma2:(33)dk≥τFk≥τε,∀k≥0.
This together with (25) shows that(34)limk⟶∞αk=0.
In addition, from Lemmas2 and 3 (ii) and the boundedness of dk, the following relation holds:(35)αk≥minβ,τρL+σFk2dk2≥minβ,τρL+στ^2>0,which contradicts (34). Therefore, (31) is true.
Second, (31) shows that there exists an infinite index set K such that limk∈KFk=0. Again, the xk is bounded and C is a closed set, so without the loss of generality, suppose that limk∈Kxk=x¯∈C. It follows from the continuity of F that(36)Fx¯=limk∈KFk=0,which shows that x¯∈S.
Finally, noticing thatx∗≔x¯ from Lemma 4, it follows that the sequence xk−x¯ is convergent, namely,(37)limk⟶∞xk−x¯=limk∈Kxk−x¯=0,which implies that the whole sequence xk converges to x¯∈S, and the proof is completed.


## 4. Numerical Experiments

In this section, the numerical performances of Algorithm1 (LJJ CGPM) for solving convex constrained nonlinear equations are tested and reported by the following two subsections.


### 4.1. Experimental Setup

In order to illustrate the effectiveness of the LJJ CGPM, we compare it with two recent CGPMs. Specifically, eight large-scale examples are solved by the LJJ CGPM method, PDY method [22], and ATTCGP method [23] in the same calculating environment. All codes were written in Matlab R2014a and run on a DELL with 4 GB RAM memory and Windows 10 operating system.For the LJJ CGPM, we use (6) and (11) as the conjugate parameter of search direction and the line search rule, respectively, and the parameters in the LJJ CGPM are chosen as ρ=0.4,β=0.99,σ=0.01,μ=1.25,andν=2.6. For the PDY [22] and the ATTCGP [23] methods, the search direction, line search rule, and selection of parameters are consistent with the original literature, respectively.For all methods, the computation will be terminated when one of the following criteria are satisfied:(38)iFk≤10−6,iidk≤10−7,oriiiItr>2000,where “Itr” refers to the total number of iterations. Define(39)Fx=f1x,f2x,⋯,fnxT,and the tested functions are listed as follows.Problem 1.
(see Wang et al. [11]). Set fix=exi−1, for i=1,2,⋯,n and C=ℝ+n. Obviously, Problem 1 has a unique solution x∗=0,0,⋯,0T.Problem 2.
(see Yu et al. [20]). Set(40)f1x=x1−ecosx1+x2/n+1,fix=xi−ecosxi−1+xi+xi+1/n+1,fnx=xn−ecosxn−1+xn/n+1,for i=2,3,⋯,n−1 and C=ℝ+n.Problem 3.
(see Yu et al. [20]). Set fix=xi−sinxi−1, for i=1,2,⋯,n and C=x∈ℝn∑i=1nxi≤n,xi≥−1,i=1,2,⋯,n. Obviously, Problem 3 is nonsmooth at x=1,⋯,1T.Problem 4.
(see Zhou and Li [13]). Set fix=2xi−sinxi, for i=1,2,⋯,n and C=ℝ+n.Problem 5.
(see Gao and He [29]). Set(41)f1x=2x1+sinx1−1,fix=2xi−1+2xi+sinxi−1,fnx=2xn+sinxn−1,for i=2,3,⋯,n−1 and C=ℝ+n.Problem 6.
(see Ou and Li [19]). Set(42)f1x=ex1−1,fix=exi+xi−1−1,for i=2,3,⋯,n and C=ℝ+n.Problem 7.
(see Gao and He [29]). Set fix=exi2+3sinxicosxi−1, for i=1,2,⋯,n and C=ℝ+n.Problem 8.
(see Gao and He [29]). Set(43)f1x=x1−ecosx1+x2/2,fix=xi−ecosxi−1+xi+xi+1/i,fnx=xn−ecosxn−1+xn/n,for i=2,3,⋯,n−1 and C=ℝ+n.
The new iterate points yielded by the quadratic program solver quadprog.m are taken from the Matlab optimization toolbox. Problems 1–8 are tested with seven initial points x1=1,1,⋯,1T,x2=0.1,0.1,⋯,0.1T,x3=1/2,1/22,⋯,1/2nT,x4=0,1/n,⋯,n−1/nT,x5=1,1/2,⋯,1/nT,x6=1/n,2/n,⋯,1T,andx7=1−1/n,1−2/n,⋯,0T. Here, the dimension of problems is chosen as 104, 5×104 and 105, respectively.
The comparison of data is listed in Tables1–8, where “Init” means the initial point, “n” is the dimension of the problem, “NF” denotes the number of function evaluations, “Tcpu” denotes the CPU time, and “F∗” is the final value of Fk when the program is stopped.
In addition, in order to show the numerical performance clearly, we adopt the profiles introduced by Dolan and Morè [38] to compare the performance on Tcpu, NF, and Itr, respectively. A brief explanation of the performance figures is as follows. Denote the whole set of np test problems by P and the set of solvers by S. Let tp,s be the Tcpu (or the NF or the Itr) required to solve problem p∈P by solver s∈S, and the comparison results between different solvers are based on the performance ratio defined from(44)rp,s=tp,smins∈Stp,s,and the performance profile for each solver s is defined by(45)ρsτ=1npsizep∈P:log2rp,s≤τ,where size A means the number of elements in the set A. Then, ρsτ is the probability for solver s∈S that a performance ratio rp,s is within a factor τ∈Rn. ρs is the (cumulative) distribution function for the performance ratio. Clearly, the top curved shape of the method is a winner. For details about the performance profile, see [38].Table 1
Numerical results on Problem1.
Init (n)LJJ CGPMPDYATTCGPItr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗x11048/23/0.025/3.939e − 0716/49/0.043/9.317e − 0720/62/0.047/6.199e − 07x21047/19/0.016/4.751e − 0715/46/0.035/5.458e − 0718/55/0.036/6.052e − 07x310410/28/0.011/4.686e − 0712/37/0.015/8.832e − 0714/43/0.017/9.585e − 07x410416/43/0.030/1.287e − 0817/52/0.036/5.220e − 0725/79/0.035/7.025e − 07x510411/31/0.021/4.987e − 0714/44/0.024/4.552e − 0716/50/0.021/7.025e − 07x610416/43/0.030/1.291e − 0817/52/0.039/5.225e − 0725/79/0.031/5.167e − 07x710416/43/0.029/1.287e − 0817/52/0.037/5.220e − 0725/79/0.034/7.025e − 07x15×1048/23/0.096/8.808e − 0718/57/0.229/9.221e − 0721/65/0.238/5.544e − 07x25×1048/21/0.078/1.062e − 0816/49/0.185/4.039e − 0719/58/0.207/5.413e − 07x35×10410/28/0.059/4.686e − 0712/37/0.080/8.832e − 0714/43/0.093/9.585e − 07x45×10416/43/0.150/2.881e − 0818/57/0.212/6.743e − 0727/86/0.187/8.395e − 07x55×10411/31/0.105/4.987e − 0714/44/0.129/4.552e − 0716/50/0.123/7.025e − 07x65×10416/43/0.151/2.883e − 0818/57/0.215/6.743e − 0727/86/0.191/8.174e − 07x75×10416/43/0.151/2.881e − 0818/57/0.215/6.743e − 0727/86/0.193/8.395e − 07x11059/25/0.185/1.246e − 0819/63/0.500/7.471e − 0721/65/0.496/7.841e − 07x21058/21/0.159/1.502e − 0816/49/0.377/5.712e − 0719/58/0.412/7.655e − 07x310510/28/0.129/4.686e − 0712/37/0.193/8.832e − 0714/43/0.186/9.585e − 07x410516/43/0.307/4.076e − 0818/57/0.436/9.535e − 0728/89/0.412/5.734e − 07x510511/31/0.207/4.987e − 0714/44/0.251/4.552e − 0716/50/0.236/7.025e − 07x610516/43/0.306/4.077e − 0818/57/0.430/9.536e − 0728/89/0.443/6.548e − 07x710516/43/0.320/4.076e − 0818/57/0.429/9.535e − 0728/89/0.419/5.734e − 07Table 2
Numerical results on Problem2.
Init (n)LJJ CGPMPDYATTCGPItr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗x11045/11/0.018/1.718e − 0818/55/0.078/3.324e − 0721/64/0.087/7.557e − 07x21045/11/0.017/2.617e − 0820/65/0.088/3.998e − 0722/67/0.092/4.606e − 07x31045/11/0.017/2.717e − 0820/65/0.091/4.150e − 0722/67/0.096/4.782e − 07x41045/11/0.017/2.236e − 0819/60/0.087/5.876e − 0721/64/0.094/9.838e − 07x51045/11/0.018/2.717e − 0820/65/0.092/4.149e − 0722/67/0.096/4.780e − 07x61045/11/0.017/2.236e − 0819/60/0.086/5.876e − 0721/64/0.093/9.838e − 07x71045/11/0.018/2.236e − 0819/60/0.086/5.876e − 0721/64/0.094/9.838e − 07x15×1045/11/0.085/3.842e − 0821/69/0.497/3.654e − 0722/67/0.479/6.759e − 07x25×1045/11/0.086/5.855e − 0823/82/0.574/7.076e − 0723/70/0.494/4.120e − 07x35×1045/11/0.085/6.078e − 0823/82/0.577/7.347e − 0723/70/0.502/4.277e − 07x45×1045/11/0.087/5.002e − 0822/77/0.561/7.324e − 0722/67/0.505/8.800e − 07x55×1045/11/0.099/6.078e − 0823/82/0.611/7.346e − 0723/70/0.541/4.277e − 07x65×1045/11/0.086/5.002e − 0822/77/0.549/7.324e − 0722/67/0.487/8.800e − 07x75×1045/11/0.087/5.002e − 0822/77/0.549/7.324e − 0722/67/0.490/8.800e − 07x11055/11/0.171/5.434e − 0823/82/1.148/6.568e − 0722/67/1.083/9.559e − 07x21055/11/0.184/8.280e − 0828/114/1.723/5.929e − 0723/70/1.219/5.826e − 07x31055/11/0.188/8.596e − 0828/114/1.564/6.155e − 0723/70/1.007/6.049e − 07x41055/11/0.171/7.074e − 0825/94/1.385/6.443e − 0723/70/1.045/4.978e − 07x51055/11/0.180/8.596e − 0828/114/1.677/6.155e − 0723/70/1.077/6.049e − 07x61055/11/0.171/7.074e − 0825/94/1.310/6.443e − 0723/70/1.023/4.978e − 07x71055/11/0.178/7.074e − 0825/94/1.319/6.443e − 0723/70/1.030/4.978e − 07Table 3
Numerical results on Problem3.
Init (n)LJJ CGPMPDYATTCGPItr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗x110414/43/0.013/4.708e − 076/19/0.006/8.620e − 0817/69/0.016/4.344e − 07x210414/43/0.011/5.456e − 0717/68/0.021/7.903e − 0717/69/0.016/4.628e − 07x310414/43/0.012/7.328e − 0718/72/0.017/4.221e − 0717/69/0.016/6.140e − 07x410414/43/0.012/3.474e − 0719/76/0.019/4.688e − 0716/65/0.017/9.030e − 07x510414/43/0.011/7.314e − 0718/72/0.018/4.221e − 0717/69/0.016/6.128e − 07x610414/43/0.011/3.474e − 0719/76/0.019/4.690e − 0716/65/0.015/9.029e − 07x710414/43/0.013/3.474e − 0719/76/0.018/4.688e − 0716/65/0.014/9.030e − 07x15×10415/46/0.058/2.722e − 0719/77/0.095/6.950e − 0717/69/0.071/9.714e − 07x25×10415/46/0.053/3.154e − 0718/72/0.083/6.418e − 0718/73/0.086/3.373e − 07x35×10415/46/0.058/4.238e − 0718/72/0.079/9.439e − 0718/73/0.076/4.476e − 07x45×10414/43/0.054/7.768e − 0720/80/0.096/3.801e − 0717/69/0.072/6.582e − 07x55×10415/46/0.055/4.236e − 0718/72/0.080/9.438e − 0718/73/0.078/4.474e − 07x65×10414/43/0.053/7.768e − 0720/80/0.092/3.801e − 0717/69/0.072/6.582e − 07x75×10414/43/0.056/7.768e − 0720/80/0.091/3.801e − 0717/69/0.079/6.582e − 07x110515/46/0.114/3.849e − 0719/77/0.191/9.829e − 0718/73/0.164/4.478e − 07x210515/46/0.122/4.461e − 0719/77/0.189/7.449e − 0718/73/0.176/4.770e − 07x310515/46/0.119/5.993e − 0719/77/0.196/9.210e − 0718/73/0.165/6.330e − 07x410515/46/0.124/2.841e − 0719/77/0.190/6.666e − 0717/69/0.164/9.308e − 07x510515/46/0.121/5.992e − 0719/77/0.202/9.209e − 0718/73/0.172/6.328e − 07x610515/46/0.121/2.841e − 0719/77/0.190/6.666e − 0717/69/0.163/9.308e − 07x710515/46/0.118/2.841e − 0719/77/0.200/6.666e − 0717/69/0.162/9.308e − 07Table 4
Numerical results on Problem4.
Init (n)LJJ CGPMPDYATTCGPItr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗x11047/18/0.007/7.770e − 0817/52/0.017/7.653e − 0720/61/0.019/8.152e − 07x21044/9/0.003/8.351e − 0815/46/0.015/5.283e − 0718/55/0.016/6.852e − 07x31047/17/0.005/4.317e − 0812/37/0.010/8.833e − 0715/46/0.012/5.849e − 07x410410/26/0.008/9.622e − 0717/52/0.015/4.108e − 0720/61/0.019/5.400e − 07x51048/21/0.006/6.150e − 0813/40/0.011/7.409e − 0716/49/0.014/4.741e − 07x610410/26/0.008/9.639e − 0717/52/0.017/4.109e − 0720/61/0.016/5.401e − 07x710410/26/0.008/9.622e − 0717/52/0.015/4.108e − 0720/61/0.016/5.400e − 07x15×1047/18/0.025/1.737e − 0719/60/0.079/5.509e − 0721/64/0.085/7.292e − 07x25×1044/9/0.015/1.867e − 0716/49/0.064/3.907e − 0719/58/0.065/6.128e − 07x35×1047/17/0.020/4.317e − 0812/37/0.038/8.833e − 0715/46/0.045/5.849e − 07x45×10411/28/0.037/2.153e − 0817/52/0.065/9.187e − 0721/64/0.081/4.830e − 07x55×1048/21/0.025/6.151e − 0813/40/0.051/7.410e − 0716/49/0.055/4.742e − 07x65×10411/28/0.038/2.154e − 0817/52/0.071/9.187e − 0721/64/0.097/4.830e − 07x75×10411/28/0.035/2.153e − 0817/52/0.067/9.187e − 0721/64/0.077/4.830e − 07x11057/18/0.050/2.457e − 0720/65/0.168/4.909e − 0722/67/0.162/4.125e − 07x21054/9/0.032/2.641e − 0716/49/0.132/5.525e − 0719/58/0.143/8.667e − 07x31057/17/0.040/4.317e − 0812/37/0.087/8.833e − 0715/46/0.096/5.849e − 07x410511/28/0.082/3.045e − 0819/60/0.179/4.622e − 0721/64/0.174/6.831e − 07x51058/21/0.062/6.151e − 0813/40/0.106/7.410e − 0716/49/0.118/4.742e − 07x610511/28/0.077/3.046e − 0819/60/0.167/4.622e − 0721/64/0.172/6.831e − 07x710511/28/0.071/3.045e − 0819/60/0.158/4.622e − 0721/64/0.161/6.831e − 07Table 5
Numerical results on Problem5.
Init (n)LJJ CGPMPDYATTCGPItr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗x110419/77/0.060/7.476e − 0727/145/0.101/8.743e − 07152/787/0.578/7.450e − 07x210423/93/0.069/7.916e − 0742/226/0.158/8.372e − 07204/1047/0.744/6.883e − 07x310419/77/0.057/5.319e − 0728/151/0.103/5.112e − 07170/876/0.610/8.184e − 07x410420/81/0.062/9.327e − 0727/144/0.103/5.294e − 07196/1008/0.699/6.472e − 07x510420/81/0.061/5.548e − 0726/138/0.098/8.101e − 07185/952/0.653/7.283e − 07x610420/81/0.062/9.324e − 0726/138/0.101/8.724e − 07196/1008/0.685/6.471e − 07x710419/77/0.057/5.444e − 0726/138/0.096/6.535e − 07160/827/0.566/6.203e − 07x15×10419/77/0.296/8.171e − 0725/133/0.487/5.286e − 07141/732/2.615/9.222e − 07x25×10421/85/0.321/7.117e − 0726/139/0.503/5.201e − 07194/997/3.551/8.547e − 07x35×10419/77/0.294/4.696e − 0727/146/0.528/9.869e − 07171/882/3.184/5.748e − 07x45×10419/77/0.297/9.870e − 0731/167/0.613/6.829e − 07185/953/3.447/8.111e − 07x55×10419/77/0.294/8.999e − 0728/150/0.543/6.333e − 07175/902/3.215/8.894e − 07x65×10419/77/0.293/9.870e − 0731/167/0.612/6.807e − 07185/953/3.397/8.111e − 07x75×10419/77/0.293/5.847e − 0726/138/0.505/9.469e − 07151/782/2.788/7.611e − 07x110519/77/0.598/8.828e − 0730/162/1.216/4.748e − 07146/758/5.586/5.631e − 07x210520/81/0.635/8.295e − 0725/133/0.993/9.282e − 07185/952/6.975/9.531e − 07x310519/77/0.596/4.759e − 0726/137/1.025/6.434e − 07162/837/6.107/6.288e − 07x410519/77/0.595/8.785e − 0727/144/1.100/8.273e − 07176/908/6.668/9.015e − 07x510519/77/0.598/8.971e − 0729/156/1.160/4.997e − 07168/867/6.344/9.717e − 07x610519/77/0.604/8.784e − 0727/143/1.074/7.645e − 07176/908/6.682/9.015e − 07x710519/77/0.604/6.266e − 0727/143/1.072/6.767e − 07143/742/5.435/8.344e − 07Table 6
Numerical results on Problem6.
Init (n)LJJ CGPMPDYATTCGPItr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗x110436/110/0.204/9.949e − 0729/117/0.075/9.510e − 0724/97/0.057/9.931e − 07x210411/34/0.026/5.671e − 0725/101/0.063/7.366e − 0725/101/0.058/8.628e − 07x310437/112/0.051/7.576e − 0752/209/0.090/9.275e − 0734/137/0.056/8.435e − 07x410420/62/0.041/7.165e − 0717/69/0.047/7.590e − 0716/65/0.032/2.846e − 07x510439/118/0.069/7.898e − 0760/241/0.127/9.418e − 0735/141/0.074/9.479e − 07x610420/62/0.042/7.506e − 0717/69/0.047/7.591e − 0716/65/0.029/2.849e − 07x710435/107/0.067/7.787e − 0731/125/0.076/8.371e − 0731/125/0.055/6.861e − 07x15×10437/113/0.376/6.935e − 0730/123/0.413/7.657e − 0723/93/0.288/8.910e − 07x25×10412/37/0.134/2.594e − 0725/101/0.333/6.388e − 0725/101/0.309/7.727e − 07x35×10437/112/0.281/7.576e − 0752/209/0.490/9.275e − 0734/137/0.311/8.435e − 07x45×10417/53/0.190/7.679e − 0718/73/0.256/5.629e − 0716/65/0.160/5.993e − 07x55×10439/118/0.365/7.898e − 0760/241/0.678/9.417e − 0735/141/0.389/9.479e − 07x65×10417/53/0.188/6.669e − 0718/73/0.259/5.629e − 0716/65/0.158/5.995e − 07x75×10435/107/0.352/9.085e − 0729/117/0.390/9.530e − 0730/121/0.286/8.457e − 07x110537/113/0.783/7.173e − 0731/130/0.890/8.423e − 0723/93/0.589/7.944e − 07x210512/37/0.280/3.661e − 0724/97/0.661/9.622e − 0725/101/0.628/7.387e − 07x310537/112/0.558/7.576e − 0752/209/1.006/9.275e − 0734/137/0.639/8.435e − 07x410517/53/0.377/6.159e − 0719/79/0.566/7.772e − 0716/65/0.334/8.412e − 07x510539/118/0.732/7.898e − 0760/241/1.351/9.417e − 0735/141/0.785/9.479e − 07x610517/53/0.374/6.332e − 0719/79/0.572/7.772e − 0716/65/0.335/8.413e − 07x710535/107/0.698/9.072e − 0730/123/0.816/8.816e − 0730/121/0.599/7.940e − 07Table 7
Numerical results on Problem7.
Init (n)LJJ CGPMPDYATTCGPItr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗x11041/4/0.007/0.000e + 0013/66/0.081/2.258e − 0719/116/0.142/8.849e − 07x210412/49/0.061/2.701e − 0712/61/0.077/6.969e − 0717/103/0.122/8.750e − 07x310410/41/0.026/5.307e − 0711/56/0.030/4.009e − 0714/85/0.041/8.584e − 07x410415/62/0.072/2.133e − 0716/83/0.086/4.425e − 0726/159/0.091/4.365e − 07x510415/59/0.066/5.118e − 0716/84/0.088/4.508e − 0718/106/0.101/8.553e − 07x610415/62/0.072/2.138e − 0716/83/0.085/4.515e − 0726/155/0.087/9.530e − 07x710415/62/0.073/2.133e − 0716/83/0.085/4.425e − 0726/159/0.089/4.365e − 07x15×1041/4/0.032/0.000e + 0015/81/0.524/3.287e − 0720/122/0.733/6.965e − 07x25×10412/49/0.308/6.039e − 0713/66/0.409/3.777e − 0718/109/0.629/6.887e − 07x35×10410/41/0.115/5.307e − 0711/56/0.157/4.009e − 0714/85/0.220/8.584e − 07x45×10415/62/0.371/4.775e − 0715/78/0.454/5.902e − 0714/92/0.297/0.000e + 00x55×10415/59/0.317/5.108e − 0716/84/0.487/4.212e − 0718/106/0.497/8.553e − 07x65×10415/62/0.375/4.777e − 0715/78/0.470/5.902e − 0715/97/0.344/0.000e + 00x75×10415/62/0.413/4.775e − 0715/78/0.575/5.902e − 0714/92/0.319/0.000e + 00x11051/4/0.065/0.000e + 0015/81/1.075/4.648e − 0720/122/1.634/9.850e − 07x210512/49/0.642/8.540e − 0713/66/1.206/5.342e − 0718/109/1.369/9.740e − 07x310510/41/0.301/5.307e − 0711/56/0.390/4.009e − 0714/85/0.484/8.584e − 07x410515/62/0.765/6.754e − 0715/78/0.923/8.346e − 0718/158/0.959/0.000e + 00x510515/59/0.627/5.107e − 0716/84/0.893/4.176e − 0718/106/0.994/8.553e − 07x610515/62/0.752/6.755e − 0715/78/0.926/8.346e − 0723/158/0.999/0.000e + 00x710515/62/0.759/6.754e − 0715/78/0.962/8.346e − 0718/158/0.941/0.000e + 00Table 8
Numerical results on Problem8.
Init (n)LJJ CGPMPDYATTCGPItr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗x110426/78/0.111/7.185e − 0725/99/0.130/7.229e − 0725/90/0.117/2.263e − 07x210425/76/0.106/9.350e − 0725/102/0.137/4.114e − 0725/90/0.127/2.314e − 07x310427/82/0.126/2.437e − 0725/102/0.132/7.922e − 0725/90/0.120/2.314e − 07x410427/82/0.116/7.926e − 0722/87/0.119/8.501e − 0725/90/0.123/2.321e − 07x510427/82/0.115/4.741e − 0725/102/0.136/4.135e − 0725/90/0.121/2.312e − 07x610427/82/0.118/8.941e − 0722/87/0.115/8.500e − 0725/90/0.121/2.321e − 07x710428/84/0.119/3.899e − 0722/87/0.115/9.315e − 0725/90/0.117/2.265e − 07x15×10428/84/0.604/1.713e − 0725/102/0.691/9.284e − 0726/93/0.638/2.269e − 07x25×10429/88/0.622/2.518e − 0728/118/0.791/4.552e − 0726/93/0.637/2.296e − 07x35×10428/85/0.599/3.829e − 0728/118/0.793/5.796e − 0726/93/0.638/2.296e − 07x45×10428/85/0.605/2.585e − 0727/113/0.767/6.928e − 0726/93/0.637/2.299e − 07x55×10427/82/0.584/9.499e − 0727/114/0.769/9.214e − 0726/93/0.642/2.295e − 07x65×10428/85/0.608/2.580e − 0727/113/0.764/6.927e − 0726/93/0.641/2.299e − 07x75×10426/78/0.555/4.908e − 0728/116/0.792/4.679e − 0726/93/0.641/2.271e − 07x110526/78/1.127/8.270e − 0726/109/1.495/5.490e − 0726/93/1.302/2.271e − 07x210527/81/1.160/4.953e − 0731/141/1.881/7.790e − 0727/96/1.320/2.292e − 07x310528/84/1.218/2.377e − 0731/141/1.874/5.803e − 0727/96/1.338/2.292e − 07x410528/85/1.212/6.599e − 0733/141/1.895/3.126e − 0727/96/1.332/2.294e − 07x510528/84/1.230/3.886e − 0730/137/1.810/8.937e − 0727/96/1.333/2.291e − 07x610528/85/1.233/6.600e − 0733/141/1.884/3.132e − 0727/96/1.336/2.294e − 07x710525/75/1.111/8.550e − 0728/121/1.715/7.546e − 0727/96/1.391/2.273e − 07


### 4.2. Numerical Testing Reports

The specific results of the numerical tests are displayed in Tables1–8. Their corresponding performance profiles are plotted in Figures 1–3 in terms of Tcpu, NF, and Itr, respectively. From Tables 1–8 and Figures 1–3, the following results are obtained.(i)
The LJJ CGPM, PDY, and ATTCGP can solve all test problems completely, which shows that abovementioned three methods are effective.(ii)
From Figures1–3, the LJJ CGPM is the top performer among the three algorithms, and this implies that the LJJ CGPM is superior to the PDY and ATTCGP totally, at least for this set of numerical experiments.Figure 1
Performance profiles on Tcpu.Figure 2
Performance profiles on NF.Figure 3
Performance profiles on Itr.Reasonably, the advantages of the LJJ CGPM are attributed to the choice of techniques (6) and (11) for the conjugate parameter of search direction and the adaptive line search strategy. At the first glance, the parameter βkLJJ (6) and line search strategy (11) are a bit complicated; however, the LJJ CGPM is very effective for solving constrained equations. Furthermore, numerical results indicate that the proposed method is competitive to similar methods for large-scale problems.


## 4.1. Experimental Setup

In order to illustrate the effectiveness of the LJJ CGPM, we compare it with two recent CGPMs. Specifically, eight large-scale examples are solved by the LJJ CGPM method, PDY method [22], and ATTCGP method [23] in the same calculating environment. All codes were written in Matlab R2014a and run on a DELL with 4 GB RAM memory and Windows 10 operating system.For the LJJ CGPM, we use (6) and (11) as the conjugate parameter of search direction and the line search rule, respectively, and the parameters in the LJJ CGPM are chosen as ρ=0.4,β=0.99,σ=0.01,μ=1.25,andν=2.6. For the PDY [22] and the ATTCGP [23] methods, the search direction, line search rule, and selection of parameters are consistent with the original literature, respectively.For all methods, the computation will be terminated when one of the following criteria are satisfied:(38)iFk≤10−6,iidk≤10−7,oriiiItr>2000,where “Itr” refers to the total number of iterations. Define(39)Fx=f1x,f2x,⋯,fnxT,and the tested functions are listed as follows.Problem 1.
(see Wang et al. [11]). Set fix=exi−1, for i=1,2,⋯,n and C=ℝ+n. Obviously, Problem 1 has a unique solution x∗=0,0,⋯,0T.Problem 2.
(see Yu et al. [20]). Set(40)f1x=x1−ecosx1+x2/n+1,fix=xi−ecosxi−1+xi+xi+1/n+1,fnx=xn−ecosxn−1+xn/n+1,for i=2,3,⋯,n−1 and C=ℝ+n.Problem 3.
(see Yu et al. [20]). Set fix=xi−sinxi−1, for i=1,2,⋯,n and C=x∈ℝn∑i=1nxi≤n,xi≥−1,i=1,2,⋯,n. Obviously, Problem 3 is nonsmooth at x=1,⋯,1T.Problem 4.
(see Zhou and Li [13]). Set fix=2xi−sinxi, for i=1,2,⋯,n and C=ℝ+n.Problem 5.
(see Gao and He [29]). Set(41)f1x=2x1+sinx1−1,fix=2xi−1+2xi+sinxi−1,fnx=2xn+sinxn−1,for i=2,3,⋯,n−1 and C=ℝ+n.Problem 6.
(see Ou and Li [19]). Set(42)f1x=ex1−1,fix=exi+xi−1−1,for i=2,3,⋯,n and C=ℝ+n.Problem 7.
(see Gao and He [29]). Set fix=exi2+3sinxicosxi−1, for i=1,2,⋯,n and C=ℝ+n.Problem 8.
(see Gao and He [29]). Set(43)f1x=x1−ecosx1+x2/2,fix=xi−ecosxi−1+xi+xi+1/i,fnx=xn−ecosxn−1+xn/n,for i=2,3,⋯,n−1 and C=ℝ+n.
The new iterate points yielded by the quadratic program solver quadprog.m are taken from the Matlab optimization toolbox. Problems 1–8 are tested with seven initial points x1=1,1,⋯,1T,x2=0.1,0.1,⋯,0.1T,x3=1/2,1/22,⋯,1/2nT,x4=0,1/n,⋯,n−1/nT,x5=1,1/2,⋯,1/nT,x6=1/n,2/n,⋯,1T,andx7=1−1/n,1−2/n,⋯,0T. Here, the dimension of problems is chosen as 104, 5×104 and 105, respectively.
The comparison of data is listed in Tables1–8, where “Init” means the initial point, “n” is the dimension of the problem, “NF” denotes the number of function evaluations, “Tcpu” denotes the CPU time, and “F∗” is the final value of Fk when the program is stopped.
In addition, in order to show the numerical performance clearly, we adopt the profiles introduced by Dolan and Morè [38] to compare the performance on Tcpu, NF, and Itr, respectively. A brief explanation of the performance figures is as follows. Denote the whole set of np test problems by P and the set of solvers by S. Let tp,s be the Tcpu (or the NF or the Itr) required to solve problem p∈P by solver s∈S, and the comparison results between different solvers are based on the performance ratio defined from(44)rp,s=tp,smins∈Stp,s,and the performance profile for each solver s is defined by(45)ρsτ=1npsizep∈P:log2rp,s≤τ,where size A means the number of elements in the set A. Then, ρsτ is the probability for solver s∈S that a performance ratio rp,s is within a factor τ∈Rn. ρs is the (cumulative) distribution function for the performance ratio. Clearly, the top curved shape of the method is a winner. For details about the performance profile, see [38].Table 1
Numerical results on Problem1.
Init (n)LJJ CGPMPDYATTCGPItr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗x11048/23/0.025/3.939e − 0716/49/0.043/9.317e − 0720/62/0.047/6.199e − 07x21047/19/0.016/4.751e − 0715/46/0.035/5.458e − 0718/55/0.036/6.052e − 07x310410/28/0.011/4.686e − 0712/37/0.015/8.832e − 0714/43/0.017/9.585e − 07x410416/43/0.030/1.287e − 0817/52/0.036/5.220e − 0725/79/0.035/7.025e − 07x510411/31/0.021/4.987e − 0714/44/0.024/4.552e − 0716/50/0.021/7.025e − 07x610416/43/0.030/1.291e − 0817/52/0.039/5.225e − 0725/79/0.031/5.167e − 07x710416/43/0.029/1.287e − 0817/52/0.037/5.220e − 0725/79/0.034/7.025e − 07x15×1048/23/0.096/8.808e − 0718/57/0.229/9.221e − 0721/65/0.238/5.544e − 07x25×1048/21/0.078/1.062e − 0816/49/0.185/4.039e − 0719/58/0.207/5.413e − 07x35×10410/28/0.059/4.686e − 0712/37/0.080/8.832e − 0714/43/0.093/9.585e − 07x45×10416/43/0.150/2.881e − 0818/57/0.212/6.743e − 0727/86/0.187/8.395e − 07x55×10411/31/0.105/4.987e − 0714/44/0.129/4.552e − 0716/50/0.123/7.025e − 07x65×10416/43/0.151/2.883e − 0818/57/0.215/6.743e − 0727/86/0.191/8.174e − 07x75×10416/43/0.151/2.881e − 0818/57/0.215/6.743e − 0727/86/0.193/8.395e − 07x11059/25/0.185/1.246e − 0819/63/0.500/7.471e − 0721/65/0.496/7.841e − 07x21058/21/0.159/1.502e − 0816/49/0.377/5.712e − 0719/58/0.412/7.655e − 07x310510/28/0.129/4.686e − 0712/37/0.193/8.832e − 0714/43/0.186/9.585e − 07x410516/43/0.307/4.076e − 0818/57/0.436/9.535e − 0728/89/0.412/5.734e − 07x510511/31/0.207/4.987e − 0714/44/0.251/4.552e − 0716/50/0.236/7.025e − 07x610516/43/0.306/4.077e − 0818/57/0.430/9.536e − 0728/89/0.443/6.548e − 07x710516/43/0.320/4.076e − 0818/57/0.429/9.535e − 0728/89/0.419/5.734e − 07Table 2
Numerical results on Problem2.
Init (n)LJJ CGPMPDYATTCGPItr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗x11045/11/0.018/1.718e − 0818/55/0.078/3.324e − 0721/64/0.087/7.557e − 07x21045/11/0.017/2.617e − 0820/65/0.088/3.998e − 0722/67/0.092/4.606e − 07x31045/11/0.017/2.717e − 0820/65/0.091/4.150e − 0722/67/0.096/4.782e − 07x41045/11/0.017/2.236e − 0819/60/0.087/5.876e − 0721/64/0.094/9.838e − 07x51045/11/0.018/2.717e − 0820/65/0.092/4.149e − 0722/67/0.096/4.780e − 07x61045/11/0.017/2.236e − 0819/60/0.086/5.876e − 0721/64/0.093/9.838e − 07x71045/11/0.018/2.236e − 0819/60/0.086/5.876e − 0721/64/0.094/9.838e − 07x15×1045/11/0.085/3.842e − 0821/69/0.497/3.654e − 0722/67/0.479/6.759e − 07x25×1045/11/0.086/5.855e − 0823/82/0.574/7.076e − 0723/70/0.494/4.120e − 07x35×1045/11/0.085/6.078e − 0823/82/0.577/7.347e − 0723/70/0.502/4.277e − 07x45×1045/11/0.087/5.002e − 0822/77/0.561/7.324e − 0722/67/0.505/8.800e − 07x55×1045/11/0.099/6.078e − 0823/82/0.611/7.346e − 0723/70/0.541/4.277e − 07x65×1045/11/0.086/5.002e − 0822/77/0.549/7.324e − 0722/67/0.487/8.800e − 07x75×1045/11/0.087/5.002e − 0822/77/0.549/7.324e − 0722/67/0.490/8.800e − 07x11055/11/0.171/5.434e − 0823/82/1.148/6.568e − 0722/67/1.083/9.559e − 07x21055/11/0.184/8.280e − 0828/114/1.723/5.929e − 0723/70/1.219/5.826e − 07x31055/11/0.188/8.596e − 0828/114/1.564/6.155e − 0723/70/1.007/6.049e − 07x41055/11/0.171/7.074e − 0825/94/1.385/6.443e − 0723/70/1.045/4.978e − 07x51055/11/0.180/8.596e − 0828/114/1.677/6.155e − 0723/70/1.077/6.049e − 07x61055/11/0.171/7.074e − 0825/94/1.310/6.443e − 0723/70/1.023/4.978e − 07x71055/11/0.178/7.074e − 0825/94/1.319/6.443e − 0723/70/1.030/4.978e − 07Table 3
Numerical results on Problem3.
Init (n)LJJ CGPMPDYATTCGPItr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗x110414/43/0.013/4.708e − 076/19/0.006/8.620e − 0817/69/0.016/4.344e − 07x210414/43/0.011/5.456e − 0717/68/0.021/7.903e − 0717/69/0.016/4.628e − 07x310414/43/0.012/7.328e − 0718/72/0.017/4.221e − 0717/69/0.016/6.140e − 07x410414/43/0.012/3.474e − 0719/76/0.019/4.688e − 0716/65/0.017/9.030e − 07x510414/43/0.011/7.314e − 0718/72/0.018/4.221e − 0717/69/0.016/6.128e − 07x610414/43/0.011/3.474e − 0719/76/0.019/4.690e − 0716/65/0.015/9.029e − 07x710414/43/0.013/3.474e − 0719/76/0.018/4.688e − 0716/65/0.014/9.030e − 07x15×10415/46/0.058/2.722e − 0719/77/0.095/6.950e − 0717/69/0.071/9.714e − 07x25×10415/46/0.053/3.154e − 0718/72/0.083/6.418e − 0718/73/0.086/3.373e − 07x35×10415/46/0.058/4.238e − 0718/72/0.079/9.439e − 0718/73/0.076/4.476e − 07x45×10414/43/0.054/7.768e − 0720/80/0.096/3.801e − 0717/69/0.072/6.582e − 07x55×10415/46/0.055/4.236e − 0718/72/0.080/9.438e − 0718/73/0.078/4.474e − 07x65×10414/43/0.053/7.768e − 0720/80/0.092/3.801e − 0717/69/0.072/6.582e − 07x75×10414/43/0.056/7.768e − 0720/80/0.091/3.801e − 0717/69/0.079/6.582e − 07x110515/46/0.114/3.849e − 0719/77/0.191/9.829e − 0718/73/0.164/4.478e − 07x210515/46/0.122/4.461e − 0719/77/0.189/7.449e − 0718/73/0.176/4.770e − 07x310515/46/0.119/5.993e − 0719/77/0.196/9.210e − 0718/73/0.165/6.330e − 07x410515/46/0.124/2.841e − 0719/77/0.190/6.666e − 0717/69/0.164/9.308e − 07x510515/46/0.121/5.992e − 0719/77/0.202/9.209e − 0718/73/0.172/6.328e − 07x610515/46/0.121/2.841e − 0719/77/0.190/6.666e − 0717/69/0.163/9.308e − 07x710515/46/0.118/2.841e − 0719/77/0.200/6.666e − 0717/69/0.162/9.308e − 07Table 4
Numerical results on Problem4.
Init (n)LJJ CGPMPDYATTCGPItr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗x11047/18/0.007/7.770e − 0817/52/0.017/7.653e − 0720/61/0.019/8.152e − 07x21044/9/0.003/8.351e − 0815/46/0.015/5.283e − 0718/55/0.016/6.852e − 07x31047/17/0.005/4.317e − 0812/37/0.010/8.833e − 0715/46/0.012/5.849e − 07x410410/26/0.008/9.622e − 0717/52/0.015/4.108e − 0720/61/0.019/5.400e − 07x51048/21/0.006/6.150e − 0813/40/0.011/7.409e − 0716/49/0.014/4.741e − 07x610410/26/0.008/9.639e − 0717/52/0.017/4.109e − 0720/61/0.016/5.401e − 07x710410/26/0.008/9.622e − 0717/52/0.015/4.108e − 0720/61/0.016/5.400e − 07x15×1047/18/0.025/1.737e − 0719/60/0.079/5.509e − 0721/64/0.085/7.292e − 07x25×1044/9/0.015/1.867e − 0716/49/0.064/3.907e − 0719/58/0.065/6.128e − 07x35×1047/17/0.020/4.317e − 0812/37/0.038/8.833e − 0715/46/0.045/5.849e − 07x45×10411/28/0.037/2.153e − 0817/52/0.065/9.187e − 0721/64/0.081/4.830e − 07x55×1048/21/0.025/6.151e − 0813/40/0.051/7.410e − 0716/49/0.055/4.742e − 07x65×10411/28/0.038/2.154e − 0817/52/0.071/9.187e − 0721/64/0.097/4.830e − 07x75×10411/28/0.035/2.153e − 0817/52/0.067/9.187e − 0721/64/0.077/4.830e − 07x11057/18/0.050/2.457e − 0720/65/0.168/4.909e − 0722/67/0.162/4.125e − 07x21054/9/0.032/2.641e − 0716/49/0.132/5.525e − 0719/58/0.143/8.667e − 07x31057/17/0.040/4.317e − 0812/37/0.087/8.833e − 0715/46/0.096/5.849e − 07x410511/28/0.082/3.045e − 0819/60/0.179/4.622e − 0721/64/0.174/6.831e − 07x51058/21/0.062/6.151e − 0813/40/0.106/7.410e − 0716/49/0.118/4.742e − 07x610511/28/0.077/3.046e − 0819/60/0.167/4.622e − 0721/64/0.172/6.831e − 07x710511/28/0.071/3.045e − 0819/60/0.158/4.622e − 0721/64/0.161/6.831e − 07Table 5
Numerical results on Problem5.
Init (n)LJJ CGPMPDYATTCGPItr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗x110419/77/0.060/7.476e − 0727/145/0.101/8.743e − 07152/787/0.578/7.450e − 07x210423/93/0.069/7.916e − 0742/226/0.158/8.372e − 07204/1047/0.744/6.883e − 07x310419/77/0.057/5.319e − 0728/151/0.103/5.112e − 07170/876/0.610/8.184e − 07x410420/81/0.062/9.327e − 0727/144/0.103/5.294e − 07196/1008/0.699/6.472e − 07x510420/81/0.061/5.548e − 0726/138/0.098/8.101e − 07185/952/0.653/7.283e − 07x610420/81/0.062/9.324e − 0726/138/0.101/8.724e − 07196/1008/0.685/6.471e − 07x710419/77/0.057/5.444e − 0726/138/0.096/6.535e − 07160/827/0.566/6.203e − 07x15×10419/77/0.296/8.171e − 0725/133/0.487/5.286e − 07141/732/2.615/9.222e − 07x25×10421/85/0.321/7.117e − 0726/139/0.503/5.201e − 07194/997/3.551/8.547e − 07x35×10419/77/0.294/4.696e − 0727/146/0.528/9.869e − 07171/882/3.184/5.748e − 07x45×10419/77/0.297/9.870e − 0731/167/0.613/6.829e − 07185/953/3.447/8.111e − 07x55×10419/77/0.294/8.999e − 0728/150/0.543/6.333e − 07175/902/3.215/8.894e − 07x65×10419/77/0.293/9.870e − 0731/167/0.612/6.807e − 07185/953/3.397/8.111e − 07x75×10419/77/0.293/5.847e − 0726/138/0.505/9.469e − 07151/782/2.788/7.611e − 07x110519/77/0.598/8.828e − 0730/162/1.216/4.748e − 07146/758/5.586/5.631e − 07x210520/81/0.635/8.295e − 0725/133/0.993/9.282e − 07185/952/6.975/9.531e − 07x310519/77/0.596/4.759e − 0726/137/1.025/6.434e − 07162/837/6.107/6.288e − 07x410519/77/0.595/8.785e − 0727/144/1.100/8.273e − 07176/908/6.668/9.015e − 07x510519/77/0.598/8.971e − 0729/156/1.160/4.997e − 07168/867/6.344/9.717e − 07x610519/77/0.604/8.784e − 0727/143/1.074/7.645e − 07176/908/6.682/9.015e − 07x710519/77/0.604/6.266e − 0727/143/1.072/6.767e − 07143/742/5.435/8.344e − 07Table 6
Numerical results on Problem6.
Init (n)LJJ CGPMPDYATTCGPItr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗x110436/110/0.204/9.949e − 0729/117/0.075/9.510e − 0724/97/0.057/9.931e − 07x210411/34/0.026/5.671e − 0725/101/0.063/7.366e − 0725/101/0.058/8.628e − 07x310437/112/0.051/7.576e − 0752/209/0.090/9.275e − 0734/137/0.056/8.435e − 07x410420/62/0.041/7.165e − 0717/69/0.047/7.590e − 0716/65/0.032/2.846e − 07x510439/118/0.069/7.898e − 0760/241/0.127/9.418e − 0735/141/0.074/9.479e − 07x610420/62/0.042/7.506e − 0717/69/0.047/7.591e − 0716/65/0.029/2.849e − 07x710435/107/0.067/7.787e − 0731/125/0.076/8.371e − 0731/125/0.055/6.861e − 07x15×10437/113/0.376/6.935e − 0730/123/0.413/7.657e − 0723/93/0.288/8.910e − 07x25×10412/37/0.134/2.594e − 0725/101/0.333/6.388e − 0725/101/0.309/7.727e − 07x35×10437/112/0.281/7.576e − 0752/209/0.490/9.275e − 0734/137/0.311/8.435e − 07x45×10417/53/0.190/7.679e − 0718/73/0.256/5.629e − 0716/65/0.160/5.993e − 07x55×10439/118/0.365/7.898e − 0760/241/0.678/9.417e − 0735/141/0.389/9.479e − 07x65×10417/53/0.188/6.669e − 0718/73/0.259/5.629e − 0716/65/0.158/5.995e − 07x75×10435/107/0.352/9.085e − 0729/117/0.390/9.530e − 0730/121/0.286/8.457e − 07x110537/113/0.783/7.173e − 0731/130/0.890/8.423e − 0723/93/0.589/7.944e − 07x210512/37/0.280/3.661e − 0724/97/0.661/9.622e − 0725/101/0.628/7.387e − 07x310537/112/0.558/7.576e − 0752/209/1.006/9.275e − 0734/137/0.639/8.435e − 07x410517/53/0.377/6.159e − 0719/79/0.566/7.772e − 0716/65/0.334/8.412e − 07x510539/118/0.732/7.898e − 0760/241/1.351/9.417e − 0735/141/0.785/9.479e − 07x610517/53/0.374/6.332e − 0719/79/0.572/7.772e − 0716/65/0.335/8.413e − 07x710535/107/0.698/9.072e − 0730/123/0.816/8.816e − 0730/121/0.599/7.940e − 07Table 7
Numerical results on Problem7.
Init (n)LJJ CGPMPDYATTCGPItr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗x11041/4/0.007/0.000e + 0013/66/0.081/2.258e − 0719/116/0.142/8.849e − 07x210412/49/0.061/2.701e − 0712/61/0.077/6.969e − 0717/103/0.122/8.750e − 07x310410/41/0.026/5.307e − 0711/56/0.030/4.009e − 0714/85/0.041/8.584e − 07x410415/62/0.072/2.133e − 0716/83/0.086/4.425e − 0726/159/0.091/4.365e − 07x510415/59/0.066/5.118e − 0716/84/0.088/4.508e − 0718/106/0.101/8.553e − 07x610415/62/0.072/2.138e − 0716/83/0.085/4.515e − 0726/155/0.087/9.530e − 07x710415/62/0.073/2.133e − 0716/83/0.085/4.425e − 0726/159/0.089/4.365e − 07x15×1041/4/0.032/0.000e + 0015/81/0.524/3.287e − 0720/122/0.733/6.965e − 07x25×10412/49/0.308/6.039e − 0713/66/0.409/3.777e − 0718/109/0.629/6.887e − 07x35×10410/41/0.115/5.307e − 0711/56/0.157/4.009e − 0714/85/0.220/8.584e − 07x45×10415/62/0.371/4.775e − 0715/78/0.454/5.902e − 0714/92/0.297/0.000e + 00x55×10415/59/0.317/5.108e − 0716/84/0.487/4.212e − 0718/106/0.497/8.553e − 07x65×10415/62/0.375/4.777e − 0715/78/0.470/5.902e − 0715/97/0.344/0.000e + 00x75×10415/62/0.413/4.775e − 0715/78/0.575/5.902e − 0714/92/0.319/0.000e + 00x11051/4/0.065/0.000e + 0015/81/1.075/4.648e − 0720/122/1.634/9.850e − 07x210512/49/0.642/8.540e − 0713/66/1.206/5.342e − 0718/109/1.369/9.740e − 07x310510/41/0.301/5.307e − 0711/56/0.390/4.009e − 0714/85/0.484/8.584e − 07x410515/62/0.765/6.754e − 0715/78/0.923/8.346e − 0718/158/0.959/0.000e + 00x510515/59/0.627/5.107e − 0716/84/0.893/4.176e − 0718/106/0.994/8.553e − 07x610515/62/0.752/6.755e − 0715/78/0.926/8.346e − 0723/158/0.999/0.000e + 00x710515/62/0.759/6.754e − 0715/78/0.962/8.346e − 0718/158/0.941/0.000e + 00Table 8
Numerical results on Problem8.
Init (n)LJJ CGPMPDYATTCGPItr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗Itr/NF/Tcpu/F∗x110426/78/0.111/7.185e − 0725/99/0.130/7.229e − 0725/90/0.117/2.263e − 07x210425/76/0.106/9.350e − 0725/102/0.137/4.114e − 0725/90/0.127/2.314e − 07x310427/82/0.126/2.437e − 0725/102/0.132/7.922e − 0725/90/0.120/2.314e − 07x410427/82/0.116/7.926e − 0722/87/0.119/8.501e − 0725/90/0.123/2.321e − 07x510427/82/0.115/4.741e − 0725/102/0.136/4.135e − 0725/90/0.121/2.312e − 07x610427/82/0.118/8.941e − 0722/87/0.115/8.500e − 0725/90/0.121/2.321e − 07x710428/84/0.119/3.899e − 0722/87/0.115/9.315e − 0725/90/0.117/2.265e − 07x15×10428/84/0.604/1.713e − 0725/102/0.691/9.284e − 0726/93/0.638/2.269e − 07x25×10429/88/0.622/2.518e − 0728/118/0.791/4.552e − 0726/93/0.637/2.296e − 07x35×10428/85/0.599/3.829e − 0728/118/0.793/5.796e − 0726/93/0.638/2.296e − 07x45×10428/85/0.605/2.585e − 0727/113/0.767/6.928e − 0726/93/0.637/2.299e − 07x55×10427/82/0.584/9.499e − 0727/114/0.769/9.214e − 0726/93/0.642/2.295e − 07x65×10428/85/0.608/2.580e − 0727/113/0.764/6.927e − 0726/93/0.641/2.299e − 07x75×10426/78/0.555/4.908e − 0728/116/0.792/4.679e − 0726/93/0.641/2.271e − 07x110526/78/1.127/8.270e − 0726/109/1.495/5.490e − 0726/93/1.302/2.271e − 07x210527/81/1.160/4.953e − 0731/141/1.881/7.790e − 0727/96/1.320/2.292e − 07x310528/84/1.218/2.377e − 0731/141/1.874/5.803e − 0727/96/1.338/2.292e − 07x410528/85/1.212/6.599e − 0733/141/1.895/3.126e − 0727/96/1.332/2.294e − 07x510528/84/1.230/3.886e − 0730/137/1.810/8.937e − 0727/96/1.333/2.291e − 07x610528/85/1.233/6.600e − 0733/141/1.884/3.132e − 0727/96/1.336/2.294e − 07x710525/75/1.111/8.550e − 0728/121/1.715/7.546e − 0727/96/1.391/2.273e − 07


## 4.2. Numerical Testing Reports

The specific results of the numerical tests are displayed in Tables1–8. Their corresponding performance profiles are plotted in Figures 1–3 in terms of Tcpu, NF, and Itr, respectively. From Tables 1–8 and Figures 1–3, the following results are obtained.(i)
The LJJ CGPM, PDY, and ATTCGP can solve all test problems completely, which shows that abovementioned three methods are effective.(ii)
From Figures1–3, the LJJ CGPM is the top performer among the three algorithms, and this implies that the LJJ CGPM is superior to the PDY and ATTCGP totally, at least for this set of numerical experiments.Figure 1
Performance profiles on Tcpu.Figure 2
Performance profiles on NF.Figure 3
Performance profiles on Itr.Reasonably, the advantages of the LJJ CGPM are attributed to the choice of techniques (6) and (11) for the conjugate parameter of search direction and the adaptive line search strategy. At the first glance, the parameter βkLJJ (6) and line search strategy (11) are a bit complicated; however, the LJJ CGPM is very effective for solving constrained equations. Furthermore, numerical results indicate that the proposed method is competitive to similar methods for large-scale problems.


## 5. Application in Compressed Sensing

In this section, the LJJ CGPM is applied to a typical compressed sensing scenario and compared with the PDY method [22] in terms of the mean of squared error (MSE), iterative number, and CPU time. The parameters for the LJJ CGPM are set as follows: ρ=0.55,β=10,σ=10−4,andμ=1.25,ν=2.6. Also, the parameters of the PDY method are taken from Section 4 in [22].


### 5.1. Compressed Sensing

Compressed sensing, also called compressed sampling or sparse sampling, is a typical signal sampling technique. In electronic engineering, especially in signal processing, compressed sensing is often used to acquire and reconstruct sparse or compressible signals.In this section, the attention is focused on recovering the unknown vectorx0∈ℝn from an incomplete or disturbed observation(46)b=Ax0+e,where b∈ℝk is the observation data, A∈ℝk×nk≪n is a linear mapping, and e∈ℝk is an error term. In fact, the sparse result of x0 can be obtained by solving the following convex unconstrained optimization problem:(47)minx∈ℝnτx1+12Ax−b22,where parameter τ>0, ⋅1 and ⋅2 mean the l1−norm and l2−norm, respectively. There are many algorithms to solve problem (47). One of them is the gradient projection method for sparse reconstruction proposed by Figueiredo et al. [39]. The first step of this method is to convert (47) into a convex quadratic programming problem.For any vectorx∈ℝn, it can be decomposed into the following two parts:(48)x=u−v,u≥0,v≥0,where ui=maxxi,0 and vi=max−xi,0 for all i=1,2,⋯,n. In this way, the l1−norm of a vector can be expressed as ⋅1=enTu+enTv, where enT=1,1,⋯,1∈ℝn. Therefore, problem (47) can be expressed as the following convex quadratic programming problem with nonnegative constraints(49)minu,v12b−Au−v22+τenTu+τenTv,where u≥0,v≥0. Furthermore, it is easy to get the following standard form:(50)minz≥012zTHz+cTz,where(51)z=uv,c=τen−ATbτen+ATb,H=ATA−ATA−ATAATA.Obviously,H is a semipositive definite matrix. So, problem (50) is a convex quadratic programming problem. In [40], it has been proven to be equivalent to(52)Fz=minz,Hz+c=0,z≥0,where the function F is vector valued, and the “min” is explained as componentwise minimum. From [40] Lemma 3, and [41] Lemma 2.2, we know that F:ℝ2n⟶ℝ2n is Lipschitz continuous and monotone. So, equation (52) can be solved by the LJJ CGPM.


### 5.2. Numerical Results

In these experiments, the main goal is to recover a one-dimensional sparse signal from its limited measurements with Gaussian noise, that is, to reconstruct an−length sparse signal from kk≪n observations. The Gaussian noise distributed with N0,10−4 is used in the test. n=212 and k=210 are taken for equation (46). The quality of restoration is measured by the MSE, namely,(53)MSE=1nx∗−x0,where x∗ is the restored signal and x0 is the original signal. Besides, the original signal x0 contains 24 nonzero elements randomly. The random matrix A is given by the command rand n,k in Matlab, and the observed data b is obtained by b=Ax0+e, where e is the Gaussian noise. fx=τx1+1/2Ax−b22 denotes the merit function, and the value τ is obtained by the same continuation technique for the abovementioned two algorithms (LJJ CGPM and PDY method).The iterative process starts at the measurement signal, that is,x0=ATb, and terminates when the relative change between successive iterative falls below 10−5, that is,(54)fxk−fxk−1fxk−1<10−5.Figure4 gives the original signal x0, the measurement b, and the signal x∗ reconstructed by the LJJ CGPM and PDY method. It can be seen from Figure 4 that the LJJ CGPM and PDY method almost completely recover the disturbed signal. Moreover, to visualize the performance of the two algorithms, Figures 5 and 6 are plotted, which describe the curves of MSE and objective function values over iterations and CPU time (seconds), respectively. As can be seen from Figures 5 and 6, the LJJ CGPM has advantages over the PDY method in the MSE and objective function values, indeed, and the red curves (LJJ CGPM) drop faster than the blue curves (PDY method). Overall, simple tests show that the LJJ CGPM can effectively decode sparse signals in compressed sensing.Figure 4
From top to bottom: the original signal, the measurement, and the reconstructed signal by two algorithms. (a) Original (n = 4096, number of nonzeros = 16), (b) measurement, (c) LJJ CGPM (MSE = 3.81e − 07, Iter = 42, time = 0.66 s), and (d) PDY (MSE = 3.21e − 07, Iter = 61, time = 1.20 s).
(a)(b)(c)(d)Figure 5
Plots of MSE versus iteration and CPU time.
(a)(b)Figure 6
Plots of the objective function value versus iteration and CPU time.
(a)(b)


## 5.1. Compressed Sensing

Compressed sensing, also called compressed sampling or sparse sampling, is a typical signal sampling technique. In electronic engineering, especially in signal processing, compressed sensing is often used to acquire and reconstruct sparse or compressible signals.In this section, the attention is focused on recovering the unknown vectorx0∈ℝn from an incomplete or disturbed observation(46)b=Ax0+e,where b∈ℝk is the observation data, A∈ℝk×nk≪n is a linear mapping, and e∈ℝk is an error term. In fact, the sparse result of x0 can be obtained by solving the following convex unconstrained optimization problem:(47)minx∈ℝnτx1+12Ax−b22,where parameter τ>0, ⋅1 and ⋅2 mean the l1−norm and l2−norm, respectively. There are many algorithms to solve problem (47). One of them is the gradient projection method for sparse reconstruction proposed by Figueiredo et al. [39]. The first step of this method is to convert (47) into a convex quadratic programming problem.For any vectorx∈ℝn, it can be decomposed into the following two parts:(48)x=u−v,u≥0,v≥0,where ui=maxxi,0 and vi=max−xi,0 for all i=1,2,⋯,n. In this way, the l1−norm of a vector can be expressed as ⋅1=enTu+enTv, where enT=1,1,⋯,1∈ℝn. Therefore, problem (47) can be expressed as the following convex quadratic programming problem with nonnegative constraints(49)minu,v12b−Au−v22+τenTu+τenTv,where u≥0,v≥0. Furthermore, it is easy to get the following standard form:(50)minz≥012zTHz+cTz,where(51)z=uv,c=τen−ATbτen+ATb,H=ATA−ATA−ATAATA.Obviously,H is a semipositive definite matrix. So, problem (50) is a convex quadratic programming problem. In [40], it has been proven to be equivalent to(52)Fz=minz,Hz+c=0,z≥0,where the function F is vector valued, and the “min” is explained as componentwise minimum. From [40] Lemma 3, and [41] Lemma 2.2, we know that F:ℝ2n⟶ℝ2n is Lipschitz continuous and monotone. So, equation (52) can be solved by the LJJ CGPM.


## 5.2. Numerical Results

In these experiments, the main goal is to recover a one-dimensional sparse signal from its limited measurements with Gaussian noise, that is, to reconstruct an−length sparse signal from kk≪n observations. The Gaussian noise distributed with N0,10−4 is used in the test. n=212 and k=210 are taken for equation (46). The quality of restoration is measured by the MSE, namely,(53)MSE=1nx∗−x0,where x∗ is the restored signal and x0 is the original signal. Besides, the original signal x0 contains 24 nonzero elements randomly. The random matrix A is given by the command rand n,k in Matlab, and the observed data b is obtained by b=Ax0+e, where e is the Gaussian noise. fx=τx1+1/2Ax−b22 denotes the merit function, and the value τ is obtained by the same continuation technique for the abovementioned two algorithms (LJJ CGPM and PDY method).The iterative process starts at the measurement signal, that is,x0=ATb, and terminates when the relative change between successive iterative falls below 10−5, that is,(54)fxk−fxk−1fxk−1<10−5.Figure4 gives the original signal x0, the measurement b, and the signal x∗ reconstructed by the LJJ CGPM and PDY method. It can be seen from Figure 4 that the LJJ CGPM and PDY method almost completely recover the disturbed signal. Moreover, to visualize the performance of the two algorithms, Figures 5 and 6 are plotted, which describe the curves of MSE and objective function values over iterations and CPU time (seconds), respectively. As can be seen from Figures 5 and 6, the LJJ CGPM has advantages over the PDY method in the MSE and objective function values, indeed, and the red curves (LJJ CGPM) drop faster than the blue curves (PDY method). Overall, simple tests show that the LJJ CGPM can effectively decode sparse signals in compressed sensing.Figure 4
From top to bottom: the original signal, the measurement, and the reconstructed signal by two algorithms. (a) Original (n = 4096, number of nonzeros = 16), (b) measurement, (c) LJJ CGPM (MSE = 3.81e − 07, Iter = 42, time = 0.66 s), and (d) PDY (MSE = 3.21e − 07, Iter = 61, time = 1.20 s).
(a)(b)(c)(d)Figure 5
Plots of MSE versus iteration and CPU time.
(a)(b)Figure 6
Plots of the objective function value versus iteration and CPU time.
(a)(b)


## 6. Conclusions

In this work, based on the research studies for the CGMs and some common inexact line searches, a modified conjugate parameter and an adaptive line search strategy are proposed, and then, an effective CGPM is presented for monotone nonlinear equations. Besides, the search direction of our proposed algorithm possesses the properties of sufficient descent and trust region independent of any line search techniques. Furthermore, the presented method inherits the advantages of the CGM with a simple iterative form, rapid convergence, being derivative-free, and low memory requirements. Under some suitable conditions, the global convergence of the proposed method is established, and its numerical experiments are conducted and reported, which show that our method is very effective in dealing with large-scale monotone nonlinear equations with convex constraints and theℓ1−norm regularization problem in compressive sensing.Regrettably, the parameterμ in a new adaptive line search strategy is fixed. Along with the existing line search rule, our future work is to develop the line search (11) by a dynamic adjustment technology instead of the constant μ. Moreover, summarizing the abovementioned observations and from the fact that the search direction is yielded by the CGM, how to improve the line search rule by combining with the frequently-used inexact line searches for the CGM deserves to be studied.



---

*Source: 8323865-2020-07-09.xml*