# An Improved Line Search Filter Method for the System of Nonlinear Equations

Authors: Zhong Jin; Yuqing Wang
Journal: Journal of Applied Mathematics
 (2012)

---

## Abstract

An improved line search filter algorithm for the system of nonlinear equations is presented. We divide the equations into two groups, one contains the equations that are treated as equality constraints and the square of other equations is regarded as objective function. Two groups of equations are updated at every iteration in the works by Nie (2004, 2006, and 2006), by Nie et al. (2008), and by Gu (2011), while we just update them at the iterations when it is needed indeed. As a consequence, the scale of the calculation is decreased in a certain degree. Under some suitable conditions the global convergence can be induced. In the end, numerical experiments show that the method in this paper is effective.


---

## Body

## 1. Introduction

Many applied problems are reduced to solve the system of nonlinear equations, which is one of the most basic problems in mathematics. This task has applications in many scientific fields such as physics, chemistry, and economics. More formally, the problem to be solved is stated as follows(1.1)ci(x)=0,i=1,2,…,m,
where each ci:Rn→R(i=1,2,…,m) is a smooth function.A well-known method for solving nonlinear equations is the Newton method, an iterative scheme which is locally quadratical convergent only if the initial iteration is sufficiently close to the solution. To improve the global properties, some important algorithms [1] for nonlinear equations proceed by minimizing a least square problem:
(1.2)minh(x)=c(x)Tc(x),
which can be also handled by the Newton method, while Powell [2] gives a counterexample to show a dissatisfactory fact that the iterates generated by the above least square problem may converge to a nonstationary point of h(x).Traditionally, a penalty or augmented Lagrange function is always used as a merit function to test the acceptability of the iterates. However, as we all know, there are several difficulties associated with the use of penalty function, and in particular the choice of the penalty parameter. Too low a choice may result in an optimal solution that cannot be obtained, on the other hand, too large a choice damps out the effect of the objective function. Hence, filter method has been first introduced for constrained nonlinear optimization problems in a sequential quadratic programming (SQP) trust-region algorithm by Fletcher and Leyffer [3], offering an alternative to merit functions, as a tool to guarantee global convergence in algorithms for nonlinear programming (NLP). The computational results are very promising in [3]. Furthermore, Fletcher et al. [4, 5] give the global convergence of the trust-region filter-SQP method, then Ulbrich [6] gets its superlinear local convergence. Consequently, filter method has been actually applied in many optimization techniques [7–13], for instance the pattern search method [7], the SLP method [8], the interior method [9], the bundle approaches [10, 11], and so on. Also combined with the trust-region search technique, Gould et al. extended the filter method to the system of nonlinear equations (1.1) and nonlinear least squares in [14], and to the unconstrained optimization problem with multidimensional filter technique in [15]. In addition, Wächter and Biegler [16, 17] presented line search filter methods for nonlinear equality constrained programming and the global and local convergence were given.Recently, some other ways were given to attack the problem (1.1) (see [18–22]). There are two common features in these papers, one is the filter approach that is utilized, and the other is that at every iteration the system of nonlinear equations is transformed into a constrained nonlinear programming problem and the equations are divided into two groups, some equations are treated as constraints and the others act as the objective function. For instance combined with the filter line search technique [16, 17], the system of nonlinear equations in [21] at the kth iteration xk becomes the following optimization problem with equality constraints:
(1.3)min∑i∈S1k‍ci2(x)s.t.cj(x)=0,j∈S2k,
where the sets S1k and S2k are defined as S1k={ij∣j≤n0} and S2k={ij∣j≥n0+1} for some positive constant n0>0 such that ci12(xk)≥ci22(xk)≥⋯≥cin2(xk).Motivated by the ideas and methods above, we propose an improved line search filter method for the system of nonlinear equations. We also divide the equations into two groups, one contains the equations that are treated as equality constraints and the square of other equations is regarded as objective function. Two groups of equations are updated at every iteration in those works [18–22], while we just update them at the iterations when it is needed indeed. Specifically, using similar transformed optimization problem (1.3), the difference between our method with [21] is that the sets S1k and S2k in (1.3) need not be updated at every iteration. As a consequence, the scale of the calculation is decreased in a certain degree. In our algorithm two groups of equations cannot be changed after a f-type iteration, thus in the case that |𝒜|<∞, the two groups are fixed after finite number of iterations. In addition, the filter should not be updated after a f-type iteration, so naturally the global convergence is discussed, respectively, according to whether the number of the updated filter is infinite or not. And it is shown that every limit point of the sequence of iterates generated by the algorithm is the ϵ solution to (1.1) or a local infeasible point when |𝒜|<∞. Furthermore, the global convergent property is induced under mild assumptions. In the end, numerical experiments show that the method in this paper is effective.The paper is outlined as follows. In Section2, the line search filter method is developed and its key ingredients are described. In Section 3 we prove that under suitable conditions, the method is well defined and globally convergent. Finally, some numerical results are given in Section 4.


## 2. Description of the Algorithm

To solve the system of nonlinear Equation (1.1), we also transform it into the optimization problem (1.3) where equations are divided into two groups as the one in [21], then let mk(x)=∥cS1k(x)∥22=∑i∈S1k‍ci2(x) and θk(x)=∥cS2k(x)∥22=∑i∈S2k‍ci2(x). The linearization of the KKT condition of (1.3) at the kth iteration xk is as follows:
(2.1)(BkAS2kk(AS2kk)T0)(skλk+)=-(g(xk)cS2k(xk)),
where Bk is the Hessian or approximate Hessian matrix of L(x,λ)=mk(x)+λTcS2k(x), AS2kk=∇cS2k(xk) and g(xk)=∇mk(xk).After a search directionsk has been computed, a step size αk,l∈(0,1] is determined in order to obtain the trial iteration
(2.2)xk(αk,l)=xk+αk,lsk.
More precisely, for fixed constants γm,γθ∈(0,1), we say that a trial step size αk,l provides sufficient reduction with respect to the current xk if
(2.3)θk(xk(αk,l))≤(1-γθ)θk(xk)ormk(xk(αk,l))≤mk(xk)-γmθk(xk).For the sake of a simplified notation, we define the filter in this paper not as a list but as a setℱk⊆[0,∞]×[0,∞] containing all (θ,m)-pairs which are prohibited in iteration k. We say that a trial point xk(αk,l) is acceptable to the filter if its (θ,m)-pair does not lie in the taboo region, that is, if
(2.4)(θ(xk(αk,l)),m(xk(αk,l)))∉ℱk.
At the beginning of the optimization, the filter is initialized to be empty: ℱ0=∅. Throughout the optimization the filter is then augmented in some iterations after the new iterate xk+1 has been accepted. For this, the following updating formula is used:
(2.5)ℱk+1=ℱk∪{(θ,m)∈R2:θ≥(1-γθ)θk(xk),m≥mk(xk)-γmθk(xk)}.Similar to the traditional strategy of the filter method, to avoid obtaining a feasible point but not an optimal solution, we consider the followingf-type switching condition:
(2.6)uk(αk,l)<0,-uk(αk,l)>δ[θk(xk)]sθ,
where uk(αk,l)=αk,lgkTsk, δ>0 and sθ∈(0,1).When Condition (2.6) holds, the step sk is a descent direction for current objective function. Then, instead of insisting on (2.3), the Armijo-type reduction condition is employed as follows:
(2.7)mk(xk(αk,l))≤mk(xk)+τ3uk(αk,l),
where τ3∈(0,1/2) is a fixed constant.If (2.6) and (2.7) hold for the accepted trial step size, we may call it an f-type point, and accordingly this iteration is called an f-type iteration. An f-type point should be accepted as xk+1 with no updating of the filter, that is,
(2.8)ℱk+1=ℱk.
While if a trial point xk(αk,l) does not satisfy the switching condition (2.6) but satisfies (2.3), we call it an h-type point (or accordingly an h-type iteration). An h-type point should be accepted as xk+1 with updating of the filter.In the situation, where no admissible step size can be found, the method switches to a feasibility restoration stage, whose purpose is to find a new iterate that satisfies (2.3) and is also acceptable to the current filter by trying to decrease the constraint violation. In order to detect the situation where no admissible step size can be found and the restoration phase has to be invoked, we define that
(2.9)αkmin={min{γθ,γm[θk(xk)]sθ-gkTsk},ifgkTsk<0,γθ,otherwise.We are now ready to formally state the overall algorithm for solving the the system of nonlinear Equation (1.1).Algorithm 2.1.
We have the following steps.
Step 1. Initialization: Choose an initial pointx0∈Rn, 0<ρ1<ρ2<1 and ϵ>0. Compute g0, ci(x0), S10, S20 and Ak for i∈S20. Set k=0 and ℱ0=∅.
Step 2. If∥c(xk)∥≤ϵ then stop. Otherwise compute (2.1) to obtain sk. If there exists no solution to (2.1), go to Step 6. If ∥sk∥≤ϵ then stop.
Step 3. Letαk,l=1 and l=0. Compute αkmin by (2.9).
Step 3.1. Ifαk,l<αkmin, go to Step 6. Otherwise compute xk(αk,l)=xk+αk,lsk. If xk(αk,l)∈ℱk, go to Step 3.2.Step3.1.1. Case 1. The switching condition (2.6) holds. If the reduction condition (2.7) holds, set xk+1=xk(αk,l), ℱk+1=ℱk, S1k+1=S1k, S2k+1=S2k and go to Step 5. Otherwise, go to Step 3.2.Step3.1.2. Case 2. The switching condition (2.6) is not satisfied. If (2.3) holds, set xk+1=xk(αk,l), augment the filter using (2.5) and go to Step 4. Otherwise, go to Step 3.2.
Step 3.2. Chooseαk,l+1∈[ρ1αk,l,ρ2αk,l]. Set l=l+1 and go to Step 3.1.
Step 4. ComputeS1k+1 and S2k+1 by (1.3). If (θk+1(xk+1),mk+1(xk+1))∈ℱk+1, set S1k+1=S1k and S2k+1=S2k.
Step 5. Computegk+1, Bk+1 and Ak+1. Go to Step 2 with k replaced by k+1.
Step 6. (Feasibility Restoration Stage) Findxkr=xk+αkrskr such that xkr is accepted by the filter and the infeasibility θ is reduced. Go to Step 2.


## 3. Global Convergence of Algorithm

In the reminder of this paper we denote the set of indices of those iterations in which the filter has been augmented by𝒜⊆ℕ. Let us now state the assumptions necessary for the global convergence analysis.Assumption 3.1.
The sequence{xk} generated by Algorithm 2.1 is nonempty and bounded.Assumption 3.2.
The functionsci(x), j=1,2,…,m are all twice continuously differentiable on an open set containing X.Assumption 3.3.
There exist two constantsb≥a>0 such that the matrices sequence {Bk} satisfy a∥s∥2≤sTBks≤b∥s∥2 for all k and s∈Rn.Assumption 3.4.
(
A
s
2
k
k
)
T has full column rank and ∥sk∥≤γs for all k with a positive constant γs.By Assumption3.1 we have Σi=1nci2(xk)≤Mmax. Since 0≤mk(xk)≤mk(xk)+θk(xk)=Σi=1nci2(xk) and 0≤θk(xk)≤mk(xk)+θk(xk)=Σi=1nci2(xk), then (θ,m) associated with the filter is restricted to
(3.1)ℬ=[0,Mmax]×[0,Mmax].Theorem 3.5.
If Algorithm2.1 terminates at Step 2, then an ϵ solution to (1.1) is achieved or a local infeasibility point is obtained.Proof.
The proof of this theorem can be found in [21].Next we can assume that our algorithm does not terminate finitely and an infinite sequence of points is generated.Lemma 3.6.
Under the above assumptions, there exists a solution to (2.1) with exact (or inexact) line search which satisfies the following descent conditions,
(3.2)|θk(xk+αsk)-(1-2α)θk(xk)|≤τ1α2∥sk∥2,(3.3)|mk(xk+αsk)-mk(xk)-uk(α)|≤τ2α2∥sk∥2,
where α∈(0,1), τ1, and τ2 are all positive constants independent of k.Proof.
By virtue of Taylor expansion ofci2(xk+αsk) with i∈S2, we obtain
(3.4)|ci2(xk+αsk)-ci2(xk)-2αci(xk)∇ci(xk)Tsk|=|ci2(xk+αsk)-ci2(xk)-2ci(xk)∇ci(xk)T(αsk)|=|12(αsk)T[2ci(xk+ζαsk)∇ci2(xk+ζαsk)+2∇ci(xk+ζαsk)∇ci(xk+ζαsk)T](αsk)|=|α2skT[ci(xk+ζαsk)∇ci2(xk+ζαsk)+∇ci(xk+ζαsk)∇ci(xk+ζαsk)T]sk|≤1mτ1α2∥sk∥2,
where the last inequality follows the above assumptions and ζ∈[0,1]. Also, from (2.1) we immediately get that ci(xk)+∇ci(xk)Tsk=0, that is, -2αci2(xk)-2αci(xk)∇ci(xk)Tsk=0. With |S2|≤m, thereby,
(3.5)|θk(xk+αsk)-(1-2α)θk(xk)|=|∑i∈S2(ci2(xk+αsk)-(1-2α)ci2(xk))|≤∑i∈S2|ci2(xk+αsk)-(1-2α)ci2(xk)|=∑i∈S2|ci2(xk+αsk)-ci2(xk)-2αci(xk)∇ci(xk)Tsk|≤m·1mτ1α2∥sk∥2≤τ1α2∥sk∥2,
then (3.2) consequently holds.
According to Taylor expansion ofΣi∈S1(ci2(xk+αsk)) (i.e., mk(xk+αsk)), we then have
(3.6)|∑i∈S1(ci2(xk+αsk))-∑i∈S1(ci2(xk))-αgkTsk|=|12α2(sk)T∇2∑i∈S1(ci2(xk+ραsk))sk|≤τ2α2∥sk∥2,
where the last inequality follows the first two assumptions and ρ∈[0,1]. That is to say,
(3.7)|mk(xk+αsk)-mk(xk)-uk(α)|≤τ2α2∥sk∥2,
which is just (3.3).Theorem 3.7.
If there are only finite or infinite number of iterates entering the filter, then(3.8)limk→∞θ(xk)=0.Proof.
Consider the following.
Case1 (if |𝒜|=∞). The proof is by contraction. Suppose that there exists an infinite subsequence {ki} of 𝒜 such that
(3.9)θki(xki)≥ε,
for some ε>0. At each iteration ki, (θki(xki),mki(xki)) is added to the filter which means that no other (θ,m) can be added to the filter at a later stage within the area
(3.10)[θki(xki)-γθθki(xki),θki(xki)]×[mki(xki)-γmθki(xki),mki],
and the area of each of these squares is at least γθγmε2.
Thus theℬ is completely covered by at most a finite number of such areas in contraction to the infinite subsequence {ki} satisfying (3.9). This means that (3.8) is true.
Case2 (if |𝒜|<∞). From |𝒜|<∞, we know the filter updates in a finite number, then there exists K∈ℕ, for k>K the filter does not update. As h-type iteration and restoration algorithm all need the updating of the filter, then for k>K our algorithm only executes the f-type iterations.
BecauseS1 and S2 do not change in the f-type iteration, then for k≥K+1 we have θk(x)=θK(x) and mk(x)=mK(x). Thus we obtain
(3.11)∑k=K+1∞(mk(xk)-mk(xk+1))=∑k=K+1∞(mK(xk)-mK(xk+1)).
From the switching condition (2.6) we know that
(3.12)τ3uk(αk,l)=τ3αk,lgkTsk<-τ3δ[θk(xk)]γθ,
together with the reduction condition (2.7) we get
(3.13)mk(xk(αk,l))≤mk(xk)+τ3uk(αk,l)≤mk(xk)-τ3δ[θk(xk)]γθ,
and since xk+1=xk(αk,l) for k≥K+1, this reduces to
(3.14)mk(xk+1)=mk(xk(αk,l))≤mk(xk)-τ3δ[θk(xk)]γθ.
Suppose to the contrary that there exists an infinite subsequence{ki} of 𝒜 such that
(3.15)θki(xki)≥ε,
for some ε>0. Choose an arbitrary subsequence {k-i} of {ki} satisfying k-i>K, we find that
(3.16)+∞>∑k=K+1∞(mK(xk)-mK(xk+1))≥∑i=1∞(mK(xk-i)-mK(xk-i+1))≥∑i=1∞τ3δ[θk-i(xk-i)]γθ→+∞,
which is a contraction.Lemma 3.8.
Let{xki} be a subsequence of iterates for which (2.6) hold and have the same S1 and S2. Then there exists a α^∈(0,1] such that
(3.17)mki(xki+α^ski)≤mki(xki)+α^τ3gkiTski.Proof.
Because{xki} have the same S1 and S2, mki(x) are fixed. And dki is a decent direction due to (2.6). Hence there exists a α^∈(0,1] satisfying (3.17).Lemma 3.9.
If there existsK-∈ℕ such that for k>K- it holds ∥sk∥≥(1/2)ε1, where ε1>0, then there exist K1∈ℕ and a constant ε0>0, for k>K1 such that
(3.18)gkTsk≤-ε0.Proof.
From (2.1) we obtain gk+AS2kλk++Bksk=0 and cS2k+(AS2k)Tsk=0, so
(3.19)gkTsk=-skTBksk-(λk+)TcS2k≤-a∥sk∥2-(λk+)TcS2k≤-a∥sk∥2+c1∥cS2k∥.
As limk→∞θ(xk)=0 implies limk→∞∥cS2k∥=0, then there exists K1(∈N)≥K-, for k>K1 such that ∥cS2k∥≤(aε12)/(8c1). Thus
(3.20)gkTsk≤-a∥sk∥2+c1∥cS2k∥≤-a(12ε1)2+c1(aε128c1)=-ε0,
with ε0=(aε12)/(8c1).Theorem 3.10.
Suppose that{xk} is an infinite sequence generated by Algorithm 2.1 and |𝒜|<∞, one has
(3.21)limk→∞∥cS2kk∥+∥sk∥=0.
Namely, every limit point is the ϵ solution to (1.1) or a local infeasible point. If the gradients of ci(xk) are linear independent for all k and i=1,2,…,m, then the solution to (1.1) is obtained.Proof.
As was noted in the proof of the case 2 in Theorem3.7, from |𝒜|<∞, the algorithm only executes the f-type iterations. Now the proof is by contraction. If the result is false, there should have been a constant ε1>0 and a subsequence {xki} of {xk} such that
(3.22)∥cS2kiki∥+∥ski∥>ε1,
for all ki. Without loss of generality, we can suppose there are only f-type iterations for k≥k1. Moreover, there exist the following results for sufficiently large ki.
Aslimk→∞θ(xk)=0, it is apparent that ∥cS2kiki∥≤(1/2)ε1 hold for large enough ki. It is reasonable that we have ∥ski∥≥(1/2)ε1 from (3.22), so similar to the Lemma 3.9, we can get gkiTski≤-ε0 for large enough ki. Without loss of generality, suppose gkiTski≤-ε0 for all ki.
BecauseS1 and S2 do not change in the f-type iteration, then for all ki we have θki(x)=θk1(x) and mki(x)=mk1(x). Since the f-type point satisfies the reduction condition (2.7), that is, mk(xk)-mk(xk+1)>-τ3αgkTsk, then it holds that
(3.23)∑i=1∞(mki(xki)-mki(xki+1))=∑i=1∞(mk1(xki)-mk1(xki+1))>∑i=1∞(-τ3αgkiTski).
By mk(xk) is decreasing for k≥k1 and Lemma 3.8, we get
(3.24)+∞>∑k=k1∞(mk(xk)-mk(xk+1))>∑i=1∞(mki(xki)-mki(xki+1))>∑i=1∞(-α^τ3gkiTski)>∑i=1∞α^τ3ε0→+∞,
which is a contraction.
If the gradients ofci(xk) are linear independent for all k and i=1,2,…,m, then the solution to (1.1) is obtained by virtue of the result in [23]. Thus, the result is established.Lemma 3.11.
IfgkTsk≤-ε0 for a positive constant ε0 independent of k, then there exists a constant α->0, for all k and α≤α- such that
(3.25)mk(xk+αsk)-mk(xk)≤τ3uk(α).Proof.
Letα-=((1-τ3)ε0)/τ2γs2. From (3.3), ∥sk∥≤γs and α≤α-, we obtain
(3.26)mk(xk+αsk)-mk(xk)-uk(α)≤τ2α2∥sk∥2≤τ2αα-γs2=(1-τ3)αε0≤-(1-τ3)αgkTsk=-(1-τ3)uk(α),
which shows that (3.25) is true.Lemma 3.12.
Under all existing assumptions, ifuk(α)≤-αε0 for a positive constant ε0 independent of k and α≥αk,lmin for all α∈(0,1] with (θk(xk),mk(xk))∉ℱk, then there exist γ1, γ2>0 such that (θk(xk+αsk),mk(xk+αsk))∉ℱk for all k and α≤min{γ1,γ2θk(xk)}.Proof.
Chooseγ1=ε0/τ2γs2, then α≤γ1 implies that -αε0+τ2α2γs2≤0. So from (3.3), we obtain
(3.27)mk(xk+αsk)≤mk(xk)+uk(α)+τ2α2∥sk∥2≤mk(xk)-αε0+τ2α2γs2≤mk(xk).
And chooseγ2=2/τ1γs2, then α≤γ2θk(xk) implies that -2αθk(xk)+τ1α2γs2≤0. It follows from (3.2) that
(3.28)θk(xk+αsk)≤θk(xk)-2αθk(xk)+τ2α2∥sk∥2≤θk(xk)-2αθk(xk)+τ1α2γs2≤θk(xk).
We further point out a fact according to the definition of filter. If(θ-,m-)∉ℱk and θ≤θ-, m≤m-, we note that (θ,m)∉ℱk. So from (θk(xk),mk(xk))∉ℱk, mk(xk+αsk)≤mk(xk) and θk(xk+αsk)≤θk(xk), we get (θk(xk+αsk),mk(xk+αsk))∉ℱk.Lemma 3.13.
If there existsK-∈ℕ such that for k>K- it holds ∥sk∥≥(1/2)ε1 for a constant ε1>0, then there exists K∈ℕ so that for all k>K the f-type iterate is always satisfied.Proof.
By Lemma3.9, there exists K1∈ℕ and a constant ε0>0, for k>K1 such that gkTsk≤-ε0. From the definition of αkmin, when gkTsk≤0, we have αkmin=min{γθ,(δ[θk(xk)]sθ)/(-gkTsk)}. As limk→∞θ(xk)=0, we have limk→∞(δ[θk(xk)]sθ)/ε0=0. So there exists K2 (>K1) ∈ℕ such that for k>K2 it holds
(3.29)δ[θk(xk)]sθ-gkTsk<δ[θk(xk)]sθε0<γθ,
which implies for k>K1 we have αkmin=min{γθ,(δ[θk(xk)]sθ)/(-gkTsk)}=(γm[θk(xk)]sθ)/(-gkTsk).
Becauselimk→∞θ(xk)=0, there exists K (>K2) ∈ℕ such that for k>K it follows
(3.30)θk(xk)<min{α-γ2,γ1γ2,[ρ1γ2ε0δ]1/(sθ-1)},
which shows that
(3.31)γ2θk(xk)≤min{α-,γ1,γ2θk(xk)},δ[θk(xk)]sθε0<ρ1γ2θk(xk).
Moreover, K>K2>K1 implies gkTsk≤-ε0 for all k>K.
Letβk=γ2θk(xk), by Lemmas 3.11 and 3.12, for α≤βk we obtain
(3.32)mk(xk+αsk)-mk(xk)≤τ3uk(α),(θk(xk+αsk),mk(xk+αsk))∉ℱk.
If we now denote with αk,L the first trial size satisfying (3.32), the backtracking line search procedure then implies that for α≥αk,L(3.33)α≥ρ1βk=ρ1γ2θk(xk)>δ[θk(xk)]sθε0,
from which it follows that
(3.34)α>δ[θk(xk)]sθε0>δ[θk(xk)]sθ-gkTsk>min{γθ,δ[θk(xk)]sθε0}=αkmin,-uk(α)>αε0>δ[θk(xk)]sθ.
This means that αk,L and all previous trial step sizes are f-step sizes and larger than αkmin. Therefore, αk,L is the accepted step size αk indeed. Since the trial point xk(αk,L)∉ℱk satisfies the switching condition (2.6) and the reduction condition (2.7), the iteration is a f-type iteration for each k>K. The claim is true.Theorem 3.14.
Suppose that{xk} is an infinite sequence generated by Algorithm 2.1 and |𝒜|=∞, then
(3.35)limk→∞inf[∥cS2kk∥+∥sk∥]=0.
Namely, it has an accumulation which is the ϵ solution to (1.1) or a local infeasible point. If the gradients of ci(xk) are linear independent for all k and i=1,2,…,m, then the solution to (1.1) is obtained.Proof.
Suppose by contraction that there should have been a constantε1>0 such that
(3.36)∥cS2kk∥+∥sk∥>ε1,
for all k. Furthermore, there exists the following results for sufficiently large k.
Aslimk→∞θ(xk)=0, it is apparent that ∥cS2kk∥≤(1/2)ε1 holds for large enough k. It is reasonable that we have ∥sk∥≥(1/2)ε1 for large enough k from (3.36). Therefore there exists K-∈ℕ such that for k>K- it holds ∥sk∥≥(1/2)ε1 for a constant ε1>0, then by Lemma 3.13, we know that there exists K∈ℕ such that for all k>K the f-type iterate is always satisfied, which is a contraction to |𝒜|=∞.
Similar to Theorem3.10, if the gradients of ci(xk) are linear independent for all k and i=1,2,…,m, then the solution to (1.1) is also obtained. The whole proof is completed.


## 4. Numerical Examples

In this section, we develop the implementation of Algorithm2.1 in order to observe its performance on some illustrative examples. In the whole process, the program is coded in MatLab with exact line search. The first example is from [2], which converges to a nonstationary point if the least squares approach is employed. The second example comes from [24], which Newton method fails to solve, and the final problem is given from [25]. In the tables, the notations NIT, NOF, and NOG mean the number of iterates, number of functions and number of gradients, respectively.Example 4.1.
Find a solution of the nonlinear equations system:(4.1)(x10xx+0.1+2y2)=(00).
The unique solution is(x*,y*)=(0,0). It has been proved in [2] that, under initial point (x0,y0)=(3,1), the iterates converge to the point z=(1.8016,0.0000), which is not a stationary point. Utilizing our algorithm, a sequence of points converging to (x*,y*) is obtained. We assume the error tolerance ϵ in this paper is always 1.0e-5. The detailed numerical results for Example 4.1 are listed in Table 1.Table 1
Numerical results of Example4.1.
Starting point
NIT
NOF
NOG
(
3,1
)
6
12
10
(
6,2
)
9
17
14
(
9,3
)
12
24
21Example 4.2.
Find a solution of the nonlinear equations system as follows:(4.2)(x+3y2(x-1.0)y)=(00).
The only solution of Example4.2 is (x*,y*)=(0,0). Define the line Γ={(1,y):y∈ℝ}. If the starting point (x0,y0)∈Γ, the Newton method [24] is confined to Γ. We choose two starting points which are belong to Γ in the experiments and then the (x*,y*) is obtained. The numerical results of Example 4.2 are given in Table 2.Table 2
Numerical results of Example4.2.
Starting point
NIT
NOF
NOG
(
1,0
)
2
4
8
(
1,2
)
11
18
15Example 4.3.
Consider the following system of nonlinear equations:(4.3)f1(x)=x12+x1x2+2x22-x1-x2-2,f2(x)=2x12+x1x2+3x22-x1-x2-4.
There are three solutions of above example,(1,1)T, (-1,1)T, (1,-1)T. The numerical results of Example 4.3 are given in Table 3.Table 3
Numerical results of Example4.3.
Starting point
(
0.5,0.5
)
(
-
0.5,0.5
)
(
0.5
,
-
0.5
)
NIT
5
9
7
NOF
10
12
14
NOG
9
15
10
Solution
(
1,1
)
(
-
1,1
)
(
1
,
-
1
)Example 4.4.
Consider the system of nonlinear equations:(4.4)fi(x)=-(N+1)+2xi+∑j=1,j≠iN‍xj,i=1,2,…,N-1,fN(x)=-1+∏j=1N‍xj,
with the with the initial point xi(0)=0.5, i=1,2,…,N. The solution to Example 4.4 is x*=(1,1,…,1)T. The numerical results of Example 4.4 are given in Table 4.
All results summarized show that our proposed algorithm is practical and effective.Table 4
Numerical results of Example4.4.
N
=
5
N
=
10
N
=
15
N
=
30
N
=
50
NIT
6
8
14
19
36
NOF
8
10
16
21
40
NOG
7
12
15
20
38



---

*Source: 273141-2012-11-06.xml*