# A New Modified Efficient Levenberg–Marquardt Method for Solving Systems of Nonlinear Equations

Authors: Zhenxiang Wu; Tong Zhou; Lei Li; Liang Chen; Yanfang Ma
Journal: Mathematical Problems in Engineering
 (2021)

---

## Abstract

For systems of nonlinear equations, a modified efficient Levenberg–Marquardt method with new LM parameters was developed by Amini et al. (2018). The convergence of the method was proved under the local error bound condition. In order to enhance this method, using nonmonotone technique, we propose a new Levenberg–Marquardt parameter in this paper. The convergence of the new Levenberg–Marquardt method is shown to be at least superlinear, and numerical experiments show that the new Levenberg–Marquardt algorithm can solve systems of nonlinear equations effectively.


---

## Body

## 1. Introduction

Consider the system of nonlinear equations(1)Fx=0,where the function Fx:Rn⟶Rm is continuously differentiable. In this paper, we assume that the solution set of (1) (denoted by X∗) is nonempty, with ⋅ referring to the 2-norm.Newton method is an important method to solve system (1) in [1]. At each iteration, it uses the trial step(2)dkN=−Jk−1Fk,where Fxk=Fk and Jk=F′xk is Jacobian matrix. When Jx is Lipschitz continuous and nonsingular, then the convergence of this method is quadratic at the solution. However, trial step dkN may not exist and Jk is singular or near singular. Newton method may not be well defined. To overcome this difficulty, Levenberg–Marquardt (LM) method was created by Levenberg [2] and Marquardt [3] which uses the trial step dkLM at each iteration, where(3)dkLM=−JkTJk+λkI−1JkTFk,and λk is a nonnegative constant. By introducing a nonnegative parameter λk, LM method overcomes the problem that Jk is singular or near singular; furthermore, excessive step size dk is avoided. In this case, where λk=0 and Jacobian matrix Jk is nonsingular, the LM method is reduced to Newton method.The efficiency of the LM method is affected by the parameterλk. For example, let λk=Fxk2, under the local error bound condition, the LM method is shown to have quadratic convergence by Yamashita and Fukushima in [4]. However, when the sequence xk is far away from the set X∗, Fk may be very big which may lead to large λk. It will result in a smaller LM step size, further reducing the efficiency of the algorithm. In [5], Fan used λk=μkFxkδ,δ∈1,2, where μk is updated with trust region technology in each iteration, the LM method also has quadratic convergence under some suitable conditions, and λk=μkFxkδ can alleviate the effect of the initial point being far away from the set X∗.To avoid this trouble, Amini usedλk=μkFk/1+Fk in [6]; when the sequence xk is far from the solution set and Fk is very large, λk is close to μk, which effectively controls the range of λk. Umar proposed some new LM parameters λk=Jk/Jk2, λk=JkTJk/JkTJk2 in [7]. Wang used λk=ηkJkTFkα with ηk updated by trust region techniques from iteration to iteration in [8].Ma introducedJkTFk into LM method and used a new LM parameter λk=θFk+1−θJkTFk in [9], where 0≤θ≤1. It is noticeable that λk is a convex combination of Fk and JkTFk, and the quadratic convergence of this method is proved. There are numerous other various LM methods to solve (1); interested readers are referred to [10–12] for related work. In order to discuss the range of parameter λk, inspired by [6, 8, 9], in this paper, we choose a new LM parameter as follows:(4)λk=μkθFkδ1+Fkδ+1−θJkTFkδ1+JkTFkδ,δ∈0,2,θ∈0,1,where λk is a convex combination of Fkδ/1+Fkδ and JkTFkδ/1+JkTFkδ and μk is updated with trust region technology in each iteration.Now, we set(5)ϕx=Fx2as the merit function for (1). We define the actual reduction and the predicted reduction of ϕx at the kth iteration as follows:(6)Aredk=Fk2−Fxk+dk2,(7)Predk=Fk2−Fk+Jkdk2,where dk is computed by (3). The following ratio is(8)rk=AredkPredk.Grippo applied the nonmonotone line search technique to Newton’s method in [13]; some authors have extended the nonmonotone techniques to trust region algorithm and proposed a lot of effective nonmonotone trust region methods in [14, 15]. And Amini proposed nonmonotone line search technique for the LM method in [6]. Numerical experiments show that the algorithm with the nonmonotone technique is more efficient than the algorithm without the nonmonotone technique. Inspired by these theories, we apply a nonmonotone strategy to LM method in this paper. Let us replace actual reduction (6) with the following actual reduction:(9)A¯redk=Flk2−Fxk+dk2,where(10)Flk=max0≤j≤nkFk−j,k=0,1,…,nk=minN0,k, and N0 is a positive integer constant. Obviously, by this change, Fk+1 will be compared with the max0≤j≤nkFk−j in each iteration, further leading to affect the ratio. The ratio after the change is(11)r¯k=A¯redkPredk.It can be used to decide whether the trial step is accepted and update the trust region parameterμk.The paper is organized as follows. In Section2, we present a new algorithm and then prove the global convergence of the new algorithm under some conditions. In Section 3, under the local error bound condition, the convergence of the new Levenberg–Marquardt method is shown to be at least superlinear. In Section 4, the new algorithm is an effective algorithm, which is demonstrated by numerical results. At last, we give some conclusions in Section 5.


## 2. The Efficient Algorithm and Global Convergence

In this section, firstly, we present the new efficient LM algorithm and then prove the global convergence of the new algorithm.When sequencexk is close to the solution, the steps may be too large, so we require(12)μk≥m,in the new algorithm, where m is a positive constant, and this is implemented by Step 5.Lemma 1.
For allk∈N, we have(13)Predk≥JkTFkmindk,JkTFkJkTJk.Proof.
This proof is directly derived from the important theory given by Powell in [16].
From literature [6], the following lemma can be obtained.Lemma 2.
Suppose the sequencexk be generated by Algorithm 1, then the sequence Flk converges.Algorithm 1: A modified efficient Levenberg–Marquardt algorithm.
Input: given x0∈Rn,N0>0,μ0>m>0,ε>0,0<p0<p1<p2<1,k:=0Output:Step 1. If JkTFk≤ε, stop. Otherwise, setλk=μkθFkδ/1+Fkδ+1−θJkTFkδ/1+JkTFkδ,whereθ∈0,1,δ∈0,2.Step 2. Compute the search directiondk=JkTJk+λkI−1JkTFk.Step 3. By (7), (8), and (10), compute Predk,A¯redk,r¯k.Step 4. Setxk+1=xk+dk,r¯k≥p0xk,r¯k<p0Step 5. Choose μk+1 asμk+1=4μk,r¯k<p1μk,r¯k∈p1,p2maxμk/4,m,r¯k>p2Step 6. Set k:=k+1, and go to Step 1.Next we present some of the assumptions needed in the following content.Assumption 1.
(a)
Fx is continuously differentiable and Lipschitz continuous; i.e., there exists a positive constant L2 that makes(14)Fy−Fx≤L2y−x.(b)
Jx is Lipschitz continuous; i.e., there exists a positive constant L1 such that(15)Jy−Jx≤L1y−x.Lemma 3.
If Assumption1 holds, then we have(16)Fy−Fx−Jxy−x≤L1y−x2,(17)Jx≤L2.Proof.
The proof of (17) can be found in [17]. So, we only prove (16). Using mean value theorem, there exists z∈x,y that makes(18)Fy=Fx+Jxy−x,and hence,(19)Fy−Fx−Jxy−x=Jz−Jxy−x.
According to the last equation, we can obtain(20)Fy−Fx−Jxy−x≤Jz−Jxy−x≤L1z−xy−x≤L1y−x2.
So (16) is true.Theorem 1.
Suppose that Assumption1 is true. Then, Algorithm 1 terminates in finite iterations or satisfies(21)liminfk⟶∞JkTFk=0.Proof.
Assume that the theorem is incorrect; then, there exist a positive constantε0 and a constant k0∈N that makes(22)JkTFk≥ε0,∀k≥k0.
Firstly, we prove that(23)limk⟶∞Fxlk=limk⟶∞Fxk.
Sincedk is accepted by the algorithm, we have(24)Flk2−Fxk+dk2≥p0Predk.
This, along with (17), (22), and Lemma 1, for all k≥k0, that means(25)Flk2−Fk+12≥p0JkTFkmindk,JkTFkJkTJk≥p0ε0mindk,ε0L22.
Replacingk with lk−1, for all sufficiently large k, there is(26)Fllk−12−Flk2≥p0mindlk−1,ε0L22.
From Lemma2(27)limk⟶∞Fllk−12−Flk2=0,which together with the last inequality yields(28)limk⟶∞mindlk−1,ε0L22.
ε0/L12 is a positive constant, so(29)limk⟶∞dlk−1=0.
Using Assumption1, the last equality implies that(30)limk⟶∞Fxlk=limk⟶∞Fxlk−1.
Letl^k=lk+N0+2. Using induction, for all j≥1, we can show that(31)limk⟶∞dl^k−j=0.
Forj=1, we can have from (31) that (29) is true. Assuming that (29) is true for given j, we show that (29) holds for given j+1. Let k be large enough such that l^k−j+1>0. Substituting k with l^k−j−1 and using (24), we obtain(32)Fll^k−j−12−Fxl^k−j2≥p0Predl^k−j−1.
Similarly, we can deduce that(33)limk⟶∞dl^k−j−1=0.
Therefore (31) holds. Along with Assumption 1 we imply that(34)limk⟶∞Fxl^k−j−1=limk⟶∞Fxl^k−j.
Similarly, for any givenj≥1, we have limk→∞Fxl^k−j=limk→∞Fxlk. On the one hand, for any k, we have(35)xk+1=xl^k−∑j=1l^k−k−1dl^k−j,
Using (31) and the fact that l^k−j−1≤N0+1, we have(36)limk⟶∞xk+1−xl^k=0,
With Assumption1, we conclude(37)limk⟶∞Fxlk=limk⟶∞Fxl^k=limk⟶∞Fxk.
And then (22) is proved. By using (22) and (25), we can deduce that(38)limk⟶∞dk=0.
Then, it follows from (3) in Algorithm 1, (17), and (22) that(39)μk⟶∞,ask⟶∞.
On the other hand, by (6), (8), (17), (22), (38), and Lemma 1, we can deduce that(40)rk=AredkPredk=Fk2−Fk+12Predk=1+Fk+JkdkOdk2+Odk4Predk≤1+Fk+JkdkOdk2+Odk4JkTFkmindk,JkTFk/JkTJk≤1+Odk2dk⟶1.
And then combined with (8) and (10), we can see that(41)r¯k=A¯redkPredk=Flk2−Fxk+dk2Predk≥Fk2−Fk+12Predk=rk⟶1.
Considering Algorithm1, obviously, for all large k, there exists a positive constant μ¯>m such that μk<μ¯, which conflicts with (22) and so Theorem 1 is true.


## 3. Local Convergence

Definition 1.
LetN be a subset of Rn that makes N∩X∗≠∅. We say Fx provides a local error bounded on N for (1), where xk−x¯k=distxk,X∗, if there exists a positive constant C1 such that(42)Fx≥C1distx,X∗,∀x∈N.Assumption 2.
(a)
Fx is continuously differentiable, and Fx provides a local error bound on subset Nx∗,b for problem (1), where(43)Nx∗,b=x∈Rn|x−x∗≤b,0<b<1.(b)
FxandJxare both Lipschitz continuous onNx∗,b; that is, there exist two positive constantsL1, L2that make(44)Jy−Jx≤L1y−x,∀x,y∈Nx∗,b,(45)Fy−Fx≤L2y−x,∀x,y∈Nx∗,b,which implies(46)Fy−Fx−Jxy−x≤L1y−x2,∀x,y∈Nx∗,b.Lemma 4.
Suppose that Assumption2 is true. Then, for all sufficiently large k, we have the following.(1)
There exists a positive constantM>m that makes(47)μk≤M.(2)
λk≥C3xk−x¯kδ, where C3=C1δ/2mθ+C2δ/2m1−θ.Proof.
The proof process of (1) is the same as that of Lemma 3.2 in [6], so we are not going to prove it here and only give the proof of (2).(48)Fk2=FkTFk=FkTFx¯k+Jkxk−x¯k+FkTVk,where Vk=Fk−F¯k−Jkxk−x¯k, obviously,(49)FkTJkxk−x¯k=Fk2−FkTVk.
We can obtain from Definition1 and (45) and (46) that(50)JkTFkxk−x¯k≥C12xk−x¯k2−L1L2xk−x¯k3,JkTFk≥C12xk−x¯k−L1L2xk−x¯k2≥C2xk−x¯k.
Thus we obtain(51)JkTFkδ≥C2δxk−x¯kδ.
Then we show the following inequalities:(52)Fkδ1+Fkδ≥C1δ2xk−x¯kδ,JkTFkδ1+JkTFkδ≥C2δ2xk−x¯kδ.
IfFk≤1, then the following holds:(53)Fkδ1+Fkδ≥C1δ2xk−x¯kδ,and if Fk>1, then(54)Fkδ1+Fkδ>12.
Through the above two inequalities, we have(55)Fkδ1+Fkδ≥max12,C1δ2xk−x¯kδ.
Similarly, ifJkTFk≤1, then the following holds(56)JkTFkδ1+JkTFkδ≥JkTFkδ2≥C2δ2xk−x¯kδ,and if JkTFk>1, we have(57)JkTFkδ1+JkTFkδ>12.
Hence, we obtain(58)JkTFkδ1+JkTFkδ≥max12,C2δ2xk−x¯kδ.
From Algorithm1, (52), and (58), we have(59)λk=μkθFkδ1+Fkδ+1−θJkTFkδ1+JkTFkδ≥C1δ2mθxk−x¯kδ+C2δ2m1−θxk−x¯kδ≥C3xk−x¯kδ,where C3=C1δ/2mθ+C2δ/2m1−θ. So when xk−x¯k is sufficiently small, (60)λk≥C3xk−x¯kδ.Lemma 5.
Suppose that Assumption2 is true, for all sufficiently large k. dk computed by (3) satisfies(61)dk≤Oxk−x¯k.Proof.
If we setφkd=Fk+Jkd2+λkd2, then we have from (3) that dk is a minimizer of φkd, so it follows from Algorithm 1, (12), (45), (46), and Fx¯k=0 that(62)dk2≤1λkφkxk−x¯k=1λkFk+Jkxk−x¯k2+λkxk−x¯k2≤1C3xk−x¯kδFk+Jkxk−x¯k2+λkxk−x¯k2≤L12xk−x¯k4C3xk−x¯kδ+xk−x¯k2=Oxk−x¯k2.
So there is(63)dk≤Oxk−x¯k.Lemma 6.
Suppose that Assumption2 is true, for all sufficiently large k. So, we have(64)λk≤C4xk−x¯kδ.Proof.
Firstly, we deal with these two equations:(65)Fkδ1+Fkδ≤Fkδ≤L2δxk−x¯kδ,JkTFk≤JkTFk≤L2Fxk−Fx¯k≤L22xk−x¯k.
On the other hand,(66)JkTFkδ1+JkTFkδ≤JkTFkδ≤L22δxk−x¯kδ.
We conclude(67)λk=μkθFkδ1+Fkδ+1−θJkTFkδ1+JkTFkδ≤MθL2δxk−x¯kδ+1−θL22δxk−x¯kδ≤C4xk−x¯kδ,where C4=MθL2δ+M1−θL1δL2δ.
Without loss of generality, for allx¯∈Nx∗,b∩X∗, suppose that rankJx¯=r, and we prove the local convergence of Algorithm 1 by singular value decomposition (SVD) of Jx¯.(68)Jx¯=U¯1U¯2Σ¯10V¯1T,V¯2T,where Σ¯1=diagσ¯1,…,σ¯r with σ¯1≥σ¯2≥⋯≥σ¯r>0, rankΣ¯1=r, U¯=U¯1,U¯2, and V¯=V¯1,V¯2. And assume the SVD of Jx are as follows:(69)Jx=UΣVT=U1,U2,U3Σ1Σ20V1TV2TV3T=U1Σ1V1T+U2Σ2V2T,where U=U1,U2,U3 and V=V1,V2,V3 are two orthogonal matrixes. Σ1=diagσ1,σ2,…,σr, σ1≥σ2≥⋯≥σr>0, and Σ2=diagσr+1,σr+2,…,σr+q, σr+1≥σr+2≥⋯≥σr+q>0.
SinceJx is Lipschitz continuous, by the theory of matrix perturbation [18], we have(70)diagΣ1−Σ¯1,Σ2,0≤Jk−J¯k≤L1xk−x¯k.
So there is(71)Σ1−Σ¯1≤L1xk−x¯k,Σ2≤Jk−J¯k≤L1xk−x¯k.
Sincexk converges to the set X∗, then we have L1xk−x¯k≤σ¯r/2 hold for all sufficiently large k. So combined with (71), there is(72)Σ1−1≤1σ¯r−L1xk−x¯k≤2σ¯r.Lemma 7.
Suppose Assumption2 holds; for all sufficiently large k, we have(1)
U1U1TFk≤Oxk−x¯k(2)
U2U2TFk≤Oxk−x¯k2Proof.
The proof process is similar to Lemma7 in [9], so we omit it here.Theorem 2.
Under Assumption2, let xk be a sequence generated by Algorithm 1 with trust region technique. If δ∈0,1, then xk converges superlinearly to the solution. If δ∈1,2 , then sequence xk converges quadratically to the solution.Proof.
Using (3) and (69), we obtain(73)dk=−V1Σ12+λkI−1Σ1U1TFk−V2Σ22+λkI−1Σ2U2TFk,Fk+Jkdk=Fk−U1Σ1Σ12+λkI−1Σ1U1TFk−U2Σ2Σ22+λkI−1Σ2U2TFk=λkU1Σ12+λkI−1U1TFk+λkU2Σ22+λkI−1U2TFk.
From (47), (71), Definition 1, and Lemma 5, we have(74)Fk+Jkdk≤λkU1U1TFk+U2U2TFk≤C4xk−x¯kδOxk−x¯k+Oxk−x¯k2≤C4xk−x¯k1+δ+Oxk−x¯k2≤Oxk−x¯k1+δ+Oxk−x¯k2.
From (44), (46), (74), Definition 1, and Lemma 5, we conclude that(75)C1xk+1−x¯k+1≤Fxk+1=Fxk+dk≤Oxk−x¯k1+δ+Oxk−x¯k2.
On the other hand, it is obvious that(76)xk−x¯k=distxk,X∗≤xk+1−x¯k≤xk+1−x¯k+1+dk.
It follows from (75) and Lemma 5 that xk−x¯k≤2dk≤Oxk−x¯k holds for all sufficiently large k. So, dk=Oxk−x¯k, and this is related to (75). We deduce that if δ∈0,1, dk+1=Odk1+δ. If δ∈1,2, dk+1=Odk2. So if δ∈0,1, xk converges superlinearly to the solution; otherwise, if δ∈1,2, xk converges quadratically to the solution.


## 4. Numerical Experiments

In Section5, we compare the performance of Algorithm 1 with Algorithm 2.1 (writing Algorithm 2.1 as AELM) in [6] through some numerical experiments. The test function Fx¯ is improved by the method in [19]. The form is as follows:(77)Fx¯=Fx−Jx∗AATA−1ATx−x,where A∈Rn×k1≤k≤n has full column rank, and Fx∗=0. It is certain that(78)J¯x∗=Jx∗I−AATA−1AT,where test problems Fx are nonsingular test functions from [20]. We take A=1,1,…,1T∈Rn×1, rank of J¯x∗ as n−1, and choose(79)A=11⋯11−1⋯±1,rank of J¯x∗ as n−2.We set the following parameters in the algorithms:p0=10−4,p1=0.25,p1=0.75,N0=5,μ1=1,m=10−8, θ=0,0.5 or 1, and δ=1 or 2. The algorithms are terminated when JkTFk≤10−5 or the number of iterates exceeds 104. When θ=1 and δ=1, Algorithm 1 is reduced to the AELM.By numerical experiments, we find the numerical results of Algorithm1 are the same as the numerical results of AELM in some functions. So we only list the results of other experiments in the following tables. Further, we adopt the efficiency index defined as EI in [21] to compare the performance of algorithm AELM and Algorithm 1. The results of the four experiments with rankJx∗=n−1 are shown in Tables 1 and 2, and the results of the four experiments with rankJx∗=n−2 are shown in Tables 3 and 4, respectively. We use six starting points ±100x0, ±10x0, and ±x0 for each test problem, where x0 is suggested in [20].(i)
NF stands for the quantity of function calculations(ii)
NJ stands for the quantity of Jacobian calculations(iii)
’−’ indicates that the iteration number is more than 104(iv)
E.I.=ρ1/NF, where ρ is the convergence order of algorithm.Table 1
Results on singular nonlinear equations with rankF′x∗=n−1.
Problemnx0NF/NJAELMAlgorithm withδ=1Algorithm withδ=2θ=0θ=0.5θ=0θ=0.5θ=1Helical valley function3–1007/77/78/89/97/77/7–106/66/67/77/76/66/6–11/11/11/11/11/11/119/99/99/99/99/99/9108/88/88/89/98/88/81008/88/88/88/88/88/8Discrete boundary value10–10014/1414/1414/1414/1414/1414/14–1011/1110/1011/1110/1010/1010/10–18/88/88/87/77/77/713/33/33/34/44/44/41011/1110/1010/109/99/99/91009/99/99/99/99/99/9100–10014/1413/1314/1413/1314/1414/14–1011/119/911/118/89/99/9–112/125/55/54/44/44/414/44/44/43/33/33/3107/77/77/77/75/58/810012/1211/1112/1211/1112/1212/12500–10015/1513/1315/1512/1214/1415/15–108/86/67/75/56/66/6–16/65/55/54/44/44/411/11/11/11/11/11/1108/86/67/75/56/66/610015/1512/1214/1411/1113/1314/14Discrete integral equation30–10016/1616/1616/1616/1616/1616/16–1011/1111/1111/1111/1111/1111/11–19/99/99/99/99/99/917/77/77/76/66/66/61011/1111/1111/1111/1111/1111/1110010/1010/1010/1010/1010/1010/10100–10016/1616/1616/1616/1616/1616/16–1012/1212/1212/1212/1212/1212/12–19/910/1010/109/99/99/918/87/78/87/77/77/71012/1211/1111/1111/1111/1112/1210010/1010/1010/1010/1010/1010/10500–10017/1717/1717/1717/1717/1717/17–1013/1313/1313/1313/1313/1313/13–111/1111/1111/1111/1110/1010/1019/98/89/98/88/88/81012/1212/1212/1212/1212/1212/1210010/1010/1010/1010/1010/1010/10Table 2
Numerical results for singular nonlinear equations with rankF′x∗=n−1.
Problemnx0Iters/fnorm/trust/EIAELM (times: 3.63)δ=1 (times: 7.47)δ=2 (times: 3.76)Rosenbrock2116/1.5876e − 07/1/1.04416/2.3984e − 07/1/1.04416/2.3152e − 07/1/1.0711018/1.199e − 07/1/1.03818/1.211e − 07/1/1.03818/1.2102e − 07/1/1.062810021/1.3601e − 07/1/1.03421/1.3604e − 07/1/1.03421/1.3603e − 07/1/1.053Powell badly2140/2.0905e − 07/1/1.01740/6.4701e − 07/1/1.01740/4.8829e − 07/1/1.0271025/3.4159e − 07/1/1.02825/3.6737e − 07/1/1.02825/3.0308e − 07/1/1.04510030/3.4445e − 07/1/1.02330/3.1265e − 07/1/1.02330/3.0929e − 07/1/1.037Wood4116/1.5876e − 07/1/1.04416/2.0905e − 07/1/1.04416/2.0897e − 07/1/1.0711019/1.926e − 07/1/1.03719/1.9262e − 07/1/1.03719/1.9262e − 07/1/1.05910022/2.8391e − 07/1/1.03122/2.8391ee − 07/1/1.03222/2.8391e − 07/1/1.050Helical valley319/9.8336e − 07/0/1.0079/1.5533e − 09/0/1.00796/1.4071e − 09/0/1.128108/2.7959e − 07/0/1.0908/7.0428e − 07/0/1.0909/8.3791e − 14/0/1.1281008/2.0685e − 07/0/1.0908/1.2902e − 08/0/1.0908/1.1432e − 08/0/1.013Brown almost-linear1018/7.2842e − 07/1/1.0908/7.3653e − 05/1/1.0908/7.4202e − 05/1/1.0131023/7.9915e − 07/1/1.03023/7.9915e − 05/1/1.03023/7.9915e − 05/1/1.04810045/6.5326e − 07/1/1.01545/6.5326e − 05/1/1.01545/6.5326e − 05/11.024Discrete boundary value1013/0.00016727/0/1.2593/0.00016758/0/1.2594/1.1606e − 05/0/1.3161011/5.6012e − 07/0/1.06410/1.3863e − 05/0/1.0719/1.1763e − 05/0/1.1291009/9.2906e − 07/0/1.0799/8.9129e − 06/0/1.0799/8.7568e − 06/0//1.129Discrete integral equation3017/9.5122e − 07/1/1.1037/8.4521e − 06/1/1.1036/1.1785e − 05/1/1.0011011/9.8059e − 07/1/1.06411/8.1669e − 06/1/1.06411/7.3177e − 06/1/1.00210010/1.0601e − 07/0/1.07110/1.0609e − 09/0/1.07110/1.0606e − 09/0/1.116Trigonometric1018/5.2977e − 07/0/1.0088/2.4187e − 07/0/1.008804/6.2122e − 09/1/1.0011019/2.3813e − 07/0/1.037–/19.064/0/1540/2.7155e − 09/1/1.002100–/46.227e − 07/0/1–/29.583/0/119/62.146/1/1Variably dimensioned function10114/2.2087e − 07/0/1.05014/2.2087e − 05/1/1.05014/2.2087e − 05/1/1.0811016/1.1269e − 07/1/1.04416/1.1269e − 05/1/1.04416/1.1269e − 05/1/1.07110019/3.702e − 07/1/1.03719/3.702e − 05/1/1.03719/3.702e − 05/1/1.059Broyden tridiagonal3019/2.2916e − 07/1/1.0079/2.4133e − 05/1/1.0079/2.3888e − 05/1/1.1291014/1.419e − 05/1/1.05014/1.419e − 05/1/1.05014/1.419e − 05/1/1.08110017/2.5157e − 05/1/1.04117/2.5157e − 07/1/1.04117/2.5157e − 05/1/1.066Broyden banded30112/4.2271e − 07/1/1.05912/4.2313e − 06/1/1.05912/4.2305e − 06/1/1.0951018/4.5992e − 07/1/1.03918/4.5992e − 06/1/1.03918/4.5992e − 06/1/1.06210024/2.7182e − 07/1/1.02924/2.7182e − 06/1/1.02924/2.7182e − 06/1/1.046Table 3
Results on singular nonlinear equations with rankF′x∗=n−2.
Problemnx0NF/NJAELMAlgorithm withδ=1Algorithm withδ=2θ=0θ=0.5θ=0θ=0.5θ=1Powell badly scaled2–100852/717275/250169/1581802/15212585/2168366/318–1076/64163/14469/611595/13343865/3226585/484–1183/16797/79118/1032754/22962585/2168341/27911288/1076128/111127/105−996/833711/591103/392/763/37/47/41043/8731003/33/33/33/33/31954/1622Discrete boundary value10–10014/1414/1414/1414/1414/1414/14–1011/1110/1011/1110/1010/1010/10–18/88/88/87/77/77/713/33/33/34/44/44/41011/1110/1010/109/99/99/910010/1010/1010/1010/1010/1010/10100–10014/1413/1314/1413/1314/1414/14–1011/119/911/118/89/99/9–16/65/55/54/44/44/414/44/44/43/33/33/3107/79/97/77/78/88/810013/1312/1213/1311/1112/1213/13Discrete integral equation30–10016/1616/1616/1616/1616/1616/16–1011/1111/1111/1111/1111/1111/11–19/99/99/99/99/99/917/77/77/76/66/66/61011/1111/1111/1111/1111/1111/1110012/1212/1212/1212/1212/1212/12100–10016/1616/1616/1616/1616/1616/16–1012/1212/1212/1212/1212/1212/12–110/1010/1010/109/99/99/918/87/78/87/77/77/71012/1211/1111/1111/1111/1112/1210014/1414/1414/1414/1414/1414/14Table 4
Numerical results for singular nonlinear equations with rankF′x∗=n−2.
Problemnx0Iters/fnorm/trust/EIAELM (times: 4.655)δ=1 (times: 3.374)δ=2 (times: 3.614)Rosenbrock2111/4.6197e − 05/0/1.06511/4.6202e − 05/0/1.06511/4.6202e − 05/0/1.1051013/0.00010074/0/1.05413/0.00010074/0/1.05413/0.00010074/0/1.08810017/3.4089e − 05/0/1.04117/3.4089e − 05/0/1.04121/63/3.4089e − 05/0/1.066Powell badly211288/1.0433e − 07/0/1.000127/4.9595e − 07/1/1.008996/41.8112e − 07/0/1.001103/3.3583e − 05/0/1.2593/3.3562e − 05/0/1.0287/1.6053e − 05/0/1.1691003/0.0099779/0/1.2593/0.009978/0/1.0233/0.009978/0/1.442Wood4114/3.2876e − 06/0/1.05014/3.2877e − 06/0/1.08114/3.2877e − 06/0/1.0811017/3.0842e − 06/0/1.04117/3.0842e − 06/0/1.04117/3.0842e − 06/1/1.06610020/4.5433e − 06/0/1.03520/4.5433e − 06/0/1.03520/4.5433e − 06/0/1.056Helical valley3113/5.9421e − 06/1/1.05413/5.9553e − 06/0/1.05413/5.9694e − 06/0/1.0881014/2.087e − 06/1/1.05014/2.0897e − 06/0/1.05014/2.0934e − 06/1/1.08110018/3.0231e − 06/1/1.03918/3.1777e − 06/0/1.03918/3.3522e − 06/1/1.062Brown almost-linear1018/7.2842e − 05/1/1.1478/7.3653e − 05/1/1.0908/7.4202e − 05/0/1.1471023/7.9915e − 05/1/1.03023/7.9915e − 05/1/1.03023/7.9915e − 05/1/1.04810045/6.5326e − 05/1/1.01545/6.5326e − 05/1/1.01545/6.5325e − 05/1/1.024Discrete boundary value1013/0.0001676/0/1.2593/0.00016795/0/1.2594/1.1572e − 05/0/1.3161011/5.5963e − 06/0/1.06510/48/1.383e − 05/0/1.0659/1.2359e − 05/0/1.12910010/6.7629e − 08/0/1.07110/6.3555e − 08/0/1.07110/6.1475e − 08/0//1.116Discrete integral equation3017/9.5122e − 06/1/1.1047/8.4521e − 06/1/1.1046/1.1785e − 05/1/1.2001011/9.806e − 06/1/1.06511/8.167e − 06/1/1.06511/7.3178e − 06/1/1.10510012/1.3149e − 07/0/1.05912/11.8382e − 07/0/1.05912/1.137e − 09/0/1.095Trigonometric10113/2.2087e − 05/1/1.05413/9.9955e − 06/0/1.05414/7.3496e − 06/1/1.0011031/1.113e − 09/0/1.02222/2.4389e − 08/0/1.03216/1.8831e − 07/0/1.002100–/15.261/0/11/120.79/0/119/48/170.57/0/1Variably dimensioned function10114/2.2087e − 05/1/1.05014/2.2087e − 05/1/1.05014/2.2087e − 05/1/1.0811016/1.1269e − 05/0/1.04416/1.1269e − 05/0/1.04416/1.1269e − 05/0/1.07110019/3.702e − 05/0/1.03719/3.702e − 05/0/1.03719/3.702e − 05/0/1.059Broyden tridiagonal3019/2.294e − 05/1/1.0809/2.4133e − 05/1/1.0079/2.3903e − 05/1/1.1291014/1.4189e − 05/1/1.05014/1.419e − 05/1/1.05014/1.419e − 05/1/1.08110017/2.5157e − 05/1/1.04117/2.5157e − 07/1/1.04117/2.5157e − 05/1/1.066Broyden banded30112/4.2271e − 07/1/1.05912/4.2313e − 06/1/1.05912/4.2305e − 06/1/1.0951018/4.5992e − 07/1/1.03918/4.2297e − 06/11.03918/4.5984e − 06/1/1.06210024/2.7182e − 07/1/1.02924/2.7182e − 06/1/1.02924/2.7178e − 06/1/1.046It is shown in Table1 that when θ=0.5 and δ=2, the effect of Algorithm 1 is obviously better than that of AELM. Algorithm 1 wins 40.5% of the numerical results while AELM wins 2.38%, and 57.1% of the two algorithms have the same results. The advantage of Algorithm 1 is not obvious when θ=0.5 and δ=1. Algorithm 1 can win 19% of the numerical results while AELM win 7.14%, and 73.8% of two algorithms have the same result.Table3 shows that when θ=0.5 and δ=1, Algorithm 1 and AELM have the best experimental results. Algorithm 1 win 23.3% of the numerical results, and 76.6% of the two algorithms has the same result. The advantage of Algorithm 1 is not obvious when θ=0.5 and δ=2. Algorithm 1 wins 40% of the numerical results while AELM wins 20%, and 40% of the two algorithms has the same results.Further, we adopt the EI and letθ=0.5 in the experiment. Tables 2 and 4 show that when δ=1, the experimental data EI of AELM and Algorithm 1 are similar, but when δ=1, the EI of Algorithm 1 is obviously larger than that of AELM. In addition, in terms of the experimental time, except when ranking F′x∗=n−2 and δ=1, the execution time of Algorithm 1 is longer than that of AELM. In other cases, the execution time of Algorithm 1 is close to or less than that of AELM.In general, it is shown that for most test problems, Algorithm1 performs better than AELM. So it can indicate that Algorithm 1 is more efficient than AELM to solve systems of nonlinear equations.


## 5. Conclusion

In this paper, we propose a new LM algorithm by modifying the LM parameter for systems of nonlinear equations. Through numerical experiments, we find the calculation amounts of Algorithm1 smaller than AELM in the case where θ and δ take some suitable value, which shows the effectiveness of the new Algorithm 1. Under some conditions, the global convergence of the new LM method is proved, and the local convergence of the new LM method is shown to be at least superlinear. Numerical results show that the new algorithm is efficient.



---

*Source: 5608195-2021-08-31.xml*