# A New Modified Levenberg–Marquardt Method for Systems of Nonlinear Equations

Authors: Liang Chen; Yanfang Ma
Journal: Journal of Mathematics
 (2023)

---

## Abstract

Taking a new choice of the LM parameterλk=μkJkTFkδ with δ∈0,2, we give a new modified Levenberg–Marquardt method. Under the error bound condition cdistw,S≤JwTFw which is weaker than the nonsingular of Jacobian Jw, we present that the new Levenberg–Marquardt method has at least superlinear convergence when δ∈0,1 and quadratic convergence when δ∈1,2, respectively, which indicates that our new Levenberg–Marquardt method is performed for the systems of nonlinear equations. Also, numerical experiments indicate its efficiency on a set of systems of nonlinear equations.


---

## Body

## 1. Introduction

Nonlinear systems of equations are of interest to engineers, physicists, mathematicians, and other scientists because the modulizations of many nonlinear problems arising in different fields of science are made using a system of nonlinear equations. Consider the following systems of nonlinear equations(1)Fw=0,where F:Rn⟶Rm is a continuously differentiable function, and w∈Rn is a vector. We denote by S the set of solutions for (1), which is assumed to be nonempty. Let · be the Euclidean norm, here we use 2-norm, both for vectors and matrices.There are many classical methods for solving the systems of nonlinear equations (1), including, to name a few of them, the Gauss–Newton method, the inexact Newton method, Broyden’s method, and the Levenberg–Marquardt method [1]. In this paper, we are mainly concerned with the Levenberg–Marquardt method (hereinafter referred to as the LM method) which was proposed by Levenberg [2] and Marquardt [3]. At every iteration, the trial step of the LM method is obtained by computing equations as follows:(2)JkTJk+λkId=−JkTFk,where Fk=Fwk, Jk=Jwk is the Jacobian at wk, I∈Rn×n denotes the identity matrix, and λk≥0 is the LM parameter. If Jk is nonsingular and Lipschitz continuous for the case m=n, the LM method is shown to have quadratic convergence provided that the initial point w0 is sufficiently close to the solution w∗ of (1) and the LM parameter λk is updated by an adaptive rule.As we all know, the LM method is widely used to solve nonlinear least-squares problems besides solving systems of nonlinear equations. Taking the LM parameterλk=JkTFkδ with δ∈0,1, Behling et al. [4] propose a modified LM (BMLM) method for solving the nonzero-residue nonlinear least-squares problems. And numerical experiments show that the BMLM method is efficient. But it might be low efficient for systems of nonlinear equations.Example 1.
The Powell singular function is given by(3)Fw1,w2,w3,w4=w1+10w2,5w3−w4,w2−2w32,10w1−w42.
Let the initial point isw0=1,−3,0,1T suggested by Moré et al. [5]. Set the stopping criterion as JkTFk<10−5. We test it by using the LM method proposed by Behling et al. in [4] and choose three choices of δ. The results are listed in Table 1.
From Table1, we note that the BMLM method is not perfect for the Powell singular function because there are more iteration steps, especially δ=10−4. So, the BMLM method is not always performed for the systems of nonlinear equations.
From equation (2), we observe that the LM parameter λk helps to overcome problematic cases where JkTJk is singular or nearly singular. Increasing λk decreases step size, and vice versa. Hence, if a step is unacceptable, λk should be increased until a smaller, acceptable step is found. If a step is accepted, we want to increase step size by decreasing λk, in order to proceed more quickly in the correct descent direction, speeding up convergence rate. Thus, the LM parameter plays a key role in the performance of the LM method. By choosing a suitable parameter λk, the LM method acts like the gradient descent method whenever the current iteration is far from a solution w∗ and behaves similar to the Gauss–Newton method if the current iteration is close to w∗. For this reason, several choices of the LM parameter have been given, for instance, Fk2 [6], Fkδ with δ∈1,2 [7], OJkTFk [8], JkTFk [9], θFk+1−θJkTFk with θ∈0,1 [10], and θFkδ+1−θJkTFkδ with θ∈0,1, δ∈1,2 [11], to name only a few. For more choices, readers can refer to [12–16].
For some applications, the condition of the nonsingularity of Jacobian is too strong, and some efforts have been made by using weaker conditions. For instance, a local error bound condition is proposed by Yamashita and Fukushima [6] which is weaker than nonsingularity, and the Hölderian local error bound condition is given by Vui [17] and Ngai [18] which is a generalization of the local error bound condition. Under these conditions, the LM method converges at least linearly and solves many nonlinear problems, such as the nonlinear least-squares problems [4], the systems of nonlinear equations [19–21], the linear complementarity problem [22], the system identification [23, 24], and other problems [25–28].
In [4], to establish the local convergence, Behling et al. propose an error-bound condition upon the gradient of the nonlinear least-squares function as follows:(4)cdistw,S≤JwTFw,where c is a positive constant, ∀w∈Nw∗,b with Nw∗,b⊂Rn is a neighbourhood of w∗, and distw,S is the distance from w to the set S. And the LM method is proved to have at least linearly convergence order. The error-bound condition (4) does not require the full rank of the Jacobian in a neighborhood of a stationary point. It is also weaker than the nonsingularity of Jacobian, and it can be derived from the local error bound condition ([13], Lemma 5.4).
In this paper, to solve the systems of nonlinear equations (1), we propose a new modified LM method by extending the LM parameter given by Behling et al. as(5)λk=μkJkTFkδ,where δ∈0,2, that is, we extend the exponent of their LM parameter from 0,1 to 0,2. Using the trust region technique, under the error bound condition (4) which is the same as the condition used by Behling et al., the convergence of our new LM method will also be investigated. Since the convergence criterion of the LM method is liminfk⟶+∞JkTFk=0, we always make sure that JkTFk is sufficiently small. It means that the LM method will be the Gauss–Newton method when k is sufficiently large.
The outline of this paper is organized as follows. In the next section, we present the proposed LM algorithm and analyze the local convergence under the error-bound condition (4). The global convergence is discussed in Section 3. Numerical experiments are tested using the proposed LM algorithm in Section 4. Finally, we draw a conclusion in Section 5.Table 1
Results for Powell singular function with initial pointw0=1,−3,0,1T, stopping criterion JkTFk<10−5, λk=JkTFkδ with δ=10−4 (left), δ=0.5 (middle), and δ=1 (right).
ItersJkTFkFunctionItersJkTFkFunctionItersJkTFkFunction12.2939e +021.4663e +0112.2939e +021.4663e +0112.2939e +021.4663e +0122.7810e +013.3467e +0023.1599e +014.4743e +0029.1950e +018.7473e +0033.4658e +009.9547e −0134.6835e +002.3177e +0054.0764e +002.5941e +0069.8048e −021.4095e −0179.2544e −021.3340e −0185.9456e −015.3381e −01149.9650e −032.9959e −02105.0828e −031.8397e −02104.0460e −027.8054e −02489.6615e −046.2663e −03134.8857e −043.8486e −03132.5177e −041.9426e −031979.9776e −051.3761e −03167.5193e −051.1112e −03143.8748e −055.1794e −048889.9937e −062.9662e −04216.8285e −062.2647e −04155.5404e −061.3449e −04


## 2. The Levenberg–Marquardt Method and Its Local Convergence

In this section, to solve the system of the nonlinear equations (1), we consider the LM method with the LM parameter as (5). By using trust region technology, we investigate its convergence rate near a solution.Take(6)Φw=Fw2,as the merit function for (1). The actual reduction and predicted reduction of Φ at the k-th iteration can be defined as(7)Aredk=Fk2−Fwk+dk2,Predk=Fk2−Fk+Jkdk2,respectively, where dk is obtained by (2). It follows from the result given by Powell [29] that, for all k,(8)Predk≥JkTFkmindk,JkTFkJkTJk.By trust region technique, to make sure whether accept the stepdk and how to update the LM parameter μk, we need the following ratio:(9)rk=AredkPredk.Algorithm 1: The new LM method.
Step 1. Givenw1∈Rn, 0<α2<1<α1, 0<m≤μ1, 0<p0≤p1≤p2<1,0<δ≤2, k≔1.Step 2. IfJkTFk=0, stop. Compute (5) to obtain the LM parameter. Solve the equation (2) to obtain dk.Step 3. Computerk=Aredk/predk. Setwk+1=wk+dk,ifrk≥p0,wk,otherwise.Step 4. Updateμk+1 asμk+1=α1μk,ifrk<p1,μk,ifrk∈p1,p2,maxα2μk,m,ifrk>p2.Setk:=k+1 and go to Step 2.Firstly, we give some assumptions which will be needed throughout this paper.Assumption 1.
(a)
Jw is Lipschitz continuous on Nw∗,b, i.e., there exists a positive constant L1 that makes(10)Jw−Jv≤L1w−v,∀w,v∈Nw∗,b.(b)
JwTFw provides a local error bound (w.r.t. S) on Nw∗,b, i.e., there exists a positive constant c such that(11)cdistw,S≤JwTFw,∀w∈Nw∗,b.From Assumption1 and the mean value inequality, we derive the following lemma without proof.Lemma 1.
Let Assumption1 holds, then we get(12)Fw−Fv−Jvw−v≤L1w−v2,(13)Fw−Fv≤L2w−v,(14)Jw≤L2,Fw≤β,where constants L2>0, β>0 and ∀w,v∈Nw∗,b.Denote byw¯k∈S which satisfies(15)w¯k−wk=distwk,S.Lemma 2.
Let the sequencewk is generated by Algorithm 1 and Assumption 1 holds. If wk∈Nw∗,b/2, there exists a positive constant c1 that(16)dk≤c1distwk,S.Proof.
From (15), we have(17)w¯k−w∗≤w¯k−wk+wk−w∗≤2wk−w∗≤b,which indicates that w¯k∈Nw∗,b. From (5), (11), and Assumption 1(b), we have(18)λk=μkJkTFkδ≥mcδw¯k−wkδ.
Define(19)φkd=Fk+Jkd2+λkd2.
It follows from (2) that dk is a stationary point of φkd, and also, dk is a minimizer of φkd because of the convexity of φkd. Thus, from (12) and (18), we have(20)dk2≤φkdkλk≤φkw¯k−wkλk=Fk+Jkw¯k−wk2+λkw¯k−wk2λk≤L12mcδw¯k−wk4−δ+w¯k−wk2≤L12mcδ+1w¯k−wk2.
It means that(21)dk≤c1distwk,S,where c1=L12/mcδ+1.Lemma 3.
Under Assumption1, for all sufficiently large k,(a)
There exists a constantM>m such that(22)μk≤M.(b)
There exists a constantc2>0 such that(23)λk≤c2w¯k−wkδ.Proof.
We first prove (a). It follows from Assumption1, Lemma 2, (8), and (12) that(24)Predk≥JkTFkmindk,JkTFkJkTJk≥cw¯k−wkmindk,cL22c1dk=dkOw¯k−wk.
This, together with (9), (12), (13), and Fk+Jkdk≤Fk, gives(25)rk−1=Aredk−PredkPredk=Fwk+dk2−Fk+Jkdk2Predk≤Fk+JkdkOdk2+Odk4dkOw¯k−wk⟶0.
Therefore,rk⟶1. Hence, we have that there exists a constant M>m>0 such that μk≤M holds for all large k in view of Algorithm 1.
Now, we prove (b). It follows from (14) and (13) that(26)Jw¯kTFw¯k−JkTFk≤Jw¯kTFw¯k−Fk+FkJw¯kT−JkT≤L22w¯k−wk+βL1w¯k−wk=L22+βL1w¯k−wk.
Sinceλk=μkJkTFkδ and w¯k∈Nw∗,b, we obtain (b) with c2=ML22+βL1δ.Theorem 1.
Let the sequencewk is generated by Algorithm 1 and Assumption 1 holds. Then, the sequence wk converges to a solution of (1) with convergence rate min1+δ,2.Proof.
It follows from Lemma2 and (15) that(27)wk+1−w¯k≤wk+1−wk+wk−w¯k≤1+c1w¯k−wk.
From (12) and (14), we obtain(28)JkTFwk+1−JkTFk−JkTJkwk+1−wk=JkFwk+1−Fk−Jkwk+1−wk≤L1L2wk+1−wk2.
Also(29)JkTFwk+1−JkTFk−JkTJkwk+1−wk=JkTFwk+1−Jwk+1TFwk+1+Jwk+1TFwk+1−JkTFk−JkTJkwk+1−wk≥Jwk+1TFwk+1−JkTFk−JkTJkwk+1−wk−JkTFwk+1−Jwk+1TFwk+1,which yields(30)Jwk+1TFwk+1−JkTFk−JkTJkwk+1−wk≤L1L2wk+1−wk2+JkTFwk+1−Jwk+1TFwk+1.
For everyw¯k∈S∩Nw∗,b, together with (27), we get(31)JkTFwk+1−Jwk+1TFwk+1=Jk−Jwk+1TFwk+1=Jk−Jw¯k+Jw¯k−Jwk+1TFwk+1≤Jk−Jw¯kTFwk+1+Jw¯k−Jwk+1TFwk+1≤Jk−Jw¯kTFw¯k+Jw¯kwk+1−w¯k+L22wk+1−w¯k2+Jw¯k−Jwk+1Fw¯k+Jw¯kwk+1−w¯k+L22wk+1−w¯k2≤L1L2wk−w¯kwk+1−w¯k+L1L22wk−w¯kwk+1−w¯k2+L1L2wk+1−w¯k2+L1L22wk+1−w¯k3≤L1L21+c12+c1w¯k−wk2+L1L221+c122+c1w¯k−wk3.
Sincedk=wk+1−wk and w¯k∈S∩Nw∗,b, together with Assumption 1(b), Lemma 3(b), (2), (27), (30), and (31), we obtain(32)w¯k+1−wk+1=distwk+1,S≤c−1Jwk+1TFwk+1≤c−1Jwk+1TFwk+1−JkTFk−JkTJkwk+1−wk+JkTFk+JkTJkwk+1−wk≤c−1L1L2wk+1−wk2+JkTFwk+1−Jwk+1TFwk+1+JkTFk+JkTJkdk≤c−1L1L2c12w¯k−wk2+L1L21+c12+c1w¯k−wk2+L1L221+c122+c1w¯k−wk3+c1c2w¯k−wk1+δ≤c3w¯k−wkmin1+δ,2,where c3=c−1L1L2c12+L1L21+c12+c1+L1L221+c122+c1+c1c2. This implies that wk converges to solution set S with convergence rate min1+δ,2.
It is clear that(33)w¯k−wk=distwk,S≤w¯k+1−wk≤w¯k+1−wk+1+dk.
It follows from (32) and Lemma 2 that(34)w¯k−wk≤2dk≤2c1distwk,S=2c1w¯k−wk,holds for every sufficiently large k. Therefore, deduce from (32), we have(35)dk+1≤Odkmin1+δ,2,which indicates the sequence wk converges to a solution of (1) with convergence rate min1+δ,2.Remark 1.
Theorem1 shows that our new Levenberg–Marquardt algorithm converges to a solution of (1) superlinearly when δ∈0,1, while quadratically when δ∈1,2. Thus, Algorithm 1 has at least superlinearly convergence rate, which is consistent with the conclusion obtained in [8, 9, 11].


## 3. Global Convergence

We study the global convergence of Algorithm1 in this section.Theorem 2.
Let Assumption1(a) holds. Then, Algorithm 1 terminates in finite iterations or satisfies(36)liminfk⟶∞JkTFk=0.Proof.
We show it by contradiction. Assume the result is not true, then there exists a positive constantτ such that(37)JkTFk≥τ,∀k.
Define the index set of successful iterations as(38)S=k:rk≥p0.
Now, we deduce the contradiction in two cases.Case 1.
S is infinite. From (8), (14), and (37), since Fw is nonincreasing and bounded below, we have(39)F12≥∑k=1∞Fk2−Fwk+12≥∑k∈Sp0Fk2−Fk+Jkdk2≥∑k∈Sp0JkTFkmindk,JkTFkJkTJk≥∑k∈Sp0τmindk,τL22.
Hence,(40)limk∈S,k⟶+∞dk=0.
By assumption, we know thatdk=0 for all k∈¯S. Thus,(41)limk⟶+∞dk=0.
It follows from (2), (15) and Fk is nonincreasing that(42)μk⟶+∞.
Moreover, from (8) and (13), we have(43)rk−1=Aredk−PredkPredk≤Fwk+dk2−Fk+Jkdk2Predk≤Fk+JkdkOdk2+Odk4τmindk,τ/L22⟶0.
Hence,rk⟶1. From Algorithm 1, we have that there exists a constant M>0 such that(44)μk<M,holds for all large k. This contradicts to (40).Case 2.
S is finite. From the definition of S, together with Algorithm 1, there exists a K such that(45)rk<p0,k≥K.
According to the update rule ofwk that dk⟶0. We also have rk⟶1 by the same discussion as (43), which contradicts the above inequality.


## 4. Numerical Experiments

The effectiveness of Algorithm1 is verified by some numerical experiments in this section. The first experiment is Powell singular function which is computed in Section 1, and the second experiment is a nonzero-residue NLS problem which is computed by Behling et al. in ([4], Example 5.1), while the finals are some systems of nonlinear equation proposed by Moré et al. in [5]. The LM method (BMLM) proposed by Behling et al. in [4] is computed for comparison on Powell singular function and nonzero-residueleast-squares problem.Example 2.
Recompute Example1 by using Algorithm 1 with the same conditions of Example 1. Since the range of the exponent δ of λk=μkJkTFkδ has been extended from 0,1 to 0,2, we also give the results about the exponent more than 1. The results for 0,1 and 0,2 are listed in Tables 2 and 3, respectively.
As can be shown from Tables1–3, the performance of Algorithm 1 is better than that of BMLM because of fewer iterations and improved accuracy.Table 2
Results of Algorithm1 for Powell singular function with initial point w0=1,−3,0,1T, stopping criterion JkTFk<10−5, λk=JkTFkδ with δ=10−4 (left), δ=0.5 (middle), δ=1 (right).
ItersJkTFkFunctionItersJkTFkFunctionItersJkTFkFunction12.2939e +021.4663e +0112.2939e +021.4663e +0112.2939e +021.4663e +0122.8290e +013.1722e +0022.8288e +013.1723e +0022.8270e +013.1744e +0033.5362e +007.9304e −0133.5364e +007.9308e −0133.5389e +007.9361e −0144.4203e −011.9826e −0144.4205e −011.9827e −0144.4240e −011.9840e −0155.5254e −024.9565e −0255.5257e −024.9568e −0255.5300e −024.9601e −0266.9067e −031.2391e −0266.9072e −031.2392e −0266.9125e −031.2400e −0278.6335e −043.0978e −0378.6340e −043.0980e −0378.6407e −043.1001e −0381.0792e −047.7446e −0481.0792e −047.7450e −0481.0801e −047.7502e −0491.3490e −051.9362e −0491.3491e −051.9362e −0491.3501e −051.9375e −04101.6862e −064.8405e −05101.6863e −064.8406e −05101.6876e −064.8439e −05Table 3
Results of Algorithm1 for Powell singular function with initial point w0=1,−3,0,1T, stopping criterion JkTFk<10−5, λk=JkTFkδ with δ=1.5 (left), δ=2 (right).
ItersJkTFkFunctionItersJkTFkFunction12.2939e +021.4663e +0112.2939e +021.4663e +0122.8045e +013.2215e +0022.8225e +013.8978e +0033.5733e +008.0103e −0133.8543e +008.5906e −0144.4698e –012.0026e −0144.8608e −012.1471e −0155.5873e –025.0065e −0256.0763e −025.3678e −0266.9842e –031.2516e −0267.5954e −031.3419e −0278.7302e –043.1291e −0379.4943e −043.3549e −0381.0913e –047.8227e −0481.1868e −048.3872e −0491.3641e –051.9557e −0491.4835e −052.0968e −04101.7051e –064.8892e −05101.8544e −065.2420e −05Example 3.
Consider the nonzero-residue NLS problem [4](46)minw∈RnΦw=12Fw2,where Fw=w12+w22−1,w12+w22−9T.
The minimizer setS consists of points verifying w12+w22−5=0 so that distw,S=w−5. The Jacobian is not full rank since the rank of the Jacobian will be one except at the origin while being zero at the origin. Set w0=0,5+0.03T be the starting point and JkTFk<10−8 be the termination criterion. The results are shown in Table 4.
As illustrated in Table4, Algorithm 1 achieves the required accuracy, not only faster than BMLM but also more accurate than BMLM. Besides, the points wk generated by Algorithm 1 are closer to the solution set S∗ than those of BMLM.Table 4
Numerical results for Example3 with stopping criterion JkTFk<10−8 and w0=0,5+0.03T.
δBMLMAlgorithm1Itersdistwk,S∗JkTFkItersdistwk,S∗JkTFk10−403.0000e −031.2024e −0103.0000e −031.2024e −0117.4925e −052.9972e −0312.0172e −068.0690e −0521.8275e −067.3101e −0522.1694e −128.6779e −1134.4533e −081.7813e −0641.0848e −094.3391e −0852.6414e −111.0566e −090.503.0000e −031.2024e −0103.0000e −031.2024e −0112.7707e −051.1083e −0312.0124e −068.0494e −0522.3212e −089.2849e −0729.1660e −133.6671e −1135.5911e −132.2368e −11103.0000e −031.2024e −0103.0000e −031.2024e −0111.0971e −054.3884e −0412.0107e −068.0427e −0521.4727e −105.8908e −0929.0372e −133.6152e −111.503.0000e −031.2024e −0112.0101e −068.0403e −0529.0328e −133.6138e −11203.0000e −031.2024e −0112.0099e −068.0395e −0529.0328e −133.6138e −11Example 4.
Let us consider systems of nonlinear equations created by modifying the singular problems proposed by Moré et al. [5] and have the same form as in [30].(47)F^w=Fw−Jw∗AATA−1ATw−w∗,where w∗ is its root, function Fw is the standard nonsingular function, and matrix A∈Rn×k has full column rank with 1≤k≤n. It is easy to check that F^w∗=0 and J^w∗=Jw∗I−AATA−1AT have rankn−k.
We chose the rank ofJ^w∗ to be n−1 by using(48)A∈Rn×1,AT=1,1,…,1,and n−2 by using(49)A∈Rn×2,AT=1111⋯11−11−1⋯±1.
We set the parameters asp0=10−4, p1=0.25, p2=0.75, μ1=10−5, m=10−8, α1=4, and α2=0.25 in Algorithm 1. The algorithm is terminated when JkTFk≤10−5 or the number of iterates exceeds 100n+1. To consider the global convergence of Algorithm 1, we test three starting points w0, 10w0, and 100w0, where w0 is suggested by Moré et al. in [5]. The notations listed in tables have the following meanings:(i)
NF: the function calculations amount(ii)
NJ: the Jacobian calculations amount(iii)
NJ∗n+ NF: the total calculations since the Jacobian calculations are usually n times of the function calculations(iv)
“–”: the algorithm failed to find the solution in100n+1 iterations
In accordance with the range ofδ defined in (5), we take five choices, such as δ=10−4, 0.5, 1, 1.5, and 2, associated with λk=μkJkTFkδ for Algorithm 1. The results for the first set of problems of rank n−1 with low dimensional and high dimensional are listed in Tables 5 and 6, respectively, and the second set of rank n−2 with low dimensional and high dimensional in Tables 7 and 8, respectively.
The results in Tables5–8 show that Algorithm 1 is effective for systems of nonlinear equations. Algorithm 1 is perfect when the initial point w0 is close enough to the solution w∗ just as Theorem 1 requires. However, when the initial points are 10w0 and 100w0, which mean that the initial points are not close to solution w∗, Algorithm 1 is not perfect because the number of iteration increases rapidly, especially the exponent of the LM parameter is more than 1. Also, we can see that Algorithm 1 saves the Jacobian calculations to find a solution to some nonlinear equations since NJ is less than NF as well as the total calculations decreased.Table 5
Results on the first singular test set withrankJw∗=n−1.
Functionnw0NF/NJ/NJ∗n+ NFδ=10−4δ=0.5δ=1δ=1.5δ=2Rosenbrock2115/15/4515/15/4515/15/4519/17/53153/80/3131017/17/5117/17/5118/18/54143/75/293185/96/37710020/20/6021/21/6323/22/67130/68/266—Powell singular4110/10/5010/10/5010/10/5010/10/5010/10/501013/13/6513/13/6513/13/6513/13/6517/17/8510016/16/8016/16/8016/16/8020/20/100106/106/530Wood4116/16/8016/16/8016/16/8016/16/8017/17/851019/19/9519/19/9519/19/9520/20/10026/26/13010022/22/11022/22/11022/22/11026/26/130—Helical valley318/8/328/8/328/8/328/8/329/9/36108/8/328/8/328/8/329/9/3610/10/401008/8/328/8/328/8/329/9/3612/12/48Brown almost: linear1018/8/888/8/888/8/888/8/888/8/881023/23/25323/23/25323/23/25328/28/308—10045/45/49545/45/49545/45/495——Discrete boundary value1013/3/334/4/444/4/444/4/444/4/44107/7/777/7/778/8/888/8/888/8/881009/9/999/9/999/9/999/9/9910/10/110Discrete integral equation3016/6/1866/6/1866/6/1866/6/1866/6/186107/7/2177/7/2177/7/2177/7/2176/6/18610010/10/31010/10/31010/10/31011/11/34114/14/434Trigonometric30125/17/53529/19/59934/18/57437/17/54730/9/3001056/38/119660/41/129059/42/131969/40/126957/28/89710041/35/109136/22/69643/25/79346/24/76648/23/738Variably dimensioned10114/14/15414/14/15414/14/15414/14/15418/18/1981016/16/17616/16/17616/16/17617/17/18723/23/25310019/19/20919/19/20919/19/20925/25/275—Broyden tridiagonal3019/9/2799/9/2799/9/2799/9/2799/9/2791014/14/43414/14/43414/14/43414/14/43417/17/52710017/17/52717/17/52717/17/52721/21/65185/85/2635Broyden banded30112/12/37212/12/37212/12/37212/12/37212/12/3721018/18/55818/18/55818/18/55821/21/65130/30/93010024/24/74424/24/74424/24/74434/34/1054—Table 6
Results on the first high-dimensional singular test set withrankJw∗=n−1.
Functionnw0NF/NJ/NJ∗n+ NFδ=10−4δ=0.5δ=1δ=1.5δ=2Brown almost: linear50019/9/45099/9/45099/9/450910/10/501012/12/6012Discrete boundary value50011/1/5011/1/5011/1/5011/1/5011/1/501105/5/25053/3/15033/3/15033/3/15033/3/150310010/10/50109/9/45099/9/45099/9/45099/9/4509Discrete integral equation100017/7/70077/7/70077/7/70077/7/70077/7/7007108/8/80088/8/80087/7/700710/10/1001011/11/1101110010/10/1001010/10/1001010/10/1001012/12/1201217/17/17017Trigonometric1000113/8/801318/9/901822/8/802225/8/802529/9/90291030/21/2103033/19/1903341/25/2504140/18/1804040/18/1804010035/23/2303538/27/2703867/38/3806765/38/3806569/37/37069Variably dimensioned500128/28/1402828/28/1402828/28/14028179/179/89679—1030/30/1503030/30/1503030/30/150303266/3266/1636266—10034/34/1703434/34/1703434/34/17034——Broyden tridiagonal100019/9/90099/9/90099/9/90099/9/90099/9/90091014/14/1401414/14/1401414/14/1401416/16/1601620/20/2002010017/17/1701717/17/1701717/17/1701723/23/230231972/1972/1973972Broyden banded1000112/12/1201212/12/1201212/12/1201212/12/1201213/13/130131018/18/1801818/18/1801818/18/1801822/22/22022178/178/17817810024/24/2402424/24/2402424/24/2402477/77/77077—Table 7
Results on the second singular test set withrankJw∗=n−2.
Functionnw0NF/NJ/NJ∗n+ NFδ=10−4δ=0.5δ=1δ=1.5δ=2Rosenbrock2111/11/3311/11/3311/11/3311/11/3311/11/331013/13/3913/13/3913/13/3914/14/4218/18/5410017/17/5117/17/5117/17/5120/20/60133/133/399Powell singular4110/10/5010/10/5010/10/5010/10/5010/10/501013/13/6513/13/6513/13/6513/13/6517/17/8510016/16/8016/16/8016/16/8020/20/100106/106/530Wood4114/14/7014/14/7014/14/7014/14/7015/15/751017/17/8517/17/8517/17/8519/19/9524/24/12010020/20/10020/20/10020/20/10026/26/130—Helical valley3113/13/5213/13/5213/13/5213/13/5213/13/521014/14/5614/14/5614/14/5612/12/4815/15/6010014/14/5614/14/5616/15/6182/44/21429/23/98Brown almost: linear1018/8/888/8/888/8/888/8/888/8/881023/23/25323/23/25323/23/25328/28/308—10045/45/49545/45/49545/45/495——Discrete boundary value1013/3/334/4/444/4/444/4/444/4/44107/7/777/7/778/8/888/8/888/8/8810010/10/11010/10/11010/10/11010/10/11010/10/110Discrete integral equation3016/6/1866/6/1866/6/1866/6/1866/6/186107/7/2177/7/2177/7/2177/7/2176/6/18610010/10/31010/10/31010/10/31011/11/34115/15/465Trigonometric30118/12/37821/10/32124/11/35429/10/32931/11/3611037/25/78763/41/129358/39/122881/45/143149/30/94910089/66/206957/41/128744/31/97439/21/66950/25/800Variably dimensioned10114/14/15414/14/15414/14/15414/14/15418/18/1981016/16/17616/16/17616/16/17617/17/18723/23/25310019/19/20919/19/20919/19/20925/25/275—Broyden tridiagonal3019/9/2799/9/2799/9/2799/9/2799/9/2791014/14/43414/14/43414/14/43414/14/43417/17/52710017/17/52717/17/52717/17/52721/21/65185/85/2635Broyden banded30112/12/37212/12/37212/12/37212/12/37212/12/3721018/18/55818/18/55818/18/55821/21/65130/30/93010024/24/74424/24/74424/24/74434/34/1054—Table 8
Results on the second high-dimensional singular test set withrankJw∗=n−2.
Functionnw0NF/NJ/NJ∗n+ NFδ=10−4δ=0.5δ=1δ=1.5δ=2Brown almost: linear50019/9/45099/9/45099/9/450910/10/501012/12/6012Discrete boundary value50011/1/5011/1/5011/1/5011/1/5011/1/501105/5/25053/3/15033/3/15033/3/15033/3/150310011/11/551110/10/501010/10/501010/10/501010/10/5010Discrete integral equation100017/7/70077/7/70077/7/70077/7/70077/7/7007108/8/80088/8/80087/7/700710/10/1001011/11/1101110010/10/1001015/13/1301521/12/1202132/17/1703234/19/19034Trigonometric1000113/8/801318/9/901822/8/802225/8/802529/9/90291029/21/2102929/19/1902952/30/3005250/25/2505042/19/1904210060/41/4106040/26/2604052/30/3005252/27/2705271/36/36071Variably dimensioned500128/28/1402828/28/1402828/28/14028179/179/89679—1030/30/1503030/30/1503030/30/150303266/3266/1636266—10034/34/1703434/34/1703434/34/17034——Broyden tridiagonal100019/9/90099/9/90099/9/90099/9/90099/9/90091014/14/1401414/14/1401414/14/1401416/16/1601620/20/2002010017/17/1701717/17/1701717/17/1701723/23/230231972/1972/1973972Broyden banded1000112/12/1201212/12/1201212/12/1201212/12/1201213/13/130131018/18/1801818/18/1801818/18/1801822/22/22022178/178/17817810024/24/2402424/24/2402424/24/2402477/77/77077—


## 5. Conclusion

For systems of nonlinear (1), we discuss the Levenberg–Marquardt method with a modified LM parameter which extended from the LM method proposed by Behling et al. Under an error-bound condition, the local and global convergences have been analyzed by using the trust region technique. The convergence theory shows that our new LM method achieves a solution of nonlinear equations superlinearly when δ∈0,1 and quadratically when δ∈1,2. Numerical experiments indicate that the new LM method is outperformed than BMLM on nonlinear least-squares problems, and the new LM method is efficient on a set of systems of nonlinear equations.



---

*Source: 6043780-2023-02-10.xml*