a shamanskii like self adaptive levenberg marquardt method for nonlinear equations abstract in this paper we are concerned with the shamanskii like self adaptive levenberg marquardt methods for nonlinear equations we consider two choices of levenberg marquardt parameter one of them is the standard self adaptive levenberg marquardt parameter the other is nonmonotone self adaptive levenberg marquardt parameter by using the nonmonotone technique of grippo lampariello and lucidi under the error bound condition which is weaker than nonsingularity we show that the shamanskii like self adaptive levenberg marquardt methods converge with q order m 1 numerical experiments show the new algorithms are efficient and promising

convergence of newton like methods for singular operator equations using outer inverses we present a semilocal kantorovich type analysis for newton like methods for singular operator equations using outer inverses we establish sharp generalizations of the kantorovich theory and the mysovskii theory for operator equations when the derivative is not necessarily invertible the results reduce in the case of an invertible derivative to well known theorems of kantorovich and mysovskii with no additional assumptions unlike earlier theorems which impose strong conditions the strategy of the analysis is based on banach type lemmas and perturbation bounds for outer inverses which show that the set of outer inverses to a given bounded linear operator admits selections that behave like bounded linear inverses in contrast to inner inverses or generalized inverses which do not depend continuously on perturbations of the operator we give two examples to illustrate our results and compare them with earlier results and another numerical example to relate our results to computational issues

tensor methods for nonlinear equations a new class of methods for solving systems of nonlinear equations called tensor methods is introduced tensor methods are general purpose methods intended especially for problems where the jacobian matrix at the solution is singular or ill conditioned they base each iteration on a quadratic model of the nonlinear function the standard linear model augmented by a simple second order term the second order term is selected so that the model interpolates function values from several previous iterations as well as the current function value and jacobian the tensor method requires no more function and derivative information per iteration and hardly more storage or arithmetic per iteration than a standard method based on newton s method in extensive computational tests a tensor algorithm is significantly more efficient than a similar algorithm based on the standard linear model both on standard nonsingular test problems and on problems where the jacobian at the solution is singular

convergence cones near bifurcation the performance of predictor solver continuation methods near bifurcation points is considered a general theory is developed for the convergence of two commonly used solvers namely newton s method and the chord method near singular points of a nonlinear equation this theory provides a uniform treatment of continuation for simple bifurcation multiple bifurcation and multiple limit point bifurcation problems for these types of bifurcation it is shown that there are conical domains of attraction for the above iterative methods centred on solution branches which correspond to isolated roots of the algebraic bifurcation equations

local convergence analysis of tensor methods for nonlinear equations tensor methods for nonlinear equations base each iteration upon a standard linear model augmented by a low rank quadratic term that is selected in such a way that the mode is efficient to form store and solve these methods have been shown to be very efficient and robust computationally especially on problems where the jacobian matrix at the root has a small rank deficiency this paper analyzes the local convergence properties of two versions of tensor methods on problems where the jacobian matrix at the root has a null space of rank one both methods augment the standard linear model by a rank one quadratic term we show under mild conditions that the sequence of iterates generated by the tensor method based upon an ideal tensor model converges locally and two step q superlinearly to the solution with q order 3 2 and that the sequence of iterates generated by the tensor method based upon a practial tensor model converges locally and three step q superlinearly to the solution with q order 3 2 in the same situation it is known that standard methods converge linearly with constant converging to 1 2 hence tensor methods have theoretical advantages over standard methods our analysis also confirms that tensor methods converge at least quadratically on problems where the jacobian matrix at the root is nonsingular

a modified quasi newton method for nonlinear equations abstract in this paper a modified quasi newton method is proposed for solving the nonlinear equation f x 0 which is based on a new quasi newton approach the usual quasi newton equation is b k 1 s k y k where s k x k 1 x k y k f x k 1 f x k the new quasi newton equation is b k 1 s k y k in which s k is based on the iterates x k 1 x k x k 1 and y k is based on the function values f x k 1 f x k f x k 1 the new quasi newton equation exploits additional information by assuming a quadratic relationship between the information from the last three iterates the modified quasi newton method is based on the new quasi newton equation and possess local superlinear convergence properties numerical experiments show that the modified quasi newton method is promising

comparison of fully implicit and impes formulations for simulation of water injection in fractured and unfractured media in this work we compare the fully implicit fi and implicit pressure explicit saturation impes formulations for the simulation of water injection in fractured media the system of partial differential equations is discretized within the discrete fracture framework using a control volume method a unique feature of the methodology is that there is no need for the computation of matrix fracture transfer terms the non linear system of equations resulting from the fi formulation is solved with state of the art newton and tensor methods direct and krylov iterative methods are employed to solve the system resulting from the newton linearization the performance of the fi and impes formulations is compared with numerical testing results show that the contrast between matrix and fracture properties affects the performance of both impes and fi formulations and that the tensor method outperforms all the newton solvers for the near singular jacobian matrix copyright 2006 john wiley sons ltd

computing isolated singular solutions of polynomial systems case of breadth one we present a symbolic numeric method to refine an approximate isolated singular solution hat mathbf x hat x 1 ldots hat x n of a polynomial system f f 1 ldots f n when the jacobian matrix of f evaluated at hat mathbf x has corank one approximately our new approach is based on the regularized newton iteration and the computation of differential conditions satisfied at the approximate singular solution the size of matrices involved in our algorithm is bounded by n times n the algorithm converges quadratically if hat mathbf x is close to the isolated exact singular solution

an accelerated newton method for equations with semismooth jacobians and nonlinear complementarity problems we discuss local convergence of newton s method to a singular solution x of the nonlinear equations f x 0 for f mathbb r n rightarrow mathbb r n it is shown that an existing proof of griewank concerning linear convergence to a singular solution x from a starlike domain around x for f twice lipschitz continuously differentiable and x satisfying a particular regularity condition can be adapted to the case in which f is only strongly semismooth at the solution further newton s method can be accelerated to produce fast linear convergence to a singular solution by overrelaxing every second newton step these results are applied to a nonlinear equations reformulation of the nonlinear complementarity problem ncp whose derivative is strongly semismooth when the function f arising in the ncp is sufficiently smooth conditions on f are derived that ensure that the appropriate regularity conditions are satisfied for the nonlinear equations reformulation of the ncp at x

randomized algorithms for the low rank approximation of matrices we describe two recently proposed randomized algorithms for the construction of low rank approximations to matrices and demonstrate their application inter alia to the evaluation of the singular value decompositions of numerically low rank matrices being probabilistic the schemes described here have a finite probability of failure in most cases this probability is rather negligible 10 17 is a typical value in many situations the new procedures are considerably more efficient and reliable than the classical deterministic ones they also parallelize naturally we present several numerical examples to illustrate the performance of the schemes

comment on identification in the linear errors in variables model on considere un modele etudie par kapteyn et wansbeek 1983 et on presente des conditions necessaires et suffisantes d identification

a levenberg marquardt algorithm with correction for singular system of nonlinear equations in this paper we propose a new lm algorithm for the singular system of nonlinear equations f x 0 where f x r n r n is continuously differentiable and f x is lipschitz continuous at every iteration not only a lm step but also a correction step are computed the lm parameter is taken as m email protected f email protected d where email protected 0 2 and m k is updated by the trust region technique under the local error bound condition which is weaker than nonsingularity we prove the convergence rate of the algorithm is min 2 email protected numerical results show that the new lm algorithm performs very well

regular article a fast adaptive multipole algorithm in three dimensions we present an adaptive fast multipole method for the laplace equation in three dimensions it uses both new compression techniques and diagonal forms for translation operators to achieve high accuracy at a reasonable cost

a kantorovich type convergence analysis for the gauss newton method we present a semilocal kantorovich type convergence analysis for the gauss newton method which reduces to the wellknown newton kantorovich theorem for the newton method in a natural way additionnally a classification of the nonlinear regression problem into adequate and not adequate models is obtained

consistent sets of estimates for regressions with correlated or uncorrelated measurement errors in arbitrary subsets of all variables over the last decade the problem of measurement errors in the independent variables of a regression equation has attracted renewed interest among econometricians in the fifties and sixties the problem was considered to be more or less hopeless due to its inherent underidentification e g theil 1971 apart from instrumental variables the most frequently cited textbook solution was wald s method of grouping wald 1940 recent insight into the properties of the method of grouping can be interpreted as making this method worthless in most practical cases pakes 1982 since about 1970 new approaches to the problem have been explored basically along three lines viz embedding the error ridden equation into a set of multiple equations e g zellner 1970 goldberger 1972 into a set of simultaneous equations e g hsiao 1976 geraci 1976 and using the dynamics of the equation if present e g maravall and aigner 1977 in view of the underidentification of the basic model it is clear that all these methods invoke additional information of some kind if this information takes the form of exact or stochastic knowledge of certain parameters in the model the construction of consistent estimators is fairly straightforward e g fuller 1980 kapteyn and wansbeek 1984 for an overview of the state of the art see aigner et al 1984 an approach somewhat orthogonal to the ones described above has been to take the model as it is and to use prior ideas about the size of the measurement errors to diagnose how serious the probem is examples are blomqvist 1972 hodges and moore 1972 and davies and hutton 1975 leamer 1983 starts from the opposite direction by asking how serious the measurement error problem has to be in order to render the data useless for inference that is to say when measurement error is large enough to make it impossible to put bounds on regression parameters in an empirical example he shows that even very small measurement errors in some explanatory variables would open up the possibility of perfectly collinear explanatory variables and hence make the data useless for statistical inference at least without additional prior information the most systematic analysis of the information loss caused by measurement error is due to klepper and leamer 1984 they start out by invoking a minimal amount of prior information and then ask the question under what conditions it is still possible to make some inferences regarding the vector of unknown regression parameters p in the special case where the measurement errors are assumed uncorrelated and the k 1 estimates of 3 obtained by regressing each of the k 1 variables involved i e the one dependent variable and the k independent variables on the remaining k variables are all in the same orthant one can bound the ml estimates of p in that case the convex hull of the k 1 regressions contains all possible ml estimates and any point in the hull is a possible ml estimate if the k 1 regressions are not all in the same orthant then the set of ml estimates is unbounded in that case klepper and leamer 1984 introduce extra prior information which allows them to bound the set of maximum likelihood estimates the prior information comes in two forms firstly a researcher is supposed to be able to specify a maximum value of r2 if all exogenous variables were measured accurately it is shown that if this maximum is low enough one can again bound the set of ml estimates by a convex hull secondly if

fixed point quasi newton methods this article studies iterative methods defined by x k 1 phi x k e k where x k in mathbb r n and e k lie on a space of parameters sufficient conditions are established for local convergence and for convergence at an ideal linear or superlinear rate a theory of least change secant update methods for this class of processes is developed several examples are given showing a wide range of applications of the new theory key words quasi

on the local classification of smooth maps induced by newton s method abstract the classification of germs of smooth maps f r n r n induced by the continuous newton method f x x f x is addressed in this paper this classification problem is shown to rely on an equivalence notion located between right and contact equivalences driving the classification problem from the setting of quasilinear odes to the singularity theory framework one dimensional problems and regular n dimensional cases are easily characterized and normal forms for them are given folded zeros in r n display a much richer behavior an invariant and a preliminary normal form are derived for these cases

the positive semidefiniteness of partitioned matrices abstract the positive semidefiniteness of a partitioned matrix is characterized in terms of its submatrices the result is applied to a variety of problems concerning lowner ordered matrices which need not be partitioned themselves

finding structure with randomness probabilistic algorithms for constructing approximate matrix decompositions low rank matrix approximations such as the truncated singular value decomposition and the rank revealing qr decomposition play a central role in data analysis and scientific computing this work surveys and extends recent research which demonstrates that randomization offers a powerful tool for performing low rank matrix approximation these techniques exploit modern computational architectures more fully than classical methods and open the possibility of dealing with truly massive data sets this paper presents a modular framework for constructing randomized algorithms that compute partial matrix decompositions these methods use random sampling to identify a subspace that captures most of the action of a matrix the input matrix is then compressed either explicitly or implicitly to this subspace and the reduced matrix is manipulated deterministically to obtain the desired low rank factorization in many cases this approach beats its classical competitors in terms of accuracy robustness and or speed these claims are supported by extensive numerical experiments and a detailed error analysis the specific benefits of randomized techniques depend on the computational environment consider the model problem of finding the k dominant components of the singular value decomposition of an m times n matrix i for a dense input matrix randomized algorithms require bigo mn log k floating point operations flops in contrast to bigo mnk for classical algorithms ii for a sparse input matrix the flop count matches classical krylov subspace methods but the randomized approach is more robust and can easily be reorganized to exploit multiprocessor architectures iii for a matrix that is too large to fit in fast memory the randomized techniques require only a constant number of passes over the data as opposed to bigo k passes for classical algorithms in fact it is sometimes possible to perform matrix approximation with a single pass over the data

a compact limited memory method for large scale unconstrained optimization abstract a compact limited memory method for solving large scale unconstrained optimization problems is proposed the compact representation of the quasi newton updating matrix is derived to the use in the form of limited memory update in which the vector y k is replaced by a modified vector y k so that more available information about the function can be employed to increase the accuracy of hessian approximations the global convergence of the proposed method is proved numerical tests on commonly used large scale test problems indicate that the proposed compact limited memory method is competitive and efficient

on high order model regularization for constrained optimization in two recent papers regularization methods based on taylor polynomial models for minimization were proposed that only rely on holder conditions on the higher order employed derivatives grapiglia and nesterov considered cubic regularization with a sufficient descent condition that uses the current gradient and resembles the classical armijo s criterion cartis gould and toint used taylor models with arbitrary order regularization and defined methods that tackle convex constraints employing the descent criterion that compares actual reduction with predicted reduction the methods presented in this paper consider general not necessarily taylor models of arbitrary order employ a very mild descent criterion and handle general nonnecessarily convex constraints complexity results are compatible with the ones presented in the papers mentioned above

proxies versus omitted variables in regression analysis when some variables to be used in a regression analysis contain measurement error the ols estimator will be inconsistent when these variables are left out ols will also be inconsistent in an appropriate metric the first type of inconsistency is smaller than the second type

tensor methods for large sparse nonlinear least squares problems this paper introduces tensor methods for solving large sparse nonlinear least squares problems where the jacobian either is analytically available or is computed by finite difference approximations tensor methods have been shown to have very good computational performance for small to medium sized dense nonlinear least squares problems in this paper we consider the application of tensor methods to large sparse nonlinear least squares problems where a sparse factorization of the jacobian can be stored this involves an entirely new way of solving the tensor model that is efficient for sparse problems a number of interesting linear algebraic implementation issues are addressed the test results of the tensor method applied to a set of sparse nonlinear least squares problems compared with those of the standard gauss newton method reveal that the tensor method is significantly more robust and efficient than the standard gauss newton method

sublinear convergence of the chord method at singular points we give sufficient conditions for convergence of the chord method for a class of singular problems the rate of convergence is sublinear

schur complements and statistics abstract in this paper we discuss various properties of matrices of the type s h ge 1 f which we call the schur complement of e in a e f g h the matrix e is assumed to be nonsingular when e is singular or rectangular we consider the generalized schur complements s h ge f where e is a generalized inverse of e a comprehensive account of results pertaining to the determinant the rank the inverse and generalized inverses of partitioned matrices and the inertia of a matrix is given both for schur complements and for generalized schur complements we survey the known results in a historical perspective and obtain several extensions numerous applications in numerical analysis and statistics are included the paper ends with an exhaustive bibliography of books and articles related to schur complements

computing singular solutions to nonlinear analytic systems a method to generate an accurate approximation to a singular solution of a system of complex analytic equations is presented since manyreal systems extend naturally tocomplex analytic systems this porvides a method for generating approximations to singular solutions to real systems examples include systems of polynomials and systems made up of trigonometric exponential and polynomial terms the theorem on which the method is based is proven using results from several complex variables no special conditions on the derivatives of the system such as restrictions on the rank of the jacobian matrix at the solution are required the numerical method itself is developed from techniques of homotopy continuation and 1 dimensional quadrature a specific implementation is given and the results of numerical experiments in solving five test problems are presented

fast directional multilevel algorithms for oscillatory kernels this paper introduces a new directional multilevel algorithm for solving n body or n point problems with highly oscillatory kernels these systems often result from the boundary integral formulations of scattering problems and are difficult due to the oscillatory nature of the kernel and the non uniformity of the particle distribution we address the problem by first proving that the interaction between a ball of radius r and a well separated region has an approximate low rank representation as long as the well separated region belongs to a cone with a spanning angle of o 1 r and is at a distance which is at least o r 2 away from from the ball we then propose an efficient and accurate procedure which utilizes random sampling to generate such a separated low rank representation based on the resulting representations our new algorithm organizes the high frequency far field computation by a multidirectional and multiscale strategy to achieve maximum efficiency the algorithm performs well on a large group of highly oscillatory kernels our algorithm is proved to have o n log n computational complexity for any given accuracy when the points are sampled from a two dimensional surface we also provide numerical results to demonstrate these properties

robust partial quadratic eigenvalue assignment with time delay using the receptance and the system matrices abstract in this paper we consider the robust partial quadratic eigenvalue assignment problem in vibration by active feedback control based on the receptance measurements and the system matrices we propose an optimization method for the robust and minimum norm partial quadratic eigenvalue assignment problem we provide a new cost function and the closed loop eigenvalue sensitivity and the feedback norms can be minimized simultaneously our method is also extended to the case of time delay between measurements of state and actuation of control numerical tests demonstrate the effectiveness of our method

on the resolution of the generalized nonlinear complementarity problem minimization of a differentiable function subject to box constraints is proposed as a strategy to solve the generalized nonlinear complementarity problem gncp defined on a polyhedral cone it is not necessary to calculate projections that complicate and sometimes even disable the implementation of algorithms for solving these kinds of problems theoretical results that relate stationary points of the function that is minimized to the solutions of the gncp are presented perturbations of the gncp are also considered and results are obtained related to the resolution of gncps with very general assumptions on the data these theoretical results show that local methods for box constrained optimization applied to the associated problem are efficient tools for solving the gncp numerical experiments are presented that encourage the use of this approach

an example of irregular convergence in some constrained optimization methods that use the projected hessian in this paper we give examples illustrating the behavior of the coleman conn horizontal vertical method and of successive quadratic programming with a hessian approximation exact on the tangent space of the constraints one example shows that these methods in general are not one step superlinearly convergent

deflation algorithm for the multiple roots of a system of nonlinear equations abstract t ojika s watanabe and t mitsui in preparation have been developing a subroutine package naes nonlinear algebraic equations solver for the numerical solutions of the system of nonlinear equations an algorithm in the package termed the deflation algorithm for determining multiple roots for a system of nonlinear equations is presented and its effectiveness is shown by solving a numerical example

numerical liapunov schmidt spectral methods for k determined problems we describe several generalized liapunov schmidt methods in the bordered systems formulation for operator equations and collect information concerning classification of bifurcation functions we present a short outline of spectral methods in this bifurcation context and show the convergence of the discrete defining equations and nondegeneracy conditions for spectral finite element and difference methods to the counterparts of the original operator for k determined problems this result is then applied to a model for turing patterns in a two component mixture

properties and numerical performance of quasi newton methods with modified quasi newton equations quasi newton qn equation plays a core role in contemporary nonlinear optimization the usual qn equation employs only the gradients but ignores the available function value information in this paper we derive a class of modified qn equations with a vector parameter which use both available gradient and function value information the modified quasi newton methods maintain most properties of the usual quasi newton methods meanwhile they achieve a higher order accuracy in approximating the second order curvature of the problem functions than the usual ones do numerical experiments are reported which support the theoretical analyses and show the advantages of the modified qn methods over the usual ones

on deriving the inverse of a sum of matrices available expressions are reviewed and new ones derived for the inverse of the sum of two matrices one of them being nonsingular particular attention is given to text bf a text bf ubv 1 where textbf a is nonsingular and textbf u b and textbf v may be rectangular generalized inverses of textbf a textbf ubv are also considered several statistical applications are discussed

a shamanskii like acceleration scheme for nonlinear equations at singular roots on applique une variante de la methode de shamanskii pour realiser une methode a convergence superlineaire de resolution d une classe d equations non lineaires ayant une derivee de frechet singuliere a la racine

usefulness of proxy variables in linear models with stochastic regressors abstract this paper considers a linear model with two stochastic regressors where one regressor is not observed directly but may be observed with error proxy variable the interest is focussed upon the estimation accuracy of the regression coefficient of the observable regressor the possibility of estimating this coefficient more accurately using a proxy variable in regression instead of omitting the non observable variable is considered and an approximate condition for this to occur is given and discussed

verified error bounds for isolated singular solutions of polynomial systems case of breadth one in this paper we describe how to improve the performance of the symbolic numeric method in 19 20 for computing the multiplicity structure and refining approximate isolated singular solutions in the breadth one case by introducing a parameterized deflated system with smoothing parameters we generalize the algorithm in 33 to compute verified error bounds such that a slightly perturbed polynomial system is guaranteed to have a breadth one multiple root within the computed bounds

parallel quasi newton methods for unconstrained optimization we discuss methods for solving the unconstrained optimization problem on parallel computers when the number of variables is sufficiently small that quasi newton methods can be used we concentrate mainly but not exclusively on problems where function evaluation is expensive first we discuss ways to parallelize both the function evaluation costs and the linear algebra calculations in the standard sequential secant method the bfgs method then we discuss new methods that are appropriate when there are enough processors to evaluate the function gradient and part but not all of the hessian at each iteration we develop new algorithms that utilize this information and analyze their convergence properties we present computational experiments showing that they are superior to parallelization either the bfgs methods or newton s method under our assumptions on the number of processors and cost of function evaluation finally we discuss ways to effectively utilize the gradient values at unsuccessful trial points that are available in our parallel methods and also in some sequential software packages

convergence rates for newton s method at singular points if newton s method is employed to find a root of a map from a banach space into itself and the derivative is singular at that root the convergence of the newton iterates to the root is linear rather than quadratic in this paper we give a detailed analysis of the linear convergence rates for several types of singular problems for some of these problems we describe modifications of newton s method which will restore quadratic convergence

the black box fast multipole method a new o n fast multipole formulation is proposed for non oscillatory kernels this algorithm is applicable to kernels k x y which are only known numerically that is their numerical value can be obtained for any x y this is quite different from many fast multipole methods which depend on analytical expansions of the far field behavior of k for x y large other black box or kernel independent fast multipole methods have been devised our approach has the advantage of requiring a small pre computation time even for very large systems and uses the minimal number of coefficients to represent the far field for a given l 2 tolerance error in the approximation this technique can be very useful for problems where the kernel is known analytically but is quite complicated or for kernels which are defined purely numerically

a new acceleration method for newton s method at singular points when newton s method is used to find a root of a map from a banach space into itself and the derivative is singular at that root convergence of the newton sequence is in general linear in this paper we give a modification of newton s method that under certain conditions converges superlinearly our method is applicable under more general conditions than other techniques in particular it may be used for certain quadratic problems and problems in which the dimension of the null space of the derivative is larger than one

convergence acceleration for newton s method at singular points if newton s method is employed to find a root of a map from a banach space into itself and the derivative is singular at that root linear convergence of the newton sequence to the root is the best

on solving nonlinear equations with simple singularities or nearly singular solutions the paper begins with a review of the convergence theory for newton s method near simple and other regular singularities followed by a brief discussion of the inherent difficulty of singular problems and the effect of rounding errors on the achievable solution accuracy then several ways to accelerate the slow convergence of newton s method are described in some detail these are richardson extrapolation overrelaxation bordering by a singularity condition and appending the usual linear model by a quadratic term the last two approaches require limited second derivative information which can be obtained analytically by differencing or through secant updating numerical comparisons reported in the final sections lead to the conclusion that the collection of the additional information pays off in terms of speed and accuracy

tensor methods for unconstrained optimization using second derivatives a new type of method for unconstrained optimization called a tensor method is introduced it is related in its basic philosophy to the tensor methods for nonlinear equations for schnabel and frank siam j numer anal 21 1984 pp 815 843 but beyond that the methods have significant differences the tensor method for unconstrained optimization bases each iteration upon a fourth order model of the objective function this model consists of the quadratic portion of the taylor series plus low rank third and fourth order terms that cause the model to interpolate already calculated function and gradient values from one or more previous iterates this paper also shows that the costs of forming storing and solving the tensor model are not significantly more than these costs for a standard method based upon a quadratic taylor series model test results are presented for sets of problems where the hessian at the minimizer is nonsingular and where it is singular on all the test sets the tensor method

a high order modified levenberg marquardt method for systems of nonlinear equations with fourth order convergence fan 2014 presented an accelerated modified levenberg marquardt method for nonlinear equations at every iteration the accelerated modified lm method computed not only a lm trial step but also an additional approximate lm step which employed a line search in this paper based on the accelerated modified lm method we compute the approximate lm step one more time at every iteration and obtain a high order accelerating modified levenberg marquardt method under the local error bound condition which is weaker than nonsingularity the convergence order of this new method is shown to be fourth a globally convergence is also given by the trust region technique numerical results show that the new method is efficient and could save many calculations of the jacobian

approximation of boundary element matrices this article considers the problem of approximating a general asymptotically smooth function in two variables typically arising in integral formulations of boundary value problems by a sum of products of two functions in one variable from these results an iterative algorithm for the low rank approximation of blocks of large unstructured matrices generated by asymptotically smooth functions is developed this algorithm uses only few entries from the original block and since it has a natural stopping criterion the approximative rank is not needed in advance

qualitative analysis of newton s flow the newton flow of a parameter dependent mapping is analysed in a neighbourhood of a singular point it is shown that the liapunov schmidt reduction process yields an invariant exponentially attracting manifold of the newton flow if the singular point is a fold then a normal form of the reduced 2 d newton flow is given moreover numerical experiments suggest that discrete versions of the normal form seem to be useful for an understanding of the performance of the ordinary newton method in a neighbourhood of the fold

a note on the levenberg marquardt parameter abstract in c ma l jiang some research on levenberg marquardt method for the nonlinear equations appl math comput 184 2007 1032 1040 the lm parameter at the kth iteration is chosen as k f k 2 1 j k t f k 2 where f is the residual function j is the jacobi of f and 0 1 is a constant in this note we point out that the lm parameter can be any combination of f k 2 and j k t f k 2 provided it is positive furthermore we give a more general choice of the lm parameter and show that the lm method still preserves the quadratic convergence under the local error condition which is weaker than nonsingularity

newton s method with deflation for isolated singularities of polynomial systems we present a modification of newton s method to restore quadratic convergence for isolated singular solutions of polynomial systems our method is symbolic numeric we produce a new polynomial system which has the original multiple solution as a regular root using standard bases a tool for the symbolic computation of multiplicities we show that the number of deflation stages is bounded by the multiplicity of the isolated root our implementation performs well on a large class of applications

dealing with singularities in nonlinear unconstrained optimization we propose a new trust region based optimization algorithm for solving unconstrained nonlinear problems whose second derivatives matrix is singular at a local solution we give a theoretical characterization of the singularity in this context and we propose an iterative procedure which allows to identify a singularity in the objective function during the course of the optimization algorithm and artificially adds curvature to the objective function numerical tests are performed on a set of unconstrained nonlinear problems both singular and non singular results illustrate the significant performance improvement compared to classical trust region and filter algorithms proposed in the literature the approach is also shown to be competitive with tensor methods in terms of efficiency while reaching a higher level of robustness

a higher order levenberg marquardt method for nonlinear equations in this paper we present a high order levenberg marquardt method for nonlinear equations at every iteration not only a lm step is computed but also two approximate lm steps are computed which use the previous calculated jacobian under the local error bound condition which is weaker than nonsingularity the new method has biquadratic convergence a globally convergent lm algorithm is also given by the trust region technique numerical results show that the new fourth order lm algorithm performs very well and could save many jacobian calculations especially for large scale problems

jacobian smoothing brown s method for ncp abstract jacobian smoothing brown s method for nonlinear complementarity problems ncp is studied in this paper this method is a generalization of classical brown s method it belongs to the class of jacobian smoothing methods for solving semismooth equations local convergence of the proposed method is proved in the case of a strictly complementary solution of ncp furthermore a locally convergent hybrid method for general ncp is introduced some numerical experiments are also presented

discretization of implicit odes for singular root finding problems this paper addresses the use of dynamical system theory to tackle singular root finding problems the use of continuous time methods leads to implicit differential systems when applied to singular nonlinear equations the analysis is based on a taxonomy of singularities and uses previous stability results proved in the context of quasilinear implicit odes the proposed approach provides a framework for the systematic formulation of quadratically convergent iterations to singular roots the scope of the work includes also the introduction of discrete time analysis techniques for singular problems which are based on continuous time stability and numerical stability some numerical experiments illustrate the applicability of the proposed techniques

approximate solution of the trust region problem by minimization over two dimensional subspaces the trust region problem minimization of a quadratic function subject to a spherical trust region constraint occurs in many optimization algorithms in a previous paper the authors introduced an inexpensive approximate solution technique for this problem that involves the solution of a two dimensional trust region problem they showed that using this approximation in an unconstrained optimization algorithm leads to the same theoretical global and local convergence properties as are obtained using the exact solution to the trust region problem this paper reports computational results showing that the two dimensional minimization approach gives nearly optimal reductions in then dimension quadratic model over a wide range of test cases we also show that there is very little difference in efficiency and reliability between using the approximate or exact trust region step in solving standard test problems for unconstrained optimization these results may encourage the application of similar approximate trust region techniques in other contexts

askit approximate skeletonization kernel independent treecode in high dimensions we present a fast algorithm for kernel summation problems in high dimensions such problems appear in computational physics numerical approximation nonparametric statistics and machine learning in our context the sums depend on a kernel function that is a pair potential defined on a dataset of points in a high dimensional euclidean space a direct evaluation of the sum scales quadratically with the number of points fast kernel summation methods can reduce this cost to linear complexity but the constants involved do not scale well with the dimensionality of the dataset the main algorithmic components of fast kernel summation algorithms are the separation of the kernel sum between near and far field which is the basis for pruning and the efficient and accurate approximation of the far field we introduce novel methods for pruning and for approximating the far field our far field approximation requires only kernel evaluations and does not use analytic expansions pruning is not done using bounding

geometrically isolated nonisolated solutions and their approximation a solution x x 0 of f x 0 is said to be isolated if the frechet derivative f x 0 is nonsingular it is said to be geometrically isolated if no other solution is in x x 0 leqq rho for some rho 0 isolated solutions are always geometrically isolated sufficient conditions are obtained to insure that a nonisolated solution is also geometrically isolated we then study the application of approximation methods in the general form f h x h 0 to approximate nonisolated solutions which are geometrically isolated under strong consistency conditions the results are somewhat negative the approximations may have an even number including zero or an odd number of roots near x 0 depending upon the multiplicity cf x 0 as a root if the accuracy is o h p and the multiplicity is n then the approximations have error o h p n the relation of these results to limit and bifurcation points is discussed briefly

a modified brown algorithm for solving singular nonlinear systems with rank defects a modified brown algorithm for solving a class of singular nonlinear systems f x 0 where x f rn is presented this method is constructed by combining the discreted brown algorithm with the space transforming method the second order information of f x at a point is not required calculating which is different from the tensor method and the hoy s method the q quadratic convergence of this algorithm and some numerical examples are given as well

on the compression of low rank matrices a procedure is reported for the compression of rank deficient matrices a matrix a of rank k is represented in the form a u circ b circ v where b is a k times k submatrix of a and u v are well conditioned matrices that each contain a k times k identity submatrix this property enables such compression schemes to be used in certain situations where the singular value decomposition svd cannot be used efficiently numerical examples are presented

characterization and computation of generalized turning points a minimal set of augmenting equations is given so that general parameter dependent problems together with these equations will be nonsingular at points of simple rank deficiency applications to de

tensor methods for minimizing convex functions with holder continuous higher order derivatives in this paper we study p order methods for unconstrained minimization of convex functions that are p times differentiable p geq 2 with nu holder continuous p th derivatives we propose

tensor methods for large sparse systems of nonlinear equations this paper introduces tensor methods for solving large sparse systems of nonlinear equations tensor methods for nonlinear equations were developed in the context of solving small to medium sized dense problems they base each iteration on a quadratic model of the nonlinear equations where the second order term is selected so that the model requires no more derivative or function information per iteration than standard linear model based methods and hardly more storage or arithmetic operations per iteration computational experiments on small to medium sized problems have shown tensor methods to be considerably more efficient than standard newton based methods with a particularly large advantage on singular problems this paper considers the extension of this approach to solve large sparse problems the key issue considered is how to make efficient use of sparsity in forming and solving the tensor model problem at each iteration accomplishing this turns out to require an entirely new way of solving the tensor model that successfully exploits the sparsity of the jacobian whether the jacobian is nonsingular or singular we develop such an approach and based upon it an efficient tensor method for solving large sparse systems of nonlinear equations test results indicate that this tensor method is significantly more efficient and robust than an efficient sparse newton based method in terms of iterations function evaluations and execution time 1998 the mathematical programming society inc published by elsevier science b v

a modified two steps levenberg marquardt method for nonlinear equations the modified levenberg marquardt mlm method to solve nonlinear equations was introduced by fan 2012 this method uses an addition of the levenberg marquardt step and an approximate lm step as the trial step at every iteration using a trust region technique the global and cubic convergence of the mlm method under the local error bound condition is proved fan 2012 recently fan proposed an accelerated mlm algorithm by using a line search strategy to generate a modified lm step and showed that the convergence rate of the algorithm is min 1 2 3 which results the cubic convergence for 1 fan 2014 in this paper by introducing an adaptive lm parameter for amlm algorithm we propose an efficient amlm algorithm the cubic convergence of the new algorithm is presented while numerical experiments show the new algorithm is promising

an efficient two step trust region algorithm for exactly determined consistent systems of nonlinear equations abstract in this paper we propose a modified two step trust region algorithm for solving nonlinear systems two step trust region algorithms at every iteration use a trust region step and an approximate step by saving the jacobian matrix to have fewer computations we introduce a convex combination to modify the trust region subproblems along with an additional criterion to verify whether the first step is accepted or not we establish global and quadratic convergence of the algorithm under some mild assumptions numerical results show that the modified algorithm is efficient and promising

a kernel independent adaptive fast multipole algorithm in two and three dimensions we present a new fast multipole method for particle simulations the main feature of our algorithm is that it does not require the implementation of multipole expansions of the underlying kernel and it is based only on kernel evaluations instead of using analytic expansions to represent the potential generated by sources inside a box of the hierarchical fmm tree we use a continuous distribution of an equivalent density on a surface enclosing the box to find this equivalent density we match its potential to the potential of the original sources at a surface in the far field by solving local dirichlet type boundary value problems the far field evaluations are sparsified with singular value decomposition in 2d or fast fourier transforms in 3d we have tested the new method on the single and double layer operators for the laplacian the modified laplacian the stokes the modified stokes the navier and the modified navier operators in two and three dimensions our numerical results indicate that our method compares very well with the best known implementations of the analytic fmm method for both the laplacian and modified laplacian kernels its advantage is the relative simplicity of the implementation and its immediate extension to more general kernels

a corrected levenberg marquardt algorithm with a nonmonotone line search for the system of nonlinear equations in this paper we propose a corrected levenberg marquardt method for the system of nonlinear equations in which not only an l m step and a corrected step are computed at every iteration but also a nonmonotone line search to find a new iteration point will be performed if a trial step is not accepted to ensure the global convergence of the new method a new nonmonotone line search technique is introduced for the merit function the cubic convergence of the new method is proved under the local error bound condition which is weaker than nonsingularity some numerical results are reported which shows that the algorithm is quite effective

finding a multiple zero by transformations and newton like methods many methods for finding a multiple zero x of a function f are based on transforming f to a function t for which x is a simple zero we give and analyze a general transformation which i

superlinearly convergent algorithms for solving singular equations and smooth reformulations of complementarity problems we propose a new algorithm for solving smooth nonlinear equations in the case where their solutions can be singular compared to other techniques for computing singular solutions a distinctive feature of our approach is that we do not employ second derivatives of the equation mapping in the algorithm and we do not assume their existence in the convergence analysis important examples of once but not twice differentiable equations whose solutions are inherently singular are smooth equation based reformulations of the nonlinear complementarity problems reformulations of complementarity problems serve both as illustration of and motivation for our approach and one of them we consider in detail we show that the proposed method possesses local superlinear quadratic convergence under reasonable assumptions we further demonstrate that these assumptions are in general not weaker and not stronger than regularity conditions employed in the context of other superlinearly convergent newton type algorithms for solving complementarity problems which are typically based on nonsmooth reformulations therefore our approach appears to be an interesting complement to the existing ones

analysis of newton s method at irregular singularities recent work on newton s method at singularities of the jacobian has established linear convergence under certain regularity assumptions here newton s method is analyzed in the neighborhood of irregular singularities which include all minimizers at which the hessian has a one dimensional null space depending on a parameter specific to the underlying optimization problem or system of simultaneous equations newton s method is found either to converge with a limiting ratio of about frac 2 3 or to diverge from arbitrarily close starting points or to behave in a certain sense chaotically

local dynamics of a family of quasilinear odes with folded singular equilibria abstract the local behavior of newton s method near singular equilibria can be addressed via normal form theory in a continuous time framework in this paper we analyze the dynamics of a recently obtained normal form for the continuous newton method near so called folded equilibrium points these can be classified in terms of a morse index which characterizes the nearby structure of pseudoequilibria and impasse points extremal values of this index yield the simplest cases in which all surrounding singularities are either forward or backward impasse points in more involved isotropic cases the behavior of trajectories will be distinguished by two types of separatrices one of them undergoing singularity crossing phenomena this approach leads to a complete local description of the different attraction domains which can be displayed at singular solutions

nonlinear programming via an exact penalty function asymptotic analysis in this paper we consider the final stage of a global method to solve the nonlinear programming problem we prove 2 step superlinear convergence in the process of analyzing this asymptotic behavior we compare our method theoretically to the popular successive quadratic programming approach

convergence analysis of a variant of the newton method for solving nonlinear equations the paper presents a convergence analysis of a modified newton method for solving nonlinear systems of equations the convergence results show that this method converges cubically in the nonsingular case and linearly with the rate 3 8 under some sufficient conditions when the jacobian is singular at the root the convergence theory is used to analyze the convergence behavior when the modified newton method is applied to a nonsymmetric algebraic riccati equation arising in transport theory numerical experiment confirms the theoretical results

three steps modified levenberg marquardt method with a new line search for systems of nonlinear equations three steps modified levenberg marquardt method for nonlinear equations was introduced by yang 2013 this method uses the addition of the levenberg marquardt lm step and two approximate lm steps as the trial step at every iteration using trust region technique the global and biquadratic convergence of the method is proved by yang the main aim of this paper is to introduce a new line search strategy while investigating the convergence properties of the method with this line search technique since the search direction of yang method may be not a descent direction standard line searches cannot be used directly in this paper we propose a new nonmonotone third order armijo type line search technique which guarantees the global convergence of this method while we use an adaptive lm parameter it is proved that the convergence order of the new method is biquadratic numerical results show the new algorithm is efficient and promising

an o nlogn hierarchical random compression method for kernel matrices by sampling partial matrix entries abstract in this paper we propose an o n log n hierarchical random compression method hrcm for kernel matrix compressing which only requires sampling o n log n entries of a matrix the hrcm combines the hierarchical framework of the h matrix and a randomized sampling technique of column and row spaces for far field interaction kernel matrix we show that a uniform column row sampling of a far field kernel matrix thus without the need and associated cost to pre compute a costly sampling distribution will give a low rank compression of such low rank matrix independent of the matrix size and only dependent on the separation of the source and target locations this far field random compression technique is then implemented at each level of the hierarchical decomposition for general kernel matrices resulting in an o n log n random compression method error and complexity analysis for the hrcm are included numerical results for electrostatic and low frequency helmholtz wave kernels have validated the efficiency and accuracy of the proposed method in comparison of direct o n 2 summations