a family of quasi newton methods for nonlinear equations with direct secant updates of matrix factorizations a new family of least change secant methods for solving systems of nonlinear algebraic equations is introduced at each iteration of the methods of this family the matrix b k is represented as a product of the form a k 1 r k where the matrices a k and r k have a determined structure and a k 1 r k 1 are chosen by the use of a least change secant rule the first and second methods of broyden math comp 19 1965 pp 577 593 math comp 25 1971 pp 285 294 and the methods of schubert math comp 24 1970 pp 27 30 johnson and austria siam j numer anal 20 1983 pp 315 325 and chadee tech report sol 85 8 dept of operations research stanford university stanford ca 1985 are members of this family local and superlinear convergence results are proved and particular members of the family that seem to be of practical usefulness are introduced

on secant updates for use in general constrained optimization in this paper we present two new classes of successive quadratic programming sqp secant methods for the equality constrained optimization problem one class of methods uses the sqp augmented lagrangian formulation while the other class uses the sqp lagrangian formulation we demonstrate under the standard assumptions that in both cases the bfgs and dfp versions of the algorithm are locally q superlinearly convergent to our knowledge this is the first time that either local or q superlinear convergence has been established for an sqp lagrangian secant method which uses either the bfgs or dfp updating philosophy and assumes no more than the standard assumptions since the standard assumptions do not require positive definiteness of the hessian of the lagrangian at the solution it is no surprise that our bfgs and dfp updates possess the hereditary positive definiteness property only on a proper subspace

convergence of newton like iterative methods newton like methods in which the intermediate systems of linear equations are solved by iterative techniques are examined by applying the theory of inexact newton methods radius of convergence and rate of convergence results are easily obtained the analysis is carried out in affine invariant terms the results are applicable to cases where the underlying newton like method is for example a difference newton like or update newton method

variable metric methods for unconstrainted optimization and nonlinear least squares variable metric or quasi newton methods are well known and commonly used in connection with unconstrained optimization since they have good theoretical and practical convergence properties although these methods were originally developed for small and moderate size dense problems their modifications based either on sparse partitioned or limited memory updates are very efficient on large scale sparse problems very significant applications of these methods also appear in nonlinear least squares approximation and nonsmooth optimization in this contribution we give an extensive review of variable metric methods and their use in various optimization fields

efficient estimation of sparse jacobian matrices by differences abstract in difference newton like methods for solving f x 0 the jacobian matrix f x is approximated by differences between values of f if f x is sparse a consistent partition of its columns can be exploited to approximate f x using relatively few values of f we provide a local convergence theory for the resulting methods a superlinearly convergent stable cyclic secant method in which at each iteration two values of f are required and several columns of the jacobian matrix approximation are updated simultaneously is developed

multilevel least change newton like methods for equality constrained optimization problems this paper deals with the application of multilevel least change newton like methods for solving twice continuously differentiable equality constrained optimization problems we define multilevel partial inverse least change updates multilevel least change newton like methods without derivatives and multilevel projections of fragments of the matrix for newton like methods without derivatives local andq superlinear convergence of these methods is proved the theorems here also imply local andq superlinear convergence of many standard newton like methods for nonconstrained and equality constraine optimization problems

a new structured quasi newton algorithm using partial information on hessian this paper presents a modified quasi newton method for structured unconstrained optimization the usual sqn equation employs only the gradients but ignores the available function value information several researchers paid attention to other secant conditions to get a better approximation of the hessian matrix of the objective function recently yabe et al 2007 6 proposed the modified secant condition which uses both gradient and function value information in order to get a higher order accuracy in approximating the second curvature of the objective function in this paper we derive a new progressive modified sqn equation with a vector parameter which use both available gradient and function value information that maintains most properties of the usual and modified structured quasi newton methods furthermore local and superlinear convergence of the algorithm is obtained under some reasonable conditions

two view multibody structure from motion we present an algebraic geometric approach to 3 d motion estimation and segmentation of multiple rigid body motions from noise free point correspondences in two perspective views our approach exploits the algebraic and geometric properties of the so called multibody epipolar constraint and its associated multibody fundamental matrix which are natural generalizations of the epipolar constraint and of the fundamental matrix to multiple motions we derive a rank constraint on a polynomial embedding of the correspondences from which one can estimate the number of independent motions as well as linearly solve for the multibody fundamental matrix we then show how to compute the epipolar lines from the first order derivatives of the multibody epipolar constraint and the epipoles by solving a plane clustering problem using generalized pca gpca given the epipoles and epipolar lines the estimation of individual fundamental matrices becomes a linear problem the clustering of the feature points is then automatically obtained from either the epipoles and epipolar lines or from the individual fundamental matrices although our approach is mostly designed for noise free correspondences we also test its performance on synthetic and real data with moderate levels of noise

application of the mesh independence principle to mesh refinement the mesh independence principle mip for a newton iteration applied to an operator equation and its discretization for different stepsizes h states for a fixed starting point the number of iterations of newton s method to obtain a fixed tolerance is eventually independent of h this mip is used to formulate an efficient strategy for the solution of nonlinear equations amounting to the work equivalent of two or three newton iterates for the smallest h examples are given

extrapolation methods for vector sequences this is an expository paper that describes and compares five methods for extrapolating to the limit or anti limit of a vector sequence without explicit knowledge of the sequence generator the methods are the minimal polynomial extrapolation mpe studied by cabay and jackson mesina and skelboe the reduced rank extrapolation rre of eddy which we show to be equivalent to mesina s version of mpe the vector and scalar versions of the epsilon algorithm vea sea introduced by wynn and extended by brezinski and gekeler and the topological epsilon algorithm tea of brezinski we cover the derivation and error analysis of iterated versions of the algorithms as applied to both linear and nonlinear problems and we show why these versions tend to converge quadratically we also present samples from extensive numerical testing that has led us to the following conclusions a tea in spite of its role as a theoretical link between the polynomial type and the epsilon type methods has no practical appl

a comparative study on methods for convergence acceleration of iterative vector sequences we discuss several methods for accelerating the convergence of the iterative solution of nonlinear equation systems commonly in use and point to interrelations between them in particular we investigate two of the most sophisticated schemes namely the anderson mixing and the broyden update both generalized to the consideration of arbitrarily many previous iterations for the broyden method we give a new derivation which is much simpler than that recently proposed by vanderbilt and louie we show that if the additional parameters invented by these authors in order to increase flexibility are used to optimize the convergence of the iteration process they in fact cancel out in addition we prove that in this optimal case the anderson mixing and the broyden update as applied to the inverse jacobian are fully identical thus we come to the conclusion that neither of these schemes is superior moreover we show that broyden update of the inverse jacobian is superior to updating the jacobian itself finally we propose an extension of the anderson mixing which avoids the numerical difficulties all these methods are faced with

mixture densities maximum likelihood and the em algorithm the problem of estimating the parameters which determine a mixture density has been the subject of a large diverse body of literature spanning nearly ninety years during the last two decades the

relative affine structure canonical model for 3d from 2d geometry and applications we propose an affine framework for perspective views captured by a single extremely simple equation based on a viewer centered invariant we call relative affine structure via a number of corollaries of our main results we show that our framework unifies previous work including euclidean projective and affine in a natural and simple way and introduces new extremely simple algorithms for the tasks of reconstruction from multiple views recognition by alignment and certain image coding applications

nonlinear krylov and moving nodes in the method of lines we report on some successes and problem areas in the method of lines from our work with moving node finite element methods first we report on our nonlinear krylov accelerator for the modified newton s method on the nonlinear equations of our stiff ode solver since 1990 it has been robust simple cheap and automatic on all our moving node computations we publicize further trials with it here because it should be of great general usefulness to all those solving evolutionary equations second we discuss the need for reliable automatic choice of spatially variable time steps third we discuss the need for robust and efficient iterative solvers for the difficult linearized equations jx b of our stiff ode solver here the 1997 thesis of zulu xaba has made significant progress

newton s method for a class of optimal shape design problems a class of optimal shape design problems is considered where a part of the boundary of the domain represents the free parameter the variable domain is parametrized by a class of functions in such a way that the optimal design problem results in an optimal control problem on a fixed domain the functions for the parametrization of the domain are used as controls and the corresponding states are then given by the solution of an elliptic boundary value problem on a fixed domain discretizing this control problem normally leads to a large scale optimization problem where the corresponding solution methods are characterized by the requirement of solving many boundary value problems in spite of this interesting numerical challenge until now little work has been done to derive more efficient algorithms by taking advantage of the specific structure of this kind of problem in this report newton s method in function space is derived resulting in an efficient algorithm for the discretized optimization problems by using the specific structure of these optimal shape design problems an efficient implementation of the numerical algorithm is introduced the properties of this algorithm are compared with those of the gradient method using illustrative numerical examples

krylov subspace methods for solving large unsymmetric linear systems some algorithms based upon a projection process onto the krylov subspace k sub m span r sub 0 ar sub 0 a sup m 1 r sub 0 are developed generalizing the method of conjugate gradients to unsymmetric systems these methods are extensions of arnoldi s algorithm for solving eigenvalue problems the convergence is analyzed in terms of the distance of the solution to the subspace k sub m and some error bounds are established showing in particular a similarity with the conjugate gradient method for symmetric matrices when the eigenvalues are real several numerical experiments are described and discussed

krylov subspace acceleration of nonlinear multigrid with application to recirculating flows this paper deals with the combination of two solution methods multigrid and gmres siam j sci comput 14 1993 pp 856 869 the generality and parallelizabili nonlinear pdes as the preconditioner for a nonlinear krylov subspace method we use the full approximation storage fas scheme math comp 31 1977 pp 333 390 a nonlinear multigrid method the nonlinear krylov acceleration is applied also on coarse grids so that recirculating incompressible flow problems discretized with a higher order upwind scheme can be solved efficiently

statistical models for intercepted data abstract consider a population of replaceable items of different ages and suppose that for each item there is a certain measurable attribute that is a monotone function perhaps random of the item s age to fix ideas we refer to this attribute as deterioration and we assume that as items age their level of deterioration increases we further assume that as the level of deterioration hits a certain threshold value perhaps unknown the item dies and disappears from the population so it is not available for sampling any more since items that deteriorate at a slower rate live longer any sample obtained over a short period of time would overrepresent slowly deteriorating items consequently under such circumstances longitudinal studies are often needed to statistically study deterioration as a function of age this article presents a statistical model that enables us to study the deterioration function from a single cross section snapshot of the population or some random parts of it specifically

iterative solution of integral equations by a quasi newton method abstract when broyden s type methods are applied to the solution of integral equations they can be specialized in such a way to generate in the course of the iteration low rank degenerate kernel equations we describe the features of this approach

independent motion detection in 3d scenes this paper presents an algorithmic approach to the problem of detecting independently moving objects in 3d scenes that are viewed under camera motion there are two fundamental constraints that can be exploited for the problem 1 two multiview camera motion constraint for instance the epipolar trilinear constraint and 2 shape constancy constraint previous approaches to the problem either use only partial constraints or rely on dense correspondences or flow we employ both the fundamental constraints in an algorithm that does not demand a priori availability of correspondences or flow our approach uses the plane plus parallax decomposition to enforce the two constraints it is also demonstrated that for a class of scenes called sparse 3d scenes in which genuine parallax and independent motions may be confounded how the plane plus parallax decomposition allows progressive introduction and verification of the fundamental constraints results of the algorithm on some difficult sparse 3d scenes are promising

anderson acceleration for fixed point iterations this paper concerns an acceleration method for fixed point iterations that originated in work of d g anderson j assoc comput mach 12 1965 pp 547 560 which we accordingly call anderson acceleration here this method has enjoyed considerable success and wide usage in electronic structure computations where it is known as anderson mixing however it seems to have been untried or underexploited in many other important applications moreover while other acceleration methods have been extensively studied by the mathematics and numerical analysis communities this method has received relatively little attention from these communities over the years a recent paper by h fang and y saad numer linear algebra appl 16 2009 pp 197 221 has clarified a remarkable relationship of anderson acceleration to quasi newton secant updating methods and extended it to define a broader anderson family of acceleration methods in this paper our goals are to shed additional light on anderson acceleration and to draw further attention to its usefulness as a general tool we first show that on linear problems anderson acceleration without truncation is essentially equivalent in a certain sense to the generalized minimal residual gmres method we also show that the type 1 variant in the fang saad anderson family is similarly essentially equivalent to the arnoldi full orthogonalization method we then discuss practical considerations for implementing anderson acceleration and illustrate its performance through numerical experiments involving a variety of applications

on the relation between two local convergence theories of least change secant update methods in this paper we show that the main results of the local convergence theory for least change secant update methods of dennis and walker siam j numer anal 18 1981 949 987 can be proved using the theory introduced recently by martinez math comp 55 1990 143 167 in addition we exhibit two generalizations of well known methods whose local convergence can be easily proved using martinez s theory

second order methods for optimal control of time dependent fluid flow second order methods for open loop optimal control problems governed by the two dimensional instationary navier stokes equations are investigated optimality systems based on a lagrangian formulation and adjoint equations are derived the newton and quasi newton methods as well as various variants of sqp methods are developed for applications to optimal flow control and their complexity in terms of system solves is discussed local convergence and rate of convergence are proved a numerical example illustrates the feasibility of solving optimal control problems for two dimensional instationary navier stokes equations by second order numerical methods in a standard workstation environment

deriving collinear scaling algorithms as extension of quasi newton methods and the local convergence of dfp and bfgs related collinear scaling algorithms this paper is concerned with collinear scaling algorithms for unconstrained minimization where the underlying local approximants are forced to interpolate the objective function value and gradient at only the two most recent iterates by suitably modifying the procedure of sorensen 1980 for deriving such algorithms we show that two members of the algorithm class derived related to the dfp and bfgs methods respectively are locally and q superlinearly convergent this local analysis as well as the results they yield exhibit the same sort of duality exhibited by those of broyden dennis and more 1973 and dennis and more 1974 for the dfp and bfgs methods the results in this paper also imply the local and q superlinear convergence of collinear scaling algorithms of sorensen 1982 pp 154 156 related to the dfp and bfgs methods

a note on quasi newton formulae for sparse second derivative matrices in order to apply quasi newton methods to solve unconstrained minimization calculations when the number of variables is very large it is usually necessary to make use of any sparsity in the second derivative matrix of the objective function therefore it is important to extend to the sparse case the updating formulae that occur in variable metric algorithms to revise the estimate of the second derivative matrix suitable extensions suggest themselves when the updating formulae are derived by variational methods 1 3 the purpose of the present paper is to give a new proof of a theorem of dennis and schnabel 1 that shows the effect of sparsity on updating formulae for second derivative estimates

a pointwise quasi newton method for unconstrained optimal control problems for a class of unconstrained optimal control problems we propose a quasi newton method that exploits the structure of the problem we define a new type of superlinear convergence for sequences in function spaces and prove superlinear convergence of the iterates generated by the quasi newton method in this sense

detecting motion regions in the presence of a strong parallax from a moving camera by multiview geometric constraints we present a method for detecting motion regions in video sequences observed by a moving camera in the presence of a strong parallax due to static 3d structures the proposed method classifies each image pixel into planar background parallax or motion regions by sequentially applying 2d planar homographies the epipolar constraint and a novel geometric constraint called the structure consistency constraint the structure consistency constraint being the main contribution of this paper is derived from the relative camera poses in three consecutive frames and is implemented within the plane parallax framework unlike previous planar parallax constraints proposed in the literature the structure consistency constraint does not require the reference plane to be constant across multiple views it directly measures the inconsistency between the projective structures from the same point under camera motion and reference plane change the structure consistency constraint is capable of detecting moving objects followed by a moving camera in the same direction a so called degenerate configuration where the epipolar constraint fails we demonstrate the effectiveness and robustness of our method with experimental results of real world video sequences

some results about vector extrapolation methods and related fixed point iterations abstract in this work we consider polynomial extrapolation methods for vector sequences we give a unified presentation of these methods and a new recursive algorithm for the implementation of the modified minimal polynomial extrapolation mmpe method properties operation counts and storage requirements of the algorithm will be given the second part of the work is devoted to the application of these methods for solving systems of nonlinear equations a sufficient condition for quadratic convergence is given

a short cut method for estimation in renewal processes this article proposes a simple noniterative estimator of the lifetime distribution from stationary renewal processes data which is a variation of the kaplan meier estimator the method is compared with the nonparametric maximum likelihood estimator using real and simulated data and the results show that as long as the censoring level is not too high the two estimates are remarkably close a heuristic asymptotic argument is given to support the simulation results and to justify the rationale behind the estimator the estimator s connection with the kaplan meier estimator is exploited to suggest a variance estimate and associated confidence interval which through a simulation study are found to be conservative

svd based initialization a head start for nonnegative matrix factorization we describe nonnegative double singular value decomposition nndsvd a new method designed to enhance the initialization stage of nonnegative matrix factorization nmf nndsvd can readily be combined with existing nmf algorithms the basic algorithm contains no randomization and is based on two svd processes one approximating the data matrix the other approximating positive sections of the resulting partial svd factors utilizing an algebraic property of unit rank matrices simple practical variants for nmf with dense factors are described nndsvd is also well suited to initialize nmf algorithms with sparse factors many numerical examples suggest that nndsvd leads to rapid reduction of the approximation error of many nmf algorithms

sequential quadratic programming for certain parameter identification problems we analyze the method of sequential quadratic programming for equality constrained minimization problems in hilbert spaces of functions and for the discrete approximations of such problems in the context of an elliptic parameter identification problem we show how the discretization can be constructed so as to preserve the convergence behavior of the iterates for the infinite dimensional problem in the finite dimensional approximations we use the structure of the parameter identification problem to reduce the size of the linear system for the sqp step and verify nondegeneracy of the constraints

a comparison of algorithms for maximum likelihood estimation of choice models abstract maximum likelihood estimation mle is often avoided in econometric and other statistical applications due to computational considerations despite its strong theoretical appeal recent advances in non linear optimization provide feasible alternative methods for calculating mle s especially when special structure may be exploited as for example in probabilistic choice models this paper examines a range of quasi newton methods and gives numerical comparisons based on the multinomial logit model and a non independence of irrelevant alternative model among the issues considered are step determination and methods of approximating the hessian these include standard secant updates dfp and bfgs statistical approximations bhhh and recently developed approaches which use model switching

the fundamental matrix theory algorithms and stability analysis in this paper we analyze in some detail the geometry of a pair of cameras i e a stereo rig contrarily to what has been done in the past and is still done currently for example in stereo or motion analysis we do not assume that the intrinsic parameters of the cameras are known coordinates of the principal points pixels aspect ratio and focal lengths this is important for two reasons first it is more realistic in applications where these parameters may vary according to the task active vision second the general case considered here captures all the relevant information that is necessary for establishing correspondences between two pairs of images this information is fundamentally projective and is hidden in a confusing manner in the commonlyused formalism of the essential matrix introduced by longuet higgins 1981 this paper clarifies the projective nature of the correspondence problem in stereo and shows that the epipolar geometry can be summarized in one 3 x 3 matrix of rank 2 which we propose to call the fundamental matrix after this theoretical analysis we embark on the task of estimating the fundamental matrix from point corre spondences a task which is of practical importance we analyze theoretically and compare experimentally using synthetic and real data several methods of estimation the problem of the stability of the estimation is studied from two complementary viewpoints first we show that there is an interesting relationship between the fundamental matrix and three dimensional planes which induce homographies between the images and create unstabilities in the estimation procedures second we point to a deep relation between the unstability of the estimation procedure and the presence in the scene of so called critical surfaces which have been studied in the context of motion analysis finally we conclude by stressing the fact that we believe that the fundamental matrix will play a crucial role in future applications of three dimensional computer vision by greatly increasing its versatility robustness and hence applicability to real difficult problems

orthogonal projections on convex sets for newton like methods this paper develops the fixed almost least change secant methods for solving of nonlinear problems the presented results generalize and extend some results given by dennis and walker siam j numer anal 18 1981 pp 949 987 among other things multilevel update algorithms are described

adapting broyden method to handle linear constraints imposed via lagrange multipliers various non linear equation solvers are adapted to handle linear constraints via the lagrange multiplier technique this adaptation process turns out to be quite straightforward for newton raphson methods and rank two quasi newton methods bfgs and dfp but rather more involved for broyden method in fact two broyden methods can be obtained the standard one and a modified one better adapted to the lagrange multiplier environment some numerical examples are used to assess the relative performance of the various adapted solvers these tests illustrate the superiority of the modified broyden method over the standard one

least change secant update methods for undetermined systems least change secant updates for nonsquare matrices have been addressed recently in 6 here the use of these updates in iterative procedures for the numerical solution of underde retrained systems is considered the model method is the normal flow algorithm used in homotopy or continuation methods for determining points on an implicitly defined curve a kantorovich type local convergence analysis is given which supports the use of least change secant updates in this al gorithm this analysis also provides a kantorovich type local convergence analysis for least change secant update methods in the usual case of an equal number of equations and unknowns this in turn gives a local convergence analysis for augmented jacobian algorithms which use least change secant updates in conclusion the results of some numerical experiments are given

length biasing and laws equivalent to the log normal abstract let x 0 denote a generic lifetime of a renewal process having unit mean lifetime let hat x denote the stationary total lifetime and let q 0 1 be a fixed constant we consider anew the scale invariance problem for which laws does q hat x have the same distribution as x our setting is more probabilistic than those presented hitherto and we explore connections with the log normal moment problem in particular it is shown that all explicitly known laws which have log normal moments solve our problem the notion of remaining lifetime is generalized and its scaling invariance is investigated using the notion of total lifetime two moment equivalent laws of askey are shown to have a simple representation in terms of laws equivalent to the log normal the representation involves a q gamma law which we explore in its own right an affine extension of our basic scale invariance relation arising in the theory of orthogonal polynomials is shown to be equivalent to the latter

vector extrapolation methods applications and numerical comparison the present paper is a survey of the most popular vector extrapolation methods such as the reduced rank extrapolation rre the minimal polynomial extrapolation mpe the modified minimal polynomial extrapolation mmpe the vector e algorithm vea and the topological e algorithm tea using projectors we derive a different interpretation of these methods and give some theoretical results the second aim of this work is to give a numerical comparison of the vector extrapolation methods above when they are used for practical large problems such as linear and nonlinear systems of equations

quasi newton methods and unconstrained optimal control problems we prove a mesh independence result for the bfgs method in hilbert space and apply it to a class of unconstrained optimal control problems a new fourth order discretization scheme is used in the i

a structured quasi newton algorithm for optimizing with incomplete hessian information we present a structured quasi newton algorithm for unconstrained optimization problems that have unavailable second order derivatives or hessian terms we provide a formal derivation of the well kn

local and superlinear convergence for truncated iterated projections methods least change secant updates can be obtained as the limit of iterated projections based on other secant updates we show that these iterated projections can be terminated or truncated after any positive number of iterations and the local and the superlinear rate of convergence are still maintained the truncated iterated projections method is used to find sparse and symmetric updates that are locally and superlinearly convergent

on simulation and optimization of macroeconometric models abstract this paper develops adapted methods for solving and optimizing deterministic nonlinear discrete dynamic macroeconometric models we give computational results concerning a belgian model of about 500 nonlinear equations per time period which leads us to a large scale minimization problem with several thousands of nonlinear constraints we use new optimization techniques based on partially known quasi newton hessian matrix updates in a reduced gradient framework we also discuss different techniques for dealing with bound constraints arising form different model formulations

direct secant updates of matrix factorizations this paper presents a new context for using the sparse broyden update method to solve systems of nonlinear equations the setting for this work is that a newton like algorithm is assumed to be available which incorporates a workable strategy for improving poor initial guesses and providing a satisfactory jacobian matrix approximation whenever required the total cost of obtaining each jacobian matrix or the cost of factoring it to solve for the newton step is assumed to be sufficiently high to make it attractive to keep the same jacobian approximation for several steps this paper suggests the extremely convenient and apparently effective technique of applying the sparse broyden update directly to the matrix factors in the iterations between reevaluations in the hope that fewer fresh factorizations will be required the strategy is shown to be locally and q superlinearly convergent and some encouraging numerical results are presented

slot nozzle design with specified pressure in a given subregion by using embedding method in this paper we study a shape optimization problem connected with controlled pressure in a subregion of a 2d slot nozzle feasible shapes for this problem are characterized only with upper walls this enables us to seek for the optimum in a class of functions and their derivatives instead of a class of shapes the shape optimization problem can be written as an optimal control problem then the resulting distributed control problem is expressed in a measure theoretical form in fact an infinite dimensional linear programming problem the optimal measure representing optimal shape is approximated by the solution of a finite dimensional linear programming problem a brief sensitivity analysis on model parameters is given

parameter identification in constitutive models via optimization with a posteriori error control in this paper we outline a computational technique for the calibration of macroscopic constitutive laws with automatic error control in the most general situation the state variables of the constitutive law as well as the material parameters are spatially non homogeneous the experimental observations are given in space time based on an appropriate dual problem we compute a posteriori the discretization error contributions from approximations of the parameter state and costate elds in space time for an arbitrarily chosen goal oriented error measure of engineering signi cance such a measure can be used in an adaptive strategy not discussed in this paper to meet a prede ned error tolerance an important observation is that the jacobian matrix associated with the resulting newton method is used in principle in solving the dual problem rather than treating the jacobian in a monolithic fashion we utilize a sequential solution strategy whereby the fe topology of the discretized state problem is used repeatedly moreover the proposed solution strategy lends itself naturally to the computation of rst and second order sensitivities which are obtained with little extra computational effort numerical results are given for the prototype model of con ned aquifer ow with spatially non homogeneous permeability the ef ciency of the optimization strategy and the effectivity of the error computation are assessed

convergence theorems for least change secant update methods the purpose of this paper is to present a convergence analysis of least change secant methods in which part of the derivative matrix being approximated is computed by other means the theorems and proofs given here can be viewed as generalizations of those given by broyden dennis more j inst math appl 12 1973 pp 223 246 and by dennis more math comp 28 1974 pp 549 560 the analysis is done in the orthogonal projection setting of dennis schnabel siam rev 21 1980 pp 443 459 and many readers might feel that it is easier to understand the theorems here readily imply local and q superlinear convergence of all the standard methods in addition to proving these results for the first time for the sparse symmetric method of marwil and toint and the nonlinear least squares method of dennis gay welsch

distribution functions invariant under residual lifetime and length biased sampling the equation f qx int limits 0 x left 1 f left u right right du rm x mathbin lower 3ex hbox buildrel over smash scriptstyle vphantom x rm 0 where f is a distribution function d f arises when the limiting d f of the residual lifetime in a renewal process is a scaled version of the general lifetime d f f the equation g qx int limits 0 x ug left du right rm x mathbin lower 3ex hbox buildrel over smash scriptstyle vphantom x rm 0 on the other hand arises when the limiting d f of the total lifetime in a renewal process is a scaled version of the general lifetime d f g

a quasi newton method for elliptic boundary value problems for some boundary value problems for equations of the form nabla 2 u f x u nabla u 0 with linear boundary conditions we give a convergence analysis for a quasi newton method that converges superlinearly in the c 1 norm this scheme for infinite dimensional problems becomes a method for the finite dimensional problems that are produced by discretizing the differential equation this method is a generalization of a method proposed by hart and soul for discretizations of two point boundary value problems

determining the epipolar geometry and its uncertainty a review two images of a single scene object are related by the epipolar geometry which can be described by a 3 3 singular matrix called the essential matrix if images internal parameters are known or the fundamental matrix otherwise it captures all geometric information contained in two images and its determination is very important in many applications such as scene modeling and vehicle navigation this paper gives an introduction to the epipolar geometry and provides a complete review of the current techniques for estimating the fundamental matrix and its uncertainty a well founded measure is proposed to compare these techniques projective reconstruction is also reviewed the software which we have developed for this review is available on the internet

an infinite dimensional convergence theory for reduced sqp methods in hilbert space we develop a general convergence theory for a class of reduced successive quadratic programming sqp methods for infinite dimensional equality constrained optimization problems in hilbert space the methods under consideration approximate second order information of the lagrangian restricted to the null space of h the derivative of the constraints in particular a sufficient condition for two step superlinear convergence will be given and a general result for secant methods will be established which can serve as a tool to prove superlinear convergence in an infinite dimensional framework this result is applied to prove local two step superlinear convergence of reduced sqp methods under the assumption that exact reduced second order information is known up to a compact perturbation to test our convergence theory we consider optimal control problems and we formulate a reduced quasi newton algorithm which presents a new approach to an efficient solution of these problems our main motivation for st

local convergence of quasi newton methods for b differentiable equations we study local convergence of quasi newton methods for solving systems of nonlinear equations defined by b differentiable functions we extend the classical linear and superlinear convergence results for general quasi newton methods as well as for broyden s method we also show how broyden s method may be applied to nonlinear complementarity problems and illustrate its computational performance on two small examples

least change secant updates of nonsquare matrices the notion of least change secant updates is extended to apply to nonsquare matrices in a way appropriate for quasi newton methods used to solve systems of nonlinear equations that depend on parameters extensions of the widely used least change secant updates for square matrices are given a local convergence analysis for certain paradigm iterations is outlined as motivation for the use of these updates and numerical experiments involving these iterations are discussed

quasi newton methods for solving underdetermined nonlinear simultaneous equations abstract we analyze iterative processes of type xk 1 xk xk ek f xk for solving f x 0 f r n r m m n parameters ek are updated at each iteraction using least change secant update procedures we prove local linear and superlinear convergence results we introduce two new superlinearly convergent methods of this type and one linearly convergent quasi newton generalization of cimmino s parallel algorithm for solving linear systems some numerical experiments are presented

a sparse quasi newton update derived variationally with a nondiagonally weighted frobenius norm the problem of symmetric sparse updating is considered from a variational point of view and a new class of sparse symmetric quasi newton updating formulae is derived this class results from the use of a nondiagonally weighted frobenius norm the computa tion of the update involves only one positive definite and symmetric linear system that has the same sparsity pattern as the problem itself

on an efficient k step iterative method for nonlinear equations this paper is devoted to the construction and analysis of an efficient k step iterative method for nonlinear equations the main advantage of this method is that it does not need to evaluate any high order frechet derivative moreover all the k step have the same matrix in particular only one lu decomposition is required in each iteration we study the convergence order the efficiency and the dynamics in order to motivate the proposed family we prove using some recurrence relations a semilocal convergence result in banach spaces finally a numerical application related to nonlinear conservative systems is presented

maximum efficiency for a family of newton like methods with frozen derivatives and some applications a generalized k step iterative application of newton s method with frozen derivative is studied and used to solve a system of nonlinear equations the maximum computational efficiency is computed a sequence that approximates the order of convergence is generated for the examples and it numerically confirms the calculation of the order of the method and computational efficiency this type of method appears in many applications where the authors have heuristically chosen a given number of steps with frozen derivatives an example is shown in which the total variation tv minimization model is approximated using the schemes described in this paper

fixed point quasi newton methods this article studies iterative methods defined by x k 1 phi x k e k where x k in mathbb r n and e k lie on a space of parameters sufficient conditions are established for local convergence and for convergence at an ideal linear or superlinear rate a theory of least change secant update methods for this class of processes is developed several examples are given showing a wide range of applications of the new theory key words quasi

accelerating with rank one updates abstract consider the iteration xk 1 xk h b axk for solving ax b a is n x n nonsingular we discuss rank one updates to improve h as an approximation to a 1 during the iteration the update kills and reduces singular values of i ah and thus speeds up the convergence the algorithm terminates after at most n sweeps and if all n sweeps are needed then a 1 has been computed

analysis of some vector extrapolation methods for solving systems of linear equations in this paper we consider some vector extrapolation methods for solving nonsymmetric systems of linear equations when applied to sequences generated linearly these methods namely the minimal polynomial extrapolation mpe and the reduced rank extrapolation rre are krylov subspaces methods and are respectively equivalent to the method of arnoldi and to the gcr and gmres by considering the geometrical aspect of these methods we derive new expressions for their residual norms and give a relationship between them this allows us to compare the two methods using this new approach we will show that for nonsingular skew symmetric matrices the gmres stagnates every two iterations and the restarted version gmres m m ge 2 is always convergent finally the incomplete forms are considered and some convergence results are given

quasi newton updates with bounds we develop a quasi newton method which preserves known bounds on the jacobian matrix we show that this update can be computed with the same amount of work as competitive methods in particular we prove that the number of operations required to obtain this update is proportional to the number of nonzeros in the sparsity pattern of the jacobian matrix the method is also shown to share the local convergence properties of broyden s and schubert s method

on the local convergence study for an efficient k step iterative method research was supported in part by programa de apoyo a ia investigacion de ia fundacion seneca agencia de ciencia y tecnologia de la region de murcia 19374 pi 14 by the project of generalitat valenciana prometeo 2016 089 and the projects mtm2015 64382 p mineco feder mtm2014 52016 c2 1 p and mtm2014 52016 c2 2 p of the spanish ministry of science and innovation