on regularization and active set methods with complexity for constrained optimization the main objective of this research is to introduce a practical method for smooth bound constrained optimization that possesses worst case evaluation complexity o varepsilon 3 2 for finding an varepsilon approximate first order stationary point when the hessian of the objective function is lipschitz continuous as other well established algorithms for optimization with box constraints the algorithm proceeds visiting the different faces of the domain aiming to reduce the norm of an internal projected gradient and abandoning active constraints when no additional progress is expected in the current face the introduced method emerges as a particular case of a method for minimization with linear constraints moreover the linearly constrained minimization algorithm is an instance of a minimization algorithm with general constraints whose implementation may be unaffordable when the constraints are complicated as a procedure for leaving faces a different method is employed that may be regarded as a

hybrid variable fidelity optimization by using a kriging based scaling function solving design problems that rely on very complex and computationally expensive calculations using standard optimization methods might not be feasible given design cycle time constraints variable fidelity methods address this issue by using lower fidelity models and a scaling function to approximate the higher fidelity models in a provably convergent framework in the past scaling functions have mainly been either first order multiplicative or additive corrections these are being extended to second order in this investigation variable metric approaches for calculating second order scaling information are developed a kriging based scaling function is introduced to better approximate the high fidelity response on a more global level an adaptive hybrid method is also developed in this investigation the adaptive hybrid method combines the additive and multiplicative approaches so that the designer does not have to determine which is more suitable prior to optimization the methodologies developed in this research are compared to existing methods using two demonstration problems the first problem is analytic whereas the second involves the design of a supercritical high lift airfoil the results demonstrate that the krigingbased scaling methods improve computational expense by lowering the number of high fidelity function calls required for convergence the results also indicate the hybrid method is both robust and effective

on the global convergence of trust region algorithms for unconstrained minimization many trust region algorithms for unconstrained minimization have excellent global convergence properties if their second derivative approximations are not too large 2 we consider how large these approximations have to be if they prevent convergence when the objective function is bounded below and continuously differentiable thus we obtain a useful convergence result in the case when there is a bound on the second derivative approximations that depends linearly on the iteration number

a trust region algorithm for heterogeneous multiobjective optimization this paper presents a trust region method for multiobjective heterogeneous optimization problems one of the objective functions is an expensive black box function given for example by a time co

global convergence of trust region algorithms for convex constrained minimization without derivatives in this work we propose a trust region algorithm for the problem of minimizing a function within a convex closed domain we assume that the objective function is differentiable but no derivatives are available the algorithm has a very simple structure and allows a great deal of freedom in the choice of the models under reasonable assumptions for derivative free schemes we prove global convergence for the algorithm that is to say that all accumulation points of the sequence generated by the algorithm are stationary

a decomposition method for large scale box constrained optimization a decomposition method for solving large scale box constrained optimization is proposed the algorithm is motivated by the successful use of the decomposition method presented by joachims for training support vector machines in particular a new technique based on the new definition kkt violating index is introduced for working set identification finally the numerical experiments and implementation details show that this method is practical for large scale problems

pattern search algorithms for bound constrained minimization we present a convergence theory for pattern search methods for solving bound constrained nonlinear programs the analysis relies on the abstract structure of pattern search methods and an understanding of how the pattern interacts with the bound constraints this analysis makes it possible to develop pattern search methods for bound constrained problems while only slightly restricting the flexibility present in pattern search methods for unconstrained problems we prove global convergence despite the fact that pattern search methods do not have explicit information concerning the gradient and its projection onto the feasible region and consequently are unable to enforce explicitly a notion of sufficient feasible decrease

non monotone trust region algorithms for nonlinear optimization subject to convex constraints this paper presents two new trust region methods for solving nonlinear optimization problems over convex feasible domains these methods are distinguished by the fact that they do not enforce strict monotonicity of the objective function values at successive iterates the algorithms are proved to be convergent to critical points of the problem from any starting point extensive numerical experiments show that this approach is competitive with the lancelot package

preconditioned conjugate gradient algorithms for nonconvex problems with box constraints the paper describes new conjugate gradient algorithms for large scale nonconvex problems with box constraints in order to speed up convergence the algorithms employ scaling matrices which transform the space of original variables into the space in which hessian matrices of the problem s functionals have more clustered eigenvalues this is done by applying limited memory bfgs updating matrices once the scaling matrix is calculated the next few conjugate gradient iterations are performed in the transformed space the box constraints are treated efficiently by the projection we also present a limited memory quasi newton method which is a special version of our general algorithm the presented algorithms have strong global convergence properties in particular they identify constraints active at a solution in a finite number of iterations we believe that they are competitive to the l bfgs b method and present some numerical results which support our claim

modified subspace limited memory bfgs algorithm for large scale bound constrained optimization in this paper a subspace limited memory bfgs algorithm for solving large scale bound constrained optimization problems is developed it is modifications of the subspace limited memory quasi newton method proposed by ni and yuan q ni y x yuan a subspace limited memory quasi newton algorithm for large scale nonlinear bound constrained optimization math comput 66 1997 1509 1520 an important property of our proposed method is that more limited memory bfgs update is used under appropriate conditions the global convergence of the method is established the implementations of the method on cute test problems are presented which indicate the modifications are beneficial to the performance of the algorithm

a trust region strategy for minimization on arbitrary domains we present a trust region method for minimizing a general differentiable function restricted to an arbitrary closed set we prove a global convergence theorem the trust region method defines difficult subproblems that are solvable in some particular cases we analyze in detail the case where the domain is a euclidean ball for this case we present numerical experiments where we consider different hessian approximations

solving karush kuhn tucker systems via the trust region and the conjugate gradient methods a popular approach to solving the karush kuhn tucker kkt system mainly arising from the variational inequality problem is to reformulate it as a constrained minimization problem with simple bounds in this paper we propose a trust region method for solving the reformulation problem with the trust region subproblems being solved by the truncated conjugate gradient cg method which is cost effective other advantages of the proposed method over existing ones include the fact that a good approximated solution to the trust region subproblem can be found by the truncated cg method and is judged in a simple way also the working matrix in each iteration is h instead of the condensed hth where h is a matrix element of the generalized jacobian of the function used in the reformulation as a matter of fact the matrix used is of reduced dimension we pay extra attention to ensure the success of the truncated cg method as well as the feasibility of the iterates with respect to the simple constraints another feature of the proposed method is that we allow the merit function value to be increased at some iterations to speed up the convergence global and superlinear quadratic convergence is shown under standard assumptions numerical results are reported on a subset of problems from the mcplib collection s p dirkse and m c ferris optim methods softw 5 1995 pp 319 345

a trust region algorithm for parametric curve and surface fitting abstract let a family of curves or surfaces be given in parametric form via the model equation x s where x rn r m and s s rd d such that the manifold m s s s is a best fit toscattered data zi i 1y rn in the sense that for some s i i 1 n the sum of the squared least distances i 1 n z i s i 2 2 from the data points to the manifold m is minimal among all such manifolds for robustness our algorithm uses a globally convergent trust region approach in which at each iteration an approximation to the objective function is minimized in a given neighborhood of the current iterate the set s may be all of rd or a closed convex subset in particular it may be chosen so that our theory is applicable to splines

a matrix free approach to build band preconditioners for large scale bound constrained optimization abstract we propose a procedure for building symmetric positive definite band preconditioners for large scale symmetric possibly indefinite linear systems when the coefficient matrix is not explicitly available but matrix vector products involving it can be computed we focus on linear systems arising in newton type iterations within matrix free versions of projected methods for bound constrained nonlinear optimization in this case the structure and the size of the matrix may significantly change in subsequent iterations and preconditioner updating algorithms that exploit information from previous steps cannot be easily applied our procedure is based on a recursive approach that incrementally improves the quality of the preconditioner while requiring a modest number of matrix vector products a strategy for dynamically choosing the bandwidth of the preconditioners is also presented numerical results are provided showing the performance of our preconditioning technique within a trust region newton method for bound constrained optimization

global convergence of a class of trust region algorithms for optimization with simple bounds this paper extends the known excellent global convergence properties of trust region algorithms for unconstrained optimization to the case where bounds on the variables are present weak conditions on the accuracy of the hessian approximations are considered it is also shown that when the strict complementarily condition holds the proposed algorithms reduce to an unconstrained calculation after finitely many iterations allowing a fast asymptotic rate of convergence

on the convergence of a new trust region algorithm in this paper we present a new trust region algorithm for general nonlinear constrained optimization problems the algorithm is based on the l infty exact penalty function under very mild conditions global convergence results for the algorithm are given local convergence properties are also studied it is shown that the penalty parameter generated by the algorithm will be eventually not less than the l 1 norm of the lagrange multipliers at the accumulation point it is proved that the method is equivalent to the sequential quadratic programming method for all large k hence superlinearly convergent results of the sqp method can be applied numerical results are also reported

a new active set algorithm for box constrained optimization an active set algorithm asa for box constrained optimization is developed the algorithm consists of a nonmonotone gradient projection step an unconstrained optimization step and a set of rules for branching between the two steps global convergence to a stationary point is established for a nondegenerate stationary point the algorithm eventually reduces to unconstrained optimization without restarts similarly for a degenerate stationary point where the strong second order sufficient optimality condition holds the algorithm eventually reduces to unconstrained optimization without restarts a specific implementation of the asa is given which exploits the recently developed cyclic barzilai borwein cbb algorithm for the gradient projection step and the recently developed conjugate gradient algorithm cg descent for unconstrained optimization numerical experiments are presented using box constrained problems in the cuter and minpack 2 test problem libraries

a trust region method for minimization of nonsmooth functions with linear constraints we introduce a trust region algorithm for minimization of nonsmooth functions with linear constraints at each iteration the objective function is approximated by a model function that satisfies a set of assumptions stated recently by qi and sun in the context of unconstrained nonsmooth optimization the trust region iteration begins with the resolution of an easy problem as in recent works of martinez and santos and friedlander martinez and santos for smooth constrained optimization in practical implementations we use the infinity norm for defining the trust region which fits well with the domain of the problem we prove global convergence and report numerical experiments related to a parameter estimation problem

a note on exploiting structure when using slack variables we show how to exploit the structure inherent in the linear algebra for constrained nonlinear optimization problems when inequality constraints have been converted to equations by adding slack variables and the problem is solved using an augmented lagrangian method

geometry of interpolation sets in derivative free optimization we consider derivative free methods based on sampling approaches for nonlinear optimization problems where derivatives of the objective function are not available and cannot be directly approximated we show how the bounds on the error between an interpolating polynomial and the true function can be used in the convergence theory of derivative free sampling methods these bounds involve a constant that reflects the quality of the interpolation set the main task of such a derivative free algorithm is to maintain an interpolation sampling set so that this constant remains small and at least uniformly bounded this constant is often described through the basis of lagrange polynomials associated with the interpolation set we provide an alternative more intuitive definition for this concept and show how this constant is related to the condition number of a certain matrix this relation enables us to provide a range of algorithms whilst maintaining the interpolation set so that this condition number or the geometry constant remain uniformly bounded we also derive bounds on the error between the model and the function and between their derivatives directly in terms of this condition number and of this geometry constant

a sequential quadratic programming based algorithm for optimization of gas networks abstract british gas uses a complex heavily looped network of pipes and controllable units compressors and regulators to transmit gas from coastal supply terminals to regional demand points computer algorithms are required for efficient management of the system this paper describes an algorithm for optimal control over periods of up to a day the problem is large scale and highly nonlinear in both objective function and constraints the method is based on sequential quadratic programming and takes account of the structure of the pipeflow equations by means of a reduced gradient technique which eliminates most of the variables from the quadratic subproblems the latter involve only simple bound constraints which are handled efficiently by a conjugate gradient active set algorithm trust region techniques permit use of the exact hessian preserving sparsity more general constraints are handled at an outer level by a truncated augmented lagrangian method results are included for some realistic problems the algorithm is generally applicable to problems with a control structure

a multivariate spectral projected gradient method for bound constrained optimization in this paper we consider a multivariate spectral projected gradient mspg method for bound constrained optimization combined with a quasi newton property the multivariate spectral projected gradient method allows an individual adaptive step size along each coordinate direction on the basis of nonmonotone line search global convergence is established a numerical comparison with the traditional spg method shows that the method is promising

a globally convergent lagrangian barrier algorithm for optimization with general inequality constraints and simple bounds we consider the global and local convergence properties of a class of lagrangian barrier methods for solving nonlinear programming problems in such methods simple bound constraints may be treated separately from more general constraints the objective and general constraint functions are combined in a lagrangian barrier function a sequence of such functions are approximately minimized within the domain defined by the simple bounds global convergence of the sequence of generated iterates to a first order stationary point for the original problem is established furthermore possible numerical difficulties associated with barrier function methods are avoided as it is shown that a potentially troublesome penalty parameter is bounded away from zero this paper is a companion to previous work of ours on augmented lagrangian methods

on the resolution of the generalized nonlinear complementarity problem minimization of a differentiable function subject to box constraints is proposed as a strategy to solve the generalized nonlinear complementarity problem gncp defined on a polyhedral cone it is not necessary to calculate projections that complicate and sometimes even disable the implementation of algorithms for solving these kinds of problems theoretical results that relate stationary points of the function that is minimized to the solutions of the gncp are presented perturbations of the gncp are also considered and results are obtained related to the resolution of gncps with very general assumptions on the data these theoretical results show that local methods for box constrained optimization applied to the associated problem are efficient tools for solving the gncp numerical experiments are presented that encourage the use of this approach

a globally convergent stabilized sqp method sequential quadratic programming sqp methods are a popular class of methods for nonlinearly constrained optimization they are particularly effective for solving a sequence of related problems such as those arising in mixed integer nonlinear programming and the optimization of functions subject to differential equation constraints recently there has been considerable interest in the formulation of stabilized sqp methods which are specifically designed to handle degenerate optimization problems existing stabilized sqp methods are essentially local in the sense that both the formulation and analysis focus on the properties of the methods in a neighborhood of a solution a new sqp method is proposed that has favorable global convergence properties yet under suitable assumptions is equivalent to a variant of the conventional stabilized sqp method in the neighborhood of a solution the method combines a primal dual generalized augmented lagrangian function with a flexible line search to obtain a seque

finite size effect in seven dimensional site percolation on a hypercubic lattice with l7 sites and l up to 21 the site percolation threshold for the spanning of a cluster from top to bottom is found to approach its asymptotic value with a 1 l law in contrast to the ising curie temperature in five dimensions

optimization by direct search new perspectives on some classical and modern methods direct search methods are best known as unconstrained optimization techniques that do not explicitly use derivatives direct search methods were formally proposed and widely applied in the 1960s but fell out of favor with the mathematical optimization community by the early 1970s because they lacked coherent mathematical analysis nonetheless users remained loyal to these methods most of which were easy to program some of which were reliable in the past fifteen years these methods have seen a revival due in part to the appearance of mathematical analysis as well as to interest in parallel and distributed com puting this review begins by briefly summarizing the history of direct search methods and considering the special properties of problems for which they are well suited our focus then turns to a broad class of methods for which we provide a unifying framework that lends itself to a variety of convergence results the underlying principles allow general ization to handle bound constraints and linear constraints we also discuss extensions to problems with nonlinear constraints

two step algorithms for nonlinear optimization with structured applications in this paper we propose extensions to trust region algorithms in which the classical step is augmented with a second step that we insist yields a decrease in the value of the objective function the classical convergence theory for trust region algorithms is adapted to this class of two step algorithms the algorithms can be applied to any problem with variable s whose contribution to the objective function is a known functional form in the nonlinear programming package lancelot they have been applied to update slack variables and variables introduced to solve minimax problems leading to enhanced optimization efficiency extensive numerical results are presented to show the effectiveness of these techniques

new adaptive move limit management strategy for approximate optimization part 2 approximations play an important role in multidisciplinary design optimization by offering system behavior information at a relatively low cost most approximate optimization strategies are sequential in which an optimization of an approximate problem subject to design variable move limits is iteratively repeated until convergence the move limits are imposed to restrict the optimization to regions of the design space in which the approximations provide meaningful information to ensure convergence of the sequence of approximate optimizations to a karush kuhn tucker solution a move limit management strategy is required in a companion paper issues of move limit management are reviewed and a new adaptive strategy for move limit management is developed wujek b a and renaud j e new adaptive move limit management strategy for approximate optimization part 1 aiaa journal vol 36 no 10 1998 pp 1911 1921 with its basis in the provably convergent trust region methodology the trust region ratio approximation method tram strategy utilizes available gradient information and employs a backtracking process using various two point approximation techniques to provide a flexible move limit adjustment factor the tram strategy is successfully implemented in application to several multidisciplinary design optimization test problems in addition a comprehensive study comparing the performance of the tram strategy to existing move limit strategies is conducted results indicate that application of the tram strategy results in increased efficiency for approximate optimization processes these implementation studies highlight the ability of the tram strategy to control the amount of approximation error and efficiently manage the convergence to a karush kuhn tucker solution

a penalty free type nonmonotone trust region method for nonlinear constrained optimization in this paper we propose a penalty free type nonmonotone trust region method for solving general nonlinear programming problem the algorithmic framework yields global convergence without using any penalty function we analyze the global convergence of the main algorithm for the degenerate problems and also give the globally convergent results under the linear independence constraint qualification the preliminary numerical tests are reported

second order nonsmooth optimization for h synthesis the standard way to compute h feedback controllers uses algebraic riccati equations and is therefore of limited applicability here we present a new approach to the h output feedback control design problem which is based on nonlinear and nonsmooth mathematical programming techniques our approach avoids the use of lyapunov variables and is therefore flexible in many practical situations

reformulation of variational inequalities on a simplex and compactification of complementarity problems many variational inequality problems vips can be reduced by a compactification procedure to a vip on the canonical simplex reformulations of this problem are studied including smooth reformulations with simple constraints and unconstrained reformulations based on the penalized fischer burmeister function it is proved that bounded level set results hold for these reformulations under quite general assumptions on the operator therefore it can be guaranteed that minimization algorithms generate bounded sequences and under monotonicity conditions these algorithms necessarily find solutions of the original problem some numerical experiments are presented

finite size scaling for percolation above six dimensions we consider percolation in d dimensions for finite samples of linear size l theoretical arguments are presented to show that for d 6 the shift in the percolation threshold pc l pc behaves like a l2 b ld3 for periodic boundary conditions a 0 these predictions are consistent with recent simulations is seven dimensions

an active set newton algorithm for large scale nonlinear programs with box constraints a new algorithm for large scale nonlinear programs with box constraints is introduced the algorithm is based on an efficient identification technique of the active set at the solution and on a nonmonotone stabilization technique it possesses global and superlinear convergence properties under standard assumptions a new technique for generating test problems with known characteristics is also introduced the implementation of the method is described along with computational results for large scale problems

a trust region method for norm constrained problems a trust region method for the solution of nonlinear optimization problems with norm constraints is analyzed such optimization problems arise in parameter identification and nonlinear eigenvalue problems the characteristics of these applications motivate the formulation and analysis of the trust region method the algorithms studied here allow for inexact gradient information and the use of subspace methods for the approximate solution of subproblems characterizations and the descent properties of trust region steps are given criteria for the existence of successful iterations under inexact gradient information and under the use of subspace methods are established and global convergence of the method is proven

a subspace limited memory quasi newton algorithm for large scale nonlinear bound constrained optimization in this paper we propose a subspace limited memory quasi newton method for solving large scale optimization with simple bounds on the variables the limited memory quasi newton method is used to update the variables with indices outside of the active set while the projected gradient method is used to update the active variables the search direction consists of three parts a subspace quasi newton direction and two subspace gradient and modified gradient directions our algorithm can be applied to large scale problems as there is no need to solve any subproblems the global convergence of the method is proved and some numerical results are also given

newton s method for large bound constrained optimization problems we analyze a trust region version of newton s method for bound constrained problems our approach relies on the geometry of the feasible set not on the particular representation in terms of constraints the convergence theory holds for linearly constrained problems and yields global and superlinear convergence without assuming either strict complementarity or linear independence of the active constraints we also show that the convergence theory leads to an efficient implementation for large bound constrained problems

percolation in high dimensions is not understood abstract the number of spanning clusters in four to nine dimensions does not fully follow the expected size dependence for random percolation

convergence properties of trust region methods for linear and convex constraints we develop a convergence theory for convex and linearly constrained trust region methods which only requires that the step between iterates produce a sufficient reduction in the trust region subproblem global convergence is established for general convex constraints while the local analysis is for linearly constrained problems the main local result establishes that if the sequence converges to a nondegenerate stationary point then the active constraints at the solution are identified in a finite number of iterations as a consequence of the identification properties we develop rate of convergence results by assuming that the step is a truncated newton method our development is mainly geometrical this approach allows the development of a convergence theory without any linear independence assumptions

global minimization using an augmented lagrangian method with variable lower level constraints a novel global optimization method based on an augmented lagrangian framework is introduced for continuous constrained nonlinear optimization problems at each outer iteration k the method requires the varepsilon k global minimization of the augmented lagrangian with simple constraints where varepsilon k to varepsilon global convergence to an varepsilon global minimizer of the original problem is proved the subproblems are solved using the bb method numerical experiments are presented

least frobenius norm updating of quadratic models that satisfy interpolation conditions quadratic models of objective functions are highly useful in many optimization algorithms they are updated regularly to include new information about the objective function such as the difference between two gradient vectors we consider the case however when each model interpolates some function values so an update is required when a new function value replaces an old one we let the number of interpolation conditions m say be such that there is freedom in each new quadratic model that is taken up by minimizing the frobenius norm of the second derivative matrix of the change to the model this variational problem is expressed as the solution of an m n 1 m n 1 system of linear equations where n is the number of variables of the objective function further the inverse of the matrix of the system provides the coefficients of quadratic lagrange functions of the current interpolation problem a method is presented for updating all these coefficients in m n 2 operations which allows the model to be updated too an extension to the method is also described that suppresses the constant terms of the lagrange functions these techniques have a useful stability property that is investigated in some numerical experiments

an active set quasi newton method with projected search for bound constrained minimization we analyze an active set quasi newton method for large scale bound constrained problems our approach combines the accurate active set identification function and the projected search both of these strategies permit fast change in the working set the limited memory method is employed to update the inactive variables while the active variables are updated by simple rules a further division of the active set enables the global convergence of the new algorithm numerical tests demonstrate the efficiency and performance of the present strategy and its comparison with some existing active set strategies

mean variance approximations to expected utility abstract it is often asserted that the application of mean variance analysis assumes normal gaussian return distributions or quadratic utility functions this common mistake confuses sufficient versus necessary conditions for the applicability of modern portfolio theory if one believes as does the author that choice should be guided by the expected utility maxim then the necessary and sufficient condition for the practical use of mean variance analysis is that a careful choice from a mean variance efficient frontier will approximately maximize expected utility for a wide variety of concave risk averse utility functions this paper reviews a half century of research on mean variance approximations to expected utility the many studies in this field have been generally supportive of mean variance analysis subject to certain initially unanticipated caveats

finite element model updating for structural applications abstract a novel method for performing model updating on finite element models is presented the approach is particularly tailored to modal analyses of buildings by which the lowest frequencies obtained by using sensors and system identification approaches need to be matched to the numerical ones predicted by the model this is done by optimizing some unknown material parameters such as mass density and young s modulus of the materials and or the boundary conditions which are often known only approximately in particular this is the case when considering historical buildings the straightforward application of a general purpose optimizer can be impractical given the large size of the model involved in the paper we show that by slightly modifying the projection scheme used to compute the eigenvalues at the lowest end of the spectrum one can obtain local parametric reduced order models that embedded in a trust region scheme form the basis for a reliable and efficient specialized algorithm we describe an optimization strategy based on this approach and we provide numerical experiments that confirm its effectiveness and accuracy

convergence of quasi newton matrices generated by the symmetric rank one update quasi newton algorithms for unconstrained nonlinear minimization generate a sequence of matrices that can be considered as approximations of the objective function second derivatives this paper gives conditions under which these approximations can be proved to converge globally to the true hessian matrix in the case where the symmetric rank one update formula is used the rate of convergence is also examined and proven to be improving with the rate of convergence of the underlying iterates the theory is confirmed by some numerical experiments that also show the convergence of the hessian approximations to be substantially slower for other known quasi newton formulae

correction to the paper on global convergence of a class of trust region algorithms for optimization with simple bounds a correction is made to the paper entitled global convergence of a class of trust region algorithms for optimization with simple bounds corm gould and toint siam j numer anal 25 1988 pp 433 460 first an error is pointed out in the proof of the fact that the correct set of active bounds is determined after a finite number of iterations a new proof of the relevant theorem is given

trust region algorithm with two subproblems for bound constrained problems abstract we discuss the trust region algorithm for bound constrained problems and propose the corresponding trust region algorithm with two subproblems the computational cost is reduced by properly using bound constrained subproblem without trust region constraint we prove that this algorithm is globally convergent numerical results on the cuter test problems show that the new algorithm is as reliable as the traditional one and more efficient

a trust region method for parabolic boundary control problems in this paper we develop a trust region algorithm for constrained parabolic boundary control problems for the computation of a trust region step we propose an iterative scheme which is a projected form of the steihaug trust region conjugate gradient method to ensure the good local convergence properties in the terminal phase a smoothing step at each iteration is added this step and the projection require the modification of the standard trust region algorithm and its convergence proof the algorithm has sup norm convergence in the terminal phase and l2 convergence in the global phase the results are illustrated for a parabolic boundary control problem

an implicit filtering algorithm for optimization of functions with many local minima in this paper we describe and analyze an algorithm for certain box constrained optimization problems that may have several local minima a paradigm for these problems is one in which the function to be minimized is the sum of a simple function such as a convex quadratic and high frequency low amplitude terms that cause local minima away from the global minimum of the simple function our method is gradient based and therefore the performance can be improved by use of quasi newton methods

a global convergence theory for the celis dennis tapia trust region algorithm for constrained optimization a global convergence theory for a class of trust region algorithms for solving the equality constrained optimization problem is presented this theory is sufficiently general that it holds for any algorithm that generates steps giving at least a fraction of cauchy decrease in the quadratic model of the constraints and that uses the augmented lagrangian as a merit function this theory is used to establish global convergence of the 1984 celis dennis tapia algorithm with a different scheme for updating the penalty parameter the behavior of the penalty parameter is also discussed

a new subspace limited memory bfgs algorithm for large scale bound constrained optimization an active set limited memory bfgs algorithm for large scale bound constrained optimization is introduced the active sets are based on guessing technique to be identified at each iteration the search direction in free subspace is determined by limited memory bfgs l bfgs algorithm which provides an efficient means for attacking large scale optimization problems the implementations of the method on cute test problems are described

convergence properties of minimization algorithms for convex constraints using a structured trust region in this paper we present a class of trust region algorithms for minimization problems within convex feasible regions in which the structure of the problem is explicitly used in the definition of the trust region this development is intended to reflect the possibility that some parts of the problem may be more accurately modelled than others a common occurrence in large scale nonlinear applications after describing the structured trust region mechanism we prove global convergence for all algorithms in our class

upper critical dimension of the negative weight percolation problem by means of numerical simulations we investigate the geometric properties of loops on hypercubic lattice graphs in dimensions d 2 through 7 where edge weights are drawn from a distribution that allows for positive and negative weights we are interested in the appearance of system spanning loops of total negative weight the resulting negative weight percolation nwp problem is fundamentally different from conventional percolation as we have seen in previous studies of this model for the two dimensional case here we characterize the transition for hypercubic systems where the aim of the present study is to get a grip on the upper critical dimension d u of the nwp problem for the numerical simulations we employ a mapping of the nwp model to a combinatorial optimization problem that can be solved exactly by using sophisticated matching algorithms we characterize the loops via observables similar to those in percolation theory and perform finite size scaling analyses e g three dimensional hypercubic systems with side length up to l 56 sites in order to estimate the critical properties of the nwp phenomenon we find our numerical results consistent with an upper critical dimension d u 6 for the nwp problem

convergence to second order stationary points in inequality constrained optimization we propose a new algorithm for the nonlinear inequality constrained minimization problem and prove that it generates a sequence converging to points satisfying the kkt second order necessary conditions for optimality the algorithm is a line search algorithm using directions of negative curvature and it can be viewed as a nontrivial extension of corresponding known techniques from unconstrained to constrained problems the main tools employed in the definition and in the analysis of the algorithm are a differentiable exact penalty function and results from the theory of lc1 functions

on the accurate identification of active constraints we consider nonlinear programs with inequality constraints and we focus on the problem of identifying those constraints which will be active at an isolated local solution the correct identification of active constraints is important from both a theoretical and a practical point of view such an identification removes the combinatorial aspect of the problem and locally reduces the inequality constrained minimization problem to an equality constrained problem which can be more easily dealt with we present a new technique which identifies active constraints in a neighborhood of a solution and which requires neither complementary slackness nor uniqueness of the multipliers we also present extensions to variational inequalities and numerical examples illustrating the identification technique

an efficient algorithm for large scale nonlinear programming problems with simple bounds on the variables in this paper we present a new conjugate gradient algorithm for nonlinear programming problems with simple bounds on the variables the method is an extension of our conjugate gradient algorithm described in r pytlak ima j numer anal 14 1994 pp 443 460 the simple constraints on the variables are treated by a projection under mild assumptions the method is globally convergent the algorithm has the property whereby many constraints can leave or enter an active set of constraints at one iteration if the strict complementarity condition is satisfied the active set at a solution is identified in a finite number of iterations the algorithm has been tested on problems with more than 10 000 variables

partitioned variable metric updates for large structured optimization problems this paper presents a minimization method based on the idea of partitioned updating of the hessian matrix in the case where the objective function can be decomposed in a sum of convex element functions this situation occurs in a large class of practical problems including nonlinear finite elements calculations some theoretical and algorithmic properties of the update are discussed and encouraging numerical results are presented

a direct active set algorithm for large sparse quadratic programs with simple bounds we show how a direct active set method for solving definite and indefinite quadratic programs with simple bounds can be efficiently implemented for large sparse problems all of the necessary factorizations can be carried out in a static data structure that is set up before the numeric computation begins the space required for these factorizations is no larger than that required for a single sparse cholesky factorization of a matrix with the same sparsity structure as the hessian of the quadratic we propose several improvements to this basic algorithm a new way to find a search direction in the indefinite case that allows us to free more than one variable at a time and a new heuristic method for finding a starting point these ideas are motivated by the two norm trust region problem additionally we also show how projection techniques can be used to add several constraints to the active set at each iteration our experimental results show that an algorithm with these improvements runs much faster than the basic algorithm for positive definite problems and finds local minima with lower function values for indefinite problems

superlinear and quadratic convergence of affine scaling interior point newton methods for problems with simple bounds without strict complementarity assumption a class of affine scaling interior point methods for bound constrained optimization problems is introduced which are locally q superlinear or q quadratic convergent it is assumed that the strong second order sufficient optimality conditions at the solution are satisfied but strict complementarity is not required the methods are modifications of the affine scaling interior point newton methods introduced by t f coleman and y li math programming 67 189 224 1994 there are two modifications one is a modification of the scaling matrix the other one is the use of a projection of the step to maintain strict feasibility rather than a simple scaling of the step a comprehensive local convergence analysis is given a simple example is presented to illustrate the pitfalls of the original approach by coleman and li in the degenerate case and to demonstrate the performance of the fast converging modifications developed in this paper

mesh independence of the gradient projection method for optimal control problems the gradient projection method and its application to optimal control have been analyzed with regard to the rate of convergence quite extensively here another aspect of this method is considered in finite dimension the set of active indices is identified after finitely many iterations under mild nondegeneracy assumptions however it is clear that this property is restricted to the finite dimensional case in this paper for example a sequence of discretized optimal control problems is considered and it is observed that the number of steps to identify all active indices increases with the refinement of the discretization a result analogous to the finite dimensional result is valid in this situation if identification of active indices is understood in the correct light this paper shows that if a different termination criterion is imposed then the number of necessary steps for termination is indeed mesh independent numerical observations illustrating this result are reported for various examples fr

global convergence of trust region interior point algorithms for infinite dimensional nonconvex minimization subject to pointwise bounds a class of interior point trust region algorithms for infinite dimensional nonlinear optimization subject to pointwise bounds in l p banach spaces 2 le p le infty is formulated and analyzed the problem formulation is motivated by optimal control problems with l p controls and pointwise control constraints the interior point trust region algorithms are generalizations of those recently introduced by coleman and li siam j optim 6 1996 pp 418 445 for finite dimensional problems many of the generalizations derived in this paper are also important in the finite dimensional context all first and second order global convergence results known for trust region methods in the finite dimensional setting are extended to the infinite dimensional framework of this paper

high order evaluation complexity for convexly constrained optimization with non lipschitzian group sparsity terms this paper studies high order evaluation complexity for partially separable convexly constrained optimization involving non lipschitzian group sparsity terms in a nonconvex objective function we propose a partially separable adaptive regularization algorithm using a pth order taylor model and show that the algorithm needs at most o epsilon p 1 p q 1 evaluations of the objective function and its first p derivatives whenever they exist to produce an epsilon delta approximate qth order stationary point our algorithm uses the underlying rotational symmetry of the euclidean norm function to build a lipschitzian approximation for the non lipschitzian group sparsity terms which are defined by the group ell 2 ell a norm with a in 0 1 the new result shows that the partially separable structure and non lipschitzian group sparsity terms in the objective function do not affect the worst case evaluation complexity order

a limited memory quasi newton trust region method for box constrained optimization by means of wolfe conditions strategy we propose a quasi newton trust region method to solve box constrained optimization problems this method is an adequate combination of the compact limited memory bfgs and the trust region direction while the generated point satisfies the wolfe conditions and therefore maintains a positive definite approximation to the hessian of the objective function the global convergence and the quadratic convergence of this method are established under suitable conditions finally we compare our algorithms iwtral and ibwtral with an active set trust region algorithm astral xu and burke 2007 on the cutest box constrained test problems presented by gould et al 2015 numerical results show that the presented method is competitive and totally interesting for solving box constrained optimization

a hybrid method for the nonlinear least squares problem with simple bounds abstract this paper presents a new method with trust region technique for solving the nonlinear least squares problem with lower and upper bounds on the variables the proposed method constructs trust region constraints that are ellipses centered at the iterative points in such a way that they lie in the interior of the feasible region thus the method belongs to the class of interior point methods and hence we may expect that the generated sequence approaches a solution smoothly without the combinatorial complications inherent to traditional active set methods we establish a convergence theorem for the proposed method and show its practical efficiency by numerical experiments