two level linear programming decentralized planning has long been recognized as an important decision making problem many approaches based on the concepts of large scale system decomposition have generally lacked the ability to model the type of truly independent subsystems which often exist in practice multilevel programming models partition control over decision variables among ordered levels within a hierarchical planning structure a planner at one level of the hierarchy may have his objective function and set of feasible decisions determined in part by other levels however his control instruments may allow him to influence the policies at other levels and thereby improve his own objective function this paper examines the special case of the two level linear programming problem geometric characterizations and algorithms are presented with some examples the goal is to demonstrate the tractability of such problems and motivate a wider interest in their study

convergence of the restricted nelder mead algorithm in two dimensions the nelder mead algorithm a longstanding direct search method for unconstrained optimization published in 1965 is designed to minimize a scalar valued function f of n real variables using only function values without any derivative information each nelder mead iteration is asso ciated with a nondegenerate simplex defined by n 1 vertices and their function values a typical iteration produces a new simplex by replacing the worst vertex by a new point despite the method s widespread use theoretical results have been limited for strictly convex objective functions of one variable with bounded level sets the algorithm always converges to the minimizer for such func tions of two variables the diameter of the simplex converges to zero but examples constructed by mckinnon show that the algorithm may converge to a nonminimizing point this paper considers the restricted nelder mead algorithm a variant that does not allow expansion steps in two dimen sions we show that for any nondegenerate starting simplex and any twice continuously differentiable function with positive definite hessian and bounded level sets the algorithm always converges to the minimizer the proof is based on treating the method as a discrete dynamical system and relies on several techniques that are nonstandard in convergence proofs for unconstrained optimization

benchmarking optimization software with performance profiles we propose performance profiles distribution functions for a performance metric as a tool for benchmarking and comparing optimization software we show that performance profiles combine the best features of other tools for performance evaluation

a smoothing method for mathematical programs with equilibrium constraints the mathematical program with equilibrium constraints mpec is an optimization problem with variational inequality constraints mpec problems include bilevel programming problems as a particular case and have a wide range of applications mpec problems with strongly monotone variational inequalities are considered in this paper they are transformed into an equivalent one level nonsmooth optimization problem then a sequence of smooth regular problems that progressively approximate the nonsmooth problem and that can be solved by standard available software for constrained optimization is introduced it is shown that the solutions stationary points of the approximate problems converge to a solution stationary point of the original mpec problem numerical results showing viability of the approach are reported

stability in the presence of degeneracy and error estimation given an approximation to a local minimizer to a nonlinear optimization problem and to associated multipliers we obtain a tight error estimate in terms of the violation of the first order conditions our results apply to degenerate optimization problems where independence of the active constraint gradients and strict complementarity can be violated

mesh adaptive direct search algorithms for constrained optimization this paper addresses the problem of minimization of a nonsmooth function under general nonsmooth constraints when no derivatives of the objective or constraint functions are available we introduce the mesh adaptive direct search mads class of algorithms which extends the generalized pattern search gps class by allowing local exploration called polling in an asymptotically dense set of directions in the space of optimization variables this means that under certain hypotheses including a weak constraint qualification due to rockafellar mads can treat constraints by the extreme barrier approach of setting the objective to infinity for infeasible points and treating the problem as unconstrained the main gps convergence result is to identify limit points hat x where the clarke generalized derivatives are nonnegative in a finite set of directions called refining directions although in the unconstrained case nonnegative combinations of these directions span the whole space the fact that there can only be finitely many gps refining directions limits rigorous justification of the barrier approach to finitely many linear constraints for gps the main result of this paper is that the general mads framework is flexible enough to allow the generation of an asymptotically dense set of refining directions along which the clarke derivatives are nonnegative we propose an instance of mads for which the refining directions are dense in the hypertangent cone at hat x with probability 1 whenever the iterates associated with the refining directions converge to a single hat x the instance of mads is compared to versions of gps on some test problems we also illustrate the limitation of our results with examples

on the convergence of the multidirectional search algorithm this paper presents the convergence analysis for the multidirectional search algorithm a direct search method for unconstrained minimization the analysis follows the classic lines of proofs of convergence for gradient related methods the novelty of the argument lies in the fact that explicit calculation of the gradient is unnecessary although it is assumed that the function is continuously differentiable over some subset of the domain the proof can be extended to treat most nonsmooth cases of interest the argument breaks down only at points where the derivative exists but is not continuous finally it is shown how a general convergence theory can be developed for an entire class of direct search methods which includes such methods as the factorial design algorithm and the pattern search algorithm that share a key feature of the multidirectional search algorithm

exact penalization of mathematical programs with equilibrium constraints we study theoretical and computational aspects of an exact penalization approach to mathematical programs with equilibrium constraints mpecs in the first part we prove that a mangasarian fromovitz type condition ensures the existence of a stable local error bound at the root of a real valued nonnegative piecewise smooth function a specification to nonsmooth formulations of equilibrium constraints e g complementarity conditions or normal equations provides conditions which guarantee the existence of a nonsmooth exact penalty function for mpecs in the second part we study a trust region minimization method for a class of composite nonsmooth functions which comprises exact penalty functions arising from mpecs we prove a global convergence result for the general method and incorporate a penalty update rule a further specification results in an sqp trust region method for mpecs based on an ell 1 penalty function

fortified descent simplicial search method a general approach we propose a new simplex based direct search method for unconstrained minimization of a real valued function f of n variables as in other methods of this kind the intent is to iteratively improve an n dimensional simplex through certain reflection expansion contraction steps the method has three novel features first a user chosen integer bar m k specifies the number of good vertices to be retained in constructing the initial trial simplices reflected then either expanded or contracted at iteration k second a trial simplex is accepted only when it satisfies the criteria of fortified descent which are stronger than the criterion of strict descent used in most direct search methods third the number of additional function evaluations needed to check a trial reflected expanded simplex for fortified descent can be controlled if one of the initial trial simplices satisfies the fortified descent criteria it is accepted as the new simplex otherwise the simplex is shrunk a fraction of the way toward a best vertex and the process is restarted etc until either a trial simplex is accepted or the simplex effectively has shrunk to a single point we prove several theoretical properties of the new method if f is continuously differentiable bounded below and uniformly continuous on its lower level set and we choose bar m k with the same value at all iterations k then every cluster point of the generated sequence of iterates is a stationary point the same conclusion holds if the function is continuously differentiable bounded below and we choose bar m k 1 at all iterations k

pattern search algorithms for bound constrained minimization we present a convergence theory for pattern search methods for solving bound constrained nonlinear programs the analysis relies on the abstract structure of pattern search methods and an understanding of how the pattern interacts with the bound constraints this analysis makes it possible to develop pattern search methods for bound constrained problems while only slightly restricting the flexibility present in pattern search methods for unconstrained problems we prove global convergence despite the fact that pattern search methods do not have explicit information concerning the gradient and its projection onto the feasible region and consequently are unable to enforce explicitly a notion of sufficient feasible decrease

expressing complementarity problems in an algebraic modeling language and communicating them to solvers diverse problems in optimization engineering and economics have natural formulations in terms of complementarity conditions which state in their simplest form that either a certain nonnegative variable must be zero or a corresponding inequality must hold with equality or both a variety of algorithms has been devised for solving problems expressed in terms of complementarity conditions it is thus attractive to consider extending algebraic modeling languages which are widely used for sending ordinary equations and inequality constraints to solvers so that they can express complementarity problems directly we describe an extension to the ampl modeling language that can express the most common complementarity conditions in a concise and flexible way through the introduction of a single new complements operator we present details of an efficient implementation that incorporates an augmented presolve phase to simplify complementarity problems and that converts complementarity conditions to a canonical form convenient for solvers

trust region model management in multidisciplinary design optimization a common engineering practice is the use of approximation models in place of expensive computer simulations to drive a multidisciplinary design process based on nonlinear programming techniques the use of approximation strategies is designed to reduce the number of detailed costly computer simulations required during optimization while maintaining the pertinent features of the design problem this paper overviews the current state of the art in model management strategies for approximate optimization model management strategies coordinate the interaction between the optimization and the fidelity of the approximation models so as to ensure that the process converges to a solution of the original design problem approximations play an important role in multidisciplinary design optimization mdo by offering system behavior information at a relatively low cost most approximate mdo strategies are sequential in which an optimization of an approximate problem subject to design variable move limits is iteratively repeated until convergence the move limits or trust region are imposed to restrict the optimization to regions of the design space in which the approximations provide meaningful information in order to insure convergence of the sequence of approximate optimizations to a karush kuhn tucker solution a trust region model management or move limit strategy is required in this paper recent developments in approximate mdo strategies and issues of trust region model management in mdo are reviewed

estimation of distribution algorithm for a class of nonlinear bilevel programming problems in this paper a novel evolutionary algorithm called estimation of distribution algorithm eda is proposed for solving a special class of nonlinear bilevel programming problems blpps in which the lower level problem is a convex programming problem for each given upper level decision this special type of blpp is transformed into a equivalent single level constrained optimization problem using the karush kuhn er conditions of the lower level problem then we propose an eda based on the statistical information of the superior candidate solutions to solve the transformed problem we stress that the new population of individuals is sampled from the probabilistic distribution of those superior solutions thus one of the main advantages of eda over most other meta heuristics is its ability to adapt the operators to the structure of the problem although adaptation in eda is usually limited by the initial choice of the probabilistic model in addition two specific rules are established in the initialization procedure to make use of the hierarchical structure of blpps and to handle the constraints moreover without requiring the differentiability of the objective function or the convexity of the search space of the equivalent problem the proposed algorithm can address nonlinear blpps with non differentiable or non convex upper level objective function and upper level constraint functions finally the proposed algorithm has been applied to 16 benchmark problem in five of these problems all of the upper level variables and lower level variables are 10 dimensional the numerical results compared with those of other methods reveal the feasibility and effectiveness of the proposed algorithm

a new regularization method for mathematical programs with complementarity constraints with strong convergence properties mathematical programs with equilibrium or complementarity constraints mpecs form a difficult class of optimization problems the feasible set has a very special structure and violates most of the standard constraint qualifications therefore one typically applies specialized algorithms in order to solve mpecs one very prominent class of specialized algorithms are the regularization or relaxation methods the first regularization method for mpecs is due to scholtes siam j optim 11 2001 pp 918 936 but in the meantime there exist a number of different regularization schemes which try to relax the difficult constraints in different ways however almost all regularization methods converge to c stationary points only which is a very weak stationarity concept an exception is a recent method by kadrani dussault and benchakroun siam j optim 20 2009 pp 78 103 whose limit points are shown to be m stationary here we provide a new regularization method which also converges to m sta

on the global convergence of derivative free methods for unconstrained optimization in this paper starting from the study of the common elements that some globally convergent direct search methods share a general convergence theory is established for unconstrained minimization methods employing only function values the introduced convergence conditions are useful for developing and analyzing new derivative free algorithms with guaranteed global convergence as examples we describe three new algorithms which combine pattern and line search approaches

on the convergence of pattern search algorithms we introduce an abstract definition of pattern search methods for solving nonlinear unconstrained optimization problems our definition unifies an important collection of optimization methods that neither compute nor explicitly approximate derivatives we exploit our characterization of pattern search methods to establish a global convergence theory that does not enforce a notion of sufficient decrease our analysis is possible because the iterates of a pattern search method lie on a scaled translated integer lattice this allows us to relax the classical requirements on the acceptance of the step at the expense of stronger conditions on the form of the step and still guarantee global convergence

on the convergence of grid based methods for unconstrained optimization the convergence of direct search methods for unconstrained minimization is examined in the case where the underlying method can be interpreted as a grid or pattern search over successively refined meshes an important aspect of the main convergence result is that translation rotation scaling and shearing of the successive grids are allowed

quantization of abelian varieties distributional sections and the transition from kahler to real polarizations abstract we study the dependence of geometric quantization of the standard symplectic torus on the choice of invariant polarization real and mixed polarizations are interpreted as degenerate complex structures using a weak version of the equations of covariant constancy and the weil brezin expansion to describe distributional sections we give a unified analytical description of the quantization spaces for all non negative polarizations the blattner kostant sternberg bks pairing maps between half form corrected quantization spaces for different polarizations are shown to be transitive and related to an action of sp 2 g r moreover these maps are shown to be unitary

an outer approximation algorithm for a class of mixed integer nonlinear programs an outer approximation algorithm is presented for solving mixed integer nonlinear programming problems of a particular class linearity of the integer or discrete variables and convexity of the nonlinear functions involving continuous variables are the main features in the underlying mathematical structure based on principles of decomposition outer approximation and relaxation the proposed algorithm effectively exploits the structure of the problems and consists of solving an alternating finite sequence of nonlinear programming subproblems and relaxed versions of a mixed integer linear master program convergence and optimality properties of the algorithm are presented as well as a general discussion on its implementation numerical results are reported for several example problems to illustrate the potential of the proposed algorithm for programs in the class addressed in this paper finally a theoretical comparison with generalized benders decomposition is presented on the lower bounds predicted by the relaxed master programs

pattern search methods for linearly constrained minimization we extend pattern search methods to linearly constrained minimization we develop a general class of feasible point pattern search algorithms and prove global convergence to a karush kuhn tucker point as in the case of unconstrained minimization pattern search methods for linearly constrained problems accomplish this without explicit recourse to the gradient or the directional derivative of the objective key to the analysis of the algorithms is the way in which the local search patterns conform to the geometry of the boundary of the feasible region

concrete structure design using mixed integer nonlinear programming with complementarity constraints we present a mixed integer nonlinear programming minlp formulation to achieve minimum cost designs for reinforced concrete rc structures that satisfy building code requirements the objective function includes material and labor costs for concrete steel reinforcing bars and formwork according to typical contractor methods restrictions enforce correct geometry of the cross section dimensions for each element and relative sizes of cross section dimensions of elements within the structure other restrictions define a stiffness and displacement correlation among all structural elements via finite element analysis the design of minimum cost rc structures introduces a new class of optimization problems namely mixed integer nonlinear programs with complementarity constraints the complementarity constraints are used to model rc element strength and american concrete institute code required safety factors we reformulate the complementarity constraints as nonlinear equations and show that the resulting

pattern search ranking and selection algorithms for mixed variable simulation based optimization the class of generalized pattern search gps algorithms for mixed variable optimization is extended to problems with stochastic objective functions because random noise in the objective function makes it more difficult to compare trial points and ascertain which points are truly better than others replications are needed to generate sufficient statistical power to draw conclusions rather than comparing pairs of points the approach taken here augments pattern search with a ranking and selection r s procedure which allows for comparing many function values simultaneously asymptotic convergence for the algorithm is established numerical issues are discussed and performance of the algorithm is studied on a set of test problems

a new branch and bound algorithm for solving quadratic programs with linear complementarity constraints in order to find a global solution for a quadratic program with linear complementarity constraints qplcc more quickly than some existing methods we consider to embed a local search method into a global search method to say more specifically in a branch and bound algorithm for solving qplcc when we find a new feasible solution to the problem we utilize an extreme point algorithm to obtain a locally optimal solution which can provide a better bound and help us to trim more branches so the global algorithm can be accelerated a preliminary numerical experiment was conducted which supports the new algorithm

constrained minimum ber multiuser detection a new linear multiuser detector that directly minimizes the bit error rate ber subject to a set of reasonable constraints is proposed it is shown that the constrained ber cost function has a unique global minimum this allows us to develop an efficient newton barrier method for finding the coefficients of the proposed detector using information about timing amplitudes channels and the signature signals of all users although the new detector cannot be shown to be optimal among linear multiuser detectors without the constraints imposed extensive simulations demonstrate that it achieves the lowest ber furthermore in some cases the ber of the proposed detector can be significantly lower than that of the decorrelating and mmse detectors

the gelfand cetlin system and quantization of the complex flag manifolds the construction of angle action variables for collective completely integrable systems is described and the associated bohr sommerfeld sets are determined the quantization method of sniatycki applied to such systems gives formulas for multiplicities for the gelfand cetlin system on complex flag manifolds we show that these formulas give the correct answers for the multiplicities of the associated representations

convex two level optimization in this paper a model for a two level planning problem is presented in the form of a static stackelberg game by assumption play is sequential and noncooperative however the leader can influence the actions of the followers through a set of coordination variables while the followers responses may partly determine the leader s payoff under certain convexity assumptions it is shown that the feasible region induced by the leader is continuous in the original problem variables this observation coupled with two corollary results are used as a basis for a hybrid algorithm which clings to the inducible region until a local optimum is found a branching scheme is then employed to located other segments of the region eventually terminating with the global optimum a number of examples are given to highlight the results while the algorithm s performance is tested in a comparison with two other procedures

geometry of interpolation sets in derivative free optimization we consider derivative free methods based on sampling approaches for nonlinear optimization problems where derivatives of the objective function are not available and cannot be directly approximated we show how the bounds on the error between an interpolating polynomial and the true function can be used in the convergence theory of derivative free sampling methods these bounds involve a constant that reflects the quality of the interpolation set the main task of such a derivative free algorithm is to maintain an interpolation sampling set so that this constant remains small and at least uniformly bounded this constant is often described through the basis of lagrange polynomials associated with the interpolation set we provide an alternative more intuitive definition for this concept and show how this constant is related to the condition number of a certain matrix this relation enables us to provide a range of algorithms whilst maintaining the interpolation set so that this condition number or the geometry constant remain uniformly bounded we also derive bounds on the error between the model and the function and between their derivatives directly in terms of this condition number and of this geometry constant

objective derivative free methods for constrained optimization we propose feasible descent methods for constrained minimization that do not make explicit use of the derivative of the objective function the methods iteratively sample the objective function value along a finite set of feasible search arcs and decrease the sampling stepsize if an improved objective function value is not sampled the search arcs are obtained by projecting search direction rays onto the feasible set and the search directions are chosen such that a subset approximately generates the cone of first order feasible variations at the current iterate we show that these methods have desirable convergence properties under certain regularity assumptions on the constraints in the case of linear constraints the projections are redundant and the regularity assumptions hold automatically numerical experience with the methods in the linearly constrained case is reported

generalized pattern searches with derivative information a common question asked by users of direct search algorithms is how to use derivative information at iterates where it is available this paper addresses that question with respect to generalized pattern search gps methods for unconstrained and linearly constrained optimization specifically this paper concentrates on the gps pollstep polling is done to certify the need to refine the current mesh and it requires o n function evaluations in the worst case we show that the use of derivative information significantly reduces the maximum number of function evaluations necessary for pollsteps even to a worst case of a single function evaluation with certain algorithmic choices given here furthermore we show that rather rough approximations to the gradient are sufficient to reduce the pollstep to a single function evaluation we prove that using these less expensive pollsteps does not weaken the known convergence properties of the method all of which depend only on the pollstep

mathematical programs with complementarity constraints we study mathematical programs with complementarity constraints several stationarity concepts based on a piecewise smooth formulation are presented and compared the concepts are related to stat

genetic algorithm based on simplex method for solving linear quadratic bilevel programming problem the bilevel programming problems are useful tools for solving the hierarchy decision problems in this paper a genetic algorithm based on the simplex method is constructed to solve the linear quadratic bilevel programming problem lqbp by use of kuhn tucker conditions of the lower level programming the lqbp is transformed into a single level programming which can be simplified to a linear programming by the chromosome according to the rule thus in our proposed genetic algorithm only the linear programming is solved by the simplex method to obtain the feasibility and fitness value of the chromosome finally the feasibility of the proposed approach is demonstrated by the example

a heuristic iterated subspace minimization method with pattern search for unconstrained optimization recently an increasing attention was paid on different procedures for an unconstrained optimization problem when the information of the first derivatives is unavailable or unreliable in this paper we consider a heuristic iterated subspace minimization method with pattern search for solving such unconstrained optimization problems the proposed method is designed to reduce the total number of function evaluations for the implementation of high dimensional problems meanwhile it keeps the advantages of general pattern search algorithm i e the information of the derivatives is not needed at each major iteration of such a method a low dimensional manifold the iterated subspace is constructed and an approximate minimizer of the objective function in this manifold is then determined by a pattern search method numerical results on some classic test examples are given to show the efficiency of the proposed method in comparison with a conventional pattern search method and a derivative free method

on the solution of a minimum weight elastoplastic problem involving displacement and complementarity constraints this paper deals with a special class of minimum weight design problem involving discretized structures holonomic reversible plasticity and constraints on displacements a key feature of the optimization problem is the presence of complementarity conditions involving the orthogonality of two sign constrained vectors this problem falls within the important class of so called mathematical programs with equilibrium constraints mpecs two simple algorithms are proposed to solve our synthesis problem and application is illustrated by some examples concerning truss like structures

convergence of mesh adaptive direct search to second order stationary points a previous analysis of second order behavior of generalized pattern search algorithms for unconstrained and linearly constrained minimization is extended to the more general class of mesh adaptive direct search mads algorithms for general constrained optimization because of the ability of mads to generate an asymptotically dense set of search directions we are able to establish reasonable conditions under which a subsequence of mads iterates converges to a limit point satisfying second order necessary or sufficient optimality conditions for general set constrained optimization problems

direct search methods on parallel machines direct search methods are methods designed to solve unconstrained minimization problems of the form mathop min limits x in mathbb r n f x where f n these methods are distinguished by the fact that they neither use nor require explicit derivative information the search for a local minimizer is driven solely by function information popular methods in this class include the factorial design algorithm of box 1 the pattern search algorithm of hooke and jeeves 4 and the simplex method of neider and mead 5

shape functional optimization with restrictions boosted with machine learning techniques shape optimization is a widely used technique in the design phase of a product current ongoing improvement policies require a product to fulfill a series of conditions from the perspective of mechanical resistance fatigue natural frequency impact resistance etc all these conditions are translated into equality or inequality restrictions which must be satisfied during the optimization process that is necessary in order to determine the optimal shape this article describes a new method for shape optimization that considers any regular shape as a possible shape thereby improving on traditional methods limited to straight profiles or profiles established a priori our focus is based on using functional techniques and this approach is based on representing the shape of the object by means of functions belonging to a finite dimension functional space in order to resolve this problem the article proposes an optimization method that uses machine learning techniques for functional data in order to represent the perimeter of the set of feasible functions and to speed up the process of evaluating the restrictions in each iteration of the algorithm the results demonstrate that the functional approach produces better results in the shape optimization process and that speeding up the algorithm using machine learning techniques ensures that this approach does not negatively affect design process response times

spherical mesh adaptive direct search for separating quasi uncorrelated sources by range based independent component analysis it is seemingly paradoxical to the classical definition of the independent component analysis ica that in reality the true sources are often not strictly uncorrelated with this in mind this letter concerns a framework to extract quasi uncorrelated sources with finite supports by optimizing a range based contrast function under unit norm constraints to handle the inherent scaling indeterminacy of ica but without orthogonality constraints albeit the appealing contrast properties of the range based function e g the absence of mixing local optima the function is not differentiable everywhere unfortunately there is a dearth of literature on derivative free optimizers that effectively handle such a nonsmooth yet promising contrast function this is the compelling reason for the design of a nonsmooth optimization algorithm on a manifold of matrices having unit norm columns with the following objectives to ascertain convergence to a clarke stationary point of the contrast function and adhere to the necessary unit norm constraints more naturally the proposed nonsmooth optimization algorithm crucially relies on the design and analysis of an extension of the mesh adaptive direct search mads method to handle locally lipschitz objective functions defined on the sphere the applicability of the algorithm in the ica domain is demonstrated with simulations involving natural face aerial and texture images

effects of finite precision arithmetic on interior point methods for nonlinear programming we show that the effects of finite precision arithmetic in forming and solving the linear system that arises at each iteration of primal dual interior point algorithms for nonlinear programming are benign provided that the iterates satisfy centrality and feasibility conditions of the type usually associated with path following methods when we replace the standard assumption that the active constraint gradients are independent by the weaker mangasarian fromovitz constraint qualification rapid convergence usually is attainable even when cancellation and roundoff errors occur during the calculations in deriving our main results we prove a key technical result about the size of the exact primal dual step this result can be used to modify existing analysis of primal dual interior point methods for convex programming making it possible to extend the superlinear local convergence results to the nonconvex case

solving bilevel programs with the kkt approach bilevel programs bl form a special class of optimization problems they appear in many models in economics game theory and mathematical physics bl programs show a more complicated structure than standard finite problems we study the so called kkt approach for solving bilevel problems where the lower level minimality condition is replaced by the kkt or the fj condition this leads to a special structured mathematical program with complementarity constraints we analyze the kkt approach from a generic viewpoint and reveal the advantages and possible drawbacks of this approach for solving bl problems numerically

a derivative free filter method for solving nonlinear complementarity problems in this work nonlinear complementarity problems ncps are considered some ncps which come from practice have no derivatives or it is very difficult to obtain their derivatives derivative free methods are hence considered generalized pattern search gps method as a kind of derivative free method plays a very significant role we combine generalized pattern search method with filter method to attack ncps because one can analyze the theory of generalized pattern search filter method without sufficient reduction conditions and avoid using of derivatives some interesting results are obtained in this work without sufficient reduction conditions furthermore generalized pattern search filter method is efficient to some special problems

a neurodynamic optimization approach to bilevel quadratic programming this paper presents a neurodynamic optimization approach to bilevel quadratic programming bqp based on the karush kuhn tucker kkt theorem the bqp problem is reduced to a one level mathematical program subject to complementarity constraints mpcc it is proved that the global solution of the mpcc is the minimal one of the optimal solutions to multiple convex optimization subproblems a recurrent neural network is developed for solving these convex optimization subproblems from any initial state the state of the proposed neural network is convergent to an equilibrium point of the neural network which is just the optimal solution of the convex optimization subproblem compared with existing recurrent neural networks for bqp the proposed neural network is guaranteed for delivering the exact optimal solutions to any convex bqp problems moreover it is proved that the proposed neural network for bilevel linear programming is convergent to an equilibrium point in finite time finally three numerical examples are elaborated to substantiate the efficacy of the proposed approach

geometric quantization of symplectic manifolds with respect to reducible non negative polarizations the leafwise complex of a reducible non negative polarization with values in the prequantum bundle on a prequantizable symplectic manifold is studied the cohomology groups of this complex is shown to vanish in rank less than the rank of the real part of the non negative polarization the bohr sommerfeld set for a reducible non negative polarization is defined a factorization theorem is proved for these reducible non negative polarizations for compact symplectic manifolds it is shown that the above complex has finite dimensional cohomology groups more over a lefschetz fixed point theorem and an index theorem for these non elliptic complexes is proved as a corollary of the index theorem we deduce that the cardinality of the bohr sommerfeld set for any reducible real polarization on a compact symplectic manifold is determined by the volume and the dimension of the manifold

theoretical and numerical comparison of relaxation methods for mathematical programs with complementarity constraints mathematical programs with equilibrium constraints mpecs are difficult optimization problems whose feasible sets do not satisfy most of the standard constraint qualifications hence mpecs cause difficulties both from a theoretical and a numerical point of view as a consequence a number of mpec tailored solution methods have been suggested during the last decade which are known to converge under suitable assumptions among these mpec tailored solution schemes the relaxation methods are certainly one of the most prominent class of solution methods several different relaxation schemes are available in the meantime and the aim of this paper is to provide a theoretical and numerical comparison of these schemes more precisely in the theoretical part we improve the convergence theorems of several existing relaxation methods there we also take a closer look at the properties of the feasible sets of the relaxed problems and show which standard constraint qualifications are satisfied for these relaxed problems finally the numerical comparison is based on the macmpec test problem collection

convergence properties of the nelder mead simplex method in low dimensions the nelder mead simplex algorithm first published in 1965 is an enormously popular direct search method for multidimensional unconstrained minimization despite its widespread use essentially no theoretical results have been proved explicitly for the nelder mead algorithm this paper presents convergence properties of the nelder mead algorithm applied to strictly convex functions in dimensions 1 and 2 we prove convergence to a minimizer for dimension 1 and various limited convergence results for dimension 2 a counterexample of mckinnon gives a family of strictly convex functions in two dimensions and a set of initial conditions for which the nelder mead algorithm converges to a nonminimizer it is not yet known whether the nelder mead method can be proved to converge to a minimizer for a more specialized class of convex functions in two dimensions

parameter identification of quasibrittle materials as a mathematical program with equilibrium constraints the identification of the cohesive fracture parameters of quasibrittle materials forms the focus of the present work this important and challenging task is formulated as an inverse problem requiring the solution of a special type of constrained optimization problem known as a mathematical program with equilibrium constraints mpec a key feature of which is the presence of complementarity conditions involving the orthogonality of two nonnegative vectors we propose a simple scheme based on transforming the mpec into a standard nonlinear programming problem through appropriate smoothing of the complementarity constraints for solving the parameter identification problem some computational results using actual experimentally obtained data are presented to assess the effectiveness of the approach

an interior point method for mathematical programs with complementarity constraints mpccs interior point methods for nonlinear programs nlps are adapted for solution of mathematical programs with complementarity constraints mpccs the constraints of the mpcc are suitably relaxed so as to guarantee a strictly feasible interior for the inequality constraints the standard primal dual algorithm has been adapted with a modified step calculation the algorithm is shown to be superlinearly convergent in the neighborhood of the solution set under assumptions of mpcc licq strong stationarity and upper level strict complementarity the modification can be easily accommodated within most nonlinear programming interior point algorithms with identical local behavior numerical experience is also presented and holds promise for the proposed method

analysis of direct searches for discontinuous functions it is known that the clarke generalized directional derivative is nonnegative along the limit directions generated by directional direct search methods at a limit point of certain subsequences of unsuccessful iterates if the function being minimized is lipschitz continuous near the limit point in this paper we generalize this result for discontinuous functions using rockafellar generalized directional derivatives upper subderivatives we show that rockafellar derivatives are also nonnegative along the limit directions of those subsequences of unsuccessful iterates when the function values converge to the function value at the limit point this result is obtained assuming that the function is directionally lipschitz with respect to the limit direction it is also possible under appropriate conditions to establish more insightful results by showing that the sequence of points generated by these methods eventually approaches the limit point along the locally best branch or step function when the number of steps is equal to two the results of this paper are presented for constrained optimization and illustrated numerically

convergence properties of a regularization scheme for mathematical programs with complementarity constraints we study the convergence behavior of a sequence of stationary points of a parametric nlp which regularizes a mathematical program with equilibrium constraints mpec in the form of complementarity conditions accumulation points are feasible points of the mpec they are c stationary if the mpec linear independence constraint qualification holds they are m stationary if in addition an approaching subsequence satisfies second order necessary conditions and they are b stationary if in addition an upper level strict complementarity condition holds these results complement recent results of fukushima and pang convergence of a smoothing continuation method for mathematical programs with equilibrium constraints in ill posed variational problems and regularization techniques springer verlag new york 1999 we further show that every local minimizer of the mpec which satisfies the linear independence upper level strict complementarity and a second order optimality condition can be embedded into a locally unique piecewise smooth curve of local minimizers of the parametric nlp

on using the elastic mode in nonlinear programming approaches to mathematical programs with complementarity constraints we investigate the possibility of solving mathematical programs with complementarity constraints mpccs using algorithms and procedures of smooth nonlinear programming although mpccs do not satisfy a constraint qualification we establish sufficient conditions for their lagrange multiplier set to be nonempty mpccs that have nonempty lagrange multiplier sets and satisfy the quadratic growth condition can be approached by the elastic mode with a bounded penalty parameter in this context the elastic mode transforms mpcc into a nonlinear program with additional variables that has an isolated stationary point and local minimum at the solution of the original problem which in turn makes it approachable by sequential quadratic programming sqp algorithms one such algorithm is shown to achieve local linear convergence once the problem is relaxed under stronger conditions we also prove superlinear convergence to the solution of an mpcc using an adaptive elastic mode approach for an sqp algorithm recently analyzed in an mpcc context in r fletcher s leyffer s sholtes and d ralph local convergence of sqp methods for mathematical programs with equilibrium constraints tech report na 210 university of dundee dundee uk 2002 our assumptions are more general since we do not use a critical assumption from that reference in addition we show that the elastic parameter update rule will not interfere locally with the superlinear convergence once the penalty parameter is appropriately chosen

the c index a new stability concept for quadratic programs with complementarity constraints we introduce nondegeneracy and the c index for c stationary points of a qpcc that is for a mathematical program with a quadratic objective function and linear complementarity constraints the c index characterizes the qualitative local behavior of a qpcc around a nondegenerate c stationary point the article focuses on the structure of the c stationary set of qpccs depending on a real parameter we show that for generic qpcc data the c index changes exactly at turning points of the c stationary set and that it changes by exactly one to illustrate this concept we introduce and analyze two homotopy methods for finding c stationary points numerical results illustrate that for randomly generated test problems the two homotopy methods very often identify b stationary points

a hybrid genetic pattern search augmented lagrangian method for constrained global optimization abstract hybridization of genetic algorithms with local search approaches can enhance their performance in global optimization genetic algorithms as most population based algorithms require a considerable number of function evaluations this may be an important drawback when the functions involved in the problem are computationally expensive as it occurs in most real world problems thus in order to reduce the total number of function evaluations local and global techniques may be combined moreover the hybridization may provide a more effective trade off between exploitation and exploration of the search space in this study we propose a new hybrid genetic algorithm based on a local pattern search that relies on an augmented lagrangian function for constraint handling the local search strategy is used to improve the best approximation found by the genetic algorithm convergence to an e global minimizer is proved numerical results and comparisons with other stochastic algorithms using a set of benchmark constrained problems are provided

generalized pattern search methods for linearly equality constrained optimization problems torczon and several persons have designed pattern search methods for unconstrained optimization problem bound constrained and linearly constrained optimization problems in this paper we shall consider linear equality constrained problems as unconstrained ones using generalized pattern search algorithm except for a feasible starting point

new sequential and parallel derivative free algorithms for unconstrained minimization this paper presents sequential and parallel derivative free algorithms for finding a local minimum of smooth and nonsmooth functions of practical interest it is proved that under mild assumptions a sufficient decrease condition holds for a nonsmooth function based on this property the algorithms explore a set of search directions and move to a point with a sufficiently lower functional value if the function is strictly differentiable at its limit points a sub sequence of points generated by the algorithm converges to a first order stationary point nabla f x 0 if the function is convex around its limit points convergence of a subsequence to a point with nonnegative directional derivatives on a set of search directions is ensured preliminary numerical results on sequential algorithms show that they compare favorably with the recently introduced pattern search methods

sample size lower bounds in pac learning by algorithmic complexity theory this paper focuses on a general setup for obtaining sample size lower bounds for learning concept classes under fixed distribution laws in an extended pac learning framework these bounds do not depend on the running time of learning procedures and are information theoretic in nature they are based on incompressibility methods drawn from kolmogorov complexity and algorithmic probability theories

a markovian extension of valiant s learning model an occam algorithm learning model maintains a tentative hypothesis consistent with past observations and when a new observation is inconsistent with the current hypothesis updates to the next simplest hypothesis consistent with all observations in previous work observations were assumed to be stochastically independent this paper initiates study of such models under weaker markovian assumptions on the observations in the special case where the sequence of hypotheses satisfies a monotonicity condition it is shown that the number of mistakes in classifying the first t observations is o t log 1 i where i is the stationary probability of the initial state i of the markov chain

a generalized second order derivative in nonsmooth optimization in this work a new notion of generalized second order directional derivative and generalized hessian for nonsmooth real valued functions is studied the general properties of these mathematical objects are investigated together with some calculus rules that may facilitate their practical computation two applications of these derivatives in optimization theory are considered first to obtaining necessary and sufficient second order optimality conditions for problems with or without constraints and second to extending the newton method for the minimization of a mathcal c 1 1 function

finding effective points by surrogate models with overcomplete bases we develop algorithms to find the so called effective points eps email protected email protected r n such that the corresponding responses f x r belong to a specific region of interest roi examples of an roi include extreme values bounded intervals and positivity we are especially interested in the problem defined by the following characteristics i the definition of f x is either complicated or implicit ii the response surface f x does not fit simple patterns and iii computational costs of function evaluations is high to solve this problem we iteratively approximate the true yet unknown response surface with simplified surrogate models and then use the surrogate models to predict the possible eps unlike interpolation schemes the surrogate models are formed by linear combinations of a set of overcomplete bases and they are not obliged to fit the known response value a numerical example that involves finding positive lyapunov exponents of a dynamical system shows that the algorithm is efficient and practical

analysis of generalized pattern searches this paper contains a new convergence analysis for the lewis and torczon generalized pattern search gps class of methods for unconstrained and linearly constrained optimization this analysis is motivated by a desire to understand the successful behavior of the algorithm under hypotheses that are satisfied by many practical problems specifically even if the objective function is discontinuous or extended valued the methods find a limit point with some minimizing properties simple examples show that the strength of the optimality conditions at a limit point depends not only on the algorithm but also on the directions it uses and on the smoothness of the objective at the limit point in question the contribution of this paper is to provide a simple convergence analysis that supplies detail about the relation of optimality conditions to objective smoothness properties and to the defining directions for the algorithm and it gives previous results as corollaries

model order reduction techniques with a posteriori error control for nonlinear robust optimization governed by partial differential equations we consider a nonlinear optimization problem governed by partial differential equations with uncertain parameters it is addressed by a robust worst case formulation the resulting optimization problem is of bilevel structure and is difficult to treat numerically we propose an approximate robust formulation that employs linear and quadratic approximations to speed up the computation reduced order models based on proper orthogonal decomposition in combination with a posteriori error estimators are developed the proposed strategy is then applied to the optimal placement of a permanent magnet in the rotor of a synchronous machine with moving rotor numerical results are presented to validate the presented approach

ant colony pattern search algorithms for unconstrained and bound constrained optimization abstract ant colony optimization is a class of metaheuristics which succeed in np hard combinational optimization problems rather than continuous optimization problems we present and analyze a class of ant colony algorithms for unconstrained and bound constrained optimization on r n ant colony pattern search algorithms apsas apsas use the ant colony framework guided by objective function heuristic pheromone to perform local searches whereas global search is handled by pattern search algorithms the analysis results of apsas prove that they have a probabilistic weak stationary point convergence theory apsas present interesting emergent properties as it was shown through some analytical test functions

pattern search algorithms for mixed variable programming many engineering optimization problems involve a special kind of discrete variable that can be represented by a number but this representation has no significance such variables arise when a decision involves some situation like a choice from an unordered list of categories this has two implications the standard approach of solving problems with continuous relaxations of discrete variables is not available and the notion of local optimality must be defined through a user specified set of neighboring points we present a class of direct search algorithms to provide limit points that satisfy some appropriate necessary conditions for local optimality for such problems we give a more expensive version of the algorithm that guarantees additional necessary optimality conditions a small example illustrates the differences between the two versions a real thermal insulation system design problem illustrates the efficacy of the user controls for this class of algorithms

variable metric method for minimization this is a method for determining numerically local minima of differentiable functions of several variables in the process of locating each minimum a matrix which characterizes the behavior of the function about the minimum is determined for a region in which the function depends quadratically on the variables no more than n iterations are required where n is the number of variables by suitable choice of starting values and without modification of the procedure linear constraints can be imposed upon the variables

smooth sqp methods for mathematical programs with nonlinear complementarity constraints mathematical programs with nonlinear complementarity constraints are reformulated using better posed but nonsmooth constraints we introduce a class of functions parameterized by a real scalar to approximate these nonsmooth problems by smooth nonlinear programs this smoothing procedure has the extra benefits that it often improves the prospect of feasibility and stability of the constraints of the associated nonlinear programs and their quadratic approximations we present two globally convergent algorithms based on sequential quadratic programming sqp as applied in exact penalty methods for nonlinear programs global convergence of the implicit smooth sqp method depends on existence of a lower level nondegenerate strictly complementary limit point of the iteration sequence global convergence of the explicit smooth sqp method depends on a weaker property i e existence of a limit point at which a generalized constraint qualification holds we also discuss some practical matters relating to computer implementations

lifting mathematical programs with complementarity constraints we present a new smoothing approach for mathematical programs with complementarity constraints based on the orthogonal projection of a smooth manifold we study regularity of the lifted feasible set and since the corresponding optimality conditions are inherently degenerate introduce a regularization approach involving a novel concept of tilting stability a correspondence between the c index in the original problem and the quadratic index in the lifted problem is shown in particular a local minimizer of the mathematical program with complementarity constraints may numerically be found by minimization of the lifted smooth problem we report preliminary computational experience with the lifting approach

cgrs an advanced hybrid method for global optimization of continuous functions closely coupling extended random search and conjugate gradient method abstract a new hybrid method for global optimization of continuous functions is proposed it is a combination of an extended random search method and a descent method random search is used as the global search strategy a newly developed distribution based region control makes use of already detected local minima to refine this search strategy the approach resembles classical step size control in deterministic optimization the descent method is embedded as a local search strategy for the detection of local minima a special realization of this approach is presented in this paper and called cgrs in cgrs the conjugate gradient method is utilized as descent method the proof of global convergence in probability for cgrs is given and extended to other descent methods used in the hybrid optimization approach in order to demonstrate the numerical properties of the approach test sets of multidimensional non convex optimization problems are solved the results are compared to well established hybrid methods for global optimization the new algorithm shows a high success rate with good and adjustable solution precision parameter tuning is not necessary but of course possible the new method proves to be efficient in terms of computational costs

second order behavior of pattern search previous analyses of pattern search algorithms for unconstrained and linearly constrained minimization have focused on proving convergence of a subsequence of iterates to a limit point satisfying either directional or first order necessary conditions for optimality depending on the smoothness of the objective function in a neighborhood of the limit point even though pattern search methods require no derivative information we are able to prove some limited directional second order results although not as strong as classical second order necessary conditions these results are stronger than the first order conditions that many gradient based methods satisfy under fairly mild conditions we can eliminate from consideration all strict local maximizers and an entire class of saddle points

regularized virtual fields method for mechanical properties identification of composite materials abstract this paper presents a new material parameters identification method based on the virtual fields method vfm where a regularization term coming from a micromechanical model has been added in this regularized virtual fields method rvfm the system of equations associated to the vfm is solved within a constrained optimization framework the method was applied to virtual scanning electron microscope in situ tests aiming at determining the local properties of uni directional composites loaded in the transverse direction and for which the strain fields were obtained through digital image correlation dic the method was tested for composites with various volume fractions and properties contrasts and under different boundary conditions the potentials and limitations of the algorithm in the presence of noisy images were investigated and compared to that of the vfm and the other finite element based methods the corresponding results indicate that the proposed rvfm is more efficient than the other identification techniques in terms of both accuracy and computational time an optimum size of region of interest was also determined by taking into account dic magnification requirements and based on the accuracy of the identified parameters this kind of study can be very useful for determining a priori the level of resolution in the imaging system required for a favorable accuracy of the identified parameters

simulation based uniform value function estimates of markov decision processes the value function of a markov decision process mdp assigns to each policy its expected discounted reward this expected reward can be estimated as the empirical average of the reward over many independent simulation runs we derive bounds on the number of runs needed for the uniform convergence of the empirical average to the expected reward for a class of policies in terms of the vapnik chervonenkis or p dimension of the policy class further we show through a counterexample that whether we get uniform convergence or not for an mdp depends on the simulation method used uniform convergence results are also obtained for the average reward case for partially observed markov decision processes and can be easily extended to markov games the results can be viewed as a contribution to empirical process theory and as an extension of the probably approximately correct pac learning theory for partially observable mdps and markov games

multiplier convergence in trust region methods with application to convergence of decomposition methods for mpecs we study piecewise decomposition methods for mathematical programs with equilibrium constraints mpecs for which all constraint functions are linear at each iteration of a decomposition method one step of a nonlinear programming scheme is applied to one piece of the mpec to obtain the next iterate our goal is to understand global convergence to b stationary points of these methods when the embedded nonlinear programming solver is a trust region scheme and the selection of pieces is determined using multipliers generated by solving the trust region subproblem to this end we study global convergence of a linear trust region scheme for linearly constrained nlps that we call a trust search method the trust search has two features that are critical to global convergence of decomposition methods for mpecs a robustness property with respect to switching pieces and a multiplier convergence result that appears to be quite new for trust region methods these combine to clarify and strengthen global convergence of decomposition methods without resorting either to additional conditions such as eventual inactivity of the trust region constraint or more complex methods that require a separate subproblem for multiplier estimation

a globally convergent augmented lagrangian pattern search algorithm for optimization with general constraints and simple bounds we give a pattern search method for nonlinearly constrained optimization that is an adaption of a bound constrained augmented lagrangian method first proposed by conn gould and toint siam j numer anal 28 1991 pp 545 572 in the pattern search adaptation we solve the bound constrained subproblem approximately using a pattern search method the stopping criterion proposed by conn gould and toint for the solution of the subproblem requires explicit knowledge of derivatives such information is presumed absent in pattern search methods however we show how we can replace this with a stopping criterion based on the pattern size in a way that preserves the convergence properties of the original algorithm in this way we proceed by successive inexact bound constrained minimization without knowing exactly how inexact the minimization is as far as we know this is the first provably convergent direct search method for general nonlinear programming

an algorithm for solving the general bilevel programming problem the conflict that naturally arises in a hierarchical system can often be modeled as a multistage optimization problem this paper considers the sequential uncooperative problem in which two decision makers wish to maximize their own objective functions over a feasible region defined by interactive strategy sets the resultant problem is known as the bilevel program such programs are inherently nonconvex and resistant to standard nonlinear programming solution techniques such as piecewise linearization and convex underestimating envelopes alternatively a grid search algorithm is offered which exhibits the desirable property of monotonicity the algorithm is based on two sets of necessary conditions previously developed and combined here to provide an operational check for stationarity and local optimality potential solutions are obtained from a parameterized master program whose feasible region approximates that of the original problem as the one dimensional parameter is varied over the unit interval

learning dynamical systems in a stationary environment abstract we consider the problem of learning the input output relation of a dynamical system from noisy data our method rests on the use of a smooth simultaneous estimator which generalizes the standard empirical estimator in a stationary environment our algorithm is shown to select a model which exhibits the probably approximately correct pac property under very mild conditions this contribution should be thought of as a first attempt to extend concepts developed in learning theory to the field of system identification where due to the presence of the system dynamics the typical i i d assumption on the data made in learning theory is not satisfied

pattern search methods for user provided points application to molecular geometry problems this paper deals with the application of pattern search methods to the numerical solution of a class of molecular geometry problems with important applications in molecular physics and chemistry the goal is to find a configuration of a cluster or a molecule with minimum total energy the minimization problems in this class of molecular geometry problems have no constraints and the objective function is smooth the difficulties arise from the existence of several local minima and especially from the expensive function evaluation total energy and the possible nonavailability of first order derivatives we introduce a pattern search approach that attempts to exploit the physical nature of the problem by using energy lowering geometrical transformations and to take advantage of parallelism without the use of derivatives numerical results for a particular instance of this new class of pattern search methods are presented showing the promise of our approach the new pattern search methods can be used in any other context where there is a user provided scheme to generate points leading to a potential objective function decrease