convergence of the restricted nelder mead algorithm in two dimensions the nelder mead algorithm a longstanding direct search method for unconstrained optimization published in 1965 is designed to minimize a scalar valued function f of n real variables using only function values without any derivative information each nelder mead iteration is asso ciated with a nondegenerate simplex defined by n 1 vertices and their function values a typical iteration produces a new simplex by replacing the worst vertex by a new point despite the method s widespread use theoretical results have been limited for strictly convex objective functions of one variable with bounded level sets the algorithm always converges to the minimizer for such func tions of two variables the diameter of the simplex converges to zero but examples constructed by mckinnon show that the algorithm may converge to a nonminimizing point this paper considers the restricted nelder mead algorithm a variant that does not allow expansion steps in two dimen sions we show that for any nondegenerate starting simplex and any twice continuously differentiable function with positive definite hessian and bounded level sets the algorithm always converges to the minimizer the proof is based on treating the method as a discrete dynamical system and relies on several techniques that are nonstandard in convergence proofs for unconstrained optimization

numerical analysis of mathcal v mathcal u decomposition mathcal u gradient and mathcal u hessian approximations advances in bundle methods for nonsmooth optimization have lead to the development of mathcal v mathcal u decompositions the mathcal u gradient and the mathcal u hessian these variational analysis constructs have proven extremely valuable and lead to the development of the superlinearly convergent mathcal v mathcal u algorithm for nonsmooth optimization in this paper we examine these constructs from the viewpoint of derivative free optimization we show that given a finite max function f x max i 0 1 ldots m f i x and a black box which returns function values for each f i it is possible to construct approximations of the mathcal v mathcal u decompositions mathcal u gradient and mathcal u hessian the approximations do not require excessive black box calls and the accuracy of the approximations is directly related to the accuracy of the approximate gradient and hessians for each f i

shape equations for axisymmetric vesicles a clarification we derive the shape equations for axisymmetric vesicles and show that they are identical to the general shape equation ou yang zhong can and w helfrich phys rev a 39 5280 1989 specialized to axisymmetry we consider three different topologies an axisymmetric membrane segment suspended between two circular rings and closed vesicles of spherical and toroidal topology we point out that the shape equations are independent of the variational method used

reducing the number of function evaluations in mesh adaptive direct search algorithms the mesh adaptive direct search mads class of algorithms is designed for nonsmooth optimization where the objective function and constraints are typically computed by launching a time consuming computer simulation each iteration of a mads algorithm attempts to improve the current best known solution by launching the simulation at a finite number of trial points common implementations of mads generate 2n trial points at each iteration where n is the number of variables in the optimization problem the objective of the present work is to dynamically reduce that number we present an algorithmic framework that reduces the number of simulations to exactly n 1 without impacting the theoretical guarantees from the convergence analysis numerical experiments are conducted for several different contexts the results suggest that these strategies allow the new algorithms to reach a better solution with fewer function evaluations

mesh adaptive direct search algorithms for constrained optimization this paper addresses the problem of minimization of a nonsmooth function under general nonsmooth constraints when no derivatives of the objective or constraint functions are available we introduce the mesh adaptive direct search mads class of algorithms which extends the generalized pattern search gps class by allowing local exploration called polling in an asymptotically dense set of directions in the space of optimization variables this means that under certain hypotheses including a weak constraint qualification due to rockafellar mads can treat constraints by the extreme barrier approach of setting the objective to infinity for infeasible points and treating the problem as unconstrained the main gps convergence result is to identify limit points hat x where the clarke generalized derivatives are nonnegative in a finite set of directions called refining directions although in the unconstrained case nonnegative combinations of these directions span the whole space the fact that there can only be finitely many gps refining directions limits rigorous justification of the barrier approach to finitely many linear constraints for gps the main result of this paper is that the general mads framework is flexible enough to allow the generation of an asymptotically dense set of refining directions along which the clarke derivatives are nonnegative we propose an instance of mads for which the refining directions are dense in the hypertangent cone at hat x with probability 1 whenever the iterates associated with the refining directions converge to a single hat x the instance of mads is compared to versions of gps on some test problems we also illustrate the limitation of our results with examples

superlinear convergence and implicit filtering in this paper we show how the implicit filtering algorithm can be coupled with the bfgs quasi newton update to obtain a superlinearly convergent iteration if the noise in the objective function decays sufficiently rapidly as the optimal point is approached in this way we give insight into the observations of good performance in practice of quasi newton methods when they are coupled with implicit filtering we also report on numerical experiments that show how an implementation of implicit filtering that exploits these new results can improve the performance of the algorithm

pattern search algorithms for bound constrained minimization we present a convergence theory for pattern search methods for solving bound constrained nonlinear programs the analysis relies on the abstract structure of pattern search methods and an understanding of how the pattern interacts with the bound constraints this analysis makes it possible to develop pattern search methods for bound constrained problems while only slightly restricting the flexibility present in pattern search methods for unconstrained problems we prove global convergence despite the fact that pattern search methods do not have explicit information concerning the gradient and its projection onto the feasible region and consequently are unable to enforce explicitly a notion of sufficient feasible decrease

frames and grids in unconstrained and linearly constrained optimization a nonsmooth approach this paper describes a class of frame based direct search methods for unconstrained and linearly constrained optimization a template is described and analyzed using clarke s break nonsmooth calculus this provides a unified and simple approach to earlier results for grid and frame based methods and also provides partial convergence results when the objective function is not smooth undefined in some places or both the template also covers many new methods which combine elements of previous ideas using frames and grids these new methods include grid based simple descent algorithms which allow moving to points off the grid at every iteration and can automatically control the grid size provided function values are available the concept of a grid is also generalized to that of an admissible set which allows sets for example with circular symmetries the method is applied to linearly constrained problems using a simple barrier approach

robust pid controller tuning based on the constrained particle swarm optimization this paper proposes a novel tuning strategy for robust proportional integral derivative pid controllers based on the augmented lagrangian particle swarm optimization alpso first the problem of pid controller tuning satisfying multiple h performance criteria is considered which is known to suffer from computational intractability and conservatism when any existing method is adopted in order to give some remedy to such a design problem without using any complicated manipulations the alpso based robust gain tuning scheme for pid controllers is introduced it does not need any conservative assumption unlike the conventional methods and often enables us to find the desired pid gains just by solving the constrained optimization problem in a straightforward way however it is difficult to guarantee its effectiveness in a theoretical way because pso is essentially a stochastic approach therefore it is evaluated by several simulation examples which demonstrate that the proposed approach works well to obtain pid controller parameters satisfying the multiple h performance criteria

a steady state lattice boltzmann model for incompressible flows in this paper we propose a steady state lattice boltzmann equation which is independent of time for steady incompressible flows on the basis of the steady state lattice boltzmann equation we find a way of modeling some steady state problems using the lattice boltzmann model in further study we investigate steady state incompressible flows with the method of the higher order moment of equilibrium distribution functions and a series of partial differential equations for different space scales by assuming r constant we obtain the momentum equation for incompressible steady state flow the numerical results show that the new method is effective

on the convergence of pattern search algorithms we introduce an abstract definition of pattern search methods for solving nonlinear unconstrained optimization problems our definition unifies an important collection of optimization methods that neither compute nor explicitly approximate derivatives we exploit our characterization of pattern search methods to establish a global convergence theory that does not enforce a notion of sufficient decrease our analysis is possible because the iterates of a pattern search method lie on a scaled translated integer lattice this allows us to relax the classical requirements on the acceptance of the step at the expense of stronger conditions on the form of the step and still guarantee global convergence

parallel space decomposition of the mesh adaptive direct search algorithm this paper describes a parallel space decomposition psd technique for the mesh adaptive direct search mads algorithm mads extends a generalized pattern search for constrained nonsmooth optimization problems the objective of the present work is to obtain good solutions to larger problems than the ones typically solved by mads the new method psd mads is an asynchronous parallel algorithm in which the processes solve problems over subsets of variables the convergence analysis based on the clarke calculus is essentially the same as for the mads algorithm a practical implementation is described and some numerical results on problems with up to 500 variables illustrate the advantages and limitations of psd mads

on the convergence of grid based methods for unconstrained optimization the convergence of direct search methods for unconstrained minimization is examined in the case where the underlying method can be interpreted as a grid or pattern search over successively refined meshes an important aspect of the main convergence result is that translation rotation scaling and shearing of the successive grids are allowed

a method for stochastic constrained optimization using derivative free surrogate pattern search and collocation recent advances in coupling novel optimization methods to large scale computing problems have opened the door to tackling a diverse set of physically realistic engineering design problems a large computational overhead is associated with computing the cost function for most practical problems involving complex physical phenomena such problems are also plagued with uncertainties in a diverse set of parameters we present a novel stochastic derivative free optimization approach for tackling such problems our method extends the previously developed surrogate management framework smf to allow for uncertainties in both simulation parameters and design variables the stochastic collocation scheme is employed for stochastic variables whereas kriging based surrogate functions are employed for the cost function this approach is tested on four numerical optimization problems and is shown to have significant improvement in efficiency over traditional monte carlo schemes problems with multiple probabilistic constraints are also discussed

curvature and topology in smectic a liquid crystals considerations of rotational invariance in one dimensionally modulated systems such as smectics a necessitate nonlinearities in the free energy the presence of these nonlinearities is critical for determining the layer configurations around defects we generalize our recent construction for finding exact minima of an approximate nonlinear free energy to the full rotationally invariant smectic free energy our construction exhibits the detailed connection between mean curvature gaussian curvature and layer spacing for layers without gaussian curvature we reduce the euler lagrange equation to an equation governing the evolution of a surface as an example we determine the layer profile and free energy of an edge dislocation

precise damping conditions for global asymptotic stability for nonlinear second order systems global asymptotic stability of the rest point for nonlinear equations has been treated by a number of authors in this paper we unify and extend previous work by obtaining various necessary and sucient conditions for the asymptotic stability as t 1 of solutions of quasi variational systems of the form rl t u u 0 0 r ul t u u 0 q t u u 0

pattern search methods for linearly constrained minimization we extend pattern search methods to linearly constrained minimization we develop a general class of feasible point pattern search algorithms and prove global convergence to a karush kuhn tucker point as in the case of unconstrained minimization pattern search methods for linearly constrained problems accomplish this without explicit recourse to the gradient or the directional derivative of the objective key to the analysis of the algorithms is the way in which the local search patterns conform to the geometry of the boundary of the feasible region

unconstrained derivative free optimization by successive approximation we present an algorithmic framework for unconstrained derivative free optimization based on dividing the search space in regions partitions every partition is assigned a representative point the representative points form a grid a piecewise constant approximation to the function subject to optimization is constructed using a partitioning and its corresponding grid the convergence of the framework to a stationary point of a continuously differentiable function is guaranteed under mild assumptions the proposed framework is appropriate for upgrading heuristics that lack mathematical analysis into algorithms that guarantee convergence to a local minimizer a convergent variant of the nelder mead algorithm that conforms to the given framework is constructed the algorithm is compared to two previously published convergent variants of the nm algorithm the comparison is conducted on the more garbow hillstrom set of test problems and on four variably dimensional functions with dimension up to 100 the results of the comparison show that the proposed algorithm outperforms both previously published algorithms

pattern search ranking and selection algorithms for mixed variable simulation based optimization the class of generalized pattern search gps algorithms for mixed variable optimization is extended to problems with stochastic objective functions because random noise in the objective function makes it more difficult to compare trial points and ascertain which points are truly better than others replications are needed to generate sufficient statistical power to draw conclusions rather than comparing pairs of points the approach taken here augments pattern search with a ranking and selection r s procedure which allows for comparing many function values simultaneously asymptotic convergence for the algorithm is established numerical issues are discussed and performance of the algorithm is studied on a set of test problems

hybrid pattern search and simulated annealing for fuzzy production planning problems in this paper the hybridization of ps pattern search method and sa simulated annealing are incorporated in the optimization process this is in order to look for the global optimal solution for the fitness function and decision variables as well as minimum computational cpu time the real strength of sa approach has been tested in a case study problem of industrial production planning this is due to the great advantage of sa for being easily escaping from trapped in local minima by accepting up hill move through a probabilistic procedure in the final stages of optimization process in the ph d thesis by vasant 2008 4 16 different techniques were provided of heuristic and meta heuristic approaches in solving industrial production problems with nonlinear cubic objective functions eight decision variables and 29 constraints in this paper fuzzy technological problems have been solved using hybrid techniques of pattern search and simulated annealing hsaps the simulated and computational results are compared to other various evolutionary techniques

stationarity results for generating set search for linearly constrained optimization we present a new generating set search gss approach for minimizing functions subject to linear constraints gss is a class of direct search optimization methods that includes generalized pattern search one of our main contributions in this paper is a new condition to define the set of conforming search directions that admits several computational advantages for continuously differentiable functions we also derive a bound relating a measure of stationarity which is equivalent to the norm of the gradient of the objective in the unconstrained case and a parameter used by gss algorithms to control the lengths of the steps with the additional assumption that the derivative is lipschitz we obtain a big o bound as a consequence of this relationship we obtain subsequence convergence to a kkt point even though gss algorithms lack explicit gradient information numerical results indicate that the bound provides a reasonable estimate of stationarity

finding local optima of high dimensional functions using direct search methods this paper focuses on a subclass of box constrained non linear optimization problems we are particularly concerned with settings where gradient information is unreliable or too costly to calculate and the function evaluations themselves are very costly this encourages the use of derivative free optimization methods and especially a subclass of these referred to as direct search methods the thrust of our investigation is twofold first we implement and evaluate a number of traditional direct search methods according to the premise that they should be suitable as local optimizers when used in a metaheuristic framework second we introduce a new direct search method based on scatter search designed to remedy the lack of a good derivative free method for solving problems of high dimensions our new direct search method has convergence properties comparable to those of existing methods in addition to being able to solve larger problems more effectively

inverted catenoid as a fluid membrane with two points pulled together under inversion in any interior point a catenoid transforms into a deflated compact geometry which touches at two points its poles the catenoid is a minimal surface and as such is an equilibrium shape of a symmetric fluid membrane the conformal symmetry of the hamiltonian implies that inverted minimal surfaces are also equilibrium shapes however they will exhibit curvature singularities at their poles such singularities are the geometrical signature of the external forces required to pull the poles together these forces will set up stresses in the inverted shapes tuning the force corresponds geometrically to the translation of the point of inversion for any fixed surface area there will be a maximum force the associated shape is a symmetric discocyte lowering the external force will induce a transition from the discocyte to a cup shaped stomatocyte

a pattern search filter method for nonlinear programming without derivatives this paper formulates and analyzes a pattern search method for general constrained optimization based on filter methods for step acceptance roughly a filter method accepts a step that improves either the objective function value or the value of some function that measures the constraint violation the new algorithm does not compute or approximate any derivatives penalty constants or lagrange multipliers a key feature of the new algorithm is that it preserves the division into search and local poll steps which allows the explicit use of inexpensive surrogates or random search heuristics in the search step it is shown here that the algorithm identifies limit points at which optimality conditions depend on local smoothness of the functions and to a greater extent on the choice of a certain set of directions stronger optimality conditions are guaranteed for smoother functions and in the constrained case for a fortunate choice of the directions on which the algorithm depends these directional conditions generalize those given previously for linear constraints but they do not require a feasible starting point in the absence of general constraints the proposed algorithm and its convergence analysis generalize previous work on unconstrained bound constrained and linearly constrained generalized pattern search the algorithm is illustrated on some test examples and on an industrial wing planform engineering design application

geometry of interpolation sets in derivative free optimization we consider derivative free methods based on sampling approaches for nonlinear optimization problems where derivatives of the objective function are not available and cannot be directly approximated we show how the bounds on the error between an interpolating polynomial and the true function can be used in the convergence theory of derivative free sampling methods these bounds involve a constant that reflects the quality of the interpolation set the main task of such a derivative free algorithm is to maintain an interpolation sampling set so that this constant remains small and at least uniformly bounded this constant is often described through the basis of lagrange polynomials associated with the interpolation set we provide an alternative more intuitive definition for this concept and show how this constant is related to the condition number of a certain matrix this relation enables us to provide a range of algorithms whilst maintaining the interpolation set so that this condition number or the geometry constant remain uniformly bounded we also derive bounds on the error between the model and the function and between their derivatives directly in terms of this condition number and of this geometry constant

membrane geometry with auxiliary variables and quadratic constraints consider a surface described by a hamiltonian which depends only on the metric and extrinsic curvature induced on the surface the metric and the curvature along with the basis vectors which connect them to the embedding functions defining the surface are introduced as auxiliary variables by adding appropriate constraints all of them quadratic the response of the hamiltonian to a deformation in each of the variables is determined and the relationship between the multipliers implementing the constraints and the conserved stress tensor of the theory established for the purpose of illustration a fluid membrane described by a hamiltonian quadratic in curvature is considered

an algorithm model for mixed variable programming in this paper we consider a particular class of nonlinear optimization problems involving both continuous and discrete variables the distinguishing feature of this class of nonlinear mixed variable optimization problems is that the structure and the number of variables of the problem depend on the values of some discrete variables in particular we define a general algorithm model for the solution of this class of problems that draws inspiration from the approach recently proposed by audet and dennis siam j optim 11 2001 pp 573 594 and is based on the strategy of combining in a suitable way a local search with respect to the continuous variables and a local search with respect to the discrete variables we prove global convergence of the algorithm model without specifying the local continuous search but only identifying some reasonable requirements moreover we define a particular derivative free algorithm for solving mixed variable programming problems where the continuous variables are linearly constrained and derivative information is not available finally we report numerical results obtained by the proposed algorithm in solving a real optimal design problem these results show the effectiveness of the approach

global optimization by an improved differential evolutionary algorithm a hybrid differential evolutionary de algorithm for global optimization is proposed in the new algorithm the stochastic properties of chaotic systems are used to spread the individuals in search spaces as much as possible the pattern search method is employed to speed up the local exploiting and the de operators are used to jump to a better point the global convergence is proved three typical chaotic systems are investigated in detail numerical experiments on benchmark examples including 13 high dimensional functions demonstrate that the new method achieved an improved success rate and final solution with less computational effort

chaos driven evolutionary algorithms for the task of pid control chaos driven differential evolution algorithm and self organizing migrating algorithm are presented in this paper for the task of pid proportional integral derivative controller optimization the dissipative chaotic lozi map is embedded as a number generator inside de and soma in order to avoid local optima stagnation and embed a superior search strategy three unique pid controller problems are presented and successfully resolved using these new approaches the obtained results compare favorably with published results

generalized pattern searches with derivative information a common question asked by users of direct search algorithms is how to use derivative information at iterates where it is available this paper addresses that question with respect to generalized pattern search gps methods for unconstrained and linearly constrained optimization specifically this paper concentrates on the gps pollstep polling is done to certify the need to refine the current mesh and it requires o n function evaluations in the worst case we show that the use of derivative information significantly reduces the maximum number of function evaluations necessary for pollsteps even to a worst case of a single function evaluation with certain algorithmic choices given here furthermore we show that rather rough approximations to the gradient are sufficient to reduce the pollstep to a single function evaluation we prove that using these less expensive pollsteps does not weaken the known convergence properties of the method all of which depend only on the pollstep

generalized pattern search methods for a class of nonsmooth optimization problems with structure we propose a generalized pattern search gps method to solve a class of nonsmooth minimization problems where the set of nondifferentiability is included in the union of known hyperplanes and therefore is highly structured both unconstrained and linearly constrained problems are considered at each iteration the set of poll directions is enforced to conform to the geometry of both the nondifferentiability set and the boundary of the feasible region near the current iterate this is the key issue to guarantee the convergence of certain subsequences of iterates to points which satisfy first order optimality conditions numerical experiments on some classical problems validate the method

convergence of mesh adaptive direct search to second order stationary points a previous analysis of second order behavior of generalized pattern search algorithms for unconstrained and linearly constrained minimization is extended to the more general class of mesh adaptive direct search mads algorithms for general constrained optimization because of the ability of mads to generate an asymptotically dense set of search directions we are able to establish reasonable conditions under which a subsequence of mads iterates converges to a limit point satisfying second order necessary or sufficient optimality conditions for general set constrained optimization problems

optimization by direct search new perspectives on some classical and modern methods direct search methods are best known as unconstrained optimization techniques that do not explicitly use derivatives direct search methods were formally proposed and widely applied in the 1960s but fell out of favor with the mathematical optimization community by the early 1970s because they lacked coherent mathematical analysis nonetheless users remained loyal to these methods most of which were easy to program some of which were reliable in the past fifteen years these methods have seen a revival due in part to the appearance of mathematical analysis as well as to interest in parallel and distributed com puting this review begins by briefly summarizing the history of direct search methods and considering the special properties of problems for which they are well suited our focus then turns to a broad class of methods for which we provide a unifying framework that lends itself to a variety of convergence results the underlying principles allow general ization to handle bound constraints and linear constraints we also discuss extensions to problems with nonlinear constraints

direct search methods on parallel machines direct search methods are methods designed to solve unconstrained minimization problems of the form mathop min limits x in mathbb r n f x where f n these methods are distinguished by the fact that they neither use nor require explicit derivative information the search for a local minimizer is driven solely by function information popular methods in this class include the factorial design algorithm of box 1 the pattern search algorithm of hooke and jeeves 4 and the simplex method of neider and mead 5

spherical mesh adaptive direct search for separating quasi uncorrelated sources by range based independent component analysis it is seemingly paradoxical to the classical definition of the independent component analysis ica that in reality the true sources are often not strictly uncorrelated with this in mind this letter concerns a framework to extract quasi uncorrelated sources with finite supports by optimizing a range based contrast function under unit norm constraints to handle the inherent scaling indeterminacy of ica but without orthogonality constraints albeit the appealing contrast properties of the range based function e g the absence of mixing local optima the function is not differentiable everywhere unfortunately there is a dearth of literature on derivative free optimizers that effectively handle such a nonsmooth yet promising contrast function this is the compelling reason for the design of a nonsmooth optimization algorithm on a manifold of matrices having unit norm columns with the following objectives to ascertain convergence to a clarke stationary point of the contrast function and adhere to the necessary unit norm constraints more naturally the proposed nonsmooth optimization algorithm crucially relies on the design and analysis of an extension of the mesh adaptive direct search mads method to handle locally lipschitz objective functions defined on the sphere the applicability of the algorithm in the ica domain is demonstrated with simulations involving natural face aerial and texture images

inexact restoration method for derivative free optimization with smooth constraints a new method is introduced for solving constrained optimization problems in which the derivatives of the constraints are available but the derivatives of the objective function are not the method is based on the inexact restoration framework by means of which each iteration is divided in two phases in the first phase one considers only the constraints in order to improve feasibility in the second phase one minimizes a suitable objective function subject to a linear approximation of the constraints the second phase must be solved using derivative free methods an algorithm introduced recently by kolda lewis and torczon for linearly constrained derivative free optimization is employed for this purpose under usual assumptions convergence to stationary points is proved a computer implementation is described and numerical experiments are presented

finding optimal algorithmic parameters using derivative free optimization the objectives of this paper are twofold we devise a general framework for identifying locally optimal algorithmic parameters algorithmic parameters are treated as decision variables in a problem for which no derivative knowledge or existence is assumed a derivative free method for optimization seeks to minimize some measure of performance of the algorithm being fine tuned this measure is treated as a black box and may be chosen by the user examples are given in the text the second objective is to illustrate this framework by specializing it to the identification of locally optimal trust region parameters in unconstrained optimization the derivative free method chosen to guide the process is the mesh adaptive direct search a generalization of pattern search methods we illustrate the flexibility of the latter and in particular make provision for surrogate objectives locally optimal parameters with respect to overall computational time on a set of test problems are identified each function call may take several hours and may not always return a predictable result a tailored surrogate function is used to guide the search towards a local solution the parameters thus identified differ from traditionally used values and allow one to solve a problem that remained otherwise unsolved in a reasonable time using traditional values

on the global convergence of a filter sqp algorithm a mechanism for proving global convergence in sqp filter methods for nonlinear programming nlp is described such methods are characterized by their use of the dominance concept of multiobjective optimization instead of a penalty parameter whose adjustment can be problematic the main point of interest is to demonstrate how convergence for nlp can be induced without forcing sufficient descent in a penalty type merit function the proof relates to a prototypical algorithm within which is allowed a range of specific algorithm choices associated with the hessian matrix representation updating the trust region radius and feasibility restoration

convergence properties of the nelder mead simplex method in low dimensions the nelder mead simplex algorithm first published in 1965 is an enormously popular direct search method for multidimensional unconstrained minimization despite its widespread use essentially no theoretical results have been proved explicitly for the nelder mead algorithm this paper presents convergence properties of the nelder mead algorithm applied to strictly convex functions in dimensions 1 and 2 we prove convergence to a minimizer for dimension 1 and various limited convergence results for dimension 2 a counterexample of mckinnon gives a family of strictly convex functions in two dimensions and a set of initial conditions for which the nelder mead algorithm converges to a nonminimizer it is not yet known whether the nelder mead method can be proved to converge to a minimizer for a more specialized class of convex functions in two dimensions

firefly algorithm approach based on chaotic tinkerbell map applied to multivariable pid controller tuning nowadays a variety of controllers used in process industries are still of the proportional integral derivative pid types pid controllers have the advantage of simple structure good stability and high reliability a relevant issue for pid controllers design is the accurate and efficient tuning of parameters in this context several approaches have been reported in the literature for tuning the parameters of pid controllers using evolutionary algorithms mainly for single input single output systems the systematic design of multi loop or decentralized pid control for multivariable processes to meet certain objectives simultaneously is still a challenging task this paper proposes a new chaotic firefly algorithm approach based on tinkerbell map cfa to tune multi loop pid multivariable controllers the firefly algorithm is a metaheuristic algorithm based on the idealized behavior of the flashing characteristics of fireflies to validate the performance of the proposed pid control design a multi loop multivariable pid structure for a binary distillation column plant wood and berry column model and an industrial scale polymerization reactor are taken simulation results indicate that a suitable set of pid parameters can be calculated by the proposed cfa besides some comparison results of a genetic algorithm a particle swarm optimization approach traditional firefly algorithm modified firefly algorithm and the proposed cfa to tune multi loop pid controllers are presented and discussed

newton s method for large bound constrained optimization problems we analyze a trust region version of newton s method for bound constrained problems our approach relies on the geometry of the feasible set not on the particular representation in terms of constraints the convergence theory holds for linearly constrained problems and yields global and superlinear convergence without assuming either strict complementarity or linear independence of the active constraints we also show that the convergence theory leads to an efficient implementation for large bound constrained problems

revisiting asynchronous parallel pattern search for nonlinear optimization we present a new asynchronous parallel pattern search apps method which is different from that developed previously by hough kolda and torczon apps efficiently uses parallel and distributed computing platforms to solve science and engineering design optimization problems where derivatives are unavailable and cannot be approximated the original apps was designed to be fault tolerant as well as asynchronous and was based on a peer to peer design each process was in charge of a single fixed search direction our new version is based instead on a manager worker paradigm though less fault tolerant the resulting algorithm is more flexible in its use of distributed computing resources we further describe how to incorporate a zero order sufficient decrease condition and handle bound constraints convergence theory for all situations unconstrained and bound constrained as well as simple and sufficient decrease is developed we close with a discussion of how the new apps will better facilitate the future incorporation of linear and nonlinear constraints

brief paper multivariable pid control with set point weighting via bmi optimisation the paper focuses on the design of multivariable pid controllers with set point weighting the advantage of this pid structure is that the responses of the system to disturbances and to changes in the set point can be adjusted separately the proposed design methods rely on the transformation of the tuning of the controller gains into a static output feedback sof problem hence multivariable pid controllers can be designed by solving an optimisation problem with bilinear matrix inequalities bmis the paper addresses the design of both time invariant and gain scheduled robust controllers all of the tuning methods discussed through the paper are based on a pid structure with filtered derivative term thus guaranteeing the well posedness of the closed loop system

analysis of direct searches for discontinuous functions it is known that the clarke generalized directional derivative is nonnegative along the limit directions generated by directional direct search methods at a limit point of certain subsequences of unsuccessful iterates if the function being minimized is lipschitz continuous near the limit point in this paper we generalize this result for discontinuous functions using rockafellar generalized directional derivatives upper subderivatives we show that rockafellar derivatives are also nonnegative along the limit directions of those subsequences of unsuccessful iterates when the function values converge to the function value at the limit point this result is obtained assuming that the function is directionally lipschitz with respect to the limit direction it is also possible under appropriate conditions to establish more insightful results by showing that the sequence of points generated by these methods eventually approaches the limit point along the locally best branch or step function when the number of steps is equal to two the results of this paper are presented for constrained optimization and illustrated numerically

a self adaptive chaotic differential evolution algorithm using gamma distribution for unconstrained global optimization evolutionary algorithms eas have yielded promising results for solving nonlinear non differentiable and multi modal optimization problems due to its population based nature eas can avoid being trapped in a local optimum and consequently have the ability to find global optimal solutions as a novel evolutionary technique differential evolution de has received increasing attention and wide applications in a variety of fields de algorithm uses an efficient way of self adapting mutation using small populations for function optimization over continuous spaces the potentialities of de are its simple structure easy use convergence property quality of solution and robustness in this paper an effective self adaptive de algorithm based on gaussian probability distribution gamma distribution and chaotic sequence degc for solving continuous global optimization problems is proposed the proposed degc algorithm is tested on several benchmark functions from the usual literature numerical results comparisons with a classical de approach and a self adaptive de approach demonstrate the effectiveness and efficiency of the proposed degc algorithm

generating set search methods for piecewise smooth problems we consider a direct search approach for solving nonsmooth minimization problems where the objective function is locally lipschitz continuous and piecewise continuously differentiable on a finite family of polyhedra a generating set search method is proposed which is named structured because the structure of the set of nondifferentiability near the current iterate is exploited to define the search directions at each iteration some numerical results are presented to validate the approach

generalized pattern search methods for linearly equality constrained optimization problems torczon and several persons have designed pattern search methods for unconstrained optimization problem bound constrained and linearly constrained optimization problems in this paper we shall consider linear equality constrained problems as unconstrained ones using generalized pattern search algorithm except for a feasible starting point

continuous global optimization through the generation of parametric curves in this paper we develop a new approach for solving a large class of global optimization problems the objective function is only continuous non smooth and non lipschitzian defined on a rectangle of r n this approach is based on the generation in the feasible set of a family of parametrized curves satisfying certain properties combined with the one dimensional evtushenko algorithm to accelerate the corresponding mixed algorithm we have incorporated in a variant a pattern search type deterministic local optimization method and in another variant a new stochastic local optimization method both variants have been applied to several typical test problems a comparison with some well known methods is highlighted through numerical experiments

an implicit filtering algorithm for optimization of functions with many local minima in this paper we describe and analyze an algorithm for certain box constrained optimization problems that may have several local minima a paradigm for these problems is one in which the function to be minimized is the sum of a simple function such as a convex quadratic and high frequency low amplitude terms that cause local minima away from the global minimum of the simple function our method is gradient based and therefore the performance can be improved by use of quasi newton methods

self organizing genetic algorithm based tuning of pid controllers this paper proposes a self organizing genetic algorithm soga with good global search properties and a high convergence speed first we introduce a new dominant selection operator that enhances the action of the dominant individuals along with a cyclical mutation operator that periodically varies the mutation probability in accordance with evolution generation found in biological evolutionary processes next the soga is constructed using the two operators mentioned above the results of a nonlinear regression analysis demonstrate that the self organizing genetic algorithm is able to avoid premature convergence with a higher convergence speed and also indicate that it possesses self organization properties finally the new algorithm is used to optimize proportional integral derivative pid controller parameters our simulation results indicate that a suitable set of pid parameters can be calculated by the proposed soga

analysis of generalized pattern searches this paper contains a new convergence analysis for the lewis and torczon generalized pattern search gps class of methods for unconstrained and linearly constrained optimization this analysis is motivated by a desire to understand the successful behavior of the algorithm under hypotheses that are satisfied by many practical problems specifically even if the objective function is discontinuous or extended valued the methods find a limit point with some minimizing properties simple examples show that the strength of the optimality conditions at a limit point depends not only on the algorithm but also on the directions it uses and on the smoothness of the objective at the limit point in question the contribution of this paper is to provide a simple convergence analysis that supplies detail about the relation of optimality conditions to objective smoothness properties and to the defining directions for the algorithm and it gives previous results as corollaries

pattern search algorithms for mixed variable programming many engineering optimization problems involve a special kind of discrete variable that can be represented by a number but this representation has no significance such variables arise when a decision involves some situation like a choice from an unordered list of categories this has two implications the standard approach of solving problems with continuous relaxations of discrete variables is not available and the notion of local optimality must be defined through a user specified set of neighboring points we present a class of direct search algorithms to provide limit points that satisfy some appropriate necessary conditions for local optimality for such problems we give a more expensive version of the algorithm that guarantees additional necessary optimality conditions a small example illustrates the differences between the two versions a real thermal insulation system design problem illustrates the efficacy of the user controls for this class of algorithms

globally convergent evolution strategies in this paper we show how to modify a large class of evolution strategies es s for unconstrained optimization to rigorously achieve a form of global convergence meaning convergence to stationary points independently of the starting point the type of es under consideration recombines the parent points by means of a weighted sum around which the offspring points are computed by random generation one relevant instance of such an es is covariance matrix adaptation es cma es the modifications consist essentially of the reduction of the size of the steps whenever a sufficient decrease condition on the function values is not verified when such a condition is satisfied the step size can be reset to the step size maintained by the es s themselves as long as this latter one is sufficiently large we suggest a number of ways of imposing sufficient decrease for which global convergence holds under reasonable assumptions in particular density of certain limit directions in the unit sphere given a limited budget of function evaluations our numerical experiments have shown that the modified cma es is capable of further progress in function values moreover we have observed that such an improvement in efficiency comes without weakening significantly the performance of the underlying method in the presence of several local minimizers

axially symmetric membranes with polar tethers axially symmetric equilibrium configurations of the conformally invariant willmore energy are shown to satisfy an equation that is two orders lower in derivatives of the embedding functions than the equilibrium shape equation not one as would be expected on the basis of axial symmetry modulo a translation along the axis this equation involves a single free parameter c if c 0 a geometry with spherical topology will possess curvature singularities at its poles the physical origin of the singularity is identified by examining the noether charge associated with the translational invariance of the energy it is consistent with an external axial force acting at the poles a one parameter family of exact solutions displaying a discocyte to stomatocyte transition is described

parameter estimation of coupled water and energy balance models based on stationary constraints of surface states 1 we use a conditional averaging approach to estimate the parameters of a land surface water and energy balance model and then use the estimated parameters to partition net radiation into latent sensible and ground heat fluxes and precipitation into evapotranspiration and drainage plus runoff through conditional averaging of the modeled fluxes with respect to soil moisture and temperature we write an objective function that approximates the temperature and moisture dependent errors of the modeled fluxes in terms of atmospheric forcing e g precipitation and radiation surface states moisture s and temperature ts and model parameters the novelty of the approach is that the error term is estimated without comparison to measured fluxes instead it is inferred from the deviation of the conditionally averaged tendency terms expectation and expectation from zero since each of these terms equals zero in stationary systems but diverges from zero in the presence of misspecified parameters minimization of the approximated error yields parameters for model applications this strategy was previously studied for simple water balance models using soil moisture conditional averaging here we extend the idea to include energy balance fluxes and surface temperature conditioning the method is tested at two ameriflux sites vaira ranch california and kendall grassland arizona the estimated fluxes using only observed forcing and state variables are in reasonable agreement with field measurements because this method is based on conditional averages it can be applied to situations with subsampled or missing data that is continuous integration in time is not required

benchmarking derivative free optimization algorithms we propose data profiles as a tool for analyzing the performance of derivative free optimization solvers when there are constraints on the computational budget we use performance and data profiles together with a convergence test that measures the decrease in function value to analyze the performance of three solvers on sets of smooth noisy and piecewise smooth problems our results provide estimates for the performance difference between these solvers and show that on these problems the model based solver tested performs better than the two direct search solvers tested

asymptotic stability in functional differential equations by liapunov functionals we consider the asymptotic stability in a system of functional differential equations x t f t xt by liapunov functionals v the work generalizes some well known results in the literature in that we only require the derivative of v to be negative definite on a sequence of intervals i sn tn we also show that it is not necessary to require a uniform upper bound on v for nonuniform asymptotic stability

modeling and model predictive control of a nonlinear hydraulic system this paper deals with modeling and control of a hydraulic three tank system a process of creating a computer model in the matlab simulink environment is described and optimal pid proportional integral derivative and model predictive controllers are proposed the modeling starts with the creation of an initial mathematical model based on a first principles approach then the initial model is enhanced to obtain better correspondence with a real time system and parameters of the modified system are identified from measurements the real time system contains nonlinearities which cannot be neglected and therefore are identified and included in the final mathematical model the resulting model is used for a control design as the real time system has long time constants usage of the simulink model dramatically speeds up the design process optimal pid and model predictive controllers mpc are proposed and compared model predictive controllers for both a linearized model and the nonlinear system are proposed the techniques described are not limited to one particular modeling problem but can be used as an illustrative example for the modeling of many technological processes

a globally convergent augmented lagrangian pattern search algorithm for optimization with general constraints and simple bounds we give a pattern search method for nonlinearly constrained optimization that is an adaption of a bound constrained augmented lagrangian method first proposed by conn gould and toint siam j numer anal 28 1991 pp 545 572 in the pattern search adaptation we solve the bound constrained subproblem approximately using a pattern search method the stopping criterion proposed by conn gould and toint for the solution of the subproblem requires explicit knowledge of derivatives such information is presumed absent in pattern search methods however we show how we can replace this with a stopping criterion based on the pattern size in a way that preserves the convergence properties of the original algorithm in this way we proceed by successive inexact bound constrained minimization without knowing exactly how inexact the minimization is as far as we know this is the first provably convergent direct search method for general nonlinear programming

a simplified version of mamdani s fuzzy controller the natural logic controller this paper proposes the natural logic controller nlc that it comes through a very important simplification of the mamdani s fuzzy controller mfc allowing easy design for single input single output siso regulation problems usually fuzzy controllers are built with two classical signals of process the error and its rate of change they use a moderate number of fuzzy subsets and fuzzy rules the main features of the nlc approach are that use the minimal fuzzy partition only two fuzzy subsets per variable and it use the minimal fuzzy rule base only two rules the nonlinear resulting fuzzy controller is the simplest one with an analytically well defined input output mapping and accepting a linear approximation at origin it allows easy extension to more than two signals of process some properties of nonlinear mapping of nlc are analyzed and some results are also presents on testing stability when nlc is used on a linear process a special attention is addressed to the two inputs nlc case where stability can be tested using the circle criterion finally two application examples are discussed in details

spherical vesicles distorted by a grafted latex bead an exact solution we present an exact solution to the problem of the global shape description of a spherical vesicle distorted by a grafted latex bead this solution is derived by treating the nonlinearity in bending elasticity through the topological bogomol nyi decomposition technique and elastic compatibility we recover the hat model approximation in the limit of a small latex bead and find that the region antipodal to the grafted latex bead flattens we also derive the appropriate shape equation using the variational principle and relevant constraints

numerical approximation of the voigt regularization for incompressible navier stokes and magnetohydrodynamic flows we study voigt regularizations for the navier stokes equations nses and magnetohydrodynamic mhd equations in the presence of physical boundary conditions in particular we develop the first finite element numerical algorithms for these systems prove stability and convergence of the algorithms and test them computationally on problems of practical interest it is found that unconditionally stable implementations of the voigt regularization can be made from a simple change to existing nse and mhd codes and moreover optimal convergence of the developed algorithms solutions to physical solutions can be obtained if lower order mixed finite elements are used finally we show that for several benchmark problems the voigt regularization on a coarse mesh produces good approximations to nse and mhd systems that is the voigt regularization provides accurate reduced order models for nse and mhd flows