kriging sparse polynomial dimensional decomposition surrogate model with adaptive refinement uncertainty quantification and global sensitivity analysis problems are made more difficult in the case of applications which involve expensive computer simulations this because a limited amount of simulations is available to build a sufficiently accurate metamodel of the quantities of interest in this work a numerical technique for the construction of a low cost and accurate metamodel is proposed having in mind applications with expensive computer codes two main points are intro duced firstly a technique which couples universal kriging with sparse polynomial dimensional decomposition pdd to build a metamodel with improved accuracy the polynomials selected by the adaptive pdd representation are used as a sparse basis to build an universal kriging surrogate model the second is a strategy derived from anisotropic mesh adaptation to adaptively add a fixed number of new training points to an existing design of experiments the convergence of the proposed algorithm is analyzed and assessed on different test functions with an increasing size of the input space finally the algorithm is used to propagate uncertainties in two high dimensional real problems related to the atmospheric reentry

mesh adaptive direct search algorithms for constrained optimization this paper addresses the problem of minimization of a nonsmooth function under general nonsmooth constraints when no derivatives of the objective or constraint functions are available we introduce the mesh adaptive direct search mads class of algorithms which extends the generalized pattern search gps class by allowing local exploration called polling in an asymptotically dense set of directions in the space of optimization variables this means that under certain hypotheses including a weak constraint qualification due to rockafellar mads can treat constraints by the extreme barrier approach of setting the objective to infinity for infeasible points and treating the problem as unconstrained the main gps convergence result is to identify limit points hat x where the clarke generalized derivatives are nonnegative in a finite set of directions called refining directions although in the unconstrained case nonnegative combinations of these directions span the whole space the fact that there can only be finitely many gps refining directions limits rigorous justification of the barrier approach to finitely many linear constraints for gps the main result of this paper is that the general mads framework is flexible enough to allow the generation of an asymptotically dense set of refining directions along which the clarke derivatives are nonnegative we propose an instance of mads for which the refining directions are dense in the hypertangent cone at hat x with probability 1 whenever the iterates associated with the refining directions converge to a single hat x the instance of mads is compared to versions of gps on some test problems we also illustrate the limitation of our results with examples

superlinear convergence and implicit filtering in this paper we show how the implicit filtering algorithm can be coupled with the bfgs quasi newton update to obtain a superlinearly convergent iteration if the noise in the objective function decays sufficiently rapidly as the optimal point is approached in this way we give insight into the observations of good performance in practice of quasi newton methods when they are coupled with implicit filtering we also report on numerical experiments that show how an implementation of implicit filtering that exploits these new results can improve the performance of the algorithm

modeling an augmented lagrangian for blackbox constrained optimization constrained blackbox optimization is a difficult problem with most approaches coming from the mathematical programming literature the statistical literature is sparse especially in addressing problems with nontrivial constraints this situation is unfortunate because statistical methods have many attractive properties global scope handling noisy objectives sensitivity analysis and so forth to narrow that gap we propose a combination of response surface modeling expected improvement and the augmented lagrangian numerical optimization framework this hybrid approach allows the statistical model to think globally and the augmented lagrangian to act locally we focus on problems where the constraints are the primary bottleneck requiring expensive simulation to evaluate and substantial modeling effort to map out in that context our hybridization presents a simple yet effective solution that allows existing objective oriented statistical approaches like those based on gaussian process surrogates

on the convergence of grid based methods for unconstrained optimization the convergence of direct search methods for unconstrained minimization is examined in the case where the underlying method can be interpreted as a grid or pattern search over successively refined meshes an important aspect of the main convergence result is that translation rotation scaling and shearing of the successive grids are allowed

non fragile control design for interval valued fuzzy systems against nonlinear actuator faults abstract in this paper we consider the reliable non fragile h control design problem for a class of discrete time interval valued fuzzy systems with actuator faults more precisely the actuator fault model considered in the controller design consists of linear and nonlinear fault terms the main focus of this paper is to design a non fragile state feedback controller that is characterized by membership functions described by upper and lower membership functions and nonlinear weighting functions it should be mentioned that the proposed interval valued fuzzy system and the controller do not share same membership functions a novel set of sufficient conditions is obtained by transforming the controller design problem into a multi objective convex optimization problem based on the linear matrix inequality and the lyapunov functional approaches which ensures the robust asymptotic stability of the addressed system with a desired h performance index finally two numerical examples including an application oriented model mass spring damping system are presented to demonstrate the effectiveness of the proposed design technique

pattern search ranking and selection algorithms for mixed variable simulation based optimization the class of generalized pattern search gps algorithms for mixed variable optimization is extended to problems with stochastic objective functions because random noise in the objective function makes it more difficult to compare trial points and ascertain which points are truly better than others replications are needed to generate sufficient statistical power to draw conclusions rather than comparing pairs of points the approach taken here augments pattern search with a ranking and selection r s procedure which allows for comparing many function values simultaneously asymptotic convergence for the algorithm is established numerical issues are discussed and performance of the algorithm is studied on a set of test problems

accelerated random search for constrained global optimization assisted by radial basis function surrogates abstract the accelerated random search ars algorithm appel et al 2004 is a stochastic algorithm for the global optimization of black box functions subject only to bound constraints ars iteratively samples uniformly within a box centered at the current best point if the sample point is worse than the current best point then the size of the box is reduced thereby making it more likely to generate an improving solution if the current best point is not yet a local minimum otherwise if the sample point is an improvement over the current best point then the size of the box is reset to the initial value so that the box covers the entire search space ars has been shown theoretically and numerically to converge to the global minimum faster than the pure random search prs algorithm on bound constrained problems this article develops the constrained ars cars algorithm which is an extension of ars for optimization problems with black box inequality constraints under some mild conditions on the constrained optimization problem we prove the convergence to the global minimum in a probabilistic sense of a class of stochastic search algorithms called scattered uniform random search scat urs algorithms which includes cars as a special case to improve performance of cars on computationally expensive simulation based problems we incorporate radial basis function rbf surrogate models to approximate the objective and constrained functions and refer to the resulting algorithm as cars rbf numerical experiments show that cars consistently and substantially outperforms both the pure random search prs and accelerated particle swarm optimization apso algorithms yang 2010 on 31 test problems with dimensions ranging from 2 to 20 moreover cars rbf is an improvement over cars outperforms a sequential penalty derivative algorithm and competes with constrlmsrbf regis 2011 on the same test problems finally sensitivity analysis of cars rbf to the type of rbf model used shows that the cubic rbf model generally yields the best results on the test problems among six types of rbf models considered including the thin plate spline gaussian and multiquadric models

isospectral pairs of metrics on balls spheres and other manifolds with different local geometries the first isospectral pairs of metrics are constructed on the most simple simply connected domains namely on balls and spheres this long standing problem concerning the existence of such pairs has been solved by a new method called anticommutator technique among the wide range of such pairs the most striking examples are provided on the spheres s4k 1 where k 3 one of these metrics is homogeneous since it is the metric on the geodesic sphere of a 2 point homogeneous space while the other is locally inhomogeneous these examples demonstrate the surprising fact that no information about the isometries is encoded in the spectrum of the laplacian acting on functions in other words the group of isometries even the local homogeneity property is lost to the nonaudible in the debate of audible versus nonaudible geometry in this article we settle a long standing and intensively studied problem of spectral geometry by constructing the first examples of nontrivial isospectral metrics on the simplest simply connected manifolds namely on balls and spheres this result is achieved by a new isospectral construction technique called anticommutator technique it provides a whole cornucopia of such examples not just on balls and spheres but on other types of manifolds too the most striking examples are constructed on the geodesic spheres of the hyperbolic quaternionic 2 point homogeneous spaces the induced metrics on these spheres are obviously homogeneous while the metrics constructed isospectrally to these are locally inhomogeneous this shows the weakness of the information about the local geometry encoded in the spectrum of laplacian acting on functions

a pattern search filter method for nonlinear programming without derivatives this paper formulates and analyzes a pattern search method for general constrained optimization based on filter methods for step acceptance roughly a filter method accepts a step that improves either the objective function value or the value of some function that measures the constraint violation the new algorithm does not compute or approximate any derivatives penalty constants or lagrange multipliers a key feature of the new algorithm is that it preserves the division into search and local poll steps which allows the explicit use of inexpensive surrogates or random search heuristics in the search step it is shown here that the algorithm identifies limit points at which optimality conditions depend on local smoothness of the functions and to a greater extent on the choice of a certain set of directions stronger optimality conditions are guaranteed for smoother functions and in the constrained case for a fortunate choice of the directions on which the algorithm depends these directional conditions generalize those given previously for linear constraints but they do not require a feasible starting point in the absence of general constraints the proposed algorithm and its convergence analysis generalize previous work on unconstrained bound constrained and linearly constrained generalized pattern search the algorithm is illustrated on some test examples and on an industrial wing planform engineering design application

a naive multi scale search algorithm for global optimization problems this paper proposes a multi scale search algorithm for solving global optimization problems given a finite number of function evaluations we refer to this algorithm as the naive multi scale search optimization nmso nmso looks for the optimal solution by optimistically partitioning the search space over multiple scales in a hierarchical fashion based on a weak assumption about the function smoothness we present a theoretical analysis on its finite time and asymptotic convergence an empirical assessment of the algorithm has been conducted on the noiseless black box optimization benchmarking bbob testbed and compared with the state of the art optimistic as well as stochastic algorithms moreover the efficacy of nmso has been validated on the black box optimization competition within the gecco 15 conference where it has secured the third place out of twenty eight participating algorithms overall nmso is suitable for problems with limited function evaluations low dimensionality search space and objective functions that are separable or multi modal otherwise it is comparable with the top performing algorithms

generalized pattern searches with derivative information a common question asked by users of direct search algorithms is how to use derivative information at iterates where it is available this paper addresses that question with respect to generalized pattern search gps methods for unconstrained and linearly constrained optimization specifically this paper concentrates on the gps pollstep polling is done to certify the need to refine the current mesh and it requires o n function evaluations in the worst case we show that the use of derivative information significantly reduces the maximum number of function evaluations necessary for pollsteps even to a worst case of a single function evaluation with certain algorithmic choices given here furthermore we show that rather rough approximations to the gradient are sufficient to reduce the pollstep to a single function evaluation we prove that using these less expensive pollsteps does not weaken the known convergence properties of the method all of which depend only on the pollstep

optimization by direct search new perspectives on some classical and modern methods direct search methods are best known as unconstrained optimization techniques that do not explicitly use derivatives direct search methods were formally proposed and widely applied in the 1960s but fell out of favor with the mathematical optimization community by the early 1970s because they lacked coherent mathematical analysis nonetheless users remained loyal to these methods most of which were easy to program some of which were reliable in the past fifteen years these methods have seen a revival due in part to the appearance of mathematical analysis as well as to interest in parallel and distributed com puting this review begins by briefly summarizing the history of direct search methods and considering the special properties of problems for which they are well suited our focus then turns to a broad class of methods for which we provide a unifying framework that lends itself to a variety of convergence results the underlying principles allow general ization to handle bound constraints and linear constraints we also discuss extensions to problems with nonlinear constraints

a comparison of three methods for selecting values of input variables in the analysis of output from a computer code two types of sampling plans are examined as alternatives to simple random sampling in monte carlo studies these plans are shown to be improvements over simple random sampling with respect to variance for a class of estimators which includes the sample mean and the empirical distribution function

spherical mesh adaptive direct search for separating quasi uncorrelated sources by range based independent component analysis it is seemingly paradoxical to the classical definition of the independent component analysis ica that in reality the true sources are often not strictly uncorrelated with this in mind this letter concerns a framework to extract quasi uncorrelated sources with finite supports by optimizing a range based contrast function under unit norm constraints to handle the inherent scaling indeterminacy of ica but without orthogonality constraints albeit the appealing contrast properties of the range based function e g the absence of mixing local optima the function is not differentiable everywhere unfortunately there is a dearth of literature on derivative free optimizers that effectively handle such a nonsmooth yet promising contrast function this is the compelling reason for the design of a nonsmooth optimization algorithm on a manifold of matrices having unit norm columns with the following objectives to ascertain convergence to a clarke stationary point of the contrast function and adhere to the necessary unit norm constraints more naturally the proposed nonsmooth optimization algorithm crucially relies on the design and analysis of an extension of the mesh adaptive direct search mads method to handle locally lipschitz objective functions defined on the sphere the applicability of the algorithm in the ica domain is demonstrated with simulations involving natural face aerial and texture images

sequential penalty derivative free methods for nonlinear constrained optimization we consider the problem of minimizing a continuously differentiable function of several variables subject to smooth nonlinear constraints we assume that the first order derivatives of the objective function and of the constraints can be neither calculated nor explicitly approximated hence every minimization procedure must use only a suitable sampling of the problem functions these problems arise in many industrial and scientific applications and this motivates the increasing interest in studying derivative free methods for their solution the aim of the paper is to extend to a derivative free context a sequential penalty approach for nonlinear programming this approach consists in solving the original problem by a sequence of approximate minimizations of a merit function where penalization of constraint violation is progressively increased in particular under some standard assumptions we introduce a general theoretical result regarding the connections between the sampling technique and the updating of the penalization which are able to guarantee convergence to stationary points of the constrained problem on the basis of the general theoretical result we propose a new method and prove its convergence to stationary points of the constrained problem the computational behavior of the method has been evaluated both on a set of test problems and on a real application the obtained results and the comparison with other well known derivative free software show the viability of the proposed sequential penalty approach

finding optimal algorithmic parameters using derivative free optimization the objectives of this paper are twofold we devise a general framework for identifying locally optimal algorithmic parameters algorithmic parameters are treated as decision variables in a problem for which no derivative knowledge or existence is assumed a derivative free method for optimization seeks to minimize some measure of performance of the algorithm being fine tuned this measure is treated as a black box and may be chosen by the user examples are given in the text the second objective is to illustrate this framework by specializing it to the identification of locally optimal trust region parameters in unconstrained optimization the derivative free method chosen to guide the process is the mesh adaptive direct search a generalization of pattern search methods we illustrate the flexibility of the latter and in particular make provision for surrogate objectives locally optimal parameters with respect to overall computational time on a set of test problems are identified each function call may take several hours and may not always return a predictable result a tailored surrogate function is used to guide the search towards a local solution the parameters thus identified differ from traditionally used values and allow one to solve a problem that remained otherwise unsolved in a reasonable time using traditional values

analysis of generalized pattern searches this paper contains a new convergence analysis for the lewis and torczon generalized pattern search gps class of methods for unconstrained and linearly constrained optimization this analysis is motivated by a desire to understand the successful behavior of the algorithm under hypotheses that are satisfied by many practical problems specifically even if the objective function is discontinuous or extended valued the methods find a limit point with some minimizing properties simple examples show that the strength of the optimality conditions at a limit point depends not only on the algorithm but also on the directions it uses and on the smoothness of the objective at the limit point in question the contribution of this paper is to provide a simple convergence analysis that supplies detail about the relation of optimality conditions to objective smoothness properties and to the defining directions for the algorithm and it gives previous results as corollaries

globally convergent evolution strategies in this paper we show how to modify a large class of evolution strategies es s for unconstrained optimization to rigorously achieve a form of global convergence meaning convergence to stationary points independently of the starting point the type of es under consideration recombines the parent points by means of a weighted sum around which the offspring points are computed by random generation one relevant instance of such an es is covariance matrix adaptation es cma es the modifications consist essentially of the reduction of the size of the steps whenever a sufficient decrease condition on the function values is not verified when such a condition is satisfied the step size can be reset to the step size maintained by the es s themselves as long as this latter one is sufficiently large we suggest a number of ways of imposing sufficient decrease for which global convergence holds under reasonable assumptions in particular density of certain limit directions in the unit sphere given a limited budget of function evaluations our numerical experiments have shown that the modified cma es is capable of further progress in function values moreover we have observed that such an improvement in efficiency comes without weakening significantly the performance of the underlying method in the presence of several local minimizers

regularized virtual fields method for mechanical properties identification of composite materials abstract this paper presents a new material parameters identification method based on the virtual fields method vfm where a regularization term coming from a micromechanical model has been added in this regularized virtual fields method rvfm the system of equations associated to the vfm is solved within a constrained optimization framework the method was applied to virtual scanning electron microscope in situ tests aiming at determining the local properties of uni directional composites loaded in the transverse direction and for which the strain fields were obtained through digital image correlation dic the method was tested for composites with various volume fractions and properties contrasts and under different boundary conditions the potentials and limitations of the algorithm in the presence of noisy images were investigated and compared to that of the vfm and the other finite element based methods the corresponding results indicate that the proposed rvfm is more efficient than the other identification techniques in terms of both accuracy and computational time an optimum size of region of interest was also determined by taking into account dic magnification requirements and based on the accuracy of the identified parameters this kind of study can be very useful for determining a priori the level of resolution in the imaging system required for a favorable accuracy of the identified parameters

a globally convergent augmented lagrangian pattern search algorithm for optimization with general constraints and simple bounds we give a pattern search method for nonlinearly constrained optimization that is an adaption of a bound constrained augmented lagrangian method first proposed by conn gould and toint siam j numer anal 28 1991 pp 545 572 in the pattern search adaptation we solve the bound constrained subproblem approximately using a pattern search method the stopping criterion proposed by conn gould and toint for the solution of the subproblem requires explicit knowledge of derivatives such information is presumed absent in pattern search methods however we show how we can replace this with a stopping criterion based on the pattern size in a way that preserves the convergence properties of the original algorithm in this way we proceed by successive inexact bound constrained minimization without knowing exactly how inexact the minimization is as far as we know this is the first provably convergent direct search method for general nonlinear programming