a stochastic approach to global optimization of nonlinear programming problem with many equality constraints many practical problems often lead to large nonconvex nonlinear programming problems that have many equality constraints the global optimization algorithms of these problems have received much attention over the last few years generally stochastic algorithms are suitable for these problems but not efficient when there are too many equality constraints therefore a global optimization algorithm for solving these problems is proposed in this paper the new algorithm based on a feasible set strategy uses a stochastic algorithm and a deterministic local algorithm the convergence of the algorithm is analyzed this algorithm is applied to practical problem and the numerical results illustrate the accuracy and efficiency of the algorithm

exact sequential filtering smoothing and prediction for nonlinear systems of the cost function 2 as the duration of the process increases and additional observation vectors are obtained the first algorithm proceeds via an imbedding on the process length and the final stare vector the second algorithm proceeds via an imbedding on the process length and the final observation vector each algorithm generates optimal least cost filtered and smoothed state estimates together with optimal one step ahead state predictions section 4 clarifies the relationship between the exact sequential filtering smoothing and prediction fsp procedures developed in sections 2 and 3 and the exact sequential fsp procedure previously developed in kalaba and tesfatsion l briefly the present procedures focus on the direct updating of the minimized least squares cost function 2 as the duration of the process increases and additional observations are obtained apart from the basic requirement that a unique cost minimizing solution x

global optimization and simulated annealing in this paper we are concerned with global optimization which can be defined as the problem of finding points on a bounded subset of n in which some real valued functionf assumes its optimal maximal or minimal value we present a stochastic approach which is based on the simulated annealing algorithm the approach closely follows the formulation of the simulated annealing algorithm as originally given for discrete optimization problems the mathematical formulation is extended to continuous optimization problems and we prove asymptotic convergence to the set of global optima furthermore we discuss an implementation of the algorithm and compare its performance with other well known algorithms the performance evaluation is carried out for a standard set of test functions from the literature

transition functions paths and path integrals the main theme of this expository paper is the relation between analysis and probability in the context of diffusion theory section 1 discusses in rather heuristic fashion the very satisfying solution to the problem of describing diffusion processes which kolmogorov achieved via pde theory the theory of partial differential equations and his criterion for path continuity section 2 describes how ito calculus totally transformed the subject by allowing us to construct the sample paths of a diffusion process x by solving an sde stochastic differential equation driven by brownian motion of course sdes have great intrinsic importance too as noisy perturbations of nonlinear dynamical systems though 2 begins heuristically the mathematics is then tightened up this paper is after all a tribute to the man whose greatest contribution to science is his setting probability theory on a rigorous foundation once kolmogorov s precise language is available 2 then takes a quick sight seeing trip through some of the great developments by doob ito and their successors in an age in which so many do simulations of ito equations i have explained precisely in the briefest possible fashion what the exact theory is it is easy and usable you will just have time for a snapshot of how brownian motion on the orthonormal frame bundle is linked to index theorems and of what the malliavin calculus is about you will however be advised on guide books on these and other areas including physicists favourites large deviations measure valued diffusions etc so that you can later explore at your leisure confession the paper consists very largely of selected tracks remixed from the album rogers williams 1987 diffusions markov processes and martingales chichester wiley tributes to kolmogorov s work in probability and statistics have appeared in every book ever written on probability and in ann probability 17 1989 815 964 ann statist 18 1990 987 1031 bull lond math soc 22 1990 31 100 teor veroyatnost primenen 34 1989 no 1 usp mat nauk 43 1988 no 6 the official biography by shiryaev will be a wonderful volume for me this paper is a further expression of my thanks to kolmogorov and as he would have wished to levy doob and ito too

a derivative free variant called dfsa of dekkers and aarts continuous simulated annealing algorithm abstract we propose a derivative free implementation of dekkers and aarts continuous simulated annealing sa essential in dfsa is the random direction local technique that we introduce we study the convergence properties of the resulting algorithm and test its performance on a set of 50 problems numerical results are presented which show the robustness of the algorithm numerical comparisons with sa two derivative free simulated annealing algorithms and three population based global optimization algorithms show that the derivative free sa dfsa offers a reasonable alternative to some recent global optimization algorithms especially for problems requiring a direct search type algorithm

sequential stopping rules for the multistart algorithm in global optimisation in this paper a sequential stopping rule is developed for the multistart algorithm a statistical model for the values of the observed local maxima of an objective function is introduced in the framework of bayesian non parametric statistics a suitablea priori distribution is proposed which is general enough and which leads to computationally manageable expressions for thea posteriori distribution sequential stopping rules of thek step look ahead kind are then explicitly derived and their numerical effectiveness compared

hit and run algorithms for the identification of nonredundant linear inequalities two probabilistic hit and run algorithms are presented to detect nonredundant constraints in a full dimensional system of linear inequalities the algorithms proceed by generating a random sequence of interior points whose limiting distribution is uniform and by searching for a nonredundant constraint in the direction of a random vector from each point in the sequence in the hypersphere directions algorithm the direction vector is drawn from a uniform distribution on a hypersphere in the computationally superior coordinate directions algorithm a search is carried out along one of the coordinate vectors the algorithms are terminated through the use of a bayesian stopping rule computational experience with the algorithms and the stopping rule will be reported

diffusions for global optimizations we seek a global minimum of u 0 1 n to r the solution to d dt x t nabla u x t will find local minima the solution to dx t nabla u x t dt sqrt 2t dw t where w is standard n dimensional brownian motion and the boundaries are reflecting will concentrate near the global minima of u at least when temperature t is small the equilibrium distribution for x t is gibbs with density pi t x alpha exp u x t this suggests setting t t t downarrow 0 to find the global minima of u we give conditions on u x and t t such that the solution to dx t nabla u x t dt sqrt 2t dw t converges weakly to a distribution concentrated on the global minima of u

towards ideal multistart a stochastic approach for locating the minima of a continuous function inside a bounded domain a stochastic global optimization method based on multistart is presented in this the local search is conditionally applied with a probability that takes in account the topology of the objective function at the detail offered by the current status of exploration as a result the number of unnecessary local searches is drastically limited yielding an efficient method results of its application on a set of common test functions are reported along with a performance comparison against other established methods of similar nature

probabilistic optimization for conceptual rainfall runoff models a comparison of the shuffled complex evolution and simulated annealing algorithms automatic optimization algorithms are used routinely to calibrate conceptual rainfall runoff crr models the goal of calibration is to estimate a feasible and unique global set of parameter estimates that best fit the observed runoff data most if not all optimization algorithms have difficulty in locating the global optimum because of response surfaces that contain multiple local optima with regions of attraction of differing size discontinuities and long ridges and valleys extensive research has been undertaken to develop efficient and robust global optimization algorithms over the last 10 years this study compares the performance of two probabilistic global optimization methods the shuffled complex evolution algorithm sce ua and the three phase simulated annealing algorithm sa sx both algorithms are used to calibrate two parameter sets of a modified version of boughton s 1984 sfb model using data from two australian catchments that have low and high runoff yields for the reduced well identified parameter set the algorithms have a similar efficiency for the low yielding catchment but sce ua is almost twice as robust although the robustness of the algorithms is similar for the high yielding catchment sce ua is six times more efficient than sa sx when fitting the full parameter set the performance of sa sx deteriorated markedly for both catchments these results indicated that sce ua s use of multiple complexes and shuffling provided a more effective search of the parameter space than sa sx s single simplex with stochastic step acceptance criterion especially when the level of parameterization is increased examination of the response surface for the low yielding catchment revealed some reasons why sce ua outperformed sa sx and why probabilistic optimization algorithms can experience difficulty in locating the global optimum

on the treatment of thermal and quantum annealing by stochastic variational means we consider the control of a diffusion process represented by a stochastic differential equation of the ito type the diffusion process is one of a random walk on the configuration space of a multimodal function and the goal of the control is to determine the optimum variation of the temperature in order to anneal into the global minimum of the function such optimal controls are derived using stochastic variational means by the viscosity solution of a second order hamilton jacobi bellman equation or by minimizing a feynman kac wiener path integral the extension to quantized annealing and the possibilities of a differential game theoretic formulation for extra diffusive annealing are also considered

an exact sequential solution procedure for a class of discrete time nonlinear estimation problems an exact procedure is developed for sequentially updating the optimal solution for a general discrete time nonlinear least squares estimation problem as the process length increases and new observations are obtained the optimal sequential estimation equations are derived by means of an imbedding on two physically meaningful parameters namely the duration of the dynamical process and the value of the final observation the optimal sequential estimation equations are contrasted with the approximate sequential estimation equations which would be obtained via extended kalman filtering

metamodeling radial basis functions versus polynomials abstract for many years metamodels have been used in simulation to provide approximations to the input output functions provided by a simulation model in this paper metamodels based on radial basis functions are applied to approximate test functions known from the literature these tests were conducted to gain insights into the behavior of these metamodels their ease of computation and their ability to capture the shape and minima of the test functions these metamodels are compared against polynomial metamodels by using surface and contour graphs of the error function difference between metamodel and the given function and by evaluating the numerical stability of the required computations full factorial and latin hypercube designs were used to fit the metamodels graphical and statistical methods were used to analyze the test results factorial designs were generally more successful for fitting the test functions as compared to latin hypercube designs radial basis function metamodels using factorial and latin hypercube designs provided better fit than polynomial metamodels using full factorial designs

a parameter free filled function for unconstrained global optimization the filled function method is considered as an efficient method to find the global minimum of multidimensional functions a number of filled functions were proposed recently most of which have one or two adjustable parameters however there is no efficient criterion to choose the parameter appropriately in this paper we propose a filled function without parameter and this function includes neither exponential terms nor logarithmic terms so it is superior to the traditional ones theories of the filled function are investigated and an algorithm which does not compute gradients during minimizing the filled function is presented moreover the numerical experiments demonstrate the efficiency of the proposed filled function

developing a fuzzy bicluster regression to estimate heat tolerance in plants by chlorophyll fluorescence this paper presents a straightforward and useful fuzzy regression approach to estimate heat tolerance of plants by chlorophyll fluorescence measurement the chlorophyll fluorescence measurement is an indicator of functional change of photosynthesis and is sensitive to temperature using the fluorescence temperature curves the experimenter may determine the heat tolerance tc of plants by intersections of two linear regression lines however as traditional statistical regression analysis shows the experiment may contain uncertain factors or phenomena such as leaf nature and growth environment which concludes that data may vary among individual plants and different species this research presents a fuzzy bicluster regression fbcr analysis with genetic algorithms which helps derive a fuzzy intersection set and fuzzy heat tolerance of plants in addition to the traditional statistical regression analysis a fuzzy clustering concept and simultaneously optimal determination of data clusters is also developed especially when there are nonlinear inflections in data curves due to the imperative use of linear regression models the traditional regression analysis may become unable to sufficiently model the uncertainties exhibited the fbcr analysis can resolve this problem effectively due to the nonlinear tolerance of the system even in a linear model to demonstrate the fbcr analysis it was applied to estimate the heat tolerance of five plant species the results derived appeared to be more suitable than that of the conventional method the approach may provide a useful means for the experimenters to derive more credible results from their chlorophyll fluorescence temperature data

newton type minimization via the lanczos method this paper discusses the use of the linear conjugate gradient method developed via the lanczos method in the solution of large scale unconstrained minimization problems at each iteration of a newton type method the direction of search is defined as the solution of a quadratic subproblem when the number of variables is very large this subproblem may be solved using the linear conjugate gradient method of hestenes and stiefel we show how the equivalent lanczos characterization of the linear conjugate gradient method may be exploited to define a modified newton method which can be applied to problems that do not necessarily have positive definite hessian matrices at all points of the region of interest this derivation also makes it possible to compute a negative curvature direction at a stationary point the idea of a truncated newton method is to perform only a limited number of iterations of the quadratic subproblem this effectively gives a search direction that interpolates between the steepest de

a stochastic technique for global optimization abstract a stochastic technique for multiextremal optimization is discussed the technique derives from the adoption of sequential stopping rules for the well known multistart algorithm the stopping rules are developed in a bayesian nonparametric framework a class of multiextremal test functions is introduced and results obtained through the application of the proposed techniques to this class are presented the results show the effectiveness of the proposed stopping rules

quasi random initial population for genetic algorithms the selection of the initial population in a population based heuristic optimizationmethod is important since it affects the search for several iterations and often has an influence on the final solution if no a priori information about the optima is available the initial population is often selected randomly using pseudorandom numbers usually however it is more important that the points are as evenly distributed as possible than that they imitate random points in this paper we study the use of quasi random sequences in the initial population of a genetic algorithm sample points in a quasi random sequence are designed to have good distribution properties here a modified genetic algorithm using quasi random sequences in the initial population is tested by solving a large number of continuous benchmark problems from the literature the numerical results of two implementations of genetic algorithms using different quasi random sequences are compared to those of a traditional implementation using pseudorandom numbers the results obtained are promising

on random pattern test length the testing of large logic networks with random patterns is examined work by previous workers for single faults is extended to a class of multiple fault situations not only is the problem of fault detection in the presence of nonmasking multiple faults treated but the question of distinguishing between them is also examined it is shown that a test that merely exposes each fault has a high probability of distinguishing between the faults the relationships between quality diagnostic resolution and random pattern test length are developed the results have application to self test schemes that use random patterns as stimuli

a simulated annealing driven multi start algorithm for bound constrained global optimization a derivative free simulated annealing driven multi start algorithm for continuous global optimization is presented we first propose a trial point generation scheme in continuous simulated annealing which eliminates the need for the gradient based trial point generation we then suitably embed the multi start procedure within the simulated annealing algorithm we modify the derivative free pattern search method and use it as the local search in the multi start procedure we study the convergence properties of the algorithm and test its performance on a set of 50 problems numerical results are presented which show the robustness of the algorithm numerical comparisons with a gradient based simulated annealing algorithm and three population based global optimization algorithms show that the new algorithm could offer a reasonable alternative to many currently available global optimization algorithms specially for problems requiring direct search type algorithm

comparative analysis of different implementations of multiple input signature analyzers signature analysis is an accepted method of obtaining data compression for built in testing applications the author deals with a unified approach to the analysis of multiple input signature analyzers by considering them as finite state switching circuits this approach is used to investigate and compare different implementations and it is shown that there is a large range of alternatives to achieve a given characteristic polynomial particular emphasis is placed on the two most common implementations it is shown that both perform polynomial division with each input of one circuit being equivalent to a combination of inputs of the other the approach also gives a method of expressing aliasing patterns for both implementations which leads to a study of the differences with regard to certain error detection capabilities

a statistical estimate of the structure of multi extremal problems the solution of ak extremal problem is defined as the set of pairs xi i i 1 k where xt isith local minimum and i is the volume of the set of attraction of this minimum a bayesian estimate ofk and 1 k is constructed

generalized simulated annealing for function optimization a generalized simulated annealing method has been developed and applied to the optimization of functions possibly constrained having many local extrema the method is illustrated in some difftcult pedagogical examples and used to solve a problem analyzed by bates technometrics 25 pp 373 376 1983 for which we identify an improved optimum the sensitivity of the solution to changes in the constraints and in other specifications of the problem is analyzed and discussed

varying coefficient models we explore a class of regression and generalized regression models in which the coefficients are allowed to vary as smooth functions of other variables general algorithms are presented for estimating the models flexibly and some examples are given this class of models ties together generalized additive models and dynamic generalized linear models into one common framework when applied to the proportional hazards model for survival data this approach provides a new way of modelling departures from the proportional hazards assumption

asymptotic global behavior for stochastic approximation and diffusions with slowly decreasing noise effects global minimization via monte carlo the asymptotic behavior of the systems x n 1 x n a n b x n xi n a n sigma x n psi n and dy bar b y dt sqrt a t sigma y dw is studied where psi n is i i d gaussian xi n is a correlated bounded sequence of random variables and a n approx a 0 log a 1 n without xi n such algorithms are versions of the simulated annealing method for global optimization when the objective function values can only be sampled via monte carlo the discrete algorithm is a combination of stochastic approximation and simulated annealing our forms are appropriate the psi n are the annealing variables and xi n is the sampling noise for large a 0 a full asymptotic analysis is presented via the theory of large deviations mean escape time after arbitrary time n from neighborhoods of stable sets of the algorithm mean transition times after arbitrary time n from a neighborhood of one stable set to another a

global optimizations and tabu search based on memory based on the idea of tabu search that glover et al put forward a new tabu search named memory tabu search mts is proposed for solving the multiple minima problem of continuous functions two convergence theorems which show that mts asymptotically converges to the global optimal solutions in probability one under suitable conditions are given numerical results illustrate that this algorithm is efficient robust and easy to implement

specification of varying coefficient time series models via generalized flexible least squares abstract flexible least squares fls is a method for recursively estimating the time paths of the coefficients of a regression model with time varying coefficients in its standard form the fls solution is capable of capturing smooth changes of the coefficients over the sample period for time series models erratic coefficient changes for instance due to seasonal variation are possible a generalization of fls is proposed which can account for such phenomena the method is applied to artificially generated as well as real economic data specifically west german income and consumption time series are analyzed in some detail

a class of filled functions for box constrained continuous global optimization in this paper we show that the unconstrained continuous global minimization problem cannot be solved by any algorithm so we consider the box constrained continuous global minimization problem we present a definition of filled function for the problem moreover a new class of filled functions are constructed which contains only one parameter a randomized algorithm is designed to solve the box constrained continuous global minimization problem based on the filled function numerical experiments are presented to show the practicability of the algorithm

solving the unconstrained optimization problem by a variable neighborhood search this paper presents variable neighborhood search vns for the problem of finding the global minimum of a nonconvex function the variable neighborhood search which changes systematically neighborhood structures in the search for finding a better solution is used to guide a set of standard improvement heuristics this algorithm was tested on some standard test functions and successful results were obtained its performance was compared with the other algorithms and observed to be better

revealed preference tests of collectively rational consumption behavior formulations and algorithms this paper focuses on revealed preference tests of the collective model of household consumption we start by showing that the decision problems corresponding to testing collective rationality are np complete this makes the application of these tests problematic for increasingly available large r scale data sets we then present two approaches to overcome this negative result first we introduce exact algorithms based on mixed integer programming mip formulations of the collective rationality tests which can be usefully applied to medium sized data sets next we propose simulated annealing heuristics which allow for efficient testing of the collective model in the case of large data sets we illustrate our methods by a number of computational experiments based on dutch labor supply data

a common tabu search algorithm for the global optimization of engineering problems abstract a novel common tabu algorithm for global optimizations of engineering problems is presented the robustness and efficiency of the presented method are evaluated by using standard mathematical functions and by solving a practical engineering problem the numerical results show that the proposed method is i superior to the conventional tabu search algorithm in robustness and ii superior to the simulated annealing algorithm in efficiency

sequential and parallel algorithms for global minimizing functions with lipschitzian derivatives abstract in this paper sequential and parallel algorithms using derivatives for solving unconstrained one dimensional global optimization problems are described sufficient conditions of convergence to all global minimizers are established for both methods parallel algorithm conditions which guarantee significant speed up in comparison to the sequential version of the method are presented the sequential method is numerically compared with the algorithms of breiman and cutler pijavskii and strongin on a set of 20 test functions taken from literature we also present results of numerical experiments illustrating the performance of the parallel method all experiments have been executed on the parallel computer alliant fx 80

user s guide to viscosity solutions of second order partial differential equations the notion of viscosity solutions of scalar fully nonlinear partial differential equations of second order provides a framework in which startling comparison and uniqueness theorems existence theorems and theorems about continuous dependence may now be proved by very efficient and striking arguments the range of important applications of these results is enormous this article is a self contained exposition of the basic theory of viscosity solutions

stochastic global optimization methods part 1 clustering methods in this stochastic approach to global optimization clustering techniques are applied to identify local minima of a real valued objective function that are potentially global three different methods of this type are described their accuracy and efficiency are analyzed in detail

stochastic relaxation gibbs distributions and the bayesian restoration of images we make an analogy between images and statistical mechanics systems pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice like physical system the assignment of an energy function in the physical system determines its gibbs distribution because of the gibbs distribution markov random field mrf equivalence this assignment also determines an mrf image model the energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the mrf for a range of degradation mechanisms including blurring nonlinear deformations and multiplicative or additive noise the posterior distribution is an mrf with a structure akin to the image model by the analogy the posterior distribution defines another imaginary physical system gradual temperature reduction in the physical system isolates low energy states annealing or what is the same thing the most probable states under the gibbs distribution the analogous operation under the posterior distribution yields the maximum a posteriori map estimate of the image given the degraded observations the result is a highly parallel relaxation algorithm for map estimation we establish convergence properties of the algorithm and we experiment with some simple pictures for which good restorations are obtained at low signal to noise ratios

a note on the use of a fuzzy approach in adaptive partitioning algorithms for global optimization in global optimization adaptive partitioning algorithms apa operate on the basis of partitioning the feasible region into subregions sampling and evaluating each subregion and selecting one or more subregions for repartitioning the purpose of the repartitioning process is to locate a narrow neighborhood around the global optimum in this correspondence we propose to use a fuzzy approach in the assessment of subregions using random samples taken from these subregions we discuss different types of uncertainties involved in apa and we conclude that the use of a fuzzy approach in the assessment of subregions is in concurrence with apa s convergence property we provide numerical results for the fuzzy approach on 13 test functions from the literature

diffusion for global optimization in r n we seek a global minimum of u mathbb r n to mathbb r the solution to d dt x t nabla u x t will find local minima using the idea of simulated annealing we consider th

a characterization of binary decision diagrams binary decision diagrams bdds are a representation of boolean functions its use in the synthesis simulation and testing of boolean circuits has been proposed by various researchers in all these applications of bdds solutions to some fundamental computational problems are needed a characterization of bdds in terms of the complexity of these computational problems is presented a tighter bound on the size of an ordered bdd that can be computed from a given boolean circuit is presented on the basis of the results a case is made for exploring the use of repeated bdds with a small number of repeated variables and free bdds for some applications for which only ordered bdds have been used so far

a probabilistic cooperative competitive hierarchical model for global optimization abstract stochastic searching methods have been applied widely to areas such as continuous and combinatorial optimization problems in a number of disciplines many existing methods solve these problems by navigating on the surface of the possibly rugged landscape this kind of navigation is not very effective because the property of the landscape at different resolutions can be very different time spent at the beginning of the search on the detailed part of the landscape is often useless appropriate searching strategies should be adopted at different resolutions in this paper we propose a new probabilistic searching model for global optimization the main contributions of the model are 1 to provide a basis for resolution control and smoothing of search space and 2 to introduce continuous memory into stochastic search the basis of resolution control is achieved by dividing the search space into a finite number of n dimensional partitions structurally the number of partitions governs the resolution of the search space the more the partitions the finer is the search space and the more detailed and rugged is the landscape the benefits are twofold first the rugged landscape problem can be smoothed because the ruggedness is a matter of the number of partitions hence the difficulty in search due to the ruggedness of the landscape can be controlled second it provides a basis to implement algorithms that may change the view of the landscape during the search process because we can dynamically divide the search space accordingly another important feature that we use is continuous memory throughout the search process searching experience is continuously accumulated in order to shape the global picture of the search space guiding the future searching direction we present results on the algorithm performance in handling numerical function optimization the empirical results show that our new model is comparable to and in many cases performs better than that of the other advanced methods in terms of solution quality and computation required

time varying linear regression via flexible least squares abstract suppose noisy observations obtained on a process are assumed to have been generated by a linear regression model with coefficients which evolve only slowly over time if at all do the estimated time paths for the coefficients display any systematic time variation or is time constancy a reasonably satisfactory approximation a flexible least squares fls solution is proposed for this problem consisting of all coefficient sequence estimates which yield vector minimal sums of squared residual measurement and dynamic errors conditional on the given observations a procedure with fortran implementation is developed for the exact sequential updating of the fls estimates as the process length increases and new observations are obtained simulation experiments demonstrating the ability of fls to track linear quadratic sinusoidal and regime shift motions in the true coefficients despite noisy observations are reported an empirical money demand application is also summarized

stochastic global optimization methods part 11 multi level methods in part ii of our paper two stochastic methods for global optimization are described that with probability 1 find all relevant local minima of the objective function with the smallest possible number of local searches the computational performance of these methods is examined both analytically and empirically

studies of multi start clustering for global optimization global multi modal optimization problems arise in many engineering applications owing to the existence of multiple minima it is a challenge to solve the multi modal optimization problem and to identify the global minimum especially if efficiency is a concern in this paper variants of the multi start with clustering strategy are developed and studied for identifying multiple local minima in nonlinear global optimization problems the study considers the sampling procedure the use of hessian information in forming clusters the technique for cluster analysis and the local search procedure variations of multi start with clustering are applied to 15 multi modal problems a comparative study focuses on the overall search effectiveness in terms of the number of local searches performed local minima found and required function evaluations the performance of these multi start clustering algorithms ranges from very efficient to very robust copyright 2002 john wiley sons ltd

estimating the prediction function and the number of unseen species in sampling with replacement abstract a sample of n units is taken from a population consisting of an unknown number of species we are interested in estimating the number of species and the prediction function for future sampling the prediction function is defined as the expected number of new species that will be found if an additional sample of size tn is taken for any positive real number t in this paper we point out that an estimator suggested by efron and thisted lacks some essential properties of the true prediction function for example the property of alternating copositivity as a result it cannot be used for large values of t we propose an alternative estimator that possesses the essential properties and is easily obtained we illustrate our estimator with two numerical examples and a simulation study

optimal and sub optimal stopping rules for the multistart algorithm in global optimization in this paper the problem of stopping the multistart algorithm for global optimization is considered the algorithm consists of repeatedly performing local searches from randomly generated starting points the crucial point in this algorithmic scheme is the development of a stopping criterion the approach analyzed in this paper consists in stopping the sequential sampling as soon as a measure of the trade off between the cost of further local searches is greater than the expected benefit i e the possibility of discovering a better optimum stopping rules are thoroughly investigated both from a theoretical point of view and from a computational one via extensive simulation this latter clearly shows that the simple1 step look ahead rule may achieve surprisingly good results in terms of computational cost vs final accuracy

fast marching methods fast marching methods are numerical schemes for computing solutions to the nonlinear eikonal equation and related static hamilton jacobi equations based on entropy satisfying upwind schemes and fast sorting techniques they yield consistent accurate and highly efficient algorithms they are optimal in the sense that the computational complexity of the algorithms is o n log n where n is the total number of points in the domain the schemes are of use in a variety of applications including problems in shape offsetting computing distances from complex curves and surfaces shape from shading photolithographic development computing first arrivals in seismic travel times construction of shortest geodesics on surfaces optimal path planning around obstacles and visibility and reflection calculations in this paper we review the development of these techniques including the theoretical and numerical underpinnings provide details of the computational schemes including higher order versions and demonstrate the techniques in a collection of different areas

comparison of partition evaluation measures in an adaptive partitioning algorithm for global optimization abstract an adaptive partitioning algorithm with random search is proposed to locate the global optimum of multimodal functions partitioning algorithms divide the feasible region into nonoverlapping partitions in order to restrict and direct the search to the most promising region expected to contain the global optimum in such a scheme a partition evaluation measure is required to assess sub regions in order to re partition the most promising sub region and intensify the search within that area this study provides computational results on several classes of partition evaluation measures used in the assessment of samples taken from all partitions among the partition evaluation classes used in our comparison are fuzzy statistical and deterministic interval estimation measures performance in terms of solution quality is reported on an extensive set of 77 test functions collected from the literature

test length for pseudorandom testing the process of determining the required test length for a desired level of confidence for pseudorandom testing using a random sampling without replacement model is examined the differences between random and pseudorandom testing are discussed and developed the strictly random testing model is shown to be inaccurate for high confidence testing of combinational circuits a method of calculating the required test length for pseudorandom testing based upon fault detectabilities is described the result provides a very accurate prediction of required test length applicable to self test using pseudorandom inputs

a stochastic method for global optimization a stochastic method for global optimization is described and evaluated the method involves a combination of sampling clustering and local search and terminates with a range of confidence intervals on the value of the global optimum computational results on standard test functions are included as well

a dynamic clustering based differential evolution algorithm for global optimization a dynamic clustering based differential evolution algorithm cde for global optimization is proposed to improve the performance of the differential evolution de algorithm with population evolution cde algorithm gradually changes from exploring promising areas at the early stages to exploiting solution with high precision at the later stages experiments on 28 benchmark problems including 13 high dimensional functions show that the new method is able to find near optimal solutions efficiently compared with other existing algorithms cde improves solution accuracy with less computational effort

an improved simulated annealing for solving the linear constrained optimization problems in this paper we propose an improved simulated annealing isa a global optimization algorithm for solving the linear constrained optimization problems of which the main characteristics are that only one component of current solution is changed based on the gaussian distribution in each iteration and isa can directly solve the linear constrained optimization problems by solving 6 benchmark functions with the lower and upper bounds constraints and 6 functions with linear constraints isa is superior to the classical techniques for solving the problems with lower and upper bounds and reduces greatly the number of function evolutions compared with genocop with the same precision condition

a hybrid algorithm for identifying global and local minima when optimizing functions with many minima the problem of determining most global minima including some of the local ones for unconstrained non convex functions is investigated this is tackled using a hybrid approach that combines simulated annealing tabu search and a descent method this approach has the advantage of not requiring differentiability of the function the special feature of this approach is that it produces not only all the global minima with a high frequency but also those good local minima which may be of relevance to the user this is the first time such a view is put forward especially in the context of functions of continuous variables this method is tested on standard test functions used in the literature and encouraging results are obtained

stopping rules for box constrained stochastic global optimization we present three new stopping rules for multistart based methods the first uses a device that enables the determination of the coverage of the bounded search domain the second is based on the comparison of asymptotic expectation values of observable quantities to the actually measured ones the third offers a probabilistic estimate for the number of local minima inside the search domain their performance is tested and compared to that of other widely used rules on a host of test problems in the framework of multistart

a tabu search hooke and jeeves algorithm for unconstrained optimization this paper addresses the problem of finding the global minimum of a nonconvex function a new hybrid algorithm for this problem based on tabu search is developed it is hybrid in the sense that search directions are generated using tabu search strategy and then they are used in an optimization algorithm the algorithm is tested on some standard test functions and its performance is compared with existing algorithms computational results show that the proposed algorithm is very efficient and robust

work by robert kalaba on multicriteria estimation for many processes of interest particularly in the socio economic and biological sciences the underlying relationships are not yet well understood consequently aside from identities model equations tend to be rough approximations at best it therefore can be both inappropriate and fundamentally misleading to adopt as a routine practice the standard statistical inference assumption that model discrepancy terms i e model specification errors are commensurable random quantities governed by meaningful probability relations for the past twelve years robert kalaba has explored the consequences for statistical inference which result from these common sense observations one consequence in particular is that multicriteria methods take on a new and critical importance this paper explains by example his multicriteria re formulation of the basic estimation problem the reconciliation of imperfect theory with observations

an efficient algorithm for large scale global optimization of continuous functions a fast descent algorithm resorting to a stretching function technique and built on one hybrid method grsa which combines simulated annealing sa algorithm and gradient based methods for large scale global optimizations is proposed unlike the previously proposed method in which the original objective functions remain unchanged during the whole course of optimization the new method firstly constructs an auxiliary function on one local minimizer obtained by gradient based methods and then sa is executed on this constructed auxiliary function instead of on the original objective function in order that we can improve the jumping ability of sa algorithm to escape from the currently discovered local minimum to a better one from which the gradient based methods restart a new local search the above procedure is repeated until a global minimum is detected in addition corresponding to the adopted stretching technique a new next trial point generating scheme is designed it is verified by simulation especially on large scale problems that the convergence speed is greatly accelerated which is its main difference from many other reported methods that mostly cope with functions with less than 50 variables and does not apply to large scale optimization problems furthermore the new algorithm functions as a global optimization procedure with a high success probability and high solution precision

decomposition in general mathematical programming in this paper a unifying framework is presented for the generalization of the decomposition methods originally developed by benders 1962 and dantzig and wolfe 1960 these generalizations calledvariable decomposition andconstraint decomposition respectively are based on the general duality theory developed by tind and wolsey the framework presented is of a general nature since there are no restrictive conditions imposed on problem structure moreover inaccuracies and duality gaps that are encountered during computations are accounted for the two decomposition methods are proven not to cycle if certain fairly general conditions are met furthermore finite convergence can be ensured under the traditional finiteness conditions and asymptotic convergence can be guaranteed once certain continuity conditions are met the obvious symmetry between both types of decomposition methods is explained by establishing a duality relation between the two which extends a similar result in linear programming a remaining asymmetry in the asymptotic convergence results is argued to be a direct consequence of a fundamental asymmetry that resides in the tind wolsey duality theory it can be shown that in case the latter asymmetry disappears the former does too other decomposition techniques such as lagrangean decomposition and cross decomposition turn out to be captured by the general framework presented here as well

on tikhonov regularisation abstract we introduce bayesian estimation into the tikhonov regularisation process for recovering functions of high bandwidth from noisy band limited observations the method allows for the incorporation of featural constraints and requires the global minimisation of a functional derived from damped pseudospectral approximation

on computing signal probability and detection probability of stuck at faults algorithms for the following two problems are presented 1 computing detection probability of stuck at faults cdp and 2 computing signal probability csp these problems arise in the context of random testing pseudorandom testing and testability analysis of combinational circuits the algorithm for cdp combines the notion of supergates and a refinement of th algorithm for cdp presented in the work of s chakravarty and h b hunt iii 1986 the algorithm for cdp can be used to compute the exact value of detection probability of multiple stuck at faults in circuits with multiple outputs single input single output pseudo gates are inserted to model stuck at faults and derive an equivalent single output circuit cdp is thus reduced to the problem of computing the probability distribution of the output over the set of four logic values 0 1d d the algorithm for cdp uses an efficient enumeration algorithm the authors show how the enumeration algorithm can be used to refine the algorithm for csp

bayesian stopping rules for multistart global optimization methods by far the most efficient methods for global optimization are based on starting a local optimization routine from an appropriate subset of uniformly distributed starting points as the number of local optima is frequently unknown in advance it is a crucial problem when to stop the sequence of sampling and searching by viewing a set of observed minima as a sample from a generalized multinomial distribution whose cells correspond to the local optima of the objective function we obtain the posterior distribution of the number of local optima and of the relative size of their regions of attraction this information is used to construct sequential bayesian stopping rules which find the optimal trade off between reliability and computational effort

genetic algorithms for continuous optimization problems a concept of parameter space size adjustment the concept of parameter space size adjustment is proposed in order to enable successful application of genetic algorithms to continuous optimization problems performance of genetic algorithms with six different combinations of selection and reproduction mechanisms with and without parameter space size adjustment were severely tested on eleven multiminima test functions an algorithm with the best performance was employed for the determination of the model parameters of the optical constants of pt ni and cr

investigating a hybrid simulated annealing and local search algorithm for constrained optimization constrained optimization problems cop often take place in many practical applications such as kinematics chemical process optimization power systems and so on these problems are challenging in terms of identifying feasible solutions when constraints are non linear and non convex therefore finding the location of the global optimum in the non convex cop is more difficult as compared to non convex bound constrained global optimization problems this paper proposes a hybrid simulated annealing method hsa for solving the general cop hsa has features that address both feasibility and optimality issues and here it is supported by a local search procedure feasible sequential quadratic programming fsqp we develop two versions of hsa the first version hsap incorporates penalty methods for constraint handling and the second one hsad eliminates the need for imposing penalties in the objective function by tracing feasible and infeasible solution sequences independently numerical experiments show that the second version is more reliable in the worst case performance

estimating the number of species a review how many kinds are there suppose that a population is partitioned into c classes in many situations interest focuses not on estimation of the relative sizes of the classes but on estimation of c itself for example biologists and ecologists may be interested in estimating the number of species in a population of plants or animals numismatists may be concemed with estimating the number of dies used to produce an ancient coin issue and linguists may be interested in estimating the size of an author s vocabulary in this article we review the problem of statistical estimation of c many approaches have been proposed some purely data analytic and others based in sampling theory in the latter case numerous variations have been considered the population may be finite or infinite if finite samples may be taken with replacement multinomial sampling or without replacement hypergeometric sampling or by bernoulli sampling if infinite sampling may be multinomial or bernoulli or the sample may be th

a hybrid genetic algorithm with the baldwin effect here we present a new hybrid genetic algorithm hga with the baldwin effect in the hga a local search is employed to change the fitness of individuals but the acquired improvements do not change the individual itself this local search step exploits the baldwin effect some numerical applications show that this algorithm can yield the global optimum more efficiently than commonly used hgas a theorem is presented that guarantees the convergence in probability of the new hga