preconditioning and boundary conditions without h 2 estimates l 2 condition numbers and the distribution of the singular values this work deals with the behavior in the l sub 2 norm of the condition number and distribution of the l sub 2 singular values of the preconditioned operators b sub h sup minus 1 a sub h and a sub h b sub h sup minus 1 sub h where a sub h and b sub h are finite element discretizations of second order elliptic operators a and b in an earlier work manteuffel and parter proved that b sub h sup minus 1 a sub h a sub h b sub h sup minus 1 have a uniformly bounded l sub 2 condition number if and only if a sup and b sup a and b have the same boundary conditions this earlier work used the h sub 2 regularity of a and b as well as optimal l sub 2 error estimates and a quasi uniform grid for the finite element spaces in the present paper the authors first extend these condition number results to the case in which neither h sub 2 regularity and hence optimal l sub 2 error estimates nor the quasi uniformity assumption need to be satisfied instead it is assumed that the principle part of the preconditioning operator b is a scalar multiple 1 mu of the principle part of a

the limited memory conjugate gradient method in theory the successive gradients generated by the conjugate gradient method applied to a quadratic should be orthogonal however for some ill conditioned problems orthogonality is quickly lost due to rounding errors and convergence is much slower than expected a limited memory version of the nonlinear conjugate gradient method is developed the memory is used to both detect the loss of orthogonality and to restore orthogonality an implementation of the algorithm is presented based on the cg descent nonlinear conjugate gradient method limited memory cg descent l cg descent possesses a global convergence property similar to that of the memoryless algorithm but has much better practical performance numerical comparisons to the limited memory bfgs method l bfgs are given using the cuter test problems

global convergence of nonmonotone descent methods for unconstrained optimization problems global convergence results are established for unconstrained optimization algorithms that utilize a nonmonotone line search procedure this procedure allows the user to specify a flexible forcing function and includes the nonmonotone armijo rule the nonmonotone goldstein rule and the nonmonotone wolfe rule as special cases

on trust region methods for unconstrained minimization without derivatives we consider some algorithms for unconstrained minimization without derivatives that form linear or quadratic models by interpolation to values of the objective function then a new vector of variables is calculated by minimizing the current model within a trust region techniques are described for adjusting the trust region radius and for choosing positions of the interpolation points that maintain not only nonsingularity of the interpolation equations but also the adequacy of the model particular attention is given to quadratic models with diagonal second derivative matrices because numerical experiments show that they are often more efficient than full quadratic models for general objective functions finally some recent research on the updating of full quadratic models is described briefly using fewer interpolation equations than before the resultant freedom is taken up by minimizing the frobenius norm of the change to the second derivative matrix of the model a preliminary version of this method provides some very promising numerical results

a globally convergent version of the polak ribiere conjugate gradient method in this paper we propose a new line search algorithm that ensures global convergence of the polak ribiere conjugate gradient method for the unconstrained minimization of nonconvex differentiable functions in particular we show that with this line search every limit point produced by the polak ribiere iteration is a stationary point of the objective function moreover we define adaptive rules for the choice of the parameters in a way that the first stationary point along a search direction can be eventually accepted when the algorithm is converging to a minimum point with positive definite hessian matrix under strong convexity assumptions the known global convergence results can be reobtained as a special case from a computational point of view we may expect that an algorithm incorporating the step size acceptance rules proposed here will retain the same good features of the polak ribiere method while avoiding pathological situations

a nonlinear conjugate gradient method with a strong global convergence property conjugate gradient methods are widely used for unconstrained optimization especially large scale problems the strong wolfe conditions are usually used in the analyses and implementations of conjugate gradient methods this paper presents a new version of the conjugate gradient method which converges globally provided the line search satisfies the standard wolfe conditions the conditions on the objective function are also weak being similar to those required by the zoutendijk condition

non monotone trust region algorithms for nonlinear optimization subject to convex constraints this paper presents two new trust region methods for solving nonlinear optimization problems over convex feasible domains these methods are distinguished by the fact that they do not enforce strict monotonicity of the objective function values at successive iterates the algorithms are proved to be convergent to critical points of the problem from any starting point extensive numerical experiments show that this approach is competitive with the lancelot package

on the convergence of pattern search algorithms we introduce an abstract definition of pattern search methods for solving nonlinear unconstrained optimization problems our definition unifies an important collection of optimization methods that neither compute nor explicitly approximate derivatives we exploit our characterization of pattern search methods to establish a global convergence theory that does not enforce a notion of sufficient decrease our analysis is possible because the iterates of a pattern search method lie on a scaled translated integer lattice this allows us to relax the classical requirements on the acceptance of the step at the expense of stronger conditions on the form of the step and still guarantee global convergence

quadratic cost flow and the conjugate gradient method by introducing quadratic penalty terms a convex non separable quadratic network program can be reduced to an unconstrained optimization problem whose objective function is a piecewise quadratic and continuously differentiable function a conjugate gradient method is applied to the reduced problem and its convergence is proved the computation exploits the special network data structures originated from the network simplex method this algorithmic framework allows direct extension to multicommodity cost flows some preliminary computational results are presented

convergence in probability of compressed annealing we consider combinatorial optimization problems for which the formation of a neighborhood structure of feasible solutions is impeded by a set of constraints neighborhoods are recovered by relaxing the complicating constraints into the objective function within a penalty term we examine a heuristic called compressed annealing that integrates a variable penalty multiplier approach within the framework of simulated annealing we refer to the value of the penalty multiplier as pressure we analyze the behavior of compressed annealing by exploring the interaction between temperature which controls the ability of compressed annealing to climb hills and pressure which controls the height of the hills we develop a necessary and sufficient condition on the joint cooling and compression schedules for compressed annealing to converge in probability to the set of global minima our work generalizes the results of hajek 1988 in the sense that when there are no relaxed constraints our results reduce to his

a new conjugate gradient method with guaranteed descent and an efficient line search a new nonlinear conjugate gradient method and an associated implementation based on an inexact line search are proposed and analyzed with exact line search our method reduces to a nonlinear version of the hestenes stiefel conjugate gradient scheme for any inexact line search our scheme satisfies the descent condition bf g k sf t bf d k le frac 7 8 bf g k 2 moreover a global convergence result is established when the line search fulfills the wolfe conditions a new line search scheme is developed that is efficient and highly accurate efficiency is achieved by exploiting properties of linear interpolants in a neighborhood of a local minimizer high accuracy is achieved by using a convergence criterion which we call the approximate wolfe conditions obtained by replacing the sufficient decrease criterion in the wolfe conditions with an approximation that can be evaluated with greater precision in a neighborhood of a local minimum than the usual sufficient decrease criterion numerical comparisons are given with both l bfgs and conjugate gradient methods using the unconstrained optimization problems in the cute library

global convergence of general derivative free trust region algorithms to first and second order critical points in this paper we prove global convergence for first and second order stationary points of a class of derivative free trust region methods for unconstrained optimization these methods are based on the sequential minimization of quadratic or linear models built from evaluating the objective function at sample sets the derivative free models are required to satisfy taylor type bounds but apart from that the analysis is independent of the sampling techniques a number of new issues are addressed including global convergence when acceptance of iterates is based on simple decrease of the objective function trust region radius maintenance at the criticality step and global convergence for second order critical points

convergence of the polak ribiere polyak conjugate gradient method abstract in this paper we consider the global convergence of the polak ribiere polyak abbreviated prp conjugate gradient method for unconstrained optimization problems a new armijo type line search is proposed for the original prp method and some convergence properties are given under some mild conditions the new armijo type line search can make the prp method choose a suitable initial step size so as to decrease the function evaluations at each iteration and improve the performance of the prp method numerical results show that the prp method with the new armijo type line search is more efficient than other similar methods in practical computation

convergence guarantees for generalized adaptive stochastic search methods for continuous global optimization this paper presents some simple technical conditions that guarantee the convergence of a general class of adaptive stochastic global optimization algorithms by imposing some conditions on the probability distributions that generate the iterates these stochastic algorithms can be shown to converge to the global optimum in a probabilistic sense these results also apply to global optimization algorithms that combine local and global stochastic search strategies and also those algorithms that combine deterministic and stochastic search strategies this makes the results applicable to a wide range of global optimization algorithms that are useful in practice moreover this paper provides convergence conditions involving the conditional densities of the random vector iterates that are easy to verify in practice it also provides some convergence conditions in the special case when the iterates are generated by elliptical distributions such as the multivariate normal and cauchy distributions these results are then used to prove the convergence of some practical stochastic global optimization algorithms including an evolutionary programming algorithm in addition this paper introduces the notion of a stochastic algorithm being probabilistically dense in the domain of the function and shows that under simple assumptions this is equivalent to seeing any point in the domain with probability 1 this in turn is equivalent to almost sure convergence to the global minimum finally some simple results on convergence rates are also proved

fusion of hidden markov random field models and its bayesian estimation in this paper we present a hidden markov random field hmrf data fusion model the proposed model is applied to the segmentation of natural images based on the fusion of colors and textons into julesz ensembles the corresponding exploration selection estimation ese procedure for the estimation of the parameters is presented this method achieves the estimation of the parameters of the gaussian kernels the mixture proportions the region labels the number of regions and the markov hyper parameter meanwhile we present a new proof of the asymptotic convergence of the ese procedure based on original finite time bounds for the rate of convergence

the barzilai and borwein gradient method for the large scale unconstrained minimization problem the barzilai and borwein gradient method for the solution of large scale unconstrained minimization problems is considered this method requires few storage locations and very inexpensive computations furthermore it does not guarantee descent in the objective function and no line search is required recently the global convergence for the convex quadratic case has been established however for the nonquadratic case the method needs to be incorporated in a globalization scheme in this work a nonmonotone line search strategy that guarantees global convergence is combined with the barzilai and borwein method this strategy is based on the nonmonotone line search technique proposed by grippo lampariello and lucidi siam j numer anal 23 1986 pp 707 716 numerical results to compare the behavior of this method with recent implementations of the conjugate gradient method are presented these results indicate that the global barzilai and borwein method may allow some significant reduction in the number of line searches and also in the number of gradient evaluations

an assessment of nonmonotone linesearch techniques for unconstrained optimization the purpose of this paper is to discuss the potential of nonmonotone techniques for enforcing convergence of unconstrained minimization algorithms from starting points distant from the solution linesearch based algorithms are considered for both small and large problems and extensive numerical experiments show that this potential is sometimes considerable a new variant is introduced in order to limit some of the identified drawbacks of the existing techniques this variant is again numerically tested and appears to be competitive finally the impact of preconditioning on the considered methods is examined

annealed feynman kac models we analyze the concentration properties of an annealed feynman kac model in distribution space we characterize the concentration regions in terms of a variational problem involving a competition between the potential function and the mutation kernel when the temperature parameter is evanescent with time and under appropriate hypotheses the probability mass tends to concentrate on regions with minimal potential values we give a precise description of these areas using non linear semi group contractions and large deviation techniques we illustrate this annealed model with two physical interpretations related respectively to markov motions in absorbing media and interacting measure valued processes

a new class of nonlinear conjugate gradient coefficients with global convergence properties abstract nonlinear conjugate gradient cg methods have played an important role in solving large scale unconstrained optimization their wide application in many fields is due to their low memory requirements and global convergence properties numerous studies and modifications have been conducted recently to improve this method in this paper a new class of conjugate gradient coefficients k that possess global convergence properties is presented the global convergence result is established using exact line searches numerical result shows that the proposed formula is superior and more efficient when compared to other cg coefficients

relative frequencies of generalized simulated annealing we consider a class of nonhomogeneous markov chains arising in simulated annealing and related stochastic search algorithms using only elementary first principles we analyze the convergence and rate of convergence of the relative frequencies of visits to states in the markov chain we describe in detail three examples including the standard simulated annealing algorithm to show how our framework applies to specific stochastic search algorithmsthese examples have not previously been recognized to be sufficiently similar to share common analytical grounds our analysis though elementary provides the strongest sample path convergence results to date for simulated annealing type markov chains our results serve to illustrate that by taking a purely sample path view surprisingly strong statements can be made using only relatively elementary tools

the quasispecies regime for the simple genetic algorithm with ranking selection we study the simple genetic algorithm with a ranking selection mechanism linear ranking or tournament we denote by the length of the chromosomes by m the population size by pc the crossover probability and by pm the mutation probability we introduce a parameter called the selection drift which measures the selection intensity of the fittest chromosome we show that the dynamics of the genetic algorithm depend in a critical way on the parameter 1 pc 1 pm if 1 then the genetic algorithm operates in a quasispecies regime an advantageous mutant invades a positive fraction of the population with probability larger than a constant p which does not depend on m we estimate next the probability of the occurrence of a catastrophe the whole population falls below a fitness level which was previously reached by a positive fraction of the population the asymptotic results suggest the following rules 1 pc 1 pm should be slightly larger than 1 pm should be of order 1 m should be larger than ln the running time should be of exponential order in m the first condition requires that pm pc ln these conclusions must be taken with great care they come from an asymptotic regime and it is a formidable task to understand the relevance of this regime for a real world problem at least we hope that these conclusions provide interesting guidelines for the practical implementation of the simple genetic algorithm

global convergence properties of conjugate gradient methods for optimization this paper explores the convergence of nonlinear conjugate gradient methods without restarts and with practical line searches the analysis covers two classes of methods that are globally convergent on smooth nonconvex functions some properties of the fletcher reeves method play an important role in the first family whereas the second family shares an important property with the polak ribiere method numerical experiments are presented

global convergence of a two parameter family of conjugate gradient methods without line search we study the global convergence of a two parameter family of conjugate gradient methods in which the line search procedure is replaced by a fixed formula of stepsize this character is of significance if the line search is expensive in a particular application in addition to the convergence results we present computational results for various conjugate gradient methods without line search including those discussed by sun and zhang ann oper res 103 2001 161 173

convergence properties of algorithms for nonlinear optimization this paper reviews some of the most successful methods for unconstrained constrained and nondifferentiable optimization calculations particular attention is given to the contribution that theoretical analysis has made to the development of algorithms it seems that practical considerations provide the main new ideas and that subsequent theoretical studies give improvements to algorithms coherence to the subject and better understanding

on the limited memory bfgs method for large scale optimization we study the numerical performance of a limited memory quasi newton method for large scale optimization which we call the l bfgs method we compare its performance with that of the method developed by buckley and lenir 1985 which combines cycles of bfgs steps and conjugate direction steps our numerical tests indicate that the l bfgs method is faster than the method of buckley and lenir and is better able to use additional storage to accelerate convergence we show that the l bfgs method can be greatly accelerated by means of a simple scaling we then compare the l bfgs method with the partitioned quasi newton method of griewank and toint 1982a the results show that for some problems the partitioned quasi newton method is clearly superior to the l bfgs method however we find that for other problems the l bfgs method is very competitive due to its low iteration cost we also study the convergence properties of the l bfgs method and prove global convergence on uniformly convex problems

a new class of nonlinear conjugate gradient coefficients with exact and inexact line searches conjugate gradient cg methods have played an important role in solving large scale unconstrained optimization in this paper we propose a new family of cg coefficients k that possess sufficient descent conditions and global convergence properties this new k is an extension of the already proven k rmil from rivaie et al 19 a new class of nonlinear conjugate gradient coefficient with global convergence properties appl math comp 218 2012 11323 11332 global convergence result is established using both exact and inexact line searches numerical results show that the performance of the new proposed formula is quite similar to k rmil and suited to both line searches importantly the performance of this k is more efficient and superior than the other well known k

limited memory reduced hessian methods for large scale unconstrained optimization limited memory bfgs quasi newton methods approximate the hessian matrix of second derivatives by the sum of a diagonal matrix and a fixed number of rank one matrices these methods are particularly effective for large problems in which the approximate hessian cannot be stored explicitly it can be shown that the conventional bfgs method accumulates approximate curvature in a sequence of expanding subspaces this allows an approximate hessian to be represented using a smaller reduced matrix that increases in dimension at each iteration when the number of variables is large this feature may be used to define limited memory reduced hessian methods in which the dimension of the reduced hessian is limited to save storage limited memory reduced hessian methods have the benefit of requiring half the storage of conventional limited memory methods in this paper we propose a particular reduced hessian method with substantial computational advantages compared to previous reduced hessian methods numerical results from a set of unconstrained problems in the cute test collection indicate that our implementation is competitive with the limited memory codes l bfgs and l bfgs b

stochastic global optimization methods part 11 multi level methods in part ii of our paper two stochastic methods for global optimization are described that with probability 1 find all relevant local minima of the objective function with the smallest possible number of local searches the computational performance of these methods is examined both analytically and empirically

parameter imprecision in finite state finite action dynamic programs in order to model parameter imprecision associated with a problem s reward or preference structure we examine a finite state finite action dynamic program having a one step transition value function that is affine in an imprecisely known parameter for the finite horizon case we also assume that the terminal value function is affine in the imprecise parameter we assume that the parameter of interest has no dynamics no new information about its value is received once the decision process begins and its imprecision is described by set inclusion we seek the set of all parameter independent strategies that are optimal for some value of the imprecisely known parameter we present a successive approximations procedure for solving the finite horizon case and a policy iteration procedure for determining the solution of the discounted infinite horizon case these algorithms are then applied to a decision analysis problem with imprecise utility function and to a markov decision process with imprecise reward structure we also present conditions that guarantee the existence of a parameter independent strategy that maximizes with respect to all other parameter invariant strategies the minimum value of its expected reward function over all possible parameter values

convergence properties of nonlinear conjugate gradient methods recently important contributions on convergence studies of conjugate gradient methods were made by gilbert and nocedal siam j optim 2 1992 pp 21 42 they introduce a sufficient descent condition to establish global convergence results although this condition is not needed in the convergence analyses of newton and quasi newton methods gilbert and nocedal hint that the sufficient descent condition which was enforced by their two stage line search algorithm may be crucial for ensuring the global convergence of conjugate gradient methods this paper shows that the sufficient descent condition is actually not needed in the convergence analyses of conjugate gradient methods consequently convergence results on the fletcher reeves and polak ribiere type methods are established in the absence of the sufficient descent condition to show the differences between the convergence properties of fletcher reeves and polak ribiere type methods two examples are constructed showing that neither the boundedness of the level set nor the restriction beta k geq 0 can be relaxed for the polak ribiere type methods

a note on convergence of prp method with new nonmonotone line search we in this note drop a lipschitz constant in a grippo lucidi type step length rule recently proposed by shi and shen z shi j shen convergence of prp method with new nonmonotone line search applied mathematics and computation 181 1 2006 423 431 and the original convergence remains valid

preconditioning nonconforming finite element methods for treating dirichlet boundary conditions ii this work deals with theh 1 condition numbers and the distribution of theb h singular values of the preconditioned operators b h 1 a h 0 h 1 wherea h andb h are finite element discretizations of second order elliptic operators a andb respectively b is also assumed to be self adjoint and positive definite for conforming finite elements parter and wong have shown that the singular values cluster in a positive finite interval goldstein also has derived results on the spectral distribution ofb h 1 a h using a different approach as a generalization of the results of parter and wong the current work includes nonconforming finite element methods which deal with dirichlet boundary conditions it will be shown that in this more general setting the singular values also cluster in a positive finite interval in particular if the leading part ofb is the same as the leading part ofa then the singular values cluster about the point 1 two specific methods are given as applications of this theory they are the penalty method of babuska and the method of nearly zero boundary conditions of nitsche finally it will be shown that the same results can be proven by an approach generalized from the work of goldstein

new properties of a nonlinear conjugate gradient method this paper provides several new properties of the nonlinear conjugate gradient method in 5 firstly the method is proved to have a certain self adjusting property that is independent of the line search and the function convexity secondly under mild assumptions on the objective function the method is shown to be globally convergent with a variety of line searches thirdly we find that instead of the negative gradient direction the search direction defined by the nonlinear conjugate gradient method in 5 can be used to restart any optimization method while guaranteeing the global convergence of the method some numerical results are also presented

a hybrid algorithm for identifying global and local minima when optimizing functions with many minima the problem of determining most global minima including some of the local ones for unconstrained non convex functions is investigated this is tackled using a hybrid approach that combines simulated annealing tabu search and a descent method this approach has the advantage of not requiring differentiability of the function the special feature of this approach is that it produces not only all the global minima with a high frequency but also those good local minima which may be of relevance to the user this is the first time such a view is put forward especially in the context of functions of continuous variables this method is tested on standard test functions used in the literature and encouraging results are obtained

trust region methods without using derivatives worst case complexity and the nonsmooth case trust region methods are a broad class of methods for continuous optimization that found application in a variety of problems and contexts in particular they have been studied and applied for problems without using derivatives the analysis of trust region derivative free methods has focused on global convergence and they have been proven to generate a sequence of iterates converging to stationarity independently of the starting point most of such an analysis is carried out in the smooth case and moreover little is known about the complexity or global rate of these methods in this paper we start by analyzing the worst case complexity of general trust region derivative free methods for smooth functions for the nonsmooth case we propose a smoothing approach for which we prove global convergence and bound the worst case complexity effort for the special case of nonsmooth functions that result from the composition of smooth and nonsmooth convex components we show how to improve the existing resu

an efficient algorithm for large scale global optimization of continuous functions a fast descent algorithm resorting to a stretching function technique and built on one hybrid method grsa which combines simulated annealing sa algorithm and gradient based methods for large scale global optimizations is proposed unlike the previously proposed method in which the original objective functions remain unchanged during the whole course of optimization the new method firstly constructs an auxiliary function on one local minimizer obtained by gradient based methods and then sa is executed on this constructed auxiliary function instead of on the original objective function in order that we can improve the jumping ability of sa algorithm to escape from the currently discovered local minimum to a better one from which the gradient based methods restart a new local search the above procedure is repeated until a global minimum is detected in addition corresponding to the adopted stretching technique a new next trial point generating scheme is designed it is verified by simulation especially on large scale problems that the convergence speed is greatly accelerated which is its main difference from many other reported methods that mostly cope with functions with less than 50 variables and does not apply to large scale optimization problems furthermore the new algorithm functions as a global optimization procedure with a high success probability and high solution precision

preconditioning nonconforming finite element methods this paper is concerned with the h sup 1 condition number and singular value distribution of preconditioned nonself adjoint discrete elliptic operators in particular a wide class of nonconforming finite element methods are investigated the main results give a precise interval on which the singular values cluster results of this kind have an important bearing on the convergence behavior of the conjugate gradient method applied to the normal equations

preconditioning isoparametric finite element methods taking into account numerical integration this paper is concerned with the h 1 condition numbers and the b h singular value distribution of preconditioned operators b h 1a h 0 h 1 where a h and b h are isoparametric finite element discretizations of second order elliptic operators a and b by taking into account numerical integration respectively where a is nonselfad joint and possibly indefinite and b is selfadjoint and positive definite it is proved that the h 1 condition numbers of operators b h 1a h 0 h 1 are uniformly bounded and the b h singular values cluster in a positive finite interval finally the implementation of b h 1 is discussed

note on the convergence of simulated annealing algorithms generalizing the results of faigle and schrader inform process lett 27 1988 pp 189 194 a short inductive proof is given that shows that the stationary distributions of a simulated annealing algorithm converge to a distribution where nonoptimal elements are generated with probability zero provided that the weak reversibility condition of hajek math oper res 13 1988 pp 311 329 holds

preconditioning projection nonconforming element method for the lowest order raviart thomas mixed triangular element method this paper is concerned with the h 1 condition number and the b h singular value distribution of preconditioned operators b h 1a h 0 h 1 where a h is the projection nonconforming element discretization of the nonself adjoint and possibly indefinite second order elliptic operator a b h is the nonconforming element discretization of the self adjoint and positive definite second order elliptic operator b the projection nonconforming element method is equivalent to the lowest order raviart thomas mixed triangular element method for second order elliptic problems it is proved that the h 1 condition number of operators b h 1a h 0 h 1 are uniformly bounded and the b h singular values cluster in a positive finite interval finally a simple v cycle multigrid implementation of b h 1 is given

convergence of nonmonotone line search method in this paper we develop a new nonmonotone line search for general line search method and establish some global convergence theorems the new nonmonotone line search is a novel form of the nonmonotone armijo line search and allows one to choose a larger step size at each iteration which is available in constructing new line search methods and possibly reduces the function evaluations at each iteration moreover we analyze the convergence rate of some special line search methods with the new line search preliminary numerical results show that some line search methods with the new nonmonotone line search are available and efficient in practical computation

a nonmonotone line search technique and its application to unconstrained optimization a new nonmonotone line search algorithm is proposed and analyzed in our scheme we require that an average of the successive function values decreases while the traditional nonmonotone approach of grippo lampariello and lucidi siam j numer anal 23 1986 pp 707 716 requires that a maximum of recent function values decreases we prove global convergence for nonconvex smooth functions and r linear convergence for strongly convex functions for the l bfgs method and the unconstrained optimization problems in the cute library the new nonmonotone line search algorithm used fewer function and gradient evaluations on average than either the monotone or the traditional nonmonotone scheme

decision analysis with incomplete utility and probability information an approach to the solution of decision analysis problems under uncertainty with imprecise and incomplete information is presented the methodology is designed for cases in which payoffs conditional on the state of nature are known precisely but only limited or imprecise probability and utility information is available regarding a decision maker s beliefs and tastes a decision maker provides conditional payoffs optionally bounds on state probabilities bounds on the certainty equivalent for a simple lottery any known relationships between probabilities of states of nature and a series of strict preferences between pairs of vectors of conditional payoffs we assume an exponential utility function with unknown parameter the method proceeds by sequentially eliciting preferences new bounds on probabilities and or the certainty equivalent and new relationships among probabilities until the problem is solved the methodology is demonstrated through a two state and a three state example which illust

some global convergence properties of the wei yao liu conjugate gradient method with inexact line search abstract in z wei s yao l liu the convergence properties of some new conjugate gradient methods applied mathematics and computation 183 2006 1341 1350 wei et al proposed a new conjugate gradient method called wyl method which has good numerical experiments and some excellent properties such as k wyl 0 in this paper we prove that while t k 1 c 2 l g k 2 d k 2 the sufficient descent condition can be satisfied at each iteration of the wyl method based on this the global convergence of the wyl method with the standard armijo line search and the generalized wolfe powell line search is established

cgrs an advanced hybrid method for global optimization of continuous functions closely coupling extended random search and conjugate gradient method abstract a new hybrid method for global optimization of continuous functions is proposed it is a combination of an extended random search method and a descent method random search is used as the global search strategy a newly developed distribution based region control makes use of already detected local minima to refine this search strategy the approach resembles classical step size control in deterministic optimization the descent method is embedded as a local search strategy for the detection of local minima a special realization of this approach is presented in this paper and called cgrs in cgrs the conjugate gradient method is utilized as descent method the proof of global convergence in probability for cgrs is given and extended to other descent methods used in the hybrid optimization approach in order to demonstrate the numerical properties of the approach test sets of multidimensional non convex optimization problems are solved the results are compared to well established hybrid methods for global optimization the new algorithm shows a high success rate with good and adjustable solution precision parameter tuning is not necessary but of course possible the new method proves to be efficient in terms of computational costs

quantitative approximations of evolving probability measures and sequential markov chain monte carlo methods we study approximations of evolving probability measures by an interacting particle system the particle system dynamics is a combination of independent markov chain moves and importance sampling resampling steps under global regularity conditions we derive non asymptotic error bounds for the particle system approximation in a few simple examples including high dimensional product measures bounds with explicit constants of feasible size are obtained our main motivation are applications to sequential mcmc methods for monte carlo integral estimation

convergence of descent method without line search line search method is sometimes a useful and efficient technique for solving unconstrained optimization problems especially for solving small and middle scale problems however a line search procedure is necessary at each iteration leading to a significant amount of computation in order to reduce the evaluations of objective functions and gradients the line search procedure should be avoided in algorithm design in this paper we propose a new descent method without line search for unconstrained optimization problems the algorithm is required to estimate some parameters at each iteration we analyze theoretically the global convergence of the new algorithm under mild conditions these theoretical conclusions can help us in designing new efficient methods for optimization problems

minimization by random search techniques we give two general convergence proofs for random search algorithms we review the literature and show how our results extend those available for specific variants of the conceptual algorithm studied here we then exploit the convergence results to examine convergence rates and to actually design implementable methods finally we report on some computational experience

on the convergence and applications of generalized simulated annealing the convergence of the generalized simulated annealing with time inhomogeneous communication cost functions is discussed this study is based on the use of log sobolev inequalities and semigroup techniques in the spirit of a previous article by one of the authors we also propose a natural test set approach to study the global minima of the virtual energy the second part of the paper is devoted to the application of these results we propose two general markovian models of genetic algorithms and we give a simple proof of the convergence toward the global minima of the fitness function finally we introduce a stochastic algorithm that converges to the set of the global minima of a given mean cost optimization problem

a family of hybrid conjugate gradient methods for unconstrained optimization conjugate gradient methods are an important class of methods for unconstrained optimization especially for large scale problems recently they have been much studied this paper proposes a three parameter family of hybrid conjugate gradient methods two important features of the family are that i it can avoid the propensity of small steps namely if a small step is generated away from the solution point the next search direction will be close to the negative gradient direction and ii its descent property and global convergence are likely to be achieved provided that the line search satisfies the wolfe conditions some numerical results with the family are also presented

modifying the bfgs update by a new column scaling technique letb be a positive definite symmetric approximation to the second derivative matrix of the objective function of a minimization calculation we study modifications of the bfgs method that apply a scaling technique to the columns of a conjugate direction matrixz satisfyingz t bz i for a simple scaling technique similar to the ones considered by powell 1987 and 1989 we show that due to a two iteration cycle linear convergence can occur when the method is applied to a quadratic function and wolfe s line search is employed although for exact line searches quadratic termination can be proved we then suggest a different scaling technique that prevents certain columns thought to contain important curvature information from being scaled for this algorithm we prove global and superlinear convergence and demonstrate the superiority of our method over the bfgs formula on a range of illconditioned optimization problems moreover we present an implementation of our algorithm that requires only 3n 2 o n multiplications per iteration

a nonlinear conjugate gradient algorithm with an optimal property and an improved wolfe line search in this paper we seek the conjugate gradient direction closest to the direction of the scaled memoryless bfgs method and propose a family of conjugate gradient methods for unconstrained optimization an improved wolfe line search is also proposed which can avoid a numerical drawback of the original wolfe line search and guarantee the global convergence of the conjugate gradient method under mild conditions to accelerate the algorithm we introduce adaptive restarts along negative gradients based on the extent to which the function approximates some quadratic function during previous iterations numerical experiments with the cuter collection show that the proposed algorithm is promising

a new class of spectral conjugate gradient methods based on a modified secant equation for unconstrained optimization conjugate gradient methods have played a special role for solving large scale optimization problems due to the simplicity of their iteration convergence properties and their low memory requirements in this work we propose a new class of spectral conjugate gradient methods which ensures sufficient descent independent of the accuracy of the line search moreover an attractive property of our proposed methods is that they achieve a high order accuracy in approximating the second order curvature information of the objective function by utilizing the modified secant condition proposed by babaie kafaki et al s babaie kafaki r ghanbari n mahdavi amiri two new conjugate gradient methods based on modified secant equations journal of computational and applied mathematics 234 2010 1374 1386 further a global convergence result for general functions is established provided that the line search satisfies the wolfe conditions our numerical experiments indicate that our proposed methods are preferable and in general superior to the classical conjugate gradient methods in terms of efficiency and robustness

sequential decision making under uncertain future preferences we present a model of sequential decision making under uncertain future preferences assuming that the evolution of the trade off weight vector is constrained by set inclusion we consider different mechanisms for selecting the next trade off weight vector a mechanism working against the decision maker dm m a mechanism trying to aid the dm b and briefly a probabilistic mechanism p piecewise linear convex upper and lower bounds on the optimal value function are determined for mechanisms m and b conditions are given that guarantee that the optimal value function is piecewise linear and convex for all three mechanisms procedures for computing these bounds and determining associated strategies are presented a hypothetical situation involving an individual seeking promotion is used to illustrate the model and the numerical techniques

a new family of conjugate gradient methods in this paper we develop a new class of conjugate gradient methods for unconstrained optimization problems a new nonmonotone line search technique is proposed to guarantee the global convergence of these conjugate gradient methods under some mild conditions in particular polak ribiere polyak and liu storey conjugate gradient methods are special cases of the new class of conjugate gradient methods by estimating the local lipschitz constant of the derivative of objective functions we can find an adequate step size and substantially decrease the function evaluations at each iteration numerical results show that these new conjugate gradient methods are effective in minimizing large scale non convex non quadratic functions