some experiments with simulated annealing for coloring graphs abstract methods of thermodynamical simulation have been used for several famous combinatorial optimization problems for graph coloring i e partition of the node set into as few independent sets as possible we describe a method of simulation such an approach is combined with other techniques for graph coloring experiments on random graphs show evidence that this combination gives better results than anyone of the original non combined methods

global asymptotic and exponential stability of a dynamic neural system with asymmetric connection weights recently a dynamic neural system was presented and analyzed due to its good performance in optimization computation and low complexity for implementation the global asymptotic stability of such a dynamic neural system with symmetric connection weights was well studied in this note based on a new lyapunov function we investigate the global asymptotic stability of the dynamic neural system with asymmetric connection weights since the dynamic neural system with asymmetric weights is more general than that with symmetric ones the new results are significant in both theory and applications specially the new result can cover the asymptotic stability results of linear systems as special cases

cooling schedules for optimal annealing a monte carlo optimization technique called simulated annealing is a descent algorithm modified by random ascent moves in order to escape local minima which are not global minima the level of randomization is determined by a control parameter t called temperature which tends to zero according to a deterministic cooling schedule we give a simple necessary and sufficient condition on the cooling schedule for the algorithm state to converge in probability to the set of globally minimum cost states in the special case that the cooling schedule has parametric form tt c log1 t the condition for convergence is that c be greater than or equal to the depth suitably defined of the deepest local minimum which is not a global minimum state

diffusions for global optimizations we seek a global minimum of u 0 1 n to r the solution to d dt x t nabla u x t will find local minima the solution to dx t nabla u x t dt sqrt 2t dw t where w is standard n dimensional brownian motion and the boundaries are reflecting will concentrate near the global minima of u at least when temperature t is small the equilibrium distribution for x t is gibbs with density pi t x alpha exp u x t this suggests setting t t t downarrow 0 to find the global minima of u we give conditions on u x and t t such that the solution to dx t nabla u x t dt sqrt 2t dw t converges weakly to a distribution concentrated on the global minima of u

optimizing antenna array geometry for interference suppression the determination of an optimum antenna array geometry for suppressing interference is addressed an optimization problem is derived whose solution yields an optimum array geometry for a given interference environment a specific example is solved via the simulated annealing optimization algorithm and results are presented for arrays of varying configurations

free vibrations for a semilinear wave equation abstract existence and regularity of nontrivial time periodic solutions are proved for semilinear wave equations under mild smoothness monotonicity and superlinearity assumptions the forced vibration case is also treated

a class of self interacting processes with applications to games and reinforced random walks this paper studies a class of non markovian and nonhomogeneous stochastic processes on a finite state space relying on a recent paper by benaim hofbauer and sorin siam j control optim 44 2005 pp 328 348 it is shown that under certain assumptions the asymptotic behavior of occupation measures can be described in terms of a certain set valued deterministic dynamical system this provides a unified approach to simulated annealing type processes and permits the study of new models of vertex reinforced random walks and new models of learning in games such as markovian fictitious play

global asymptotic stability of recurrent neural networks with multiple time varying delays in this paper several sufficient conditions are established for the global asymptotic stability of recurrent neural networks with multiple time varying delays the lyapunov krasovskii stability theory for functional differential equations and the linear matrix inequality lmi approach are employed in our investigation the results are shown to be generalizations of some previously published results and are less conservative than existing results the present results are also applied to recurrent neural networks with constant time delays

a note on the finite time behavior of simulated annealing simulated annealing has proven to be a very successful heuristic for various combinatorial optimization problems it is a randomized algorithm that attempts to find the global optimum with high probability by local exchanges in this paper we give a new proof of the convergence of simulated annealing by applying results about rapidly mixing markov chains with this proof technique it is possible to obtain better bounds for the finite time behavior of simulated annealing than previously known

on the landscape ruggedness of the quadratic assignment problem local search based heuristics have been demonstrated to give very good results to approximately solve the quadratic assignment problem qap in this paper following the works of weinberger and stadler we introduce a parameter called the ruggedness coeffcient which measures the ruggedness of the qap landscape which is the union of a cost function and a neighborhood we give an exact expression and a sharp lower bound for this parameter we are able toderive from it that the landscape of the qap is rather flat and so it gives a theoretical justification of the effectiveness of local search based heuristics for this problem experimental results with simulated annealing are presented which con8rm this conclusion and also the influence of the ruggedness coe5cient on the quality of results obtained

the quadratic three dimensional assignment problem exact and approximate solution methods abstract this paper reports on algorithm development for solving the quadratic three dimensional assignment problem q3ap the q3ap arises for example in the implementation of a hybrid arq automatic repeat request scheme for enriching diversity among multiple packet re transmissions by optimizing the mapping of data bits to modulation symbols typical practical problem sizes would be 8 16 32 and 64 we present an exact solution method based upon a reformulation linearization technique that is one of the best available for solving the quadratic assignment problem qap our current exact algorithm is useful for q3ap instances of size 13 or smaller we also investigate four stochastic local search algorithms that provide optimum or near optimum solutions for large and difficult qap instances and adapt them for solving the q3ap the results of our experiments make it possible to get good solutions to signal mapping problems of size 8 and 16

an algorithm for quadratic assignment problems abstract facility layout and location problems with stochastic congestion in the traffic circulation system have been studied in a previous paper in this paper we present a heuristic algorithm to solve these complex problems with its straight forward approach the algorithm can solve large scale quadratic assignment problems with reasonable computing times and efficient performance computational experience for solving many test examples is also presented

nonlinear measures a new approach to exponential stability analysis for hopfield type neural networks in this paper a new concept called nonlinear measure is introduced to quantify stability of nonlinear systems in the way similar to the matrix measure for stability of linear systems based on the new concept a novel approach for stability analysis of neural networks is developed with this approach a series of new sufficient conditions for global and local exponential stability of hopfield type neural networks is presented which generalizes those existing results by means of the introduced nonlinear measure the exponential convergence rate of the neural networks to stable equilibrium point is estimated and for local stability the attraction region of the stable equilibrium point is characterized the developed approach can be generalized to stability analysis of other general nonlinear systems

cooling down levy flights let l t be a levy flights process with a stability index 0 2 and u be an external multi well potential a jump diffusion z satisfying a stochastic differential equation dz t u z t dt t dl t describes an evolution of a levy particle of an instant scale t in an external force field the scale is supposed to decrease polynomially fast i e t t for some 0 we discover two different decrease regimes if 1 fast cooling the levy particle gets trapped in one of the potential wells

ergodicity in parametric nonstationary markov chains an application to simulated annealing methods a nonstationary markov chain is weakly ergodic if the dependence of the state distribution on the starting state vanishes as time tends to infinity a chain is strongly ergodic if it is weakly ergodic and converges in distribution in this paper we show that the two ergodicity concepts are equivalent for finite chains under rather general and widely verifiable conditions we discuss applications to probabilistic analyses of general search methods for combinatorial optimization problems simulated annealing

stochastic optimization on continuous domains with finite time guarantees by markov chain monte carlo methods we introduce bounds on the finite time performance of markov chain monte carlo mcmc algorithms in solving global stochastic optimization problems defined over continuous domains it is shown that mcmc algorithms with finite time guarantees can be developed with a proper choice of the target distribution and by studying their convergence in total variation norm this work is inspired by the concept of finite time learning with known accuracy and confidence developed in statistical learning theory

guided local search and its application to the traveling salesman problem abstract the traveling salesman problem tsp is one of the most famous problems in combinatorial optimization in this paper we are going to examine how the techniques of guided local search gls and fast local search fls can be applied to the problem gls sits on top of local search heuristics and has as a main aim to guide these procedures in exploring efficiently and effectively the vast search spaces of combinatorial optimization problems gls can be combined with the neighborhood reduction scheme of fls which significantly speeds up the operations of the algorithm the combination of gls and fls with tsp local search heuristics of different eiciency and effectiveness is studied in an effort to determine the dependence of gls on the underlying local search heuristic used comparisons are made with some of the best tsp heuristic algorithms and general optimization techniques which demonstrate the advantages of gls over alternative heuristic approaches suggested for the problem

phase transition and landscape statistics of the number partitioning problem the phase transition in the number partitioning problem npp i e the transition from a region in the space of control parameters in which almost all instances have many solutions to a region in which almost all instances have no solution is investigated by examining the energy landscape of this classic optimization problem this is achieved by coding the information about the minimum energy paths connecting pairs of minima into a tree structure termed a barrier tree the leaves and internal nodes of which represent respectively the minima and the lowest energy saddles connecting those minima here we apply several measures of shape balance and symmetry as well as of branch lengths barrier heights to the barrier trees that result from the landscape of the npp aiming at identifying traces of the easy hard transition we find that it is not possible to tell the easy regime from the hard one by visual inspection of the trees or by measuring the barrier heights only the difficulty measure given by the maximum value of the ratio between the barrier height and the energy surplus of local minima succeeded in detecting traces of the phase transition in the tree in addition we show that the barrier trees associated with the npp are very similar to random trees contrasting dramatically with trees associated with the p spin glass and random energy models we also examine critically a recent conjecture on the equivalence between the npp and a truncated random energy model

assignment problems a golden anniversary survey abstract having reached the 50th golden anniversary of the publication of kuhn s seminal article on the solution of the classic assignment problem it seems useful to take a look at the variety of models to which it has given birth this paper is a limited survey of what appear to be the most useful of the variations of the assignment problem that have appeared in the literature over the past 50 years the intention here is not to identify every such paper of which there have been hundreds nor to identify the best solution procedure for each variation rather the intention is to identify what these variations are and what they are called so as to make it easier for a researcher trying to develop some variation of the assignment problem for a particular application to find the relevant literature

experimental analysis of simulated annealing based algorithms for the layout problem abstract in this paper results from an experimental analysis involving the 2 way and 3 way exchange algorithms a modified penalty algorithm the simulated annealing algorithm and a hybrid simulated annealing algorithm are presented the hybrid simulated annealing algorithm uses the modified penalty algorithm to generate an initial solution and then improves it using simulated annealing it is tested on single row layout problems with facilities of unequal area and multi row layout problems with facilities of equal area the algorithm produced solutions better than those previously reported in the literature for two of the larger single row problems and five multi row layout problems for all the other single row problems and eight of ten multi row layout test problems it produced the best known solutions the hybrid simulated annealing algorithm appears to produce better quality solutions than the simulated annealing and tabu navigation algorithms but requires more computation time than simulated annealing

the quadratic assignment problem an experimental evaluation of solution strategies since procedures for producing optimal solutions to a quadratic assignment problem are computationally infeasible for any but small problems heuristic techniques for producing approximate solutions must be employed for the solution of real practical problems this paper presents the results of experimentation that demonstrates that a solution procedure which couples a constructive initial placement technique with simple and inexpensive improvement procedures produces good solutions with very reasonable computation requirements the results indicate that such an approach would make the solution of large scale quadratic assignment problems feasible

stochastic relaxation on partitions with connected components and its application to image segmentation we present a new method of segmentation in which images are segmented in partitions with connected components we give computationally inexpensive algorithms for probability simulation and simulated annealing on the space of partitions with connected components of a general graph in particular hastings algorithms 1970 and generalized metropolis algorithms are defined to avoid heavy computation to achieve segmentation we propose a hierarchical approach which at each step minimizes a cost function on the space of partitions with connected components of a graph the algorithm is applied to segment gray level color and textured images

estimate of exponential convergence rate and exponential stability for neural networks estimates of exponential convergence rate and exponential stability are studied for a class of neural networks which includes hopfield neural networks and cellular neural networks both local and global exponential convergence are discussed theorems for estimation of the exponential convergence rate are established and the bounds on the rate of convergence are given the domains of attraction in the case of local exponential convergence are obtained simple conditions are presented for checking exponential stability of the neural networks

absolute exponential stability of recurrent neural networks with generalized activation function in this paper the recurrent neural networks rnns with a generalized activation function class is proposed in this proposed model every component of the neuron s activation function belongs to a convex hull which is bounded by two odd symmetric piecewise linear functions that are convex or concave over the real space all of the convex hulls are composed of generalized activation function classes the novel activation function class is not only with a more flexible and more specific description of the activation functions than other function classes but it also generalizes some traditional activation function classes the absolute exponential stability aest of the rnn with a generalized activation function class is studied through three steps the first step is to demonstrate the global exponential stability ges of the equilibrium point of original rnn with a generalized activation function being equivalent to that of rnn under all vertex functions of convex hull the second step transforms the rnn under every vertex activation function into neural networks under an array of saturated linear activation functions because the ges of the equilibrium point of three systems are equivalent the next stability analysis focuses on the ges of the equilibrium point of rnn system under an array of saturated linear activation functions the last step is to study both the existence of equilibrium point and the ges of the rnn under saturated linear activation functions using the theory of m matrix in the end a two neuron rnn with a generalized activation function is constructed to show the effectiveness of our results

nonstationary markov chains and convergence of the annealing algorithm we study the asymptotic behavior as timet of certain nonstationary markov chains and prove the convergence of the annealing algorithm in monte carlo simulations we find that in the limitt a nonstationary markov chain may exhibit phase transitions nonstationary markov chains in general and the annealing algorithm in particular lead to biased estimators for the expectation values of the process we compute the leading terms in the bias and the variance of the sample means estimator we find that the annealing algorithm converges if the temperaturet t goes to zero no faster thanc log t t0 ast with a computable constantc andt0 the initial time the bias and the variance of the sample means estimator in the annealing algorithm go to zero likeo t 1 e for some 0 e 1 with e 0 only in very special circumstances our results concerning the convergence of the annealing algorithm and the rate of convergence to zero of the bias and the variance of the sample means estimator provide a rigorous procedure for choosing the optimal annealing schedule this optimal choice reflects the competition between two physical effects a the adiabatic effect whereby if the temperature is loweredtoo abruptly the system may end up not in a ground state but in a nearby metastable state and b the super cooling effect whereby if the temperature is loweredtoo slowly the system will indeed approach the ground state s but may do so extremely slowly

a comment on on equilibria stability and instability of hopfield neural networks and reply it is pointed out that the main analysis results about the existence uniqueness and global asymptotic stability of the equilibrium of a continuous time hopfield type neural network given in the paper by zhi hong guan et al 2000 are special cases of relevant ones previously obtained in the literature in reply the original authors consider the reasoning of xue bin liang s comments and state that their analysis method is in fact different to existing ones

simulated annealing via sobolev inequalities we use sobolev inequalities to study the simulated annealing algorithm this approach takes advantage of the local time reversibility of the process and yields the optimal freezing schedule as well as quantitative information about the rate at which the process is tending to its ground state

low rank structure learning via nonconvex heuristic recovery in this paper we propose a nonconvex framework to learn the essential low rank structure from corrupted data different from traditional approaches which directly utilizes convex norms to measure the sparseness our method introduces more reasonable nonconvex measurements to enhance the sparsity in both the intrinsic low rank structure and the sparse corruptions we will respectively introduce how to combine the widely used lp norm 0 p 1 and log sum term into the framework of low rank structure learning although the proposed optimization is no longer convex it still can be effectively solved by a majorization minimization mm type algorithm with which the nonconvex objective function is iteratively replaced by its convex surrogate and the nonconvex problem finally falls into the general framework of reweighed approaches we prove that the mm type algorithm can converge to a stationary point after successive iterations the proposed model is applied to solve two typical problems robust principal component analysis and low rank representation experimental results on low rank structure learning demonstrate that our nonconvex heuristic methods especially the log sum heuristic recovery algorithm generally perform much better than the convex norm based method 0 p 1 for both data with higher rank and with denser corruptions

stochastic relaxation gibbs distributions and the bayesian restoration of images we make an analogy between images and statistical mechanics systems pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice like physical system the assignment of an energy function in the physical system determines its gibbs distribution because of the gibbs distribution markov random field mrf equivalence this assignment also determines an mrf image model the energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the mrf for a range of degradation mechanisms including blurring nonlinear deformations and multiplicative or additive noise the posterior distribution is an mrf with a structure akin to the image model by the analogy the posterior distribution defines another imaginary physical system gradual temperature reduction in the physical system isolates low energy states annealing or what is the same thing the most probable states under the gibbs distribution the analogous operation under the posterior distribution yields the maximum a posteriori map estimate of the image given the degraded observations the result is a highly parallel relaxation algorithm for map estimation we establish convergence properties of the algorithm and we experiment with some simple pictures for which good restorations are obtained at low signal to noise ratios

formulating and solving production planning problems production planning problems frequently involve the assignment of jobs or operations to machines the simplest model of this problem is the well known assignment problem ap however due to simplifying assumptions this model does not provide implementable solutions for many actual production planning problems extensions of the simple assignment model known as the generalized assignment problem gap and the multi resource generalized assignment problem mrgap have been developed to overcome this difficulty this paper presents an extension of the mrgap to allow splitting individual batches across multiple machines while considering the effect of setup times and setup costs the extension is important for many actual production planning problems including ones in the injection molding industry and in the metal cutting industry we formulate models which are logical extensions of previous models which ignored batch splitting for the problem we address we then give different formulations and suggest adaptations of a genetic algorithm ga and simulated annealing sa a systematic evaluation of these algorithms as well as a lagrangian relaxation lr approach is presented

simulated annealing with an optimal fixed temperature contrary to conventional belief it turns out that in some problem instances of moderate size fixed temperature simulated annealing algorithms based on a heuristic formula for determining the optimal temperature can be superior to algorithms based on cooling such a heuristic formula however often seems elusive in practical cases considered we include instances of traveling salesman quadratic assignment and graph partitioning problems where we obtain results that compare favorably to the ones known in the literature

a combined multistart annealing algorithm for continuous global optimization the application of simulated annealing to the global optimization of a function on a compact subset of rd is discussed for the langevin algorithm the class of convergent schedules depends on some a priori knowledge about the form of the function it is shown that this problem disappears by using the simplest annealing algorithm of jump type which can also be improved by performing local searches between two consecutive jump times the resulting algorithm is essentially a multistart technique controlled by an annealing schedule

simulated annealing searching for an optimal temperature schedule a sizable part of the theoretical literature on simulated annealing deals with a property called convergence which asserts that the simulated annealing chain is in the set of global minimum states of the objective function with probability tending to 1 however in practice the convergent algorithms are considered too slow whereas a number of nonconvergent ones are usually preferred we attempt a detailed analysis of various temperature schedules examples will be given of when it is both practically and theoretically justified to use boiling fixed temperature or even fast cooling schedules which have a small probability of reaching global minima applications to traveling salesman problems of various sizes are also given

heuristics for biquadratic assignment problems and their computational comparison abstract the biquadratic assignment problem biqap is a generalization of the quadratic assignment problem qap as for any hard optimization problem also for biqap a reasonable effort to cope with the problem is trying to derive heuristics which solve it suboptimally and which possibly yield a good trade off between the solution quality and the time and memory requirements in this paper we describe several heuristics for biqaps in particular pair exchange algorithms improvement methods and variants of simulated annealing and taboo search we implement these heuristics as c codes and analyze their performances

absolute exponential stability of a class of continuous time recurrent neural networks this paper presents a new result on absolute exponential stability aest of a class of continuous time recurrent neural networks with locally lipschitz continuous and monotone nondecreasing activation functions the additively diagonally stable connection weight matrices are proven to be able to guarantee aest of the neural networks the aest result extends and improves the existing absolute stability and aest ones in the literature

an improved annealing scheme for the qap abstract recently there has been some interest in the use of simulated annealing to obtain good solutions to a number of combinatorial problems this paper reports on the use of this method applied to the quadratic assignment problem i e the assignment of inter communicating objects to locations to minimize the total cost of communication between them the result is a much improved annealing scheme for this problem which performs well on a range of examples finding improved solutions for several of the largest problems available in the literature and requiring only modest amounts of computational effort

a delayed neural network for solving linear projection equations and its analysis in this paper we present a delayed neural network approach to solve linear projection equations the lyapunov krasovskii theory for functional differential equations and the linear matrix inequality lmi approach are employed to analyze the global asymptotic stability and global exponential stability of the delayed neural network compared with the existing linear projection neural network theoretical results and illustrative examples show that the delayed neural network can effectively solve a class of linear projection equations and some quadratic programming problems

new bounds for binary covering arrays using simulated annealing covering arrays cas are combinatorial structures specified as a matrix of n rows and k columns over an alphabet on v symbols such that for each set of t columns called the strength of the array every t tuple of symbols is covered recently they have been used to represent interaction test suites for software testing given that they provide economical sized test cases while still preserving important fault detection capabilities this paper introduces an improved implementation of a simulated annealing algorithm called isa for constructing cas of strengths three through six over a binary alphabet i e binary cas extensive experimentation is carried out using 127 well known benchmark instances for assessing its performance with respect to an existing simulated annealing implementation a greedy method and five state of the art algorithms the results show that our algorithm attains 104 new bounds and equals the best known solutions for the other 23 instances consuming reasonable computational time furthermore the implications of using these results as ingredients to recursive constructions are also analyzed

proposals for chunking and tabu search chunking grouping basic units of information to create higher level units is a critical aspect of human intelligence we identify special types of chunking to enhance tabu search memory structures producing improved problem solving ability and giving useful supporting information for decision makers tabu search as proposed by glover orsa journal on computing 1 3 1989 190 2 1 1990 4 has proven to be a very effective meta heuristic for hard optimization problems the search is typically based on a local improvement scheme that involves alteration of one or more attributes of fully specified solutions at each iteration in this paper we describe the use of and search for groupings of solution attributes although they are most natural in the context of the tabu search paradigm many of the proposals can also be employed in genetic algorithms and simulated annealing based algorithms chunking enables a number of search improvements including a general purpose metric for decision vectors the paper develops theory and proposals for learning about chunks and using them computational experience to date is summarized to support the contention that chunking can be an important part of effective optimization algorithms

optimization of the quadratic assignment problem using an ant colony algorithm ant algorithm is a multi agent systems inspired by the behaviors of real ant colonies function to solve optimization problems in this paper an ant colony optimization algorithm is developed to solve the quadratic assignment problem the local search process of the algorithm is simulated annealing in the exploration of the search space the evaluation of pheromones which are laid on the ground by ants is used in this work the algorithm is analyzed by using current problems in the literature and is compared with other metaheuristics

recent directions in matrix stability abstract matrix stability has been intensively investigated in the past two centuries we review work that has been done in this topic focusing on the great progress that has been achieved in the last decade or two we start with classical stability criteria of lyapunov routh and hurwitz and lienard and chipart we then study recently proven sufficient conditions for stability with particular emphasis on p matrices we investigate conditions for the existence of a stable scaling for a given matrix we review results on other types of matrix stability such as d stability additive d stability and lyapunov diagonal stability we discuss the weak principal submatrix rank property shared by lyapunov diagonally semistable matrices we also discuss the uniqueness of lyapunov scaling factors maximal lyapunov scaling factors cones of real positive semidefinite matrices and their applications to matrix stability and inertia preserving matrices

iterated local search for the quadratic assignment problem iterated local search ils is a simple and powerful stochastic local search method this article presents and analyzes the application of ils to the quadratic assignment problem qap we justify the potential usefulness of an ils approach to this problem by an analysis of the qap search space however an analysis of the run time behavior of a basic ils algorithm reveals a stagnation behavior which strongly compromises its performance to avoid this stagnation behavior we enhance the ils algorithm using acceptance criteria that allow moves to worse local optima and we propose population based ils extensions an experimental evaluation of the enhanced ils algorithms shows their excellent performance when compared to other state of the art algorithms for the qap

a genetic algorithm for the multiple choice integer program we present a genetic algorithm for the multiple choice integer program that finds an optimal solution with probability one though it is typically used as a heuristic general constraints are relaxed by a nonlinear penalty function for which the corresponding dual problem has weak and strong duality the relaxed problem is attacked by a genetic algorithm with solution representation special to the multiple choice structure nontraditional reproduction crossover and mutation operations are employed extensive computational tests for dual degenerate problem instances show that suboptimal solutions can be obtained with the genetic algorithm within running times that are shorter than those of the osl optimization routine

asymptotic behavior of some linear differential systems abstract consider the following chasing game in r amongbparticles whose trajectories are governed by the following differential equations fori s 1 2 b f i t j i q ij t f j t f i t wherefi t is the position at timetof the particleiandqij t which measures the instantaneous rate that particleiis attracted toward particlej is given byqij t pij t u i j forj i the rate function t is positive with limt t 0 u i j 0 is the cost function indicating the level of difficulty fromitojand the nonnegative numberspijfori jreflect the neighborhood structure of the particles particles are attracted to one another but such attraction becomes weaker as time goes on a sufficient condition on t is given for allfi t converging to a common finite limit

global exponential stability and global convergence in finite time of delayed neural networks with infinite gain this paper introduces a general class of neural networks with arbitrary constant delays in the neuron interconnections and neuron activations belonging to the set of discontinuous monotone increasing and possibly unbounded functions the discontinuities in the activations are an ideal model of the situation where the gain of the neuron amplifiers is very high and tends to infinity while the delay accounts for the finite switching speed of the neuron amplifiers or the finite signal propagation speed it is known that the delay in combination with high gain nonlinearities is a particularly harmful source of potential instability the goal of this paper is to single out a subclass of the considered discontinuous neural networks for which stability is instead insensitive to the presence of a delay more precisely conditions are given under which there is a unique equilibrium point of the neural network which is globally exponentially stable for the states with a known convergence rate the conditions are easily testable and independent of the delay moreover global convergence in finite time of the state and output is investigated in doing so new interesting dynamical phenomena are highlighted with respect to the case without delay which make the study of convergence in finite time significantly more difficult the obtained results extend previous work on global stability of delayed neural networks with lipschitz continuous neuron activations and neural networks with discontinuous neuron activations but without delays

generalized deterministic annealing we develop a general formalism for computing high quality low cost solutions to nonconvex combinatorial optimization problems expressible as distributed interacting local constraints for problems of this type generalized deterministic annealing gda avoids the performance related sacrifices of current techniques gda exploits the localized structure of such problems by assigning k state neurons to each optimization variable the neuron values correspond to the probability densities of k state local markov chains and may be updated serially or in parallel the markov model is derived from the markov model of simulated annealing sa although it is greatly simplified theorems are presented that firmly establish the convergence properties of gda as well as supplying practical guidelines for selecting the initial and final temperatures in the annealing process a benchmark image enhancement application is provided where the performance of gda is compared to other optimization methods the empirical data taken in conjunction with the formal analytical results suggest that gda enjoys significant performance advantages relative to current methods for combinatorial optimization

approximating the permanent a randomised approximation scheme for the permanent of a 0 1s presented the task of estimating a permanent is reduced to that of almost uniformly generating perfect matchings in a graph the latter is accomplished by simulating a markov chain whose states are the matchings in the graph for a wide class of 0 1 matrices the approximation scheme is fully polynomial i e runs in time polynomial in the size of the matrix and a parameter that controls the accuracy of the output this class includes all dense matrices those that contain sufficiently many 1 s and almost all sparse matrices in some reasonable probabilistic model for 0 1 matrices of given density for the approach sketched above to be computationally efficient the markov chain must be rapidly mixing informally it must converge in a short time to its stationary distribution a major portion of the paper is devoted to demonstrating that the matchings chain is rapidly mixing apparently the first such result for a markov chain with genuinely c

equilibria and their bifurcations in a recurrent neural network involving iterates of a transcendental function some practical models contain so complicated mathematical expressions that it is hard to determine the number and distribution of all equilibria not mentioning the qualitative properties and bifurcations of those equilibria the three node recurrent neural network system with two free weight parameters originally introduced by ruiz owens and townley in 1997 is such a system for which the equation of equilibria involves transcendental function and its iterates not computing coordinates of its equilibria in this paper we display an effective technique to determine the number and distribution of its equilibria without full information about equilibria our method enables to further study qualitative properties of those equilibria and discuss their saddle node pitchfork and hopf bifurcations by approximating center manifolds

a survey for the quadratic assignment problem the quadratic assignment problem qap one of the most difficult problems in the np hard class models many real life problems in several areas such as facilities location parallel and distributed computing and combinatorial data analysis combinatorial optimization problems such as the traveling salesman problem maximal clique and graph partitioning can be formulated as a qap in this paper we present some of the most important qap formulations and classify them according to their mathematical sources we also present a discussion on the theoretical resources used to define lower bounds for exact and heuristic algorithms we then give a detailed discussion of the progress made in both exact and heuristic solution methods including those formulated according to metaheuristic strategies finally we analyze the contributions brought about by the study of different approaches

simulated annealing for convex optimization we apply the method known as simulated annealing to the following problem in convex optimization minimize a linear function over an arbitrary convex set where the convex set is specified only by a membership oracle using distributions from the boltzmann gibbs family leads to an algorithm that needs only o n phases for instances in rn this gives an optimization algorithm that makes o n4 5 calls to the membership oracle in the worst case compared to the previous best guarantee of o n5 the benefits of using annealing here are surprising because such problems have no local minima that are not also global minima hence we conclude that one of the advantages of simulated annealing in addition to avoiding poor local minima is that in these problems it converges faster to the minima that it finds we also give a proof that under certain general conditions the boltzmann gibbs distributions are optimal for annealing on these convex problems

convergence of an annealing algorithm the annealing algorithm is a stochastic optimization method which has attracted attention because of its success with certain difficult problems including np hard combinatorial problems such as the travelling salesman steiner trees and others there is an appealing physical analogy for its operation but a more formal model seems desirable in this paper we present such a model and prove that the algorithm converges with probability arbitrarily close to 1 we also show that there are cases where convergence takes exponentially long that is it is no better than a deterministic method we study how the convergence rate is affected by the form of the problem finally we describe a version of the algorithm that terminates in polynomial time and allows a good deal of practical confidence in the solution

a heuristic for quadratic boolean programs with applications to quadratic assignment problems abstract a heuristic for quadratic boolean programs is presented computational tests with quadratic assignment problems qap showed that it finds very good suboptimal solutions in moderate time and behaves computationally stable in the appendix a fortran program for qap is listed which improves an earlier code published by burkard and derigs

a thermodynamically motivated simulation procedure for combinatorial optimization problems abstract exchange algorithms are an important class of heuristics for hard combinatorial optimization problems as e g salesman problems or quadratic assignment problems in kirkpatrick s and cerny s exchange algorithms for the travelling salesman problem and placement problem they propose to perform an exchange not only if the objective function value decreases by this exchange but also in certain cases if the objective function value increases an exchange increasing the objective function value is performed stochastically depending on the size of the increment computational tests with quadratic assignment problems revealed an excellent behaviour in such an approach suboptimal solutions differing 1 2 from the best known solutions are obtained by a simple program in short time by starting this program several times with different starting values all known minimal objective function values were reached thus this approach is well suited also for smaller computers and leads in short time to acceptable solutions

global exponential stability of neural networks with globally lipschitz continuous activations and its application to linear variational inequality problem this paper investigates the existence uniqueness and global exponential stability ges of the equilibrium point for a large class of neural networks with globally lipschitz continuous activations including the widely used sigmoidal activations and the piecewise linear activations the provided sufficient condition for ges is mild and some conditions easily examined in practice are also presented the ges of neural networks in the case of locally lipschitz continuous activations is also obtained under an appropriate condition the analysis results given in the paper extend substantially the existing relevant stability results in the literature and therefore expand significantly the application range of neural networks in solving optimization problems as a demonstration we apply the obtained analysis results to the design of a recurrent neural network rnn for solving the linear variational inequality problem vip defined on any nonempty and closed box set which includes the box constrained quadratic programming and the linear complementarity problem as the special cases it can be inferred that the linear vip has a unique solution for the class of lyapunov diagonally stable matrices and that the synthesized rnn is globally exponentially convergent to the unique solution some illustrative simulation examples are also given

traditional heuristic versus hopfield neural network approaches to a car sequencing problem this paper considers the problem of optimally sequencing different car models along an assembly line according to some contiguity constraints while ensuring that the demands for each of the models are satisfied this car sequencing problem csp is a practical np hard combinatorial optimisation problem the csp is formulated as a nonlinear integer programming problem and it is shown that exact solutions to the problem are difficult to obtain due to the indefinite quadratic form of the csp objective function two traditional heuristics steepest descent and simulated annealing are employed to solve the csp approximately several hopfield neural network hnn approaches are also presented the process of mapping an optimisation problem onto a hnn is demonstrated explicitly and modifications to the existing neural approaches are presented which guarantee feasibility of solutions further modifications are proposed to improve the solution quality by permitting escape from local minima in an attempt to locate the global optimum results from all solutions techniques are compared on a set of instances of the csp and conclusions drawn

stability analysis of hopfield type neural networks the paper applies several concepts in robust control research such as linear matrix inequalities edge theorem parameter dependent lyapunov function and popov criteria to investigate the stability property of hopfield type neural networks the existence and uniqueness of an equilibrium is formulated as a matrix determinant problem an induction scheme is used to find the equilibrium to verify whether the determinant is nonzero for a class of matrix a numerical range test is proposed several robust control techniques in particular linear matrix inequalities are used to characterize the local stability of the neural networks around the equilibrium the global stability of the hopfield neural networks is then addressed using a parameter dependent lyapunov function technique all these results are shown to generalize existing results in verifying the existence uniqueness of the equilibrium and local global stability of hopfield type neural networks

algorithms for the multi resource generalized assignment problem the multi resource generalized assignment problem is encountered when a set of tasks have to be assigned to a set of agents in a way that permits assignment of multiple tasks to an agent subject to the availability of a set of multiple resources consumed by that agent this problem differs from the generalized assignment problem in that an agent consumes not just one but a variety of resources in performing the tasks assigned to him this paper develops effective solution procedures for the multi resource generalized assignment problem various relaxations of the problem are studied and theoretical relations among these relaxations are pointed out rules for reducing problem size are discussed and are shown to be effective through computational experiments heuristic solution procedures and an efficient branch and bound procedure are developed results of computational experiments testing these procedures are reported

algodesk an experimental comparison of eight evolutionary heuristics applied to the quadratic assignment problem abstract this work compares the effectiveness of eight evolutionary heuristic algorithms applied to the quadratic assignment problem qap reputedly one of the most difficult combinatorial optimization problems qap is merely used as a carrier for the comparison we do not attempt to compare any heuristics with solving algorithms specific for it results are given both with respect to the best result achieved by each algorithm in a limited time span and to its speed of convergence to that result

generalized lyapunov approach for convergence of neural networks with discontinuous or non lipschitz activations the paper considers a class of additive neural networks where the neuron activations are modeled by discontinuous functions or by continuous non lipschitz functions some tools are developed which enable us to apply a lyapunov like approach to differential equations with discontinuous right hand side modeling the neural network dynamics the tools include a chain rule for computing the time derivative along the neural network solutions of a nondifferentiable lyapunov function and a comparison principle for this time derivative which yields conditions for exponential convergence or convergence in finite time by means of the lyapunov like approach a general result is proved on global exponential convergence toward a unique equilibrium point of the neural network solutions moreover new results on global convergence in finite time are established which are applicable to neuron activations with jump discontinuities or neuron activations modeled by means of continuous non lipschitz holder functions

simulated annealing for the qap optimal tradeoff between simulation time and solution quality abstract this paper reports on the optimization of solution quality for a simulated annealing algorithm applied to the quadratic assignment problem the optimization is performed by varying two of the parameters in the algorithm the size of the ensemble i e the number of simulations and the length of each simulation the variation is performed in such a way that the algorithm always uses the same amount of time for a given problem size it is found that the simulation length is indeed optimizable but also that a rather broad spectrum of values produce a near optimal solution quality generally it seems that the optimal simulation length is rather short compared to results found for other optimization problems

some experiments with simulated annealing techniques for packing problems abstract experiments with a variety of combinatorial optimisation problems have shown that the simulated annealing algorithm is an effective local search heuristic method the basic requirements of the algorithm are a neighbourhood structure on the set of feasible solutions and a number of parameters which govern the acceptance or rejection of new solutions generated during the search the quality of the solution is very sensitive to both these factors this paper is concerned with the application of the simulated annealing approach to packing problems and describes a series of experiments carried out to ascertain the effectiveness of the method for such problems and the most appropriate neighbourhood structure to use and the best parameters to apply

application of the simulated annealing algorithm to the combinatorial optimisation problem with permutation property an investigation of generation mechanism focusing on the generation mechanism of random permutation solutions this paper investigates the application of the simulated annealing sa algorithm to the combinatorial optimisation problems with permutation property six types of perturbation scheme for generating random permutation solutions are introduced they are proved to satisfy the asymptotical convergence requirements the results of experimental evaluations on traveling salesman problem tsp flow shop scheduling problem fsp and quadratic assignment problem qap reveal that the efficiencies of the perturbation schemes are different in searching a solution space by adopting a proper perturbation scheme the sa algorithm has shown to produce very efficient solutions to different combinatorial optimisation problems with permutation property

a bayesian approach to dynamic contours through stochastic sampling and simulated annealing in many applications of image analysis simply connected objects are to be located in noisy images during the last 5 6 years active contour models have become popular for finding the contours of such objects connected to these models are iterative algorithms for finding the minimizing energy curves making the curves behave dynamically through the iterations these approaches do however have several disadvantages the numerical algorithms that are in use constrain the models that can be used furthermore in many cases only local minima can be achieved in this paper the author discusses a method for curve detection based on a fully bayesian approach a model for image contours which allows the number of nodes on the contours to vary is introduced iterative algorithms based on stochastic sampling is constructed which make it possible to simulate samples from the posterior distribution making estimates and uncertainty measures of specific quantities available further simulated annealing schemes making the curve move dynamically towards the global minimum energy configuration are presented in theory no restrictions on the models are made in practice however computational aspects must be taken into consideration when choosing the models much more general models than the one used for active contours may however be applied the approach is applied to ultrasound images of the left ventricle and to magnetic resonance images of the human brain and show promising results

metropolis type annealing algorithms for global optimization in r d the convergence of a class of metropolis type markov chain annealing algorithms for global optimization of a smooth function u cdot on mathbb r d is established no prior information is assumed as to what bounded region contains a global minimum the analysis contained herein is based on writing the metropolis type algorithm in the form of a recursive stochastic algorithm x k 1 x k a k nabla u x k xi k b k w k where w k is a standard white gaussian sequence xi k are random variables and a k a k b k sqrt b sqrt k log log k for k large convergence results for x k are then applied from our previous work siam journal on control and optimization 29 1991 pp 999 1018 since the analysis of x k is based on the asymptotic behavior of the related langevin type markov diffusion annealing algorithm dy t nabla u y t dt c t dw t where w cdot is a standard wiener process and c t

a cost minimization approach to edge detection using simulated annealing the authors cast edge detection as a problem in cost minimization this is achieved by the formulation of a cost function that evaluates the quality of edge configurations the function is a linear sum of weighted cost factors the cost factors capture desirable characteristics of edges such as accuracy in localization thinness and continuity edges are detected by finding the edge configurations that minimize the cost function the authors give a mathematical description of edges and analyze the cost function in terms of the characteristics of the edges in minimum cost configurations through the analysis guidelines are provided on the choice of weights to achieve certain characteristics of the detected edges the cost function is minimized by the simulated annealing method a set of strategies is presented for generating candidate states and to devise a suitable temperature schedule

a comprehensive review of stability analysis of continuous time recurrent neural networks stability problems of continuous time recurrent neural networks have been extensively studied and many papers have been published in the literature the purpose of this paper is to provide a comprehensive review of the research on stability of continuous time recurrent neural networks including hopfield neural networks cohen grossberg neural networks and related models since time delay is inevitable in practice stability results of recurrent neural networks with different classes of time delays are reviewed in detail for the case of delay dependent stability the results on how to deal with the constant variable delay in recurrent neural networks are summarized the relationship among stability results in different forms such as algebraic inequality forms m matrix forms linear matrix inequality forms and lyapunov diagonal stability forms is discussed and compared some necessary and sufficient stability conditions for recurrent neural networks without time delays are also discussed concluding remarks and future directions of stability analysis of recurrent neural networks are given

constrained nonlinear optimization approaches to color signal separation separating a color signal into illumination and surface reflectance components is a fundamental issue in color reproduction and constancy this can be carried out by minimizing the error in the least squares ls fit of the product of the illumination and the surface spectral reflectance to the actual color signal when taking in account the physical realizability constraints on the surface reflectance and illumination the feasible solutions to the nonlinear ls problem should satisfy a number of linear inequalities four distinct novel optimization algorithms are presented to employ these constraints to minimize the nonlinear ls fitting error the first approach which is based on ritter s superlinear convergent method luengerger 1980 provides a computationally superior algorithm to find the minimum solution to the nonlinear ls error problem subject to linear inequality constraints unfortunately this gradient like algorithm may sometimes be trapped at a local minimum or become unstable when the parameters involved in the algorithm are not tuned properly the remaining three methods are based on the stable and promising global minimizer called simulated annealing the annealing algorithm can always find the global minimum solution with probability one but its convergence is slow to tackle this a cost effective variable separable formulation based on the concept of golub and pereyra 1973 is adopted to reduce the nonlinear ls problem to be a small scale nonlinear ls problem the computational efficiency can be further improved when the original boltzman generating distribution of the classical annealing is replaced by the cauchy distribution

recent advances in the solution of quadratic assignment problems the quadratic assignment problem qap is notoriously difficult for exact solution methods in the past few years a number of long open qaps including those posed by steinberg 1961 nugent et al 1968 and krarup 1972 were solved to optimality for the first time the solution of these problems has utilized both new algorithms and novel computing structures we describe these developments as well as recent work which is likely to result in the solution of even more difficult instances

the time complexity of maximum matching by simulated annealing the random heuristic search algorithm called simulated annealing is considered for the problem of finding the maximum cardinality matching in a graph it is shown that neither a basic form of the algorithm nor any other algorithm in a fairly large related class of algorithms can find maximum cardinality matchings such that the average time required grows as a polynomial in the number of nodes of the graph in contrast it is also shown for arbitrary graphs that a degenerate form of the basic annealing algorithm obtained by letting temperature be a suitably chosen constant produces matchings with nearly maximum cardinality in polynomial average time

global stability of complex valued recurrent neural networks with time delays since the last decade several complex valued neural networks have been developed and applied in various research areas as an extension of real valued recurrent neural networks complex valued recurrent neural networks use complex valued states connection weights or activation functions with much more complicated properties than real valued ones this paper presents several sufficient conditions derived to ascertain the existence of unique equilibrium global asymptotic stability and global exponential stability of delayed complex valued recurrent neural networks with two classes of complex valued activation functions simulation results of three numerical examples are also delineated to substantiate the effectiveness of the theoretical results

markov chains with rare transitions and simulated annealing we consider approximate stationary markov chains in which the entries of the one step transition probability matrix are known to be of different orders of magnitude and whose structure that is the orders of magnitude of the transition probabilities does not change with time for such markov chains we present a method for generating order of magnitude estimates for the t step transition probabilities for any t we then notice that algorithms of the simulated annealing type may be represented by markov chains which are approximately stationary over fairly long time intervals using our results we obtain a characterization of the convergent cooling schedules for the most general class of algorithms of the simulated annealing type

quadratic assignment problems abstract this paper surveys quadratic assignment problems qap at first several applications of this problem class are described and mathematical formulations of qaps are given then some exact solution methods and good heuristics are outlined their computational behaviour is illustrated by numerical results further recent results on the asymptotic probabilistic behaviour of qaps are outlined