global optimization and simulated annealing in this paper we are concerned with global optimization which can be defined as the problem of finding points on a bounded subset of n in which some real valued functionf assumes its optimal maximal or minimal value we present a stochastic approach which is based on the simulated annealing algorithm the approach closely follows the formulation of the simulated annealing algorithm as originally given for discrete optimization problems the mathematical formulation is extended to continuous optimization problems and we prove asymptotic convergence to the set of global optima furthermore we discuss an implementation of the algorithm and compare its performance with other well known algorithms the performance evaluation is carried out for a standard set of test functions from the literature

large deviations hypotheses testing and source coding for finite markov chains let x n n geq 1 be a finite markov chain with transition probability matrix of strictly positive entries a large deviation theorem is proved for the empirical transition count matrix and is used to get asymptotically optimal critical regions for testing simple hypotheses about the transition matrix as a corollary the error exponent in the source coding theorem for x n is obtained these results generalize the corresponding results for the independent and identically distributed case

the reliability functions of the general source with fixed length coding the reliability function problems with fixed length source coding for the general source are studied for all rates r our fundamental philosophy in doing so is to convert all of the reliability function problems to the pertinent computation problems in the large derivation probability theory it turns out that this kind of new methodology which was previously developed by han see ibid vol 43 p 1145 64 1997 enables us to establish quite compact general formulas of the reliability function for general sources including all nonstationary and or nonergodic sources with countably infinite alphabet such general formulas are presented from the information spectrum point of view

capacity of steganographic channels this work investigates a central problem in steganography that is how much data can safely be hidden without being detected to answer this question a formal definition of steganographic capacity is presented once this has been defined a general formula for the capacity is developed the formula is applicable to a very broad spectrum of channels due to the use of an information spectrum approach this approach allows for the analysis of arbitrary steganalyzers as well as nonstationary nonergodic encoder and attack channels after the general formula is presented various simplifications are applied to gain insight into example hiding and detection methodologies finally the context and applications of the work are summarized in a general discussion

distributional scaling an algorithm for structure preserving embedding of metric and nonmetric spaces we present a novel approach for embedding general metric and nonmetric spaces into low dimensional euclidean spaces as opposed to traditional multidimensional scaling techniques which minimize the distortion of pairwise distances our embedding algorithm seeks a low dimensional representation of the data that preserves the structure geometry of the original data the algorithm uses a hybrid criterion function that combines the pairwise distortion with what we call the geometric distortion to assess the geometric distortion we explore functions that reflect geometric properties our approach is different from the isomap and lle algorithms in that the discrepancy in distributional information is used to guide the embedding we use clustering algorithms in conjunction with our embedding algorithm to direct the embedding process and improve its convergence properties we test our method on metric and nonmetric data sets and in the presence of noise we demonstrate that our method preserves the structural properties of embedded data better than traditional mds and that its performance is robust with respect to clustering errors in the original data other results of the paper include accelerated algorithms for optimizing the standard mds objective functions and two methods for finding the most appropriate dimension in which to embed a given set of data

large factorial designs for product engineering and marketing research applications we propose an algorithm that incorporates both combinatorial and heuristic optimization methods for generating d efficient factorial designs this includes an approach that can automatically construct more than 115 000 orthogonal arrays including at least one array for every known specification with fewer than 144 runs the algorithm tries various initialization and iteration methods chooses the most promising method and then applies more computational effort with the chosen method the algorithm can make orthogonal and nearly orthogonal arrays and it can accommodate restrictions interactions and large designs usage requires minimal expertise and input applications in the area of optimal product design choice modeling and marketing research are discussed

probabilistic measures of adequacy of a numerical search for a global maximum abstract measures of the probability of all unobserved species are applied to the problem of assessing the adequacy of a search for a global maximum using random starting points the measures as used here estimate the probability that an iterative algorithm using a randomly selected starting point will find a solution not observed in previous random starting points the probability of an unobserved global maximum is less than or equal to this probability we used these measures to evaluate the adequacy of our search procedure for the maximum likelihood estimates of the parameters of a mixture of two normals these measures indicated that for most problems generated there was little chance that there were unobserved domains of convergence occasional problems however had appreciable estimated probabilities in such problems examination of the data suggested regions where a more focused search for unobserved domains of convergence was warranted

immune based algorithms for dynamic optimization the main problem with biologically inspired algorithms like evolutionary algorithms or particle swarm optimization when applied to dynamic optimization is to force their readiness for continuous search for new optima occurring in changing locations immune based algorithm being an instance of an algorithm that adapt by innovation seem to be a perfect candidate for continuous exploration of a search space in this paper we describe various implementations of the immune principles and we compare these instantiations on complex environments

a latent class multidimensional scaling model for two way one mode continuous rating dissimilarity data in this paper we propose a cluster mds model for two way one mode continuous rating dissimilarity data the model aims at partitioning the objects into classes and simultaneously representing the cluster centers in a low dimensional space under the normal distribution assumption a latent class model is developed in terms of the set of dissimilarities in a maximum likelihood framework in each iteration the probability that a dissimilarity belongs to each of the blocks conforming to a partition of the original dissimilarity matrix and the rest of parameters are estimated in a simulated annealing based algorithm a model selection strategy is used to test the number of latent classes and the dimensionality of the problem both simulated and classical dissimilarity data are analyzed to illustrate the model

a new clustering algorithm with multiple runs of iterative procedures abstract the k means clustering algorithm is well known and widely used recently an alternative algorithm based on an earlier idea duda and hart pattern classification and scene analysis 1973 ismail and kamel pattern recognition 22 75 89 1989 has been proposed in comprehensive experiments we verify that this alternative is superior to k means both techniques suffer from the complexity of the cost function surface they are descending and a consequent inclination to fall into poor local minima a popular solution to this problem is to rerun the chosen iterative procedure from many randomly chosen starting positions saving the best we introduce here an alternative strategy to form the starting position for such a multiple run which consists of perturbing an existing result in an attempt to improve its poor aspects this controlled configuration change has comparable complexity with random initialisation but usually produces superior clustering results comprehensive experiments confirm this result

clustering with evolution strategies tbe applicability of evolution strategies ess population based stochastic optimization techniques to optimize clustering objective functions is explored clustering objective functions are categorized into centroid and non centroid type of functions optimization of the centroid type of objective functions is accomplished by formulating them as functions of real valued parameters using ess both hard and fuzzy clustering objective functions are considered in this study applicability of ess to discrete optimization problems is extended to optimize the non centroid type of objective functions as ess are amenable to parallelization a parallel model master slave model is described in the context of the clustering problem results obtained for selected data sets substantiate the utility of ess in clustering

generalized simulated annealing for function optimization a generalized simulated annealing method has been developed and applied to the optimization of functions possibly constrained having many local extrema the method is illustrated in some difftcult pedagogical examples and used to solve a problem analyzed by bates technometrics 25 pp 373 376 1983 for which we identify an improved optimum the sensitivity of the solution to changes in the constraints and in other specifications of the problem is analyzed and discussed

inverse methods and data assimilation in nonlinear ocean models abstract an overview is given of the current status of inverse methods and data assimilation for nonlinear ocean models the inverse theory for time dependent dynamical models is formulated and the most promising solution methods like simulated annealing the representer method and sequential methods based on monte carlo simulations are discussed with special focus on applications with nonlinear dynamics a rather general model independent presentation has been used to make the methodology more accessible for different scientific areas dealing with dynamical models and data

detection of gauss markov random fields with nearest neighbor dependency the problem of hypothesis testing against independence for a gauss markov random field gmrf is analyzed assuming an acyclic dependency graph an expression for the log likelihood ratio of detection is derived assuming random placement of nodes over a large region according to the poisson or uniform distribution and nearest neighbor dependency graph the error exponent of the neyman pearson detector is derived using large deviations theory the error exponent is expressed as a dependency graph functional and the limit is evaluated through a special law of large numbers for stabilizing graph functionals the exponent is analyzed for different values of the variance ratio and correlation it is found that a more correlated gmrf has a higher exponent at low values of the variance ratio whereas the situation is reversed at high values of the variance ratio

optimization of non linear gravity models through generalized simulated annealing abstract in this paper we apply the generalized simulated annealing gsa approach to the inversion of gravity data for 2 d and 3 d density distributions we consider a modeling process where the input is the vector of model parameters m that can be density contrast mass or some coordinates and the output is described by the transformation h m d where d is the vector of data parameters we generally have access in practical problems if the vector d describes the observed actual output of the system the problem is to choose or estimate the parameters m est in order to minimize in some sense in our case in the least squares sense the difference between the observed vector d and the prescribed output of the system h m est the tests with synthetic data show the promising application of gsa in gravity inversion the results obtained suggest us that the gsa approach enables to find quickest optimization machines than the two conventional approaches boltzmann and cauchy machines

optimal sequence of quantum measurements in the sense of stein s lemma in quantum hypothesis testing we derive a necessary and sufficient condition for a sequence of quantum measurements to achieve the optimal performance in quantum hypothesis testing we discuss what quantum measurement we should perform in order to attain the optimal exponent of the second error probability under the condition that the first error probability goes to 0 as an asymptotically optimal measurement we propose a projection measurement characterized by the irreducible representation theory of the special linear group sl especially in the spin 1 2 system it is realized by the simultaneous measurement of the total momentum and a momentum of a specified direction as a by product we obtain another proof of quantum stein s lemma in addition an asymptotically optimal measurement is constructed in the quantum gaussian case and it is physically meaningful

integer valued minimax robust designs for estimation and extrapolation in heteroscedastic approximately linear models abstract we present our findings on a new approach to robust regression design this approach differs from previous investigations into this area in three respects the use of a finite design space the use of simulated annealing to carry out the numerical minimization problems and in our search for integer valued rather than continuous designs we present designs for the situation in which the response is thought to be approximately polynomial we also discuss the cases of approximate first and second order multiple regression in each case we allow for possible heteroscedasticity and also obtain minimax regression weights the results are extended to cover extrapolation of the regression response to regions outside of the design space a case study involving dose response experimentation is undertaken the optimal robust designs which protect against bias as well as variance can be roughly described as being obtained from the classical variance minimizing designs by replacing replicates with clust

exponential decay of solutions of maxwell s equations coupled with a first order ordinary differential equation for the polarization this paper is concerned with maxwell s equations involving a first order ordinary differential equation for the dielectric polarization the main goal is the exponential convergence of the electromagnetic field to an asymptotic state which is determined by the initial data this is achieved by the construction of a suitable energy functional involving a vector potential which provides an additional dissipative term for the magnetic field

numerical methods for estimating parameters in nonlinear models with errors in the variables a numerically stable implementation of the gauss newton method for computing least squares estimates of parameters and variables in explicit nonlinear models with errors in the variables is proposed the algorithm uses only orthogonal transformations and exploits the special structure of the problem moreover a partially regularized marquardt like version is described that works with a reasonable overhead of arithmetic operations and storage compared to the error free case

a cellular coevolutionary algorithm for image segmentation clustering is inherently a difficult problem both with respect to the definition of adequate models as well as to the optimization of the models we present a model for the cluster problem that does not need knowledge about the number of clusters a priori this property is among others useful in the image segmentation domain which we especially address further we propose a cellular coevolutionary algorithm for the optimization of the model within this scheme multiple agents are placed in a regular two dimensional 2 d grid representing the image which imposes neighboring relations on them the agents cooperatively consider pixel migration from one agent to the other in order to improve the homogeneity of the ensemble of the image regions they represent if the union of the regions of neighboring agents is homogeneous then the agents form alliances on the other hand if an agent discovers a deviant subject it isolates the subject in the experiments we show the effectiveness of the proposed method and compare it to other segmentation algorithms the efficiency can easily be improved by exploiting the intrinsic parallelism of the proposed method

a genetic approach to the automatic clustering problem abstract in solving the clustering problem traditional methods for example the k means algorithm and its variants usually ask the user to provide the number of clusters unfortunately the number of clusters in general is unknown to the user therefore clustering becomes a tedious trial and error work and the clustering result is often not very promising especially when the number of clusters is large and not easy to guess in this paper we propose a genetic algorithm for the clustering problem this algorithm is suitable for clustering the data with compact spherical clusters it can be used in two ways one is the user controlled clustering where the user may control the result of clustering by varying the values of the parameter w a small value of w results in a larger number of compact clusters while a large value of w results in a smaller number of looser clusters the other is an automatic clustering where a heuristic strategy is applied to find a good clustering experimental results are given to illustrate the effectiveness of this genetic clustering algorithm

global optimization in clustering using hyperbolic cross points erich novak and klaus ritter developed in 1996 a global optimization algorithm that uses hyperbolic cross points hcps in this paper we develop a hybrid algorithm for clustering called cmhcp that uses a modified version of this hcp algorithm for global search and the alternating optimization for local search the program has been tested extensively with very promising results and high efficiency this provides a nice addition to the arsenal of global optimization in clustering in the process we also analyze the smoothness of some reformulated objective functions

a common tabu search algorithm for the global optimization of engineering problems abstract a novel common tabu algorithm for global optimizations of engineering problems is presented the robustness and efficiency of the presented method are evaluated by using standard mathematical functions and by solving a practical engineering problem the numerical results show that the proposed method is i superior to the conventional tabu search algorithm in robustness and ii superior to the simulated annealing algorithm in efficiency

on the converse theorem in statistical hypothesis testing for markov chains hypothesis testing for two markov chains is considered under the constraint that the error probability of the first kind is less than or equal to exp rn the error probability of the second kind is minimized the geodesic that connects the two markov chains is defined by analyzing the geodesic the power exponents are calculated and then represented in terms of kullback leibler divergence

a maximum variance cluster algorithm we present a partitional cluster algorithm that minimizes the sum of squared error criterion while imposing a hard constraint on the cluster variance conceptually hypothesized clusters act in parallel and cooperate with their neighboring clusters in order to minimize the criterion and to satisfy the variance constraint in order to enable the demarcation of the cluster neighborhood without crucial parameters we introduce the notion of foreign cluster samples finally we demonstrate a new method for cluster tendency assessment based on varying the variance constraint parameter

j means a new local search heuristic for minimum sum of squares clustering abstract a new local search heuristic called j m eans is proposed for solving the minimum sum of squares clustering problem the neighborhood of the current solution is defined by all possible centroid to entity relocations followed by corresponding changes of assignments moves are made in such neighborhoods until a local optimum is reached the new heuristic is compared with two other well known local search heuristics k and h m eans as well as with h m eans an improved version of the latter in which degeneracy is removed moreover another heuristic which fits into the variable neighborhood search metaheuristic framework and uses j m eans in its local search step is proposed too results on standard test problems from the literature are reported it appears that j m eans outperforms the other local search methods quite substantially when many entities and clusters are considered

reduced complexity dynamic programming based on policy iteration abstract in this paper a forward method is introduced for solving the dynamic programming equations of bellman this is in contrast to most existing methods for dynamic programming which solve the problem backwards a key advantage is that the forward dynamic programming approach can be systematically simplified to provide computation optimality trade offs such trade offs are lacking in backwards iterative methods which tend to be all or nothing propositions a second advantage is that the computation is independent of the state dimension these properties together offer some promise for circumventing the curse of dimensionality on many problems of practical interest due to a strong connection with the work of bellman on policy iteration the method is denoted as the iteration in policy space ips algorithm several examples are given to demonstrate the general usefulness of the method

global well posedness and exponential stability for heterogeneous anisotropic maxwell s equations under a nonlinear boundary feedback with delay abstract we consider an initial boundary value problem for the maxwell s system in a bounded domain with a linear nonhomogeneous anisotropic instantaneous material law subject to a nonlinear silver muller type boundary feedback mechanism incorporating both an instantaneous damping and a time localized delay effect by proving the maximal monotonicity property of the underlying nonlinear generator we establish the global well posedness in an appropriate hilbert space further under suitable assumptions and geometric conditions we show the system is exponentially stable

general formulas for fixed length quantum entanglement concentration in this paper we derive general formulas for the amount of entanglement that can be concentrated from general sequences of partially entangled bipartite pure states the formulas are obtained by using an information spectrum approach the formulas obtained express the optimal rate under constant error constraints or exponential error constraints in the asymptotic framework since the formulas treat general sequences they can be applied to i i d sequences as well as sequences with correlations

single and multiobjective structural optimization in discrete continuous variables using simulated annealing a multivariable optimization technique based on the monte carlo method used in statistical mechanics studies of condensed systems is adapted for solving single and multiobjective structural optimization problems this procedure known as simulated annealing draws an analogy between energy minimization in physical systems and objective function minimization in structural systems the search for a minimum is simulated by a relaxation of the statistical mechanical system where a probabilistic acceptance criterion is used to accept or reject candidate designs to model the multiple objective functions in the problem formulation a cooperative game theoretic approach is used numerical results obtained using three different annealing strategies for the single and multiobjective design of structures with discrete continuous variables are presented the influence of cooling schedule parameters on the optimum solutions obtained is discussed simulation results indicate that in several instances the optimum solutions obtained using simulated annealing outperform the optimum solutions obtained using some gradient based and discrete optimization techniques the results also indicate that simulated annealing has substantial potential for additional applications in optimization especially for problems with mixed discrete continuous variables

a simulated annealing algorithm for the clustering problem abstract in this paper we discuss the solution of the clustering problem usually solved by the k means algorithm the problem is known to have local minimum solutions which are usually what the k means algorithm obtains the simulated annealing approach for solving optimization problems is described and is proposed for solving the clustering problem the parameters of the algorithm are discussed in detail and it is shown that the algorithm converges to a global solution of the clustering problem we also find optimal parameters values for a specific class of data sets and give recommendations on the choice of parameters for general data sets finally advantages and disadvantages of the approach are presented

experiments in projection and clustering by simulated annealing simulated annealing is a stochastic relaxation algorithm which has been used successfully to optimize functions of many variables this paper analyzes the simulated annealing algorithm when applied to the minimization of functions from two common problems encountered in exploratory pattern analysis projection and clustering the projection is a nonlinear mapping of patterns in high dimension to two dimensions the simulated annealing mapping is compared to gradient descent minimization of the same objective function as well as eigenvector projection the simulated annealing clustering is compared to a k means algorithm empirical results show that simulated annealing can produce results as good as those obtained by conventional methods but are impractical for small data sets because of the high computational cost simulated annealing does in the case of the mapping problem yield a better optimization and better retained structure for large data sets containing tight gaussian clusters

the derivative of x x and its uses the determinant of matrices of the form x x is used as the d optimal design criterion for parameter estimation in nonlinear regression and as the estimation criterion for multiresponse parameter estimation it is helpful to be able to calculate the gradient of this determinant so that sophisticated optimization methods can be used a method is given to efficiently compute the determinant using modern computational linear algebra and at the same time to provide the necessary information to compute the gradient the method is extended to cases in which a part of the x matrix is fixed examples from multiresponse parameter estimation and from experimental design are given

the strong ergodic theorem for densities generalized shannon mcmillan breiman theorem let xi x2 be a stationary process with probability densities f x1 x2 xn with respect to lebesgue measure or with respect to a markov measure with a stationary transition measure it is shown that the sequence of relative entropy densities 1 n log f xi x2 x0 converges almost surely this long conjectured result extends the l convergence obtained by moy perez and kieffer and generalizes the shannon mcmillanbreiman theorem to nondiscrete processes the heart of the proof is a new martingale inequality which shows that logarithms of densities are l dominated

opportunistic detection rules finite and asymptotic analysis opportunistic detection rules odrs are variants of fixed sample size detection rules in which the statistician is allowed to make an early decision on the alternative hypothesis opportunistically based on the sequentially observed samples from a sequential decision perspective odrs are also mixtures of one sided and truncated sequential detection rules several results regarding odrs are established in this paper in the finite regime the maximum sample size is modeled either as a fixed finite number or a geometric random variable with a fixed finite mean for both cases the corresponding bayesian formulations are investigated the former case is a slight variation of the well known finite length sequential hypothesis testing procedure in the literature whereas the latter case is new for which the bayesian optimal odr is shown to be a sequence of likelihood ratio threshold tests with two different thresholds a running threshold which is determined by solving a stationary state equation is used when future samples are still available and a terminal threshold simply the ratio between the priors scaled by costs is used when the statistician reaches the final sample and thus has to make a decision immediately in the asymptotic regime the tradeoff among the exponents of the false alarm and miss error probabilities and the normalized expected stopping time under the alternative hypothesis is completely characterized and proved to be tight via an information theoretic argument within the tradeoff region one noteworthy fact is that the performance of the stein chernoff lemma is attainable by odrs

csisza spl acute r s cutoff rates for the general hypothesis testing problem in csisza spl acute r established the concept of forward spl beta cutoff rate for the error exponent hypothesis testing problem based on independent and identically distributed i i d observations given spl beta 0 spl alpha spl ne 1 similarly for 0 spl beta 1 csisza spl acute r also established the concept of reverse spl beta cutoff rate for the correct exponent hypothesis testing problem in this work we extend csisza spl acute r s results by investigating the forward and reverse spl beta cutoff rates for the hypothesis testing between two arbitrary sources with memory we demonstrate that the lim inf re spl acute nyi spl alpha divergence rate provides the expression for the forward spl beta cutoff rate we also show that if the log likelihood large deviation spectrum admits a limit then the reverse spl beta cutoff rate equals the liminf spl alpha divergence rate where spl alpha 1 1 spl beta and 0 spl beta spl beta sub max where spl beta sub max is the largest spl beta 1 for which the lim inf 1 1 spl beta divergence rate is finite for spl beta sub max spl les spl beta 1 we show that the reverse cutoff rate is in general only upper bounded by the lim inf re spl acute nyi divergence rate unlike in where the alphabet for the source coding cutoff rate problem was assumed to be finite we assume arbitrary countable or continuous source alphabet we also provide several examples to illustrate our forward and reverse spl beta cutoff rates results and the techniques employed to establish them

an information spectrum approach to capacity theorems for the general multiple access channel the paper deals with the capacity problems for the general multiple access channel where the channel transition probabilities are arbitrary for every blocklength n the approach used here which is called the information spectrum approach is quite different from the standard typical sequence and or aep techniques the general formula for the capacity region for the general multiple access channel is thus established in order to examine the potentiality we apply it to the mixed channel to obtain the formula for its capacity region the case where input cost constraints are imposed is also considered

sampling design for classifying contaminant level using annealing search algorithms a stochastic method for sampling spatially distributed contaminant level is presented the purpose of sampling is to partition the contaminated region into zones of high and low pollutant concentration levels in particular given an initial set of observations of a contaminant within a site it is desired to find a set of additional sampling locations in a way that takes into consideration the spatial variability characteristics of the site and optimizes certain objective functions emerging from the physical regulatory and monetary considerations of the specific site cleanup process since the interest is in classifying the domain into zones above and below a pollutant threshold level a natural criterion is the cost of misclassification the resulting objective function is the expected value of a spatial loss function associated with sampling stochastic expectation involves the joint probability distribution of the pollutant level and its estimate where the latter is calculated by means of spatial estimation techniques actual computation requires the discretization of the contaminated domain as a consequence any reasonably sized problem results in combinatorics precluding an exhaustive search the use of an annealing algorithm although suboptimal can find a good set of future sampling locations quickly and efficiently in order to obtain insight about the parameters and the computational requirements of the method an example is discussed in detail the implementation of spatial sampling design in practice will provide the model inputs necessary for waste site remediation groundwater management and environmental decision making

asymptotic behavior of electromagnetic waves in nonlinear dielectric media this paper is concerned with maxwell s equations including a nonlinear dielectric polarization and a generally nonlinear law for the electric current the large time behavior of the electromagnetic field is investigated in the case of a bounded spatial domain as well as in the exterior domain case

error exponents for hypothesis testing of the general source in this correspondence we consider the simple hypothesis testing problems for general sources in the sense of han and verdu recently han established a compact formula for the supremum of achievable exponents for the second kind of error probability under the asymptotic constraint of the form mun e nr n rarr infin on the first kind of error probability mun where r is a given positive number we investigate the same hypothesis testing problems studied by han the aim of the correspondence is to give a new expression for the supremum of achievable error exponents our formula is expressed in terms of the divergences and given in quite different forms from han s expression

some guidelines for constructing exact d optimal designs on convex design spaces some problems unique to the construction of n point d optimal designs on convex design spaces are considered multiple point augmentation and exchange algorithms are shown to be more costly and less efficient than the analogous single point procedures moreover some recommendations for improving single point methods are given finally empirical evidence is found that supports the galil and kiefer option for constructing initial designs and powell s optimization method for design augmentation

an information spectrum approach to classical and quantum hypothesis testing for simple hypotheses the information spectrum analysis made by han for classical hypothesis testing for simple hypotheses is extended to a unifying framework including both classical and quantum hypothesis testing the results are also applied to fixed length source coding when loosening the normalizing condition for probability distributions and for quantum states we establish general formulas for several quantities relating to the asymptotic optimality of tests codes in terms of classical and quantum information spectra

global optimization of statistical functions with simulated annealing abstract many statistical methods rely on numerical optimization to estimate a model s parameters unfortunately conventional algorithms sometimes fail even when they do converge there is no assurance that they have found the global rather than a local optimum we test a new optimization algorithm simulated annealing on four econometric problems and compare it to three common conventional algorithms not only can simulated annealing find the global optimum it is also less likely to fail on difficult functions because it is a very robust algorithm the promise of simulated annealing is demonstrated on the four econometric problems

two approaches to obtain the strong converse exponent of quantum hypothesis testing for general sequences of quantum states we present two general approaches to obtain the strong converse exponent of simple quantum hypothesis testing for correlated quantum states one approach requires that the states satisfy a certain factorization property typical examples of such states are the temperature states of translation invariant finite range interactions on a spin chain the other approach requires the differentiability of a regularized renyi alpha divergence in the parameter alpha typical examples of such states include temperature states of non interacting fermionic lattice systems and classical irreducible markov chains in all cases we get that the strong converse exponent is equal to the hoeffding anti divergence which in turn is obtained from the regularized renyi divergences of the two states

the local minima problem in hierarchical classes analysis an evaluation of a simulated annealing algorithm and various multistart procedures hierarchical classes models are quasi order retaining boolean decomposition models for n way n mode binary data to fit these models to data rationally started alternating least squares or equivalently alternating least absolute deviations algorithms have been proposed extensive simulation studies showed that these algorithms succeed quite well in recovering the underlying truth but frequently end in a local minimum in this paper we evaluate whether or not this local minimum problem can be mitigated by means of two common strategies for avoiding local minima in combinatorial data analysis simulated annealing sa and use of a multistart procedure in particular we propose a generic sa algorithm for hierarchical classes analysis and three different types of random starts the effectiveness of the sa algorithm and the random starts is evaluated by reanalyzing data sets of previous simulation studies the reported results support the use of the proposed sa algorithm in combination with a random multistart procedure regardless of the properties of the data set under study

a genetic clustering algorithm for data with non spherical shape clusters abstract in solving clustering problem traditional methods for example the k means algorithm and its variants usually ask the user to provide the number of clusters unfortunately the number of clusters in general is unknown to the user the traditional neighborhood clustering algorithm usually needs the user to provide a distance d for the clustering this d is difficult to decide because some clusters may be compact but others may be loose in this paper we propose a genetic clustering algorithm for clustering the data whose clusters are not of spherical shape it can automatically cluster the data according to the similarities and automatically find the proper number of clusters the experimental results are given to illustrate the effectiveness of the genetic algorithm

the strong converse theorem for hypothesis testing the authors present the answer to a so called converse problem by giving the explicit form of the power exponent this strong converse should be regarded as completing a weak converse previously obtained by r e blahut 1974

asymptotically optimal matching of multiple sequences to source distributions and training sequences consider a finite set of sources each producing independent identically distributed observations that follow a unique probability distribution on a finite alphabet we study the problem of matching a finite set of observed sequences to the set of sources under the constraint that the observed sequences are produced by distinct sources in general the number of sequences n may be different from the number of sources m and only some k min m n of the observed sequences may be produced by a source from the set of sources of interest we consider two versions of the problem one in which the probability laws of the sources are known and another in which the probability laws of the sources are unspecified but one training sequence from each of the sources is available we show that both these problems can be solved using a sequence of tests that are allowed to produce no match decisions the tests ensure exponential decay of the probabilities of incorrect matching as the sequence lengths increase and minimize the no match decisions both the tests can be implemented using variants of the minimum weight matching algorithm applied to a weighted bipartite graph we also compare the performances obtained using these tests with those obtained using tests that do not consider the constraint that the sequences are produced by distinct sources for the version of the problem in which the probability laws of the sources are known we compute the rejection exponents and error exponents of the tests and show that the tests that make use of the constraint have better exponents than the tests that do not make use of this information

discrimination of two channels by adaptive methods and its application to quantum system the optimal exponential error rate for adaptive discrimination of two channels is discussed in this problem adaptive choice of input signal is allowed this problem is discussed in various settings it is proved that adaptive choice does not improve the exponential error rate in these settings these results are applied to quantum state discrimination

data visualization by multimensional scaling a deterministic annealing approach abstract multidimensional scaling addresses the problem how proximity data can be faithfully visualized as points in a low dimensional euclidean space the quality of a data embedding is measured by a stress function which compares proximity values with euclidean distances of the respective points the corresponding minimization problem is non convex and sensitive to local minima we present a novel deterministic annealing algorithm for the frequently used objective sstress and for sammon mapping derived in the framework of maximum entropy estimation experimental results demonstrate the superiority of our optimization technique compared to conventional gradient descent methods

weak asymptotic decay via a relaxed invariance principle for a wave equation with nonlinear non monotone damping this paper considers the problem of asymptotic decay as t of solutions of the wave equation u tt u a x u t u t x a bounded open connected set in n n 1 with smooth boundary u 0 on the nonlinear function is not assumed to be globally lipschitz continuous 0 y 2 y n 1 0 y 1 y 1 y n 1 0 for all y n 1 is not assumed to be monotone in y 1 under additional restrictions on the kernel of conditions are given which imply that u u t converges to 0 0 weakly in h h 1 0 l 2 as t the work generalises earlier results of dafermos and haraux where strong decay in h as t was obtained in the case y 1 y n 1 q y 1 q monotone on

tabu search method with random moves for globally optimal design optimum engineering design problems are usually formulated as non convex optimization problems of continuous variables because of the absence of convexity structure they can have multiple minima and global optimization becomes difficult traditional methods of optimization such as penalty methods can often be trapped at a local optimum the tabu search method with random moves to solve approximately these problems is introduced its reliability and efficiency are examined with the help of standard test functions by the analysis of the implementations it is seen that this method is easy to use and no derivative information is necessary it outperforms the random search method and composite genetic algorithm in particular it is applied to minimum weight design examples of a three bar truss coil springs a z section and a channel section for the channel section the optimal design using the tabu search method with random moves saved 26 14 per cent over the weight of the sumt method

a general formula for channel capacity a formula for the capacity of arbitrary single user channels without feedback not necessarily information stable stationary etc is proved capacity is shown to equal the supremum over all input processes of the input output inf information rate defined as the liminf in probability of the normalized information density the key to this result is a new converse approach based on a simple new lower bound on the error probability of m ary hypothesis tests among equiprobable hypotheses a necessary and sufficient condition for the validity of the strong converse is given as well as general expressions for spl epsiv capacity

hypothesis testing with the general source the asymptotically optimal hypothesis testing problem with general sources as the null and alternative hypotheses is studied under exponential type error constraints on the first kind of error probability our fundamental philosophy is to convert all of the hypothesis testing problems to the pertinent computation problems in the large deviation probability theory this methodologically new approach enables us to establish compact general formulas of the optimal exponents of the second kind of error and correct testing probabilities for the general sources including all nonstationary and or nonergodic sources with arbitrary abstract alphabet countable or uncountable these general formulas are presented from the information spectrum point of view

stabilization of trajectories for some weakly damped hyperbolic equations in this paper we study the phenomenon of stabilization of trajectories for a wave equation in a bounded open domain endowed with a weak dissipative mechanism by stabilization we mean that the self oscillations induced by the wave equation are damped out asymptotically and so we are left when time tends to infinity either with an equilibrium if the system is autonomous or with a forced oscillation if the system was submitted to an exterior periodic or almost periodic excitation the common point between the two situations is the extinction of selfoscillations in the presence of a weak damping and it is extremely satisfactory to discover that the difficulty is overcome in both cases by the use of one simple and elegant idea due to c m dafermos and which appeared in s precisely in the study of some partially damped hyperbolic systems the paper is divided in four sections in section 1 we recall the idea of s which will be used in sections 2 and 3 in section 2 we generalize the result of s proposition 2 31 in a framework which allows the existence of nontrivial equilibria a simple example from continuum mechanics is given in section 3 we apply the same device to establish total damping in the sense of s for a wave equation with single valued damping term and almost periodic forcing the result which we obtain is not covered by the general theorems of 7 or 8 a simple example is studied in section 4 we give two counterexamples which show that the result of section 3 cannot be extended to the framework of weak solutions or to other boundary conditions in each specific case a careful analysis will be necessary the notation which is used throughout the text is the standard one with 145 0022 0396185 3 00

compound d and d s optimum designs for determining the order of a chemical reaction estimation of the order of a chemical reaction is often at least as important as estimation of the rate of the reaction locally optimum experimental designs are found for the order and the rate separately and together compound d optimum designs provide a method for designing experiments with specified efficiency for rate or order determination a relationship between the compound designs and d optimum designs for both rate and order aids interpretation of the plots of design efficiencies bayesian designs incorporating prior uncertainty are exemplified monte carlo sampling of the prior is used to design an experiment for the esterification of acetic anhydride

optimal design in nonlinear multiresponse estimation poisson model for filter feeding summary d optimality criteria have been applied to construct locally optimal designs for a multiresponse nonlinear model simulated annealing was used to perform the needed numerical optimization calculations as this method can locate the global optimum of a function and can efficiently handle constraints in the independent variables the calculated optimal designs greatly reduce variances of model parameter estimates compared to variances from previously used empirical designs the effect of several design variables including the number of design points and the number of responses on the efficiency of the design was investigated and designs for various subsets of parameters were also calculated new directions for the design of future experiments were suggested by this analysis

a tabu search based heuristic for clustering abstract this paper considers a clustering problem where a given data set is partitioned into a certain number of natural and homogeneous subsets such that each subset is composed of elements similar to one another but different from those of any other subset for the clustering problem a heuristic algorithm is exploited by combining the tabu search heuristic with two complementary functional procedures called packing and releasing procedures the algorithm is numerically tested for its effectiveness in comparison with reference works including the tabu search algorithm the k means algorithm and the simulated annealing algorithm

the n city travelling salesman problem statistical mechanics and the metropolis algorithm in any n city travelling salesman problem there are n 1 mathord left vphantom 2 right kern nulldelimiterspace 2 possible tours we use the metropolis algorithm to generate a sequence of such tours this sequence may be viewed as the random evolution of a physical system in contact with a heat bath as the temperature is lowered the tours generated approach the optimal tour it appears that for large n one arrives within a few percent of the optimal solution in better than quadratic time

stabilization and exact boundary controllability for maxwell s equations this paper considers maxwell s equations with dissipative boundary conditions under certain geometric conditions imposed on the domain omega the results on uniform stabilization of the solutions are established exact boundary controllability is then obtained through russell s controllability via stabilizability principle

bayesian design and analysis of computer experiments use of derivatives in surface prediction this article is concerned with the problem of predicting a deterministic response function yo over a multidimensional domain t given values of yo and all of its first derivatives at a set of design sites points in t the intended application is to computer experiments in which yo is an output from a computer model of a physical system and each point in t represents a particular configuration of the input parameters it is assumed that the first derivatives are already available e g from a sensitivity analysis or can be produced by the code that implements the model a bayesian approach in which the random function that represents prior uncertainty about yo is taken to be a stationary gaussian stochastic process is used the calculations needed to update the prior given observations of yo and its first derivatives at the design sites are given and are illustrated in a small example the issue of experimental design is also discussed in particular the criterion of maximizing the reduction in entropy

stabilization of the wave equation with localized nonlinear damping abstract we prove some decay estimates of the energy of the wave equation in a bounded domain the damping is nonlinear and is effective only in a neighborhood of a suitable subset of the boundary the method of proof is direct and is based on the multipliers technique on some integral inequalities due to haraux and komornik and on a judicious idea of conrad and rao

pairwise clustering using a monte carlo markov chain abstract in this work an application of mcmc is proposed for unsupervised data classification in conjunction with a novel pairwise objective function which is shown to work well in situations where clusters to be identified have a strong overlap and the centroid oriented methods such as k means fail by construction in particular an exceptionally simple but difficult situation is addressed when cluster centroids coincide and one can differentiate between the clusters only on the basis of their variance performance of the proposed approach is tested on synthetic and real datasets

a comparison of heuristic procedures for minimum within cluster sums of squares partitioning perhaps the most common criterion for partitioning a data set is the minimization of the within cluster sums of squared deviation from cluster centroids although optimal solution procedures for within cluster sums of squares wcss partitioning are computationally feasible for small data sets heuristic procedures are required for most practical applications in the behavioral sciences we compared the performances of nine prominent heuristic procedures for wcss partitioning across 324 simulated data sets representative of a broad spectrum of test conditions performance comparisons focused on both percentage deviation from the best found wcss values as well as recovery of true cluster structure a real coded genetic algorithm and variable neighborhood search heuristic were the most effective methods however a straightforward two stage heuristic algorithm hk means also yielded exceptional performance a follow up experiment using 13 empirical data sets from the clustering literature generally supported the results of the experiment using simulated data our findings have important implications for behavioral science researchers whose theoretical conclusions could be adversely affected by poor algorithmic performances

a tabu search approach to the clustering problem in this paper we consider the problem of clustering m objects into c clusters the objects are represented by points in an n dimensional euclidean space and the objective is to classify these m points into c clusters such that the distance between points within a cluster and its center which is to be found is minimized the problem is a nonconvex program that has many local minima it has been studied by many researchers and the most well known algorithm for solving it is the k means algorithm in this paper we develop a new algorithm for solving this problem based on a tabu search technique preliminary computational experience on the developed algorithm are encouraging and compare favorably with both the k means and the simulated annealing algorithms

a global algorithm for the fuzzy clustering problem abstract the fuzzy clustering fc problem is a non convex mathematical program which usually possesses several local minima the global minimum solution of the problem is found using a simulated annealing based algorithm some preliminary computational experiments are reported and the solution is compared with that generated by the fuzzy c means algorithm

theorems on the variable length intrinsic randomness we address variable length intrinsic randomness problems in the sense of vembu and verdu 1995 for countably infinite source alphabet spl chi under the unnormalized divergence distance the normalized conditional divergence distance and the variational distance it turns out that under all three kinds of approximation measures the variable length intrinsic randomness still takes the same value called the inf entropy rate of the source

applications of the annealing algorithm to combinatorial problems in statistics summary there are several problems in statistics which can be formulated so that the desired solution is the global minimum of some explicitly defined objective function in many cases the number of candidate solutions increases exponentially with the size of the problem making exhaustive search impossible but descent procedures devised to reduce the number of solutions examined can terminate with local minima in this paper we describe the annealing algorithm a widely applicable stochastic search procedure which can escape local optima and we use the evolutionary tree problem to illustrate the method of application