vectorized search for single clusters breadth first search for a single cluster on a regular lattice is shown to be vectorizable it is applied to construct clusters in the single cluster variant of the swendsen wang algorithm on a cray ymp total cpu time has been reduced by factors of 3 5 7 in large scale applications a multiple cluster version is also described

mixture subclass discriminant analysis link to restricted gaussian model and other generalizations in this paper a theoretical link between mixture subclass discriminant analysis msda and a restricted gaussian model is first presented then two further discriminant analysis da methods i e fractional step msda fsmsda and kernel msda kmsda are proposed linking msda to an appropriate gaussian model allows the derivation of a new da method under the expectation maximization em framework em msda which simultaneously derives the discriminant subspace and the maximum likelihood estimates the two other proposed methods generalize msda in order to solve problems inherited from conventional da fsmsda solves the subclass separation problem that is the situation in which the dimensionality of the discriminant subspace is strictly smaller than the rank of the inter between subclass scatter matrix this is done by an appropriate weighting scheme and the utilization of an iterative algorithm for preserving useful discriminant directions on the other hand kmsda uses the kernel trick to separate data with nonlinearly separable subclass structure extensive experimentation shows that the proposed methods outperform conventional msda and other linear discriminant analysis variants

on the correct convergence of the em algorithm for gaussian mixtures it is well known that the em algorithm generally converges to a local maximum likelihood estimate however there have been many evidences to show that the em algorithm can converge correctly to the true parameters as long as the overlap of gaussians in the sample data is small enough this paper studies this correct convergence problem asymptotically on the em algorithm for gaussian mixtures it has been proved that the em algorithm becomes a contraction mapping of the parameters within a neighborhood of the consistent solution of the maximum likelihood when the measure of average overlap among gaussians in the original mixture is small enough and the number of samples is large enough that is if the initial parameters are set within the neighborhood the em algorithm will always converge to the consistent solution i e the expected result moreover the simulation results further demonstrate that this correct convergence neighborhood becomes larger as the average overlap becomes smaller

computing the roughening transition of ising and solid on solid models by bcsos model matching we study the roughening transition of the dual of the two dimensional 2d xy model of the discrete gaussian model of the absolute value solid on solid model and of the interface in an ising model on a three dimensional 3d simple cubic lattice the investigation relies on a renormalization group finite size scaling method that was proposed and successfully tested a few years ago the basic idea is to match the renormalization group flow of the interface observables with that of the exactly solvable body centred solid on solid bcsos model our estimates for the critical couplings are and for the xy model the discrete gaussian model and the absolute value solid on solid model respectively for the inverse roughening temperature of the ising interface we find to the best of our knowledge these are the most precise estimates for these parameters published so far

mixture densities maximum likelihood and the em algorithm the problem of estimating the parameters which determine a mixture density has been the subject of a large diverse body of literature spanning nearly ninety years during the last two decades the

on convergence and parameter selection of the em and da em algorithms for gaussian mixtures abstract the expectation maximization em for gaussian mixtures is popular as a clustering algorithm however the em algorithm is sensitive to initial values and so ueda and nakano 4 proposed the deterministic annealing em da em algorithm to improve it in this paper we investigate theoretical behaviors of the em and da em algorithms we first derive a general jacobian matrix of the da em algorithm with respect to posterior probabilities we then propose a theoretical lower bound for initialization of the annealing parameter in the da em algorithm on the other hand some researches mentioned that the em algorithm exhibits a self annealing behavior that is the equal posterior probability with small random perturbations can avoid the em algorithm to output the mass center for gaussian mixtures however there is no theoretical analysis on this self annealing property since the da em will become the em when the annealing parameter is 1 according to the jacobian matrix of the da em we can prove the self annealing property of the em algorithm for gaussian mixtures based on these results we give not only convergence behaviors of the equal posterior probabilities and initialization lower bound of the temperature parameter of the da em but also a theoretical explanation why the em algorithm for gaussian mixtures exhibits a self annealing behavior

high precision renormalization group study of the roughening transition abstract we confirm the kosterlitz thouless scenario of the roughening transition for three different solid on solid models the discrete gaussian model the absolute value solid on solid model and the dual transform of the xy model with standard cosine action the method is based on a matching of the renormalization group flow of the candidate models with the flow of a bona fide kt model the exactly solvable bcsos model the monte carlo simulations are performed using efficient cluster algorithms we obtain high precision estimates for the critical couplings and other non universal quantities for the xy model with cosine action our critical coupling estimate is xyr 1 1197 5 for the roughening coupling of the discrete gaussian and the absolute value solid on solid model we find kdgr 0 6645 6 and kasosr 0 8061 3 respectively

em algorithms for gaussian mixtures with split and merge operation abstract in order to alleviate the problem of local convergence of the usual em algorithm a split and merge operation is introduced into the em algorithm for gaussian mixtures the split and merge equations are first presented theoretically these equations show that the merge operation is a well posed problem whereas the split operation is an ill posed problem because it is the inverse procedure of the merge two methods for solving this ill posed problem are developed through the singular value decomposition and the cholesky decomposition accordingly a new modified em algorithm is constructed our experiments demonstrate that this algorithm is efficient for unsupervised color image segmentation

exact number of solutions for a class of two point boundary value problems with one dimensional p laplacian abstract in this paper exact number of solutions are obtained for the one dimensional p laplacian in a class of two point boundary value problems the interesting point is that the nonlinearity f is general form f u g u h u meanwhile some properties of the solutions are given in details the arguments are based upon a quadrature method

the solid on solid surface width around the roughening transition abstract we investigate the surface width w of solid on solid surfaces in the vicinity of the roughening temperature t r above t r w 2 is expected to diverge with the system size l like ln l however close to t r a clean ln l behavior can only be seen on extremely large lattices starting from the kosterlitz thouless renormalization group we derive an improved formula that describes the small l behavior on both sides of t r for the discrete gaussian model we used the valleys to mountains reflections cluster algorithm in order to simulate the fluctuating solid on solid surface the base plane above which the surface is defined is an l l square lattice in the simulation we took 8 l 256 the improved formula fits the numerical results very well from the analysis we estimate the roughening temperature to be t r 0 755 3

unsupervised learning of finite mixture models this paper proposes an unsupervised algorithm for learning a finite mixture model from multivariate data the adjective unsupervised is justified by two properties of the algorithm 1 it is capable of selecting the number of components and 2 unlike the standard expectation maximization em algorithm it does not require careful initialization the proposed method also avoids another drawback of em for mixture fitting the possibility of convergence toward a singular estimate at the boundary of the parameter space the novelty of our approach is that we do not use a model selection criterion to choose one among a set of preestimated candidate models instead we seamlessly integrate estimation and model selection in a single algorithm our technique can be applied to any type of parametric mixture model for which it is possible to write an em algorithm in this paper we illustrate it with experiments involving gaussian mixtures these experiments testify for the good performance of our approach

existence of a pair of periodic solutions of an o d e generalizing a problem in nonlinear elasticity via variational methods on considere l existence des solutions t periodiques de l equation differentielle x g t x ou g r r r est continue et t periodique en t sous certaines conditions sur g on demontre l existence d au moins deux solutions t periodiques on utilise une methode variationnelle

a class of two point boundary value problems we improve the results obtained by the first author in a recent paper where a problem raised by a ambrosetti h brezis and g cerami concerning the exact structure of the set of solutions for a class of two point boundary value problems was solved here we study a more general problem and get more detailed results

a symmetry theorem and applications to nonlinear partial differential equations on donne des applications d un nouveau theoreme sur les proprietes de symetrie des solutions des equations differentielles non lineaires de la forme lu f x u dans u 0 ou u n 0 ou est une region bornee ou non bornee de r n

on convergence properties of the em algorithm for gaussian mixtures we build up the mathematical connection between the expectation maximization em algorithm and gradient based approaches for maximum likelihood learning of finite gaussian mixtures we show that the em step in parameter space is obtained from the gradient via a projection matrix p and we provide an explicit expression for the matrix we then analyze the convergence of em in terms of special properties of p and provide new results analyzing the effect that p has on the likelihood surface based on these mathematical results we present a comparative discussion of the advantages and disadvantages of em and other algorithms for the learning of gaussian mixture models

robust clustering by deterministic agglomeration em of mixtures of multivariate t distributions abstract this paper presents new robust clustering algorithms which significantly improve upon the noise and initialization sensitivity of traditional mixture decomposition algorithms and simplify the determination of the optimal number of clusters in the data set the algorithms implement maximum likelihood mixture decomposition of multivariate t distributions a robust parametric extension of gaussian mixture decomposition we achieve improved convergence capability relative to the expectation maximization em approach by deriving deterministic annealing em daem algorithms for this mixture model and turning them into agglomerative algorithms going through a monotonically decreasing number of components an approach we term deterministic agglomeration em dagem two versions are derived based on two variants of daem for mixture models simulation studies demonstrate the algorithms performance for mixtures with isotropic and non isotropic covariances in two and 10 dimensions with known or unknown levels of outlier contamination

exact number of solutions of a class of two point boundary value problems involving concave and convex nonlinearities in this paper exact number of solutions are obtained for a class of two point boundary value problems involving concave and convex nonlinearities moreover some properties of the solutions are studied in details

enhanced iterative projection for subclass discriminant analysis under em alike framework mixture discriminant analysis mda and subclass discriminant analysis sda belong to the supervised classification approaches they have advantage over the standard linear discriminant analysis lda in large sample size problems since both of them divide the samples in each class into subclasses which keep locality but lda does not however since the current mda and sda algorithms perform subclass division in just one step in the original data space before solving the generalized eigenvalue problem two problems are exposed 1 they ignore the relation among classes since subclass division is performed in each isolated class 2 they cannot guarantee good performance of classifiers in the transformed space because locality in the original data space may not be kept in the transformed space to address these problems this paper presents a new approach for subclass division based on k means clustering in the projected space class by class using the iterative steps under em alike framework experiments are performed on the artificial data set the uci machine learning data sets the cenparmi handwritten numeral database the nust603 handwritten chinese character database and the terrain cover database extensive experimental results demonstrate the performance advantages of the proposed method highlightswe focus on the classification of the large sample size problems we propose a new em alike iterative projection approach emipa for subclass division our proposed approach outperforms the traditional mda and sda our approach operates subclass division and eigenvector seeking class by class our proposed approach takes only a bit more time than mda and sda

self annealing and self annihilation unifying deterministic annealing and relaxation labeling abstract deterministic annealing and relaxation labeling algorithms for classification and matching are presented and discussed a new approach self annealing is introduced to bring deterministic annealing and relaxation labeling into accord self annealing results in an emergent linear schedule for winner take all and linear assignment problems self annihilation a generalization of self annealing is capable of performing the useful function of symmetry breaking the original relaxation labeling algorithm is then shown to arise from an approximation to either the self annealing energy function or the corresponding dynamical system with this relationship in place self annihilation can be introduced into the relaxation labeling framework experimental results on synthetic matching and labeling problems clearly demonstrate the three way relationship between deterministic annealing relaxation labeling and self annealing

using em to obtain asymptotic variance covariance matrices the sem algorithm abstract the expectation maximization em algorithm is a popular and often remarkably simple method for maximum likelihood estimation in incomplete data problems one criticism of em in practice is that asymptotic variance covariance matrices for parameters e g standard errors are not automatic byproducts as they are when using some other methods such as newton raphson in this article we define and illustrate a procedure that obtains numerically stable asymptotic variance covariance matrices using only the code for computing the complete data variance covariance matrix the code for em itself and code for standard matrix operations the basic idea is to use the fact that the rate of convergence of em is governed by the fractions of missing information to find the increased variability due to missing information to add to the complete data variance covariance matrix we call this supplemented em algorithm the sem algorithm theory and particular examples reinforce the conclusion that the sem alg

on the convergence properties of the em algorithm two convergence aspects of the em algorithm are studied i does the em algorithm find a local maximum or a stationary value of the incomplete data likelihood function ii does the sequence of parameter estimates generated by em converge several convergence results are obtained under conditions that are applicable to many practical situations two useful special cases are a if the unobserved complete data specification can be described by a curved exponential family with compact parameter space all the limit points of any em sequence are stationary points of the likelihood function b if the likelihood function is unimodal and a certain differentiability condition is satisfied then any em sequence converges to the unique maximum likelihood estimate a list of key properties of the algorithm is included

asymptotic convergence rate of the em algorithm for gaussian mixtures it is well known that the convergence rate of the expectation maximization em algorithm can be faster than those of convention first order iterative algorithms when the overlap in the given mixture is small but this argument has not been mathematically proved yet this article studies this problem asymptotically in the setting of gaussian mixtures under the theoretical framework of xu and jordan 1996 it has been proved that the asymptotic convergence rate of the em algorithm for gaussian mixtures locally around the true solution is o e0 5 e where e 0 is an arbitrarily small number o x means that it is a higher order infinitesimal as x 0 and e is a measure of the average overlap of gaussians in the mixture in other words the large sample local convergence rate for the em algorithm tends to be asymptotically superlinear when e tends to zero

a q parameterized deterministic annealing em algorithm based on nonextensive statistical mechanics we propose a q parameterized deterministic annealing expectation maximization q daem algorithm for parameter estimation motivated by the concept of tsallis entropy that originates from the nonextensive statistical mechanics the q daem algorithm combines the feature of annealing algorithms to reduce initialization sensitivity and that of q em algorithms to achieve fast convergence the q em algorithm is a one parameter generalized em algorithm that has been previously proposed by the authors by interpreting the em algorithm via likelihood lower bound maximization we build the fundamental interconnections among daem q daem and statistical mechanics to illustrate the benefits of using q daem we investigate two applications finite mixture model estimation in data clustering and joint channel estimation and data detection in communication systems where we show that the q daem algorithm achieves superior performance over the em algorithm for both applications