learning gaussian mixture models with entropy based criteria in this paper we address the problem of estimating the parameters of gaussian mixture models although the expectation maximization em algorithm yields the maximum likelihood ml solution its sensitivity to the selection of the starting parameters is well known and it may converge to the boundary of the parameter space furthermore the resulting mixture depends on the number of selected components but the optimal number of kernels may be unknown beforehand we introduce the use of the entropy of the probability density function pdf associated to each kernel to measure the quality of a given mixture model with a fixed number of kernels we propose two methods to approximate the entropy of each kernel and a modification of the classical em algorithm in order to find the optimum number of components of the mixture moreover we use two stopping criteria a novel global mixture entropy based criterion called gaussianity deficiency gd and a minimum description length mdl principle based one our algorithm called entropy based em ebem starts with a unique kernel and performs only splitting by selecting the worst kernel attending to gd we have successfully tested it in probability density estimation pattern classification and color image segmentation experimental results improve the ones of other state of the art model order selection methods

discriminant analysis by gaussian mixtures fisher rao linear discriminant analysis lda is a valuable tool for multigroup classification lda is equivalent to maximum likelihood classification assuming gaussian distributions for each class in this paper we fit gaussian mixtures to each class to facilitate effective classification in non normal settings especially when the classes are clustered low dimensional views are an important by product of lda our new techniques inherit this feature we can control the within class spread of the subclass centres relative to the between class spread our technique for fitting these models permits a natural blend with nonparametric versions of lda

global asymptotic stability in a class of generalized putnam equations abstract it was conjectured that for every integer m 3 the unique equilibrium c 1 of the generalized putnam equation x n 1 i 0 m 2 x n i x n m 1 x n m x n x n 1 i 2 m x n i n 0 1 2 with positive initial conditions is globally asymptotically stable in this paper we prove this conjecture

bayesian approaches to gaussian mixture modeling a bayesian based methodology is presented which automatically penalizes overcomplex models being fitted to unknown data we show that with a gaussian mixture model the approach is able to select an optimal number of components in the model and so partition data sets the performance of the bayesian method is compared to other methods of optimal model selection and found to give good results the methods are tested on synthetic and real data sets

global behavior for a fourth order rational difference equation in this paper we use a method different from the known literature to investigate the global behavior of the following fourth order rational difference equation xn 1 xn 1xn 2xn 3 xn 1 xn 2 xn 3 a xn 1xn 2 xn 1xn 3 xn 2xn 3 1 a n 0 1 2

detecting features in spatial point processes with clutter via model based clustering abstract we consider the problem of detecting features such as minefields or seismic faults in spatial point processes when there is substantial clutter we use model based clustering based on a mixture model for the process in which features are assumed to generate points according to highly linear multivariate normal densities and the clutter arises according to a spatial poisson process nonlinear features are represented by several densities giving a piecewise linear representation hierarchical model based clustering provides a first estimate of the features and this is then refined using the em algorithm the number of features is estimated from an approximation to its posterior distribution the method gives good results for the minefield and seismic fault problems software to implement it is available on the world wide web

model based gaussian and non gaussian clustering abstract the classification maximum likelihood approach is sufficiently general to encompass many current clustering algorithms including those based on the sum of squares criterion and on the criterion of friedman and rubin 1967 however as currently implemented it does not allow the specification of which features orientation size and shape are to be common to all clusters and which may differ between clusters also it is restricted to gaussian distributions and it does not allow for noise we propose ways of overcoming these limitations a reparameterization of the covariance matrix allows us to specify that some features but not all be the same for all clusters a practical framework for non gaussian clustering is outlined and a means of incorporating noise in the form of a poisson process is described an approximate bayesian method for choosing the number of clusters is given the performance of the proposed methods is studied by simulation with encouraging results the methods are applied to the analysis of a data set arising in the study of diabetes and the results seem better than those of previous analyses rh

image feature localization by multiple hypothesis testing of gabor features several novel and particularly successful object and object category detection and recognition methods based on image features local descriptions of object appearance have recently been proposed the methods are based on a localization of image features and a spatial constellation search over the localized features the accuracy and reliability of the methods depend on the success of both tasks image feature localization and spatial constellation model search in this paper we present an improved algorithm for image feature localization the method is based on complex valued multiresolution gabor features and their ranking using multiple hypothesis testing the algorithm provides very accurate local image features over arbitrary scale and rotation we discuss in detail issues such as selection of filter parameters confidence measure and the magnitude versus complex representation and show on a large test sample how these influence the performance the versatility and accuracy of the method is demonstrated on two profoundly different challenging problems faces and license plates

on two rational difference equations abstract in this paper we study the oscillating property of positive solutions and the global asymptotic stability of the unique equilibrium of the two rational difference equations x n 1 x n x n 1 x n 2 a x n x n 1 x n 2 a n 0 1 2 and x n 1 x n x n 1 x n 2 a x n 1 x n x n 2 a n 0 1 2 where a is a nonnegative constant

a q parameterized deterministic annealing em algorithm based on nonextensive statistical mechanics we propose a q parameterized deterministic annealing expectation maximization q daem algorithm for parameter estimation motivated by the concept of tsallis entropy that originates from the nonextensive statistical mechanics the q daem algorithm combines the feature of annealing algorithms to reduce initialization sensitivity and that of q em algorithms to achieve fast convergence the q em algorithm is a one parameter generalized em algorithm that has been previously proposed by the authors by interpreting the em algorithm via likelihood lower bound maximization we build the fundamental interconnections among daem q daem and statistical mechanics to illustrate the benefits of using q daem we investigate two applications finite mixture model estimation in data clustering and joint channel estimation and data detection in communication systems where we show that the q daem algorithm achieves superior performance over the em algorithm for both applications

genetic based em algorithm for learning gaussian mixture models we propose a genetic based expectation maximization ga em algorithm for learning gaussian mixture models from multivariate data this algorithm is capable of selecting the number of components of the model using the minimum description length mdl criterion our approach benefits from the properties of genetic algorithms ga and the em algorithm by combination of both into a single procedure the population based stochastic search of the ga explores the search space more thoroughly than the em method therefore our algorithm enables escaping from local optimal solutions since the algorithm becomes less sensitive to its initialization the ga em algorithm is elitist which maintains the monotonic convergence property of the em algorithm the experiments on simulated and real data show that the ga em outperforms the em method since 1 we have obtained a better mdl score while using exactly the same termination condition for both algorithms 2 our approach identifies the number of components which were used to generate the underlying data more often than the em algorithm

discrete and continuous time probabilistic models and algorithms for inferring neuronal up and down states up and down states the periodic fluctuations between increased and decreased spiking activity of a neuronal population are a fundamental feature of cortical circuits understanding up down state dynamics is important for understanding how these circuits represent and transmit information in the brain to date limited work has been done on characterizing the stochastic properties of up down state dynamics we present a set of markov and semi markov discrete and continuous time probability models for estimating up and down states from multiunit neural spiking activity we model multiunit neural spiking activity as a stochastic point process modulated by the hidden up and down states and the ensemble spiking history we estimate jointly the hidden states and the model parameters by maximum likelihood using an expectation maximization em algorithm and a monte carlo em algorithm that uses reversible jump markov chain monte carlo sampling in the e step we apply our models and algorithms in the analysis of both simulated multiunit spiking activity and actual multi unit spiking activity recorded from primary somatosensory cortex in a behaving rat during slow wave sleep our approach provides a statistical characterization of up down state dynamics that can serve as a basis for verifying and refining mechanistic descriptions of this process

image denoising using mixtures of projected gaussian scale mixtures we propose a new statistical model for image restoration in which neighborhoods of wavelet subbands are modeled by a discrete mixture of linear projected gaussian scale mixtures mpgsm in each projection a lower dimensional approximation of the local neighborhood is obtained thereby modeling the strongest correlations in that neighborhood the model is a generalization of the recently developed mixture of gsm mgsm model that offers a significant improvement both in psnr and visually compared to the current state of the art wavelet techniques however the computation cost is very high which hampers its use for practical purposes we present a fast em algorithm that takes advantage of the projection bases to speed up the algorithm the results show that when projecting on a fixed data independent basis even computational advantages with a limited loss of psnr can be obtained with respect to the bls gsm denoising method while data dependent bases of principle components offer a higher denoising performance both visually and in psnr compared to the current wavelet based state of the art denoising methods

two rational recursive sequences two sufficient conditions are obtained for the global asymptotic stability of the following two rational difference equations x n 1 x nx n 2 ax n x n 2 n 0 1 2 and x n 1 x n 1x n 2 ax n 1 x n 2 n 0 1 2 where a 0 and the initial values x 2 x 1 x 0 0

feature selection based on the approximation of class densities by finite mixtures of special type abstract a new method of feature selection based on the approximation of class conditional densities by a mixture of parameterized densities of a special type suitable especially for multimodal data is presented no search procedure is needed when using the proposed method its performance is tested both on real simulated data

feature representation and discrimination based on gaussian mixture model probability densities practices and algorithms this study promotes the use of statistical methods in specific classification tasks since statistical methods have certain advantages which advocate their use in pattern recognition one central problem in statistical methods is estimation of class conditional probability density functions based on examples in a training set in this study maximum likelihood estimation methods for gaussian mixture models are reviewed and discussed from a practical point of view in addition good practices for utilizing probability densities in feature classification and selection are discussed for bayesian and more importantly for non bayesian tasks as a result the use of confidence information in the classification is proposed and a method for confidence estimation is presented the propositions are tested experimentally

unsupervised learning of finite mixture models this paper proposes an unsupervised algorithm for learning a finite mixture model from multivariate data the adjective unsupervised is justified by two properties of the algorithm 1 it is capable of selecting the number of components and 2 unlike the standard expectation maximization em algorithm it does not require careful initialization the proposed method also avoids another drawback of em for mixture fitting the possibility of convergence toward a singular estimate at the boundary of the parameter space the novelty of our approach is that we do not use a model selection criterion to choose one among a set of preestimated candidate models instead we seamlessly integrate estimation and model selection in a single algorithm our technique can be applied to any type of parametric mixture model for which it is possible to write an em algorithm in this paper we illustrate it with experiments involving gaussian mixtures these experiments testify for the good performance of our approach

assessing a mixture model for clustering with the integrated completed likelihood we propose an assessing method of mixture model in a cluster analysis setting with integrated completed likelihood for this purpose the observed data are assigned to unknown clusters using a maximum a posteriori operator then the integrated completed likelihood icl is approximated using the bayesian information criterion bic numerical experiments on simulated and real data of the resulting icl criterion show that it performs well both for choosing a mixture model and a relevant number of clusters in particular icl appears to be more robust than bic to violation of some of the mixture model assumptions and it can select a number of dusters leading to a sensible partitioning of the data

the geometry of mixture likelihoods a general theory in this paper certain fundamental properties of the maximum likelihood estimator of a mixing distribution are shown to be geometric properties of the likelihood set the existence support size likelihood equations and uniqueness of the estimator are revealed to be directly related to the properties of the convex hull of the likelihood set and the support hyperplanes of that hull it is shown using geometric techniques that the estimator exists under quite general conditions with a support size no larger than the number of distinct observations analysis of the convex dual of the likelihood set leads to a dual maximization problem a convergent algorithm is described the defining equations for the estimator are compared with the usual parametric likelihood equations for finite mixtures sufficient conditions for uniqueness are given part ii will deal with a special theory for exponential family mixtures

a simple nonparametric approach to estimating the distribution of random coefficients in structural models abstract we explore least squares and likelihood nonparametric mixtures estimators of the joint distribution of random coefficients in structural models the estimators fix a grid of heterogeneous parameters and estimate only the weights on the grid points an approach that is computationally attractive compared to alternative nonparametric estimators we provide conditions under which the estimated distribution function converges to the true distribution in the weak topology on the space of distributions we verify most of the consistency conditions for three discrete choice models we also derive the convergence rates of the least squares nonparametric mixtures estimator under additional restrictions we perform a monte carlo study on a dynamic programming model

deterministic annealing for density estimation by multivariate normal mixtures an approach to maximum likelihood density estimation by mixtures of multivariate normal distributions for large high dimensional data sets is presented conventionally that problem is tackled by notoriously unstable expectation maximization em algorithms we remove these instabilities by the introduction of soft constraints enabling deterministic annealing our developments are motivated by the proof that algorithmically stable fuzzy clustering methods that are derived from statistical physics analogs are special cases of em procedures

kullback proximal algorithms for maximum likelihood estimation accelerated algorithms for maximum likelihood image reconstruction are essential for emerging applications such as three dimensional 3 d tomography dynamic tomographic imaging and other high dimensional inverse problems in this paper we introduce and analyze a class of fast and stable sequential optimization methods for computing maximum likelihood estimates and study its convergence properties these methods are based on a proximal point algorithm implemented with the kullback liebler kl divergence between posterior densities of the complete data as a proximal penalty function when the proximal relaxation parameter is set to unity one obtains the classical expectation maximization em algorithm for a decreasing sequence of relaxation parameters relaxed versions of em are obtained which can have much faster asymptotic convergence without sacrifice of monotonicity we present an implementation of the algorithm using more s 1983 trust region update strategy for illustration the method is applied to a nonquadratic inverse problem with poisson distributed data

resolution based complexity control for gaussian mixture models in the domain of unsupervised learning mixtures of gaussians have become a popular tool for statistical modeling for this class of generative models we present a complexity control scheme which provides an effective means for avoiding the problem of overfitting usually encountered with unconstrained mixtures of gaussians in high dimensions according to some prespecified level of resolution as implied by a fixed variance noise model the scheme provides an automatic selection of the dimensionalities of some local signal subspaces by maximum likelihood estimation together with a resolution based control scheme for adjusting the number of mixture components we arrive at an incremental model refinement procedure within a common deterministic annealing framework which enables an efficient exploration of the model space the advantages of the resolution based framework are illustrated by experimental results on synthetic and high dimensional real world data

efficient greedy learning of gaussian mixture models this article concerns the greedy learning of gaussian mixtures in the greedy approach mixture components are inserted into the mixture one after the other we propose a heuristic for searching for the optimal component to insert in a randomized manner a set of candidate new components is generated for each of these candidates we find the locally optimal new component and insert it into the existing mixture the resulting algorithm resolves the sensitivity to initialization of state of the art methods like expectation maximization and has running time linear in the number of data points and quadratic in the final number of mixture components due to its greedy nature the algorithm can be particularly useful when the optimal number of mixture components is unknown experimental results comparing the proposed algorithm to other methods on density estimation and texture segmentation are provided

global asymptotic stability in a class of putnam type equations abstract in this paper we initiate the study of a class of putnam type equation of the form x n 1 a 1 x n a 2 x n 1 a 3 x n 2 x n 3 a 4 b 1 x n x n 1 b 2 x n 2 b 3 x n 3 b 4 n 0 1 2 where a 1 a 2 a 3 a 4 b 1 b 2 b 3 b 4 are positive constants with a 1 a 2 a 3 a 4 b 1 b 2 b 3 b 4 x 3 x 2 x 1 x 0 are positive numbers a sufficient condition is given for the global asymptotic stability of the equilibrium point c 1 of such equations

on the difference equation xn 1 pxn s xn tqxn s xn t abstract in this paper we address the difference equation x n 1 px n s x n t qx n s x n t n 0 1 with positive initial conditions we establish a sufficient condition for the global asymptotic stability of the positive equilibrium of this equation

qml based joint diagonalization of positive definite hermitian matrices in this paper a new algorithm for approximate joint diagonalization ajd of positive definite hermitian matrices is presented the ajd matrix which is assumed to be square non unitary is derived via minimization of a quasi maximum likelihood qml objective function this objective function coincides asymptotically with the maximum likelihood ml objective function hence enabling the proposed algorithm to asymptotically approach the ml estimation performance in the proposed method the rows of the ajd matrix are obtained independently in an iterative manner this feature enables direct estimation of full row rank rectangular ajd sub matrices under some mild assumptions convergence of the proposed algorithm is asymptotically guarantied such that the error norm corresponding to each row of the ajd matrix reduces significantly after the first iteration and the convergence is almost q super linear this property results rapid convergence which leads to low computational load in the proposed method the performance of the proposed algorithm is evaluated and compared to other state of the art algorithms for ajd and its practical use is demonstrated in the blind source separation and blind source extraction problems the results imply that under the assumptions of high signal to noise ratio and large amount of matrices the proposed algorithm is computationally efficient with performance similar to state of the art algorithms for ajd

approximate conditional mean particle filtering for linear nonlinear dynamic state space models we consider linear systems whose state parameters are separable into linear and nonlinear sets and evolve according to some known transition distribution and whose measurement noise is distributed according to a mixture of gaussians in doing so we propose a novel particle filter that addresses the optimal state estimation problem for the aforementioned class of systems the proposed filter referred to as the approximate conditional mean particle filter acm pf is a combination of the approximate conditional mean filter and the sequential importance sampling particle filter the algorithm development depends on approximating a mixture of gaussians distribution with a moment matched gaussian in the weight update recursion a condition indicating when this approximation is valid is given in order to evaluate the performance of the proposed algorithm we address the blind signal detection problem for an impulsive flat fading channel and the tracking of a maneuvering target in the presence of glint noise extensive computer simulations were carried out for computationally intensive implementations large number of particles the proposed algorithm offers performance that is comparable to other state of the art particle filtering algorithms in the scenario where computational horsepower is heavily constrained it is shown that the proposed algorithm offers the best performance amongst the considered algorithms for these specific examples

unsupervised texture segmentation using gabor filters abstract this paper presents a texture segmentation algorithm inspired by the multi channel filtering theory for visual information processing in the early stages of human visual system the channels are characterized by a bank of gabor filters that nearly uniformly covers the spatial frequency domain and a systematic filter selection scheme is proposed which is based on reconstruction of the input image from the filtered images texture features are obtained by subjecting each selected filtered image to a nonlinear transformation and computing a measure of energy in a window around each pixel a square error clustering algorithm is then used to integrate the feature images and produce a segmentation a simple procedure to incorporate spatial information in the clustering process is proposed a relative index is used to estimate the true number of texture categories

global behavior of solutions of the generalized lyness difference equations under quadratic perturbations we study the global asymptotic stability of solutions of the following two difference equations x n 2 x n a bx n 1 1 c x n 1 2 cx n 2 n 0 1 2 and x n 2 x n a bx n 1 d 1 c x n 1 2 d x n 1 cx n 2 n 0 1 2 where a 0 d 0 c 0 1 and the initial values x 0 x 1 0 bastien and rogalski 2004 showed if c 0 then there exist all the possible periods for the solutions of the above equations using an extension of the quasi lyapunov method we prove that the sequences generated by the first difference equation are globally asymptotically stable where 0 c 1 0 b 2 a 1 c and the initial values x 0 x 1 0 the global convergence property of the second difference equation has also been obtained for b 0 and 0 c 1 conversely the sequences generated by the above equations are increasing to for c 1 and b 0

qualitative properties for a fourth order rational difference equation abstract in this paper we use a method different from the known literature to investigate the qualitative properties of the following fourth order rational difference equation x n 1 x n x n 1 x n 3 x n x n 1 x n 3 a x n x n 1 x n x n 3 x n 1 x n 3 1 a n 0 1 2 where a 0 and the initial values x 3 x 2 x 1 x 0 0 the successive lengths of positive and negative semicycles of nontrivial solutions of the above equation is found to periodically occur that is 3 2 1 1 3 2 1 1 3 2 1 1 3 2 1 1 or 2 1 1 3 2 1 1 3 2 1 1 3 2 1 1 3 2 1 1 3 by using the rule the positive equilibrium point of the equation is verified to be globally asymptotically stable

on convergence properties of the em algorithm for gaussian mixtures we build up the mathematical connection between the expectation maximization em algorithm and gradient based approaches for maximum likelihood learning of finite gaussian mixtures we show that the em step in parameter space is obtained from the gradient via a projection matrix p and we provide an explicit expression for the matrix we then analyze the convergence of em in terms of special properties of p and provide new results analyzing the effect that p has on the likelihood surface based on these mathematical results we present a comparative discussion of the advantages and disadvantages of em and other algorithms for the learning of gaussian mixture models

mixture of spherical distributions for single view relighting we present a method for simultaneously estimating the illumination of a scene and the reflectance property of an object from single view images a single image or a small number of images taken from the same viewpoint we assume that the illumination consists of multiple point light sources and the shape of the object is known first we represent the illumination on the surface of a unit sphere as a finite mixture of von mises fisher distributions based on a novel spherical specular reflection model that well approximates the torrance sparrow reflection model next we estimate the parameters of this mixture model including the number of its component distributions and the standard deviation of them which correspond to the number of light sources and the surface roughness respectively finally using these results as the initial estimates we iteratively refine the estimates based on the original torrance sparrow reflection model the final estimates can be used to relight single view images such as altering the intensities and directions of the individual light sources the proposed method provides a unified framework based on directional statistics for simultaneously estimating the intensities and directions of an unknown number of light sources as well as the specular reflection parameter of the object in the scene

global stability and asymptotics of some classes of rational difference equations in this note we prove that the positive solutions of some classes of rational difference equations are globally asymptotically stable using a berg s result we also find asymptotics of some solutions of these equations

structure learning in conditional probability models via an entropic prior and parameter extinction we introduce an entropic prior for multinomial parameter estimation problems and solve for its maximum a posteriori map estimator the prior is a bias for maximally structured and minimally ambig

classification of fmri time series in a low dimensional subspace with a spatial prior we propose a new method for detecting activation in functional magnetic resonance imaging fmri data we project the fmri time series on a low dimensional subspace spanned by wavelet packets in order to create projections that are as non gaussian as possible our approach achieves two goals it reduces the dimensionality of the problem by explicitly constructing a sparse approximation to the dataset and it also creates meaningful clusters allowing the separation of the activated regions from the clutter formed by the background time series we use a mixture of gaussian densities to model the distribution of the wavelet packet coefficients we expect activated areas that are connected and impose a spatial prior in the form of a markov random field our approach was validated with in vivo data and realistic synthetic data where it outperformed a linear model equipped with the knowledge of the true hemodynamic response

maximum likelihood training of probabilistic neural networks a maximum likelihood method is presented for training probabilistic neural networks pnn s using a gaussian kernel or parzen window the proposed training algorithm enables general nonlinear discrimination and is a generalization of fisher s method for linear discrimination important features of maximum likelihood training for pnn s are 1 it economizes the well known parzen window estimator while preserving feedforward nn architecture 2 it utilizes class pooling to generalize classes represented by small training sets 3 it gives smooth discriminant boundaries that often are piece wise flat for statistical robustness 4 it is very fast computationally compared to backpropagation and 5 it is numerically stable the effectiveness of the proposed maximum likelihood training algorithm is assessed using nonparametric statistical methods to define tolerance intervals on pnn classification performance

absent data generating classifier for imbalanced class sizes we propose an algorithm for two class classification problems when the training data are imbalanced this means the number of training instances in one of the classes is so low that the conventional classification algorithms become ineffective in detecting the minority class we present a modification of the kernel fisher discriminant analysis such that the imbalanced nature of the problem is explicitly addressed in the new algorithm formulation the new algorithm exploits the properties of the existing minority points to learn the effects of other minority data points had they actually existed the algorithm proceeds iteratively by employing the learned properties and conditional sampling in such a way that it generates sufficient artificial data points for the minority set thus enhancing the detection probability of the minority class implementing the proposed method on a number of simulated and real data sets we show that our proposed method performs competitively compared to a set of alternative state of the art imbalanced classification algorithms

stability analysis and stabilization of discrete time non homogeneous semi markov jump linear systems a polytopic approach abstract this paper addresses the issues of stability analysis and stabilizing controller design for two classes of discrete time non homogeneous semi markov jump linear systems s mjlss the first class is concerned with the sojourn time probability mass functions that are independent of jump instants while the other considers the existence of probability mass functions of sojourn time depending on jump instants new techniques are developed respectively for both classes of systems by transforming their corresponding time varying semi markov kernels into more tractable forms based on the stochastic stability allowing for bounded sojourn time the derived stability and stabilization criteria containing the probability distribution information of random sojourn time can be numerically tested in order to reduce the possible conservatism of the obtained results a novel class of polytopic quadratic lyapunov function is constructed which depends on both the current mode and the elapsed time in the current mode finally the proposed control strategies are applied to an illustrative application of automotive electronic throttle valve to show the effectiveness of the theoretical results the results also reveal the significance of considering the inhomogeneity of s mjlss and the importance of our constructed lyapunov function

mixtures of probabilistic principal component analyzers principal component analysis pca is one of the most popular techniques for processing compressing and visualizing data although its effectiveness is limited by its global linearity while nonlinear variants of pca have been proposed an alternative paradigm is to capture data complexity by a combination of local linear pca projections however conventional pca does not correspond to a probability density and so there is no unique way to combine pca models therefore previous attempts to formulate mixture models for pca have been ad hoc to some extent in this article pca is formulated within a maximum likelihood framework based on a specific form of gaussian latent variable model this leads to a well defined mixture model for probabilistic principal component analyzers whose parameters can be determined using an expectationmaximization algorithm we discuss the advantages of this model in the context of clustering density modeling and local dimensionality reduction and we demonstrate its application to image compression and handwritten digit recognition

on the global asymptotic stability of the difference equation xn xn 1xn 2 xn 3 a xn 1 xn 2xn 3 a li and zhu applied mathematics and computation 150 2004 481 studied the difference equationx n x n 1x n 2 x n 3 ax n 1 x n 2x n 3 a n 1 2 where a 0 they claimed that the unique positive equilibrium of the equation is globally asymptotically stable unfortunately there is a subtle mistake in their proof in this paper we present a correct proof for this claim

gaussian mixture modeling by exploiting the mahalanobis distance in this paper the expectation maximization em algorithm for gaussian mixture modeling is improved via three statistical tests the first test is a multivariate normality criterion based on the mahalanobis distance of a sample measurement vector from a certain gaussian component center the first test is used in order to derive a decision whether to split a component into another two or not the second test is a central tendency criterion based on the observation that multivariate kurtosis becomes large if the component to be split is a mixture of two or more underlying gaussian sources with common centers if the common center hypothesis is true the component is split into two new components and their centers are initialized by the center of the old component candidate for splitting otherwise the splitting is accomplished by a discriminant derived by the third test this test is based on marginal cumulative distribution functions experimental results are presented against seven other em variants both on artificially generated data sets and real ones the experimental results demonstrate that the proposed em variant has an increased capability to find the underlying model while maintaining a low execution time

a presic type contractive condition and its applications abstract we study discrete dynamic systems in a complete metric space m d defined by mappings which satisfy presic type contractive conditions their counterparts in an ordered banach space are investigated and applied to solve the global asymptotic stability of the equilibriums of some nonlinear difference equations

a byy scale incremental em algorithm for gaussian mixture learning abstract gaussian mixture model has been used extensively in the fields of information processing and data analysis however its model selection i e the selection of number of components or gaussians in the mixture is still a difficult problem fortunately the new established bayesian ying yang byy harmony function provides an efficient criterion for the model selection of gaussian mixture with a set of sample data in this paper we propose a byy scale incremental em algorithm for gaussian mixture learning via a component split rule to increase the byy harmony function incrementally particularly starting from two components and adding one component sequentially via the split rule after each em procedure until a maximum number of components the algorithm increases the scale of the mixture incrementally and leads to the maximization of the byy harmony function together with the correct model selection and a good parameter estimation of the gaussian mixture it is demonstrated well by the simulation experiments that this byy scale incremental em algorithm can make both model selection and parameter estimation efficiently for gaussian mixture modeling moreover the byy scale incremental em algorithm is successfully applied to two real life data sets including iris data classification and unsupervised color image segmentation

global stability of a rational symmetric difference equation motivated by ideas from papers c cinar i yalcinkaya s stevic a note on global asymptotic stability of a family of rational equations rostock math kolloq 59 2004 41 49 and s stevic global stability and asymptotics of some classes of rational difference equations j math anal appl 316 2006 60 68 here we confirm a conjecture on a rational symmetric difference equation from the paper k berenhaut j foley s stevic the global attractivity of the rational difference equation y n y n k y n m1 y n ky n m appl math lett 20 2007 54 58

on the recursive sequence xn 1 a i lkxn 2i 1b c i lk 1xn 2i abstract the aim of this work is to investigate the global stability periodic nature oscillation and the boundedness of solutions of the difference equation x n 1 a i l k x n 2 i 1 b c i l k 1 x n 2 i n 0 1 where a b c are nonnegative real numbers and l k are nonnegative integers l k we discuss the existence of unbounded solutions under certain conditions when l 0

mimo ar system identification and blind source separation for gmm distributed sources the problem of blind source separation bss and system identification for multiple input multiple output mimo auto regressive ar mixtures is addressed in this paper two new time domain algorithms for system identification and bss are proposed based on the gaussian mixture model gmm for sources distribution both algorithms are based on the generalized expectation maximization gem method for joint estimation of the mimo ar model parameters and the gmm parameters of the sources the first algorithm is derived under the assumption of unstructured input signal statistics while the second algorithm incorporates the prior knowledge about the structure of the input signal statistics due to the statistically independent source assumption these methods are tested via simulations using synthetic and audio signals the system identification performances are tested by comparison between the state transition matrix estimation using the proposed algorithms and the well known multidimensional yule walker solution followed by an instantaneous bss method the results show that the proposed algorithms outperform the yule walker based approach the bss performances were compared to other convolutive bss methods the results show that the proposed algorithms achieve higher signal to interference ratio sir compared to the other tested methods

trust tech based expectation maximization for learning finite mixture models the expectation maximization em algorithm is widely used for learning finite mixture models despite its greedy nature most popular model based clustering techniques might yield poor clusters if the parameters are not initialized properly to reduce the sensitivity of initial points a novel algorithm for learning mixture models from multivariate data is introduced in this paper the proposed algorithm takes advantage of trust tech transformation under stability retaining equilibria characterization to compute neighborhood local maxima on the likelihood surface using stability regions basically our method coalesces the advantages of the traditional em with that of the dynamic and geometric characteristics of the stability regions of the corresponding nonlinear dynamical system of the log likelihood function two phases namely the em phase and the stability region phase are repeated alternatively in the parameter space to achieve local maxima with improved likelihood values the em phase obtains the local maximum of the likelihood function and the stability region phase helps to escape out of the local maximum by moving toward the neighboring stability regions though applied to gaussian mixtures in this paper our technique can be easily generalized to any other parametric finite mixture model the algorithm has been tested on both synthetic and real data sets and the improvements in the performance compared to other approaches are demonstrated the robustness with respect to initialization is also illustrated experimentally

global asymptotic stability for two recursive difference equations two sufficient conditions are obtained for the global asymptotic stability of the following two recursive difference equationsx n 1 x nx n 1 x n 2 ax n x n 1x n 2 a n 0 1 2 andx n 1 x n 1 x nx n 2 ax nx n 1 x n 2 a n 0 1 2 where a 0 and the initial values x 2 x 1 x 0 0 some known results are included

newtonian clustering an approach based on molecular dynamics and global optimization given a data set a dynamical procedure is applied to the data points in order to shrink and separate possibly overlapping clusters namely newton s equations of motion are employed to concentrate the data points around their cluster centers using an attractive potential constructed specially for this purpose during this process important information is gathered concerning the spread of each cluster in succession this information is used to create an objective function that maps each cluster to a local maximum global optimization is then used to retrieve the positions of the maxima that correspond to the locations of the cluster centers further refinement is achieved by applying the em algorithm to a gaussian mixture model whose construction and initialization is based on the acquired information to assess the effectiveness of our method we have conducted experiments on a plethora of benchmark data sets in addition we have compared its performance against four clustering techniques that are well established in the literature

practical bayesian density estimation using mixtures of normals abstract mixtures of normals provide a flexible model for estimating densities in a bayesian framework there are some difficulties with this model however first standard reference priors yield improper posteriors second the posterior for the number of components in the mixture is not well defined if the reference prior is used third posterior simulation does not provide a direct estimate of the posterior for the number of components we present some practical methods for coping with these problems finally we give some results on the consistency of the method when the maximum number of components is allowed to grow with the sample size

direct calculation of the information matrix via the em algorithm summary a simple explicit formula is provided for the matrix of second derivatives of the observed data log likelihood in terms of derivatives of the criterion function the conditional expectation of the complete data log likelihood given the observed data invoked by the em algorithm

modeling the manifolds of images of handwritten digits this paper describes two new methods for modeling the manifolds of digitized images of handwritten digits the models allow a priori information about the structure of the manifolds to be combined with empirical data accurate modeling of the manifolds allows digits to be discriminated using the relative probability densities under the alternative models one of the methods is grounded in principal components analysis the other in factor analysis both methods are based on locally linear low dimensional approximations to the underlying data manifold links with other methods that model the manifold are discussed

direct calculation of the information matrix via the em a simple explicit formula is provided for the matrix of second derivatives of the observed data log likelihood in terms of derivatives of the criterion function conditional expectation of the complete data log likelihood given the observed data invoked by the em algorithm

on the recursive sequence x n 1 aπ k i l x n 2 j 1 b c π k l i l x n 2 i the aim of this work is to investigate the global stability periodic nature oscillation and the boundedness of solutions of the difference equation x n email protected i lkx n 2 i email protected i lk 1x n 2 i n 0 1 where a b c are nonnegative real numbers and l k are nonnegative integers l

dichotomy of a perturbed lyness difference equation abstract we investigate in this paper the perturbed lyness difference equation bx n 2 x n x n 1 x n 2 n 0 1 2 where b are arbitrary positive real numbers and 0 and the initial values x 1 x 0 0 which is a generalization of the lyness difference equation x n 2 x n a x n 1 extensively studied it is known that for the lyness difference equation i e the perturbed lyness difference equation with 0 all its solutions are periodic or strictly oscillatory however one here finds that this perturbed lyness difference equation possesses the following dichotomy for 0 b all of its solutions are globally asymptotically stable for b all the sequences generated by it converge to we hence find that there exists the essential difference for the properties of solutions between the unperturbed lyness difference equation and the perturbed lyness difference equation

a fast improvement to the em algorithm on its own terms the em algorithm is a numerical technique for the evaluation of maximum likelihood estimates for parameters describing incomplete data models it is easy to apply in many problems and is stable but slow the algorithm fails to provide a consistent estimator of the standard errors of the maximum likelihood estimates unless the additional analysis required by the louis method is performed newton type or other gradient methods are faster and provide error estimates but tend to be unstable and require the analytical evaluation of likelihoods to derive expressions for the score function and at least approximations to the fisher information matrix the purpose of this paper is to expand on a result by fisher that permits a unification of em methodology and newton methods the evaluation of the individual observation by observation score functions of the incomplete data is a byproduct of the application of the e step of the em algorithm once these become available the fisher information matrix may be consistently estimated and the m step may be replaced by a fast newton type step

the rule of semicycle and global asymptotic stability for a fourth order rational difference equation in this paper the rule for the lengths of positive and negative semicycles of nontrivial solutions of the following fourth order rational difference equation x n 1 x n bx n 2 x n 3 b ax n b x n 2x n 3 b a n 0 1 2 where a b e 0 and the initial values x 3 x 2 x 1 x 0 e 0 to successively occur is found to be 4 3 1 2 2 1 1 1 4 3 1 2 2 1 1 1 4 3 1 2 2 1 1 1 by which the positive equilibrium point of the equation is verified to be globally asymptotically stable

on the recursive sequencexn 1 α xn 1 xn abstract we study the global stability the boundedness character and the periodic nature of the positive solutions of the difference equationxn 1 xn 1 xn where 0 and where the initial conditionsx 1andx0are arbitrary positive real numbers

on bayesian analysis of mixtures with an unknown number of components with discussion new methodology for fully bayesian mixture analysis is developed making use of reversible jump markov chain monte carlo methods that are capable of jumping between the parameter subspaces corresponding to different numbers of components in the mixture a sample from the full joint distribution of all unknown variables is thereby generated and this can be used as a basis for a thorough presentation of many aspects of the posterior distribution the methodology is applied here to the analysis of univariate normal mixtures using a hierarchical prior model that offers an approach to dealing with weak prior information while avoiding the mathematical pitfalls of using improper priors in the mixture context

on the difference equation xn p xn sqxn t xn s abstract we investigate the persistence and global attractivity of the difference equation x n p x n s qx n t x n s n 1 2 3 with positive parameters and positive initial conditions our results not only include some previously known results but apply to some difference equations that have not been investigated so far

on the dynamics of the recursive sequence xn 1 xn 1β γxn 22xn 4 γxn 2xn 42 in this paper we investigate the global behavior of the difference equation x n 1 x n email protected email protected n 2 2x n email protected n 2x n 4 2 n 0 1 with positive parameters and non negative initial conditions

training faster by separating modes of variation in batch normalized models batch normalization bn is essential to effectively train state of the art deep convolutional neural networks cnn it normalizes the layer outputs during training using the statistics of each mini batch bn accelerates training procedure by allowing to safely utilize large learning rates and alleviates the need for careful initialization of the parameters in this work we study bn from the viewpoint of fisher kernels that arise from generative probability models we show that assuming samples within a mini batch are from the same probability density function then bn is identical to the fisher vector of a gaussian distribution that means batch normalizing transform can be explained in terms of kernels that naturally emerge from the probability density function that models the generative process of the underlying data distribution consequently it promises higher discrimination power for the batch normalized mini batch however given the rectifying non linearities employed in cnn architectures distribution of the layer outputs show an asymmetric characteristic therefore in order for bn to fully benefit from the aforementioned properties we propose approximating underlying data distribution not with one but a mixture of gaussian densities deriving fisher vector for a gaussian mixture model gmm reveals that batch normalization can be improved by independently normalizing with respect to the statistics of disentangled sub populations we refer to our proposed soft piecewise version of batch normalization as mixture normalization mn through extensive set of experiments on cifar 10 and cifar 100 using both a 5 layers deep cnn and modern inception v3 architecture we show that mixture normalization reduces required number of gradient updates to reach the maximum test accuracy of the batch normalized model by sim 31 47 31 47 across a variety of training scenarios replacing even a few bn modules with mn in the 48 layers deep inception v3 architecture is sufficient to not only obtain considerable training acceleration but also better final test accuracy we show that similar observations are valid for 40 and 100 layers deep densenet architectures as well we complement our study by evaluating the application of mixture normalization to the generative adversarial networks gans where mode collapse hinders the training process we solely replace a few batch normalization layers in the generator with our proposed mixture normalization our experiments using deep convolutional gan dcgan on cifar 10 show that mixture normalized dcgan not only provides an acceleration of sim 58 58 but also reaches lower better frechet inception distance fid of 33 35 compared to 37 56 of its batch normalized counterpart

a hidden seasonal switching model for high resolution breakpoint rainfall data 1 a nonhomogeneous hidden semi markov model nhsmm for breakpoint rainfall data is proposed which extends the homogeneous hidden semi markov model hsmm of sansom and thomson 2001 to incorporate stochastic seasonality the nhsmm model is able to switch seasons at times that are earlier or later than expected and in this way is able to explain additional seasonal variability due to varying length seasons the model s hidden rainfall states align with precipitation mechanisms that are seasonally invariant but the state dynamics are assumed to vary with season recursions for constructing the likelihood are developed and the em algorithm used to fit the parameters of the model an application of the model to breakpoint rainfall measurements from invercargill new zealand is discussed and the results of fitting a number of different nhsmms are compared to those from fitting the non seasonal homogeneous hsmm

hallucinating optimal high dimensional subspaces abstract linear subspace representations of appearance variation are pervasive in computer vision this paper addresses the problem of robustly matching such subspaces computing the similarity between them when they are used to describe the scope of variations within sets of images of different possibly greatly so scales a naive solution of projecting the low scale subspace into the high scale image space is described first and subsequently shown to be inadequate especially at large scale discrepancies a successful approach is proposed instead it consists of i an interpolated projection of the low scale subspace into the high scale space which is followed by ii a rotation of this initial estimate within the bounds of the imposed downsampling constraint the optimal rotation is found in the closed form which best aligns the high scale reconstruction of the low scale subspace with the reference it is compared to the method is evaluated on the problem of matching sets of i face appearances under varying illumination and ii object appearances under varying viewpoint using two large data sets in comparison to the naive matching the proposed algorithm is shown to greatly increase the separation of between class and within class similarities as well as produce far more meaningful modes of common appearance on which the match score is based

information ratios for validating mixture analyses abstract determining the number of components in a mixture of distributions is an important but difficult problem this article introduces a procedure called minimum information ratio estimation and validation mirev which is based on a ratio of fisher information matrices the smallest eigenvalue of the information ratio matrix is used to determine the number of components a measure of uncertainty may be obtained using a bootstrap technique simulations illustrate the effectiveness of the procedure for mixtures of exponential families an expression for the observed information ratio matrix provides insight to the success of the procedure cluster analysis attempts to identify and characterize subpopulations believed to be present in a population a wide variety of methods are available including criterion optimization hierarchical methods and various heuristic methods criterion optimization techniques such as mixture analysis fuzzy clustering and partitioning methods are popular because they

blind separation of independent sources using gaussian mixture model in this paper two novel algorithms for blind separation of noiseless instantaneous linear mixture of independent sources are presented the proposed algorithms exploit nongaussianity of the independent sources by modeling their distribution using the gaussian mixture model gmm the first proposed method is based on the maximum likelihood ml estimator according to this method the sensors distribution parameters are estimated via the expectation maximization em algorithm for gmm parameter estimation and the separation matrix is estimated by applying nonorthogonal joint diagonalization of the estimated gmm covariance matrices the second proposed method is also a ml based approach according to this method the distribution parameters of the prewhitened sensors are estimated via the em algorithm for gmm parameter estimation and a unitary separation matrix is estimated by applying orthogonal joint diagonalization of the estimated gmm covariance matrices it is shown that estimation of the sensors distribution parameters amounts to obtaining a tight lower bound on the log likelihood of the separation matrix and that the joint diagonalization of the estimated gmm covariance matrices amounts to maximization of the obtained tight lower bound simulations demonstrate that the proposed methods outperform state of the art blind source separation techniques in terms of interference to signal ratio