non negative matrix factorization ill posedness and a geometric algorithm non negative matrix factorization nmf has been proposed as a mathematical tool for identifying the components of a dataset however popular nmf algorithms tend to operate slowly and do not always identify the components which are most representative of the data in this paper an alternative algorithm for performing nmf is developed using the geometry of the problem the computational costs of the algorithm are explored and it is shown to successfully identify the components of a simulated dataset the development of the geometric algorithm framework illustrates the ill posedness of the nmf problem and suggests that nmf is not sufficiently constrained to be applied successfully outside of a particular class of problems

blind spectral unmixing based on sparse nonnegative matrix factorization nonnegative matrix factorization nmf is a widely used method for blind spectral unmixing su which aims at obtaining the endmembers and corresponding fractional abundances knowing only the collected mixing spectral data it is noted that the abundance may be sparse i e the endmembers may be with sparse distributions and sparse nmf tends to lead to a unique result so it is intuitive and meaningful to constrain nmf with sparseness for solving su however due to the abundance sum to one constraint in su the traditional sparseness measured by l0 l1 norm is not an effective constraint any more a novel measure termed as s measure of sparseness using higher order norms of the signal vector is proposed in this paper it features the physical significance by using the s measure constraint smc a gradient based sparse nmf algorithm termed as nmf smc is proposed for solving the su problem where the learning rate is adaptively selected and the endmembers and abundances are simultaneously estimated in the proposed nmf smc there is no pure index assumption and no need to know the exact sparseness degree of the abundance in prior yet it does not require the preprocessing of dimension reduction in which some useful information may be lost experiments based on synthetic mixtures and real world images collected by aviris and hydice sensors are performed to evaluate the validity of the proposed method

finding imaging patterns of structural covariance via non negative matrix factorization in this paper we investigate the use of non negative matrix factorization nnmf for the analysis of structural neuroimaging data the goal is to identify the brain regions that co vary across individuals in a consistent way hence potentially being part of underlying brain networks or otherwise influenced by underlying common mechanisms such as genetics and pathologies nnmf offers a directly data driven way of extracting relatively localized co varying structural regions thereby transcending limitations of principal component analysis pca independent component analysis ica and other related methods that tend to produce dispersed components of positive and negative loadings in particular leveraging upon the well known ability of nnmf to produce parts based representations of image data we derive decompositions that partition the brain into regions that vary in consistent ways across individuals importantly these decompositions achieve dimensionality reduction via highly interpretable ways and generalize well to new data as shown via split sample experiments we empirically validate nnmf in two data sets i a diffusion tensor dt mouse brain development study and ii a structural magnetic resonance smr study of human brain aging we demonstrate the ability of nnmf to produce sparse parts based representations of the data at various resolutions these representations seem to follow what we know about the underlying functional organization of the brain and also capture some pathological processes moreover we show that these low dimensional representations favorably compare to descriptions obtained with more commonly used matrix factorization methods like pca and ica

using underapproximations for sparse nonnegative matrix factorization nonnegative matrix factorization consists in approximately factorizing a nonnegative data matrix by the product of two low rank nonnegative matrices it has been successfully applied as a data analysis technique in numerous domains e g text mining image processing microarray data analysis collaborative filtering etc we introduce a novel approach to solve nmf problems based on the use of an underapproximation technique and show its effectiveness to obtain sparse solutions this approach based on lagrangian relaxation allows the resolution of nmf problems in a recursive fashion we also prove that the underapproximation problem is np hard for any fixed factorization rank using a reduction of the maximum edge biclique problem in bipartite graphs we test two variants of our underapproximation approach on several standard image datasets and show that they provide sparse part based representations with low reconstruction error our results are comparable and sometimes superior to those obtained by two standard sparse nonnegative matrix factorization techniques

clustering based initialization for non negative matrix factorization non negative matrix factorization nmf is an unsupervised learning algorithm that can extract parts from visual data the goal of this technique is to find intuitive basis such that training examples can be faithfully reconstructed using linear combination of basis images which are restricted to non negative values thus nmf basis images can be understood as localized features that correspond better with intuitive notions of parts of images however there has been few systematic study to explore various methods for initialization of nmf algorithm which is crucial for the performance of nmf algorithm in data analysis in this paper we discuss a structured nmf initialization scheme based on the clustering method comparing with the random initialization in common use our method achieved faster convergence while maintaining the data structure and also obtained good result for the face recognition task furthermore we also proposed to use a normalized aic incorporated with our nmf initialization for rank selection of traditional nmf at the cost of much less computational load while obtaining a good performance in face recognition

nonnegative matrix factorization for spectral data analysis data analysis is pervasive throughout business engineering and science very often the data to be analyzed is nonnegative and it is often preferable to take this constraint into account in the analysis process here we are concerned with the application of analyzing data obtained using astronomical spectrometers which provide spectral data which is inherently nonnegative the identification and classification of space objects that cannot be imaged in the normal way with telescopes is an important but difficult problem for tracking thousands of objects including satellites rocket bodies debris and asteroids in orbit around the earth in this paper we develop an effective nonnegative matrix factorization algorithm with novel smoothness constraints for unmixing spectral reflectance data for space object identification and classification purposes promising numerical results are presented using laboratory and simulated datasets

fast nonnegative matrix factorization an active set like method and comparisons nonnegative matrix factorization nmf is a dimension reduction method that has been widely used for numerous applications including text mining computer vision pattern discovery and bioinformatics a mathematical formulation for nmf appears as a nonconvex optimization problem and various types of algorithms have been devised to solve the problem the alternating nonnegative least squares anls framework is a block coordinate descent approach for solving nmf which was recently shown to be theoretically sound and empirically efficient in this paper we present a novel algorithm for nmf based on the anls framework our new algorithm builds upon the block principal pivoting method for the nonnegativity constrained least squares problem that overcomes a limitation of the active set method we introduce ideas that efficiently extend the block principal pivoting method within the context of nmf computation our algorithm inherits the convergence property of the anls framework and can easily be extended to other constrained nmf formulations extensive computational comparisons using data sets that are from real life applications as well as those artificially generated show that the proposed algorithm provides state of the art performance in terms of computational speed

exploiting discriminant information in nonnegative matrix factorization with application to frontal face verification in this paper two supervised methods for enhancing the classification accuracy of the nonnegative matrix factorization nmf algorithm are presented the idea is to extend the nmf algorithm in order to extract features that enforce not only the spatial locality but also the separability between classes in a discriminant manner the first method employs discriminant analysis in the features derived from nmf in this way a two phase discriminant feature extraction procedure is implemented namely nmf plus linear discriminant analysis lda the second method incorporates the discriminant constraints inside the nmf decomposition thus a decomposition of a face to its discriminant parts is obtained and new update rules for both the weights and the basis images are derived the introduced methods have been applied to the problem of frontal face verification using the well known xm2vts database both methods greatly enhance the performance of nmf for frontal face verification

optshrink an algorithm for improved low rank signal matrix denoising by optimal data driven singular value shrinkage the truncated singular value decomposition of the measurement matrix is the optimal solution to the representation problem of how to best approximate a noisy measurement matrix using a low rank matrix here we consider the unobservable denoising problem of how to best approximate a low rank signal matrix buried in noise by optimal re weighting of the singular vectors of the measurement matrix we exploit recent results from random matrix theory to exactly characterize the large matrix limit of the optimal weighting coefficients and show that they can be computed directly from data for a large class of noise models that includes the independent identically distributed gaussian noise case our analysis brings into sharp focus the shrinkage and thresholding form of the optimal weights the nonconvex nature of the associated shrinkage function on the singular values and explains why matrix regularization via singular value thresholding with convex penalty functions such as the nuclear norm will always be suboptimal we validate our theoretical predictions with numerical simulations develop an implementable algorithm optshrink that realizes the predicted performance gains and show how our methods can be used to improve estimation in the setting where the measured matrix has missing entries

linear and nonlinear projective nonnegative matrix factorization a variant of nonnegative matrix factorization nmf which was proposed earlier is analyzed here it is called projective nonnegative matrix factorization pnmf the new method approximately factorizes a projection matrix minimizing the reconstruction error into a positive low rank matrix and its transpose the dissimilarity between the original data matrix and its approximation can be measured by the frobenius matrix norm or the modified kullback leibler divergence both measures are minimized by multiplicative update rules whose convergence is proven for the first time enforcing orthonormality to the basic objective is shown to lead to an even more efficient update rule which is also readily extended to nonlinear cases the formulation of the pnmf objective is shown to be connected to a variety of existing nmf methods and clustering approaches in addition the derivation using lagrangian multipliers reveals the relation between reconstruction and sparseness for kernel principal component analysis pca with the binary constraint useful in graph partitioning problems the nonlinear kernel pnmf provides a good approximation which outperforms an existing discretization approach empirical study on three real world databases shows that pnmf can achieve the best or close to the best in clustering the proposed algorithm runs more efficiently than the compared nmf methods especially for high dimensional data moreover contrary to the basic nmf the trained projection matrix can be readily used for newly coming samples and demonstrates good generalization

convex and semi nonnegative matrix factorizations we present several new variations on the theme of nonnegative matrix factorization nmf considering factorizations of the form x fgt we focus on algorithms in which g is restricted to containing nonnegative entries but allowing the data matrix x to have mixed signs thus extending the applicable range of nmf methods we also consider algorithms in which the basis vectors of f are constrained to be convex combinations of the data points this is used for a kernel extension of nmf we provide algorithms for computing these new factorizations and we provide supporting theoretical analysis we also analyze the relationships between our algorithms and clustering algorithms and consider the implications for sparseness of solutions finally we present experimental results that explore the properties of these new methods

statistical physics of machinery forgotten middle ground abstract statistical physics long concentrated on the equilibrium state in the thermodynamic limit and small deviations from that state machinery is spatially inhomogeneous non linear and operates far from equilibrium after our field expanded to face these problems they were soon displaced by still newer fashions it is symptomatic of this situation that the real resolution of maxwell s demon in the 1980s passed almost unnoticed we revisit a number of subjects emphasizing the need for concern with the detailed kinetics of the system at hand and the difficulties posed by magic short cuts the maximum entropy formalism minimal entropy production and the difficulties in identifying the most likely state among competing states of local stability will be taken up the blowtorch theorem demonstrates that relative stability cannot in general be decided by examining only the neighborhood of the competing states the kinetics along the pathway connecting the states must be taken into account the latter point will be connected to biological evolution circulating currents set up by the simultaneous presence of force fields and temperature gradients will be described

updating downdating the nonnegative matrix factorization the non negative matrix factorization nnmf is a recent numerical tool that given a non negative data matrix tries to obtain its factorization as the approximate product of two nonnegative matrices nowadays this factorization is being used in many science fields in some of these fields real time computation of the nnmf is required in some scenarios all data is not initially available and when new data as new rows or columns becomes available the nnmf must be recomputed recomputing the whole factorization every time is very costly and not suitable for real time applications in this paper we propose several algorithms to update the nnmf factorization taking advantage of the previously computed factorizations with similar error and lower computational cost

efficient nonnegative matrix factorization via projected newton method nonnegative matrix factorization nmf is a popular decomposition technique in pattern analysis document clustering image processing and related fields in this paper we propose a fast nmf algorithm via projected newton method pnm first we propose pnm to efficiently solve a nonnegative least squares problem which achieves a quadratic convergence rate under appropriate assumptions second in the framework of an alternating optimization method we adopt pnm as an essential subroutine to efficiently solve the nmf problem moreover by exploiting the low rank assumption of nmf we make pnm very suitable for solving nmf efficiently empirical studies on both synthetic and real world text and image data demonstrate that pnm is quite efficient to solve nmf compared with several state of the art algorithms

learning from hidden traits joint factor analysis and latent clustering dimensionality reduction techniques play an essential role in data analytics signal processing and machine learning dimensionality reduction is usually performed in a preprocessing stage that is separate from subsequent data analysis such as clustering or classification finding reduced dimension representations that are well suited for the intended task is more appealing this paper proposes a joint factor analysis and latent clustering framework which aims at learning cluster aware low dimensional representations of matrix and tensor data the proposed approach leverages matrix and tensor factorization models that produce essentially unique latent representations of the data to unravel latent cluster structure which is otherwise obscured because of the freedom to apply an oblique transformation in latent space at the same time latent cluster structure is used as prior information to enhance the performance of factorization specific contributions include several custom built problem formulations corresponding algorithms and discussion of associated convergence properties besides extensive simulations real world datasets such as reuters document data and mnist image data are also employed to showcase the effectiveness of the proposed approaches

image processing using newton based algorithm of nonnegative matrix factorization in this paper we propose a newton based algorithm for nonnegative matrix factorization in image processing we employ the new algorithm to three real world databases extensive numerical results show the feasibility and validity of the proposed algorithm

on the complexity of nonnegative matrix factorization nonnegative matrix factorization nmf has become a prominent technique for the analysis of image databases text databases and other information retrieval and clustering applications the problem is most naturally posed as continuous optimization in this report we define an exact version of nmf then we establish several results about exact nmf i that it is equivalent to a problem in polyhedral combinatorics ii that it is np hard and iii that a polynomial time local search heuristic exists

anderson acceleration for fixed point iterations this paper concerns an acceleration method for fixed point iterations that originated in work of d g anderson j assoc comput mach 12 1965 pp 547 560 which we accordingly call anderson acceleration here this method has enjoyed considerable success and wide usage in electronic structure computations where it is known as anderson mixing however it seems to have been untried or underexploited in many other important applications moreover while other acceleration methods have been extensively studied by the mathematics and numerical analysis communities this method has received relatively little attention from these communities over the years a recent paper by h fang and y saad numer linear algebra appl 16 2009 pp 197 221 has clarified a remarkable relationship of anderson acceleration to quasi newton secant updating methods and extended it to define a broader anderson family of acceleration methods in this paper our goals are to shed additional light on anderson acceleration and to draw further attention to its usefulness as a general tool we first show that on linear problems anderson acceleration without truncation is essentially equivalent in a certain sense to the generalized minimal residual gmres method we also show that the type 1 variant in the fang saad anderson family is similarly essentially equivalent to the arnoldi full orthogonalization method we then discuss practical considerations for implementing anderson acceleration and illustrate its performance through numerical experiments involving a variety of applications

analog simulation of a simple system with state dependent diffusion we have constructed an electronic simulator of a simple bistable system driven by noise whose intensity is determined by the instantaneous value of the coordinate we observe that the most probable state of the system can be reversed by altering the noise intensity only in the neighborhood of the barrier an effect pointed out by landauer many years ago in the context of discussions on entropy related stability criteria for nonequilibrium systems we compare detailed measurements on the system with the recent white noise calculations of landauer and van kampen the system also has interesting possibilities for tests of contemporary colored noise theory which we illustrate with an example

critical dynamics of the d 1 kinetic ising model the critical dynamics of the d 1 alternating bond glauber ising model and related models is studied there are two contributions to the critical slowing down one is due to the long range fluctuations near the critical point it is characterized by the dynamic exponent z 2 the other contribution is a temperature dependence of the bare time scale which is a result of short range phenomena in contrast to previous studies it is shown that all these models belong to one universality class

a flexible and efficient algorithmic framework for constrained matrix and tensor factorization we propose a general algorithmic framework for constrained matrix and tensor factorization which is widely used in signal processing and machine learning the new framework is a hybrid between alternating optimization ao and the alternating direction method of multipliers admm each matrix factor is updated in turn using admm hence the name ao admm this combination can naturally accommodate a great variety of constraints on the factor matrices and almost all possible loss measures for the fitting computation caching and warm start strategies are used to ensure that each update is evaluated efficiently while the outer ao framework exploits recent developments in block coordinate descent bcd type methods which help ensure that every limit point is a stationary point as well as faster and more robust convergence in practice three special cases are studied in detail non negative matrix tensor factorization constrained matrix tensor completion and dictionary learning extensive simulations and experiments with real data are used to showcase the effectiveness and broad applicability of the proposed framework

collective sparse symmetric non negative matrix factorization for identifying overlapping communities in resting state brain functional networks resting state functional magnetic resonance imaging rs fmri provides a valuable tool to study spontaneous brain activity using rs fmri researchers have extensively studied the organization of the brain functional network and found several consistent communities consisting of functionally connected but spatially separated brain regions across subjects however increasing evidence in many disciplines has shown that most realistic complex networks have overlapping community structure only recently has the overlapping community structure drawn increasing interest in the domain of brain network studies another issue is that the inter subject variability is often not directly reflected in the process of community detection at the group level in this paper we propose a novel method called collective sparse symmetric non negative matrix factorization cssnmf to address these issues the cssnmf approach identifies the group level overlapping communities across subjects and in the meantime preserves the information of individual variation in brain functional network organization to comprehensively validate cssnmf a simulated fmri dataset with ground truth a real rs fmri dataset with two repeated sessions and another different real rs fmri dataset have been used for performance comparison in the experiment experimental results show that the proposed cssnmf method accurately and stably identifies group level overlapping communities across subjects as well as individual differences in network organization with neurophysiologically meaningful interpretations this research extends our understanding of the common underlying community structures and individual differences in community strengths in brain functional network organization

a multilevel approach for nonnegative matrix factorization nonnegative matrix factorization nmf the problem of approximating a nonnegative matrix with the product of two low rank nonnegative matrices has been shown to be useful in many applications such as text mining image processing and computational biology in this paper we explain how algorithms for nmf can be embedded into the framework of multilevel methods in order to accelerate their initial convergence this technique can be applied in situations where data admit a good approximate representation in a lower dimensional space through linear transformations preserving nonnegativity several simple multilevel strategies are described and are experimentally shown to speed up significantly three popular nmf algorithms alternating nonnegative least squares multiplicative updates and hierarchical alternating least squares on several standard image datasets

phase transitions on fractals ii sierpinski gaskets for pt i see ibid vol 16 p 1267 1983 the authors construct and investigate a family of fractals which are generalisations of the sierpinski gaskets sgs to all euclidean dimensionalities these fractal lattices have a finite order of ramification and can be considered marginal between one dimensional and higher dimensional geometries physical models defined on them are exactly solvable the authors argue that short range spin models on the sg show no finite temperature phase transitions as examples they solve a few spin models and study the resistor network and percolation problems on these lattices

svd based initialization a head start for nonnegative matrix factorization we describe nonnegative double singular value decomposition nndsvd a new method designed to enhance the initialization stage of nonnegative matrix factorization nmf nndsvd can readily be combined with existing nmf algorithms the basic algorithm contains no randomization and is based on two svd processes one approximating the data matrix the other approximating positive sections of the resulting partial svd factors utilizing an algebraic property of unit rank matrices simple practical variants for nmf with dense factors are described nndsvd is also well suited to initialize nmf algorithms with sparse factors many numerical examples suggest that nndsvd leads to rapid reduction of the approximation error of many nmf algorithms

uniqueness of nonnegative tensor approximations we show that for a nonnegative tensor a best nonnegative rank r approximation is almost always unique its best rank one approximation may always be chosen to be a best nonnegative rank one approximation and the set of nonnegative tensors with nonunique best rank one approximations forms an algebraic hypersurface we show that the last part holds true more generally for real tensors and thereby determine a polynomial equation so that a real or nonnegative tensor that does not satisfy this equation is guaranteed to have a unique best rank one approximation we also establish an analogue for real or nonnegative symmetric tensors in addition we prove a singular vector variant of the perron frobenius theorem for positive tensors and apply it to show that a best nonnegative rank r approximation of a positive tensor can never be obtained by deflation as an aside we verify that the euclidean distance ed discriminants of the segre variety and the veronese variety are hypersurfaces and give defining equations of these ed discriminants

non negative matrix factorization with sparseness constraints non negative matrix factorization nmf is a recently developed technique for finding parts based linear representations of non negative data although it has successfully been applied in several applications it does not always result in parts based representations in this paper we show how explicitly incorporating the notion of sparseness improves the found decompositions additionally we provide complete matlab code both for standard nmf and for our extension our hope is that this will further the application of these methods to solving novel data analysis problems

a convex semi nonnegative matrix factorisation approach to fuzzy c means clustering we propose an alternative approach to fuzzy c means clustering which eliminates the weighting exponent parameter of conventional algorithms it is based on a particular convex factorisation of data matrix the proposed method is invariant under certain linear transformations of the data including principal component analysis we tested its accuracy using both synthetic data and real datasets and compared it to that provided by the usual fuzzy c means algorithm we were able to ascertain that our proposal can be a credible yet easier alternative to this approach to fuzzy clustering moreover it showed no noticeable sensitivity to the initial guess of the partition matrix

a comparison of block pivoting and interior point algorithms for linear least squares problems with nonnegative variables in this paper we discuss the use of block principal pivoting and predictor corrector methods for the solution of large scale linear least squares problems with nonnegative variables nvlsq we also describe two implementations of these algorithms that are based on the normal equations and corrected seminormal equations csne approaches we show that the method of normal equations should be employed in the implementation of the predictor corrector algorithm this type of approach should also be used in the implementation of the block principal pivoting method but a switch to the csne method may be useful in the last iterations of the algorithm computational experience is also included in this paper and shows that both the predictor corrector and the block principal pivoting algorithms are quite efficient to deal with large scale nvlsq problems

nonnegative ranks decompositions and factorizations of nonnegative matrices the nonnegative rank of a nonnegative matrix is the smallest number of nonnegative rank one matrices into which the matrix can be decomposed additively such decompositions are useful in diverse scientific disciplines we obtain characterizations and bounds and show that the nonnegative rank can be computed exactly over the reals by a finite algorithm

non negative matrix factorization revisited uniqueness and algorithm for symmetric decomposition non negative matrix factorization nmf has found numerous applications due to its ability to provide interpretable decompositions perhaps surprisingly existing results regarding its uniqueness properties are rather limited and there is much room for improvement in terms of algorithms as well uniqueness aspects of nmf are revisited here from a geometrical point of view both symmetric and asymmetric nmf are considered the former being tantamount to element wise non negative square root factorization of positive semidefinite matrices new uniqueness results are derived e g it is shown that a sufficient condition for uniqueness is that the conic hull of the latent factors is a superset of a particular second order cone checking this condition is shown to be np complete yet this and other results offer insights on the role of latent sparsity in this context on the computational side a new algorithm for symmetric nmf is proposed which is very different from existing ones it alternates between procrustes rotation and projection onto the non negative orthant to find a non negative matrix close to the span of the dominant subspace simulation results show promising performance with respect to the state of art finally the new algorithm is applied to a clustering problem for co authorship data yielding meaningful and interpretable results

efficient nonnegative matrix factorization by dc programming and dca in this letter we consider the nonnegative matrix factorization nmf problem and several nmf variants two approaches based on dc difference of convex functions programming and dca dc algorithm are developed the first approach follows the alternating framework that requires solving at each iteration two nonnegativity constrained least squares subproblems for which dca based schemes are investigated the convergence property of the proposed algorithm is carefully studied we show that with suitable dc decompositions our algorithm generates most of the standard methods for the nmf problem the second approach directly applies dca on the whole nmf problem two algorithms one computing all variables and one deploying a variable selection strategy are proposed the proposed methods are then adapted to solve various nmf variants including the nonnegative factorization the smooth regularization nmf the sparse regularization nmf the multilayer nmf the convex convex hull nmf and the symmetric nmf we also show that our algorithms include several existing methods for these nmf variants as special versions the efficiency of the proposed approaches is empirically demonstrated on both real world and synthetic data sets it turns out that our algorithms compete favorably with five state of the art alternating nonnegative least squares algorithms

on the convergence of multiplicative update algorithms for nonnegative matrix factorization nonnegative matrix factorization nmf is useful to find basis information of nonnegative data currently multiplicative updates are a simple and popular way to find the factorization however for the common nmf approach of minimizing the euclidean distance between approximate and true values no proof has shown that multiplicative updates converge to a stationary point of the nmf optimization problem stationarity is important as it is a necessary condition of a local minimum this paper discusses the difficulty of proving the convergence we propose slight modifications of existing updates and prove their convergence techniques invented in this paper may be applied to prove the convergence for other bound constrained optimization problems

a new active set method for nonnegative matrix factorization nonnegative matrix factorization nmf is powerful in data analysis by recovering hidden features from redundant data recently various extensions of nmf have been proposed to further enhance the classification results in this paper we present a novel proximal alternating nonnegative least squares panls framework for nmf we show the globally convergent property that any accumulation point of panls is a stationary point of nmf a new active set method based on the framework is proposed to solve each subproblem which reduces to unconstrained optimization after finite steps and converges to the unique minimizer of the subproblem the panls framework and the associated new active set method can be easily extended to variants of nmf extensive computational experiments using both synthetic data and real data show that our proposed method is attractive in terms of computational speed the solution quality and the flexibility to extensions of nmf

minimum volume constrained nonnegative matrix factorization enhanced ability of learning parts nonnegative matrix factorization nmf with minimum volume constraint mvc is exploited in this paper our results show that mvc can actually improve the sparseness of the results of nmf this sparseness is l0 norm oriented and can give desirable results even in very weak sparseness situations thereby leading to the significantly enhanced ability of learning parts of nmf the close relation between nmf sparse nmf and the mvc nmf is discussed first then two algorithms are proposed to solve the mvc nmf model one is called quadratic programming mvc nmf qp mvc nmf which is based on quadratic programming and the other is called negative glow mvc nmf ng mvc nmf because it uses multiplicative updates incorporating natural gradient ingeniously the qp mvc nmf algorithm is quite efficient for small scale problems and the ng mvc nmf algorithm is more suitable for large scale problems simulations show the efficiency and validity of the proposed methods in applications of blind source separation and human face images analysis

on reduced rank nonnegative matrix factorization for symmetric nonnegative matrices abstract let v r m n be a nonnegative matrix the nonnegative matrix factorization nnmf problem consists of finding nonnegative matrix factors w r m r and h r r n such that v wh lee and seung proposed two algorithms one of which finds nonnegative w and h such that v wh f is minimized after examining the case in which r 1 about which a complete characterization of the solution is possible we consider the case in which m n and v is symmetric we focus on questions concerning when the best approximate factorization results in the product wh being symmetric and on cases in which the best approximation cannot be a symmetric matrix finally we show that the class of positive semidefinite symmetric nonnegative matrices v generated via a soules basis admit for every 1 r rank v a nonnegative factorization wh which coincides with the best approximation in the frobenius norm to v in r n n of rank not exceeding r an example of applications in which nnmf factorizations for nonnegative symmetric matrices v arise is video and other media summarization technology where v is obtained from a distance matrix

mean first passage time approach to the problem of optimal barrier subdivision for kramer s escape rate we consider the effect of subdividing the potential barrier along the reaction coordinate on kramer s escape rate for a model potential using the known supersymmetric potential approach we show the existence of an optimal number of subdivisions that maximizes the rate we cast the problem as a mean first passage time problem of a biased random walker and obtain equivalent results we briefly summarize the results of our investigation on the increase in the escape rate by placing a blow torch in the unstable part of one of the potential wells

descent algorithms on oblique manifold for source adaptive ica contrast a riemannian manifold optimization strategy is proposed to facilitate the relaxation of the orthonormality constraint in a more natural way in the course of performing independent component analysis ica that employs a mutual information based source adaptive contrast function despite the extensive development of manifold techniques catering to the orthonormality constraint only a limited number of works have been dedicated to oblique manifold ob algorithms to intrinsically handle the normality constraint which has been empirically shown to be superior to other riemannian and euclidean approaches imposing the normality constraint implicitly in line with the ica definition essentially guarantees a substantial improvement in the solution accuracy by way of increased degrees of freedom while searching for an optimal unmixing ica matrix in contrast with the orthonormality constraint designs of the steepest descent conjugate gradient with hager zhang or a hybrid update parameter quasi newton and cost effective quasi newton methods intended for ob are presented in this paper their performance is validated using natural images and systematically compared with the popular state of the art approaches in order to assess the performance effects of the choice of algorithm and the use of a riemannian rather than euclidean framework we surmount the computational challenge associated with the direct estimation of the source densities using the improved fast gauss transform in the evaluation of the contrast function and its gradient the proposed ob schemes may find applications in the offline image signal analysis wherein on one hand the computational overhead can be tolerated and on the other the solution quality holds paramount interest

improving non negative matrix factorizations through structured initialization in this paper we explore a recent iterative compression technique called non negative matrix factorization nmf several special properties are obtained as a result of the constrained optimization problem of nmf for facial images the additive nature of nmf results in a basis of features such as eyes noses and lips we explore various methods for efficiently computing nmf placing particular emphasis on the initialization of current algorithms we propose using spherical k means clustering to produce a structured initialization for nmf we demonstrate some of the properties that result from this initialization and develop an efficient way of choosing the rank of the low dimensional nmf representation

the nmf problem and lattice subspaces abstract suppose that a is a nonnegative n m real matrix the nmf problem is the determination of two nonnegative real matrices f v so that a f v with intermediate dimension p smaller than m i n n m in this article we present a general method for the determination of two nonnegative real factors f v of a based on the theory of lattice subspaces and positive bases

subtractive clustering for seeding non negative matrix factorizations non negative matrix factorization is a multivariate analysis method which is proven to be useful in many areas such as bio informatics molecular pattern discovery pattern recognition document clustering and so on it seeks a reduced representation of a multivariate data matrix into the product of basis and encoding matrices possessing only non negative elements in order to learn the so called part based representations of data all algorithms for computing non negative matrix factorization are iterative therefore particular emphasis must be placed on a proper initialization of nmf because of its local convergence the problem of selecting appropriate starting matrices becomes more complex when data possess special meaning as in document clustering in this paper we propose the adoption of the subtractive clustering algorithm as a scheme to generate initial matrices for non negative matrix factorization algorithms comparisons with other commonly adopted initializations of non negative matrix factorization algorithms have been performed and the proposed scheme reveals to be a good trade off between effectiveness and speed moreover the effectiveness of the proposed initialization to suggest a number of basis for nmf when data distances are estimated is illustrated when nmf is used for solving clustering problems where the number of groups in which the data are grouped is not known a priori the influence of a proper rank factor on the interpretability and the effectiveness of the results are also discussed

sparse non negative tensor factorization using columnwise coordinate descent many applications in computer vision biomedical informatics and graphics deal with data in the matrix or tensor form non negative matrix and tensor factorization which extract data dependent non negative basis functions have been commonly applied for the analysis of such data for data compression visualization and detection of hidden information factors in this paper we present a fast and flexible algorithm for sparse non negative tensor factorization sntf based on columnwise coordinate descent ccd different from the traditional coordinate descent which updates one element at a time ccd updates one column vector simultaneously our empirical results on higher mode images such as brain mri images gene expression images and hyperspectral images show that the proposed algorithm is 1 2 orders of magnitude faster than several state of the art algorithms

projected newton methods for optimization problems with simple constraints we consider the problem min f x x geqq 0 and propose algorithms of the form x k 1 x k alpha k d k nabla f x k where cdot denotes projection on the positive orthant alpha k is a stepsize chosen by an armijo like rule and d k is a positive definite symmetric matrix which is partly diagonal we show that d k can be calculated simply on the basis of second derivatives of f so that the resulting newton like algorithm has a typically superlinear rate of convergence with other choices of d k convergence at a typically linear rate is obtained the algorithms are almost as simple as their unconstrained counterparts they are well suited for problems of large dimension such as those arising in optimal control while being competitive with existing methods for low dimensional problems the effectiveness of the newton like algorithm is demonstrated via computational examples involving as many as 10 000 variables extensions to general linearly constrained pr

learning sparse representations by non negative matrix factorization and sequential cone programming we exploit the biconvex nature of the euclidean non negative matrix factorization nmf optimization problem to derive optimization schemes based on sequential quadratic and second order cone programming we show that for ordinary nmf our approach performs as well as existing state of the art algorithms while for sparsity constrained nmf as recently proposed by p o hoyer in jmlr 5 2004 it outperforms previous methods in addition we show how to extend nmf learning within the same optimization framework in order to make use of class membership information in supervised learning problems

a nonconvex splitting method for symmetric nonnegative matrix factorization convergence analysis and optimality symmetric nonnegative matrix factorization symnmf has important applications in data analytics problems such as document clustering community detection and image segmentation in this paper we propose a novel nonconvex variable splitting method for solving symnmf the proposed algorithm is guaranteed to converge to the set of karush kuhn tucker kkt points of the nonconvex symnmf problem furthermore it achieves a global sublinear convergence rate we also show that the algorithm can be efficiently implemented in parallel further sufficient conditions are provided that guarantee the global and local optimality of the obtained solutions extensive numerical results performed on both synthetic and real datasets suggest that the proposed algorithm converges quickly to a local minimum solution

ellipsoidal rounding for nonnegative matrix factorization under noisy separability we present a numerical algorithm for nonnegative matrix factorization nmf problems under noisy separability an nmf problem under separability can be stated as one of finding all vertices of the convex hull of data points the research interest of this paper is to find the vectors as close to the vertices as possible in a situation in which noise is added to the data points our algorithm is designed to capture the shape of the convex hull of data points by using its enclosing ellipsoid we show that the algorithm has correctness and robustness properties from theoretical and practical perspectives correctness here means that if the data points do not contain any noise the algorithm can find the vertices of their convex hull robustness means that if the data points contain noise the algorithm can find the near vertices finally we apply the algorithm to document clustering and report the experimental results

critical dynamics of the alternating bond kinetic ising model the critical dynamics of the alternating bond kinetic ising model on a chain is examined it is shown that previous nonuniversal results for the dynamical critical exponent z for single spin flip dynamics should be reinterpreted the critical slowing down is due to two distinct contributions which when separated indicate that the value of z is universal the exponent remains universal even when two spin flip dynamics are considered

non negative matrix factorization algorithm for the deconvolution of one dimensional chromatograms in chromatogram analysis overlapped chromatograms are difficult to analyze if they are not resolved the conventional multivariate resolution techniques do not give accurate results when the chromatograms are severely overlapped in this work ml nmfdiv modified non negative matrix factorization nmf with divergence objective algorithm has been proposed for the separation of severely overlapped chromatograms of acetone and acrolein mixture before applying nmf principal component analysis pca is applied to determine number of components in the mixture taken most of the nmf algorithms used so far for chromatogram separation do not converge to a stable limit point and no uniqueness in the results to get unique results instead of random initialization three different initialization methods namely robust initialization nndsvd non negative double singular value decomposition based initialization and efa evolving factor analysis based initializations have been used in this work and the performances are compared the multiplicative update of already existing nmfdiv algorithm has been modified and proposed in this work as ml nmfdiv nmfdiv with modified multiplicative update for overlapped chromatogram separation to improve the convergence the proposed ml nmfdiv algorithm is applied on the simulated and experimental chromatograms obtained for acetone and acrolein mixture the results of proposed ml nmfdiv are compared with existing multivariate curve resolution alternating least square mcr als method

nonnegative matrix factorization for rapid recovery of constituent spectra in magnetic resonance chemical shift imaging of the brain we present an algorithm for blindly recovering constituent source spectra from magnetic resonance mr chemical shift imaging csi of the human brain the algorithm which we call constrained nonnegative matrix factorization cnmf does not enforce independence or sparsity instead only requiring the source and mixing matrices to be nonnegative it is based on the nonnegative matrix factorization nmf algorithm extending it to include a constraint on the positivity of the amplitudes of the recovered spectra this constraint enables recovery of physically meaningful spectra even in the presence of noise that causes a significant number of the observation amplitudes to be negative we demonstrate and characterize the algorithm s performance using sup 31 p volumetric brain data comparing the results with two different blind source separation methods bayesian spectral decomposition bsd and nonnegative sparse coding nnsc we then incorporate the cnmf algorithm into a hierarchical decomposition framework showing that it can be used to recover tissue specific spectra given a processing hierarchy that proceeds coarse to fine we demonstrate the hierarchical procedure on sup 1 h brain data and conclude that the computational efficiency of the algorithm makes it well suited for use in diagnostic work up

critical dynamics and universality in kinetic ising models without translational invariance the critical dynamics of the glauber ising model on nontranslationally invariant lattices is studied both a quasi periodic and a fractal geometry are considered the distribution of inverse relaxation times rho 1 tau is calculated using a generating function method the distribution consists of bands with an internal self similar structure in the limit 1 tau to 0 rho 1 tau diverges with a universal exponent related to the dynamic critical exponent z the width of the lowest frequency band is determined by a nonuniversal bare time scale which is related to the presence of metastable states

efficient and non convex coordinate descent for symmetric nonnegative matrix factorization given a symmetric nonnegative matrix a symmetric nonnegative matrix factorization symnmf is the problem of finding a nonnegative matrix h usually with much fewer columns than a such that a approx hh t symnmf can be used for data analysis and in particular for various clustering tasks unlike standard nmf which is traditionally solved by a series of quadratic convex subproblems we propose to solve symnmf by directly solving the nonconvex problem namely minimize vert a hh t vert 2 which is a fourth order nonconvex problem in this paper we propose simple and very efficient coordinate descent schemes which solve a series of fourth order univariate subproblems exactly we also derive convergence guarantees for our methods and show that they perform favorably compared to recent state of the art methods on synthetic and real world datasets especially on large and sparse input matrices

a numerically stable solver for positive semidefinite quadratic programs based on nonnegative least squares this paper proposes a new algorithm for solving convex quadratic programs qp subject to linear inequality and equality constraints the method extends an approach recently proposed by the author for solving strictly convex qp s using nonnegative least squares by making it numerically more robust and able to handle also the nonstrictly convex case equality constraints and warm starting from an initial guess robustness is achieved by introducing an outer proximal point iteration scheme that regularizes the problem without altering the solution and by adaptively weighting the least squares problems encountered while solving the problem the performance of the resulting qp solver in terms of speed and robustness in the single precision arithmetic is assessed against other optimization algorithms tailored to embedded model predictive control applications

projected gradient methods for nonnegative matrix factorization nonnegative matrix factorization nmf can be formulated as a minimization problem with bound constraints although bound constrained optimization has been studied extensively in both theory and practice so far no study has formally applied its techniques to nmf in this letter we propose two projected gradient methods for nmf both of which exhibit strong optimization properties we discuss efficient implementations and demonstrate that one of the proposed methods converges faster than the popular multiplicative update approach a simple matlab code is also provided