nonnegative local coordinate factorization for image representation recently nonnegative matrix factorization nmf has become increasingly popular for feature extraction in computer vision and pattern recognition nmf seeks two nonnegative matrices whose product can best approximate the original matrix the nonnegativity constraints lead to sparse parts based representations that can be more robust than nonsparse global features to obtain more accurate control over the sparseness in this paper we propose a novel method called nonnegative local coordinate factorization nlcf for feature extraction nlcf adds a local coordinate constraint into the standard nmf objective function specifically we require that the learned basis vectors be as close to the original data points as possible in this way each data point can be represented by a linear combination of only a few nearby basis vectors which naturally leads to sparse representation extensive experimental results suggest that the proposed approach provides a better representation and achieves higher accuracy in image clustering

structural and functional discovery in dynamic networks with non negative matrix factorization time series of graphs are increasingly prevalent in modern data and pose unique challenges to visual exploration and pattern extraction this paper describes the development and application of matrix factorizations for exploration and time varying community detection in time evolving graph sequences the matrix factorization model allows the user to home in on and display interesting underlying structure and its evolution over time the methods are scalable to weighted networks with a large number of time points or nodes and can accommodate sudden changes to graph topology our techniques are demonstrated with several dynamic graph series from both synthetic and real world data including citation and trade networks these examples illustrate how users can steer the techniques and combine them with existing methods to discover and display meaningful patterns in sizable graphs over many time points

nonnegative matrix factorization and i divergence alternating minimization abstract in this paper we consider the nonnegative matrix factorization nmf problem given an elementwise nonnegative matrix v r m n find for assigned k nonnegative matrices w r m k and h r k n such that v wh exact nontrivial nonnegative factorizations do not always exist hence it is interesting to pose the approximate nmf problem the criterion which is commonly employed is i divergence between nonnegative matrices the problem becomes that of finding for assigned k the factorization wh closest to v in i divergence an iterative algorithm em like for the construction of the best pair w h has been proposed in the literature in this paper we interpret the algorithm as an alternating minimization procedure a la csiszar tusnady and investigate some of its stability properties nmf is widespreading as a data analysis method in applications for which the positivity constraint is relevant there are other data analysis methods which impose some form of nonnegativity we discuss here the connections between nmf and archetypal analysis

nonlinear non negative component analysis algorithms in this paper general solutions for nonlinear non negative component analysis for data representation and recognition are proposed motivated by a combination of the non negative matrix factorization nmf algorithm and kernel theory which has lead to a recently proposed nmf algorithm in a polynomial feature space we propose a general framework where one can build a nonlinear non negative component analysis method using kernels the so called projected gradient kernel non negative matrix factorization pgknmf in the proposed approach arbitrary positive definite kernels can be adopted while at the same time it is ensured that the limit point of the procedure is a stationary point of the optimization problem moreover we propose fixed point algorithms for the special case of gaussian radial basis function rbf kernels we demonstrate the power of the proposed methods in face and facial expression recognition applications

projected gradient methods for nonnegative matrix factorization nonnegative matrix factorization nmf can be formulated as a minimization problem with bound constraints although bound constrained optimization has been studied extensively in both theory and practice so far no study has formally applied its techniques to nmf in this letter we propose two projected gradient methods for nmf both of which exhibit strong optimization properties we discuss efficient implementations and demonstrate that one of the proposed methods converges faster than the popular multiplicative update approach a simple matlab code is also provided

approximation of nonnegative systems by finite impulse response convolutions we pose the deterministic nonparametric approximation problem for scalar nonnegative input output systems via finite impulse response convolutions based on repeated observations of input output signal pairs the problem is converted into a nonnegative matrix factorization with special structure for which we use csiszar s i divergence as the criterion of optimality conditions are given on the input output data that guarantee the existence and uniqueness of the minimum we propose an algorithm of the alternating minimization type for i divergence minimization and study its asymptotic behavior for the case of noisy observations we give the large sample properties of the statistical version of the minimization problem numerical experiments confirm the asymptotic results and exhibit the fast convergence of the proposed algorithm

a note on the convergence analysis of lms adaptive filters with gaussian data necessary and sufficient conditions for the convergence of lms least mean squares adaptive filters with gaussian data have been established by l l horowitz and k d senne 1981 with the recent support of a feuer and e weinstein 1985 a feature of both of these studies is the necessity to investigate bounds on the roots of rather unwieldy characteristic equations the authors show how such an investigation can be avoided through the use of a theorem of f r gantmacher 1959 in formally applying this theorem similar results to those of the above studies are obtained in a precise and straightforward manner

non negative matrix factorization with sparseness constraints non negative matrix factorization nmf is a recently developed technique for finding parts based linear representations of non negative data although it has successfully been applied in several applications it does not always result in parts based representations in this paper we show how explicitly incorporating the notion of sparseness improves the found decompositions additionally we provide complete matlab code both for standard nmf and for our extension our hope is that this will further the application of these methods to solving novel data analysis problems

graph regularized nonnegative matrix factorization for data representation matrix factorization techniques have been frequently applied in information retrieval computer vision and pattern recognition among them nonnegative matrix factorization nmf has received considerable attention due to its psychological and physiological interpretation of naturally occurring data whose representation may be parts based in the human brain on the other hand from the geometric perspective the data is usually sampled from a low dimensional manifold embedded in a high dimensional ambient space one then hopes to find a compact representation which uncovers the hidden semantics and simultaneously respects the intrinsic geometric structure in this paper we propose a novel algorithm called graph regularized nonnegative matrix factorization gnmf for this purpose in gnmf an affinity graph is constructed to encode the geometrical information and we seek a matrix factorization which respects the graph structure our empirical study shows encouraging results of the proposed algorithm in comparison to the state of the art algorithms on real world problems

symmetric nonnegative matrix factorization algorithms and applications to probabilistic clustering nonnegative matrix factorization nmf is an unsupervised learning method useful in various applications including image processing and semantic analysis of documents this paper focuses on symmetric nmf snmf which is a special case of nmf decomposition three parallel multiplicative update algorithms using level 3 basic linear algebra subprograms directly are developed for this problem first by minimizing the euclidean distance a multiplicative update algorithm is proposed and its convergence under mild conditions is proved based on it we further propose another two fast parallel methods snmf and snmf algorithms all of them are easy to implement these algorithms are applied to probabilistic clustering we demonstrate their effectiveness for facial image clustering document categorization and pattern clustering in gene expression

on the convergence of multiplicative update algorithms for nonnegative matrix factorization nonnegative matrix factorization nmf is useful to find basis information of nonnegative data currently multiplicative updates are a simple and popular way to find the factorization however for the common nmf approach of minimizing the euclidean distance between approximate and true values no proof has shown that multiplicative updates converge to a stationary point of the nmf optimization problem stationarity is important as it is a necessary condition of a local minimum this paper discusses the difficulty of proving the convergence we propose slight modifications of existing updates and prove their convergence techniques invented in this paper may be applied to prove the convergence for other bound constrained optimization problems

minimum volume constrained nonnegative matrix factorization enhanced ability of learning parts nonnegative matrix factorization nmf with minimum volume constraint mvc is exploited in this paper our results show that mvc can actually improve the sparseness of the results of nmf this sparseness is l0 norm oriented and can give desirable results even in very weak sparseness situations thereby leading to the significantly enhanced ability of learning parts of nmf the close relation between nmf sparse nmf and the mvc nmf is discussed first then two algorithms are proposed to solve the mvc nmf model one is called quadratic programming mvc nmf qp mvc nmf which is based on quadratic programming and the other is called negative glow mvc nmf ng mvc nmf because it uses multiplicative updates incorporating natural gradient ingeniously the qp mvc nmf algorithm is quite efficient for small scale problems and the ng mvc nmf algorithm is more suitable for large scale problems simulations show the efficiency and validity of the proposed methods in applications of blind source separation and human face images analysis

on reduced rank nonnegative matrix factorization for symmetric nonnegative matrices abstract let v r m n be a nonnegative matrix the nonnegative matrix factorization nnmf problem consists of finding nonnegative matrix factors w r m r and h r r n such that v wh lee and seung proposed two algorithms one of which finds nonnegative w and h such that v wh f is minimized after examining the case in which r 1 about which a complete characterization of the solution is possible we consider the case in which m n and v is symmetric we focus on questions concerning when the best approximate factorization results in the product wh being symmetric and on cases in which the best approximation cannot be a symmetric matrix finally we show that the class of positive semidefinite symmetric nonnegative matrices v generated via a soules basis admit for every 1 r rank v a nonnegative factorization wh which coincides with the best approximation in the frobenius norm to v in r n n of rank not exceeding r an example of applications in which nnmf factorizations for nonnegative symmetric matrices v arise is video and other media summarization technology where v is obtained from a distance matrix

logical matrix factorization with application to topological structure analysis of boolean network this note investigates the logical matrix factorization with application to the topological structure analysis of boolean networks first the concepts of both factorization and rank are defined for logical matrices and two factorization problems are then studied using nonsingular logical matrix transformations several necessary and sufficient conditions for the factorization of a given logical matrix are presented second the logical matrix factorization is applied to the topological structure analysis of a given boolean network and a size reduced structure equivalent logical network is constructed for the given system it is shown that the topological structure including all the fixed points and cycles of the resulting size reduced logical network is the same as that of the original boolean network the study of an illustrative example shows that the new results presented in this note are effective in analyzing the topological structure of boolean networks

discriminant nonnegative tensor factorization algorithms nonnegative matrix factorization nmf has proven to be very successful for image analysis especially for object representation and recognition nmf requires the object tensor with valence more than one to be vectorized this procedure may result in information loss since the local object structure is lost due to vectorization recently in order to remedy this disadvantage of nmf methods nonnegative tensor factorizations ntf algorithms that can be applied directly to the tensor representation of object collections have been introduced in this paper we propose a series of unsupervised and supervised ntf methods that is we extend several nmf methods using arbitrary valence tensors moreover by incorporating discriminant constraints inside the ntf decompositions we present a series of discriminant ntf methods the proposed approaches are tested for face verification and facial expression recognition where it is shown that they outperform other popular subspace approaches

path dependent hamilton jacobi equations in infinite dimensions abstract we propose notions of minimax and viscosity solutions for a class of fully nonlinear path dependent pdes with nonlinear monotone and coercive operators on hilbert space our main result is well posedness existence uniqueness and stability for minimax solutions a particular novelty is a suitable combination of minimax and viscosity solution techniques in the proof of the comparison principle one of the main difficulties the lack of compactness in infinite dimensional hilbert spaces is circumvented by working with suitable compact subsets of our path space as an application our theory makes it possible to employ the dynamic programming approach to study optimal control problems for a fairly general class of delay evolution equations in the variational framework furthermore differential games associated to such evolution equations can be investigated following the krasovski subbotin approach similarly as in finite dimensions

global discriminative based nonnegative spectral clustering based on spectral graph theory spectral clustering is an optimal graph partition problem it has been proven that the spectral clustering is equivalent to nonnegative matrix factorization nmf under certain conditions based on the equivalence some spectral clustering methods are proposed but the global discriminative information of the dataset is neglected in this paper based on the equivalence between spectral clustering and nmf we simultaneously maximize the between class scatter matrix and minimize the within class scatter matrix to enhance the discriminating power we integrate the geometrical structure and discriminative structure in a joint framework with a global discriminative regularization term added into the nonnegative matrix factorization framework we propose two novel spectral clustering methods named global discriminative based nonnegative and spectral clustering gdbnsc ncut and gdbnsc rcut these new spectral clustering algorithms can preserve both the global geometrical structure and global discriminative structure the intrinsic geometrical information of the dataset is detected and clustering quality is improved with enhanced discriminating power in addition the proposed algorithms also have very good abilities of handling out of sample data experimental results on real word data demonstrate that the proposed algorithms outperform some state of the art methods with good clustering qualities we maximize the between class scatter matrix meanwhile we minimize the within class scatter matrix we integrate the geometrical structure and discriminative structure in a framework a global discriminative regularization term added into the objective functions the new algorithms can preserve the global geometrical and discriminative structure

variants of non negative least mean square algorithm and convergence analysis due to the inherent physical characteristics of systems under investigation non negativity is one of the most interesting constraints that can usually be imposed on the parameters to estimate the non negative least mean square algorithm nnlms was proposed to adaptively find solutions of a typical wiener filtering problem but with the side constraint that the resulting weights need to be non negative it has been shown to have good convergence properties nevertheless certain practical applications may benefit from the use of modified versions of this algorithm in this paper we derive three variants of nnlms each variant aims at improving the nnlms performance regarding one of the following aspects sensitivity of input power unbalance of convergence rates for different weights and computational cost we study the stochastic behavior of the adaptive weights for these three new algorithms for non stationary environments this study leads to analytical models to predict the first and second order moment behaviors of the weights for gaussian inputs simulation results are presented to illustrate the performance of the new algorithms and the accuracy of the derived models

constrained nonnegative matrix factorization for image representation nonnegative matrix factorization nmf is a popular technique for finding parts based linear representations of nonnegative data it has been successfully applied in a wide range of applications such as pattern recognition information retrieval and computer vision however nmf is essentially an unsupervised method and cannot make use of label information in this paper we propose a novel semi supervised matrix decomposition method called constrained nonnegative matrix factorization cnmf which incorporates the label information as additional constraints specifically we show how explicitly combining label information improves the discriminating power of the resulting matrix decomposition we explore the proposed cnmf method with two cost function formulations and provide the corresponding update solutions for the optimization problems empirical experiments demonstrate the effectiveness of our novel algorithm in comparison to the state of the art approaches through a set of evaluations based on real world applications

local coordinate concept factorization for image representation learning sparse representation of high dimensional data is a state of the art method for modeling data matrix factorization based techniques such as nonnegative matrix factorization and concept factorization cf have shown great advantages in this area especially useful for image representation both of them are linear learning problems and lead to a sparse representation of the images however the sparsity obtained by these methods does not always satisfy locality conditions for example the learned new basis vectors may be relatively far away from the original data thus we may not be able to achieve the optimal performance when using the new representation for other learning tasks such as classification and clustering in this paper we introduce a locality constraint into the traditional cf by requiring the concepts basis vectors to be as close to the original data points as possible each datum can be represented by a linear combination of only a few basis concepts thus our method is able to achieve sparsity and locality simultaneously we analyze the complexity of our novel algorithm and demonstrate the effectiveness in comparison with the state of the art approaches through a set of evaluations based on real world applications