a novel framework for the nmf methods with experiments to unmixing signals and feature representation abstract non negative matrix factorization nmf can be used in clustering feature representation or blind source separation many nmf methods have been developed including least squares ls error kullback leibler kl divergence itakura saito is divergence bregman divergence divergence divergence divergence convex constrained graph regularized nmfs the main contribution of this paper is to develop a framework to generalize the existing nmf methods and also provide new nmf methods this paper constructs a general optimization model and develops a generic updating rule with a simple structure using a surrogate function which possesses similar properties as the standard nmf methods the experimental results obtained using several standard databases demonstrate the power of the work in which some new methods provide performance superior to that of the other existing methods

modified principal component analysis an integration of multiple similarity subspace models we modify the conventional principal component analysis pca and propose a novel subspace learning framework modified pca mpca using multiple similarity measurements mpca computes three similarity matrices exploiting the similarity measurements 1 mutual information 2 angle information and 3 gaussian kernel similarity we employ the eigenvectors of similarity matrices to produce new subspaces referred to as similarity subspaces a new integrated similarity subspace is then generated using a novel feature selection approach this approach needs to construct a kind of vector set termed weak machine cell wmc which contains an appropriate number of the eigenvectors spanning the similarity subspaces combining the wrapper method and the forward selection scheme mpca selects a wmc at a time that has a powerful discriminative capability to classify samples mpca is very suitable for the application scenarios in which the number of the training samples is less than the data dimensionality mpca outperforms the other state of the art pca based methods in terms of both classification accuracy and clustering result in addition mpca can be applied to face image reconstruction mpca can use other types of similarity measurements extensive experiments on many popular real world data sets such as face databases show that mpca achieves desirable classification results as well as has a powerful capability to represent data

finding imaging patterns of structural covariance via non negative matrix factorization in this paper we investigate the use of non negative matrix factorization nnmf for the analysis of structural neuroimaging data the goal is to identify the brain regions that co vary across individuals in a consistent way hence potentially being part of underlying brain networks or otherwise influenced by underlying common mechanisms such as genetics and pathologies nnmf offers a directly data driven way of extracting relatively localized co varying structural regions thereby transcending limitations of principal component analysis pca independent component analysis ica and other related methods that tend to produce dispersed components of positive and negative loadings in particular leveraging upon the well known ability of nnmf to produce parts based representations of image data we derive decompositions that partition the brain into regions that vary in consistent ways across individuals importantly these decompositions achieve dimensionality reduction via highly interpretable ways and generalize well to new data as shown via split sample experiments we empirically validate nnmf in two data sets i a diffusion tensor dt mouse brain development study and ii a structural magnetic resonance smr study of human brain aging we demonstrate the ability of nnmf to produce sparse parts based representations of the data at various resolutions these representations seem to follow what we know about the underlying functional organization of the brain and also capture some pathological processes moreover we show that these low dimensional representations favorably compare to descriptions obtained with more commonly used matrix factorization methods like pca and ica

heterogeneous tensor decomposition for clustering via manifold optimization tensor clustering is an important tool that exploits intrinsically rich structures in real world multiarray or tensor datasets often in dealing with those datasets standard practice is to use subspace clustering that is based on vectorizing multiarray data however vectorization of tensorial data does not exploit complete structure information in this paper we propose a subspace clustering algorithm without adopting any vectorization process our approach is based on a novel heterogeneous tucker decomposition model taking into account cluster membership information we propose a new clustering algorithm that alternates between different modes of the proposed heterogeneous tensor model all but the last mode have closed form updates updating the last mode reduces to optimizing over the multinomial manifold for which we investigate second order riemannian geometry and propose a trust region algorithm numerical experiments show that our proposed algorithm compete effectively with state of the art clustering algorithms that are based on tensor factorization

nonnegative matrix factorization and i divergence alternating minimization abstract in this paper we consider the nonnegative matrix factorization nmf problem given an elementwise nonnegative matrix v r m n find for assigned k nonnegative matrices w r m k and h r k n such that v wh exact nontrivial nonnegative factorizations do not always exist hence it is interesting to pose the approximate nmf problem the criterion which is commonly employed is i divergence between nonnegative matrices the problem becomes that of finding for assigned k the factorization wh closest to v in i divergence an iterative algorithm em like for the construction of the best pair w h has been proposed in the literature in this paper we interpret the algorithm as an alternating minimization procedure a la csiszar tusnady and investigate some of its stability properties nmf is widespreading as a data analysis method in applications for which the positivity constraint is relevant there are other data analysis methods which impose some form of nonnegativity we discuss here the connections between nmf and archetypal analysis

multiple graph regularized nonnegative matrix factorization non negative matrix factorization nmf has been widely used as a data representation method based on components to overcome the disadvantage of nmf in failing to consider the manifold structure of a data set graph regularized nmf grnmf has been proposed by cai et al by constructing an affinity graph and searching for a matrix factorization that respects graph structure selecting a graph model and its corresponding parameters is critical for this strategy this process is usually carried out by cross validation or discrete grid search which are time consuming and prone to overfitting in this paper we propose a grnmf called multigrnmf in which the intrinsic manifold is approximated by a linear combination of several graphs with different models and parameters inspired by ensemble manifold regularization factorization metrics and linear combination coefficients of graphs are determined simultaneously within a unified object function they are alternately optimized in an iterative algorithm thus resulting in a novel data representation algorithm extensive experiments on a protein subcellular localization task and an alzheimer s disease diagnosis task demonstrate the effectiveness of the proposed algorithm

nonnegative local coordinate factorization for image representation recently nonnegative matrix factorization nmf has become increasingly popular for feature extraction in computer vision and pattern recognition nmf seeks two nonnegative matrices whose product can best approximate the original matrix the nonnegativity constraints lead to sparse parts based representations that can be more robust than nonsparse global features to obtain more accurate control over the sparseness in this paper we propose a novel method called nonnegative local coordinate factorization nlcf for feature extraction nlcf adds a local coordinate constraint into the standard nmf objective function specifically we require that the learned basis vectors be as close to the original data points as possible in this way each data point can be represented by a linear combination of only a few nearby basis vectors which naturally leads to sparse representation extensive experimental results suggest that the proposed approach provides a better representation and achieves higher accuracy in image clustering

manifold regularization a geometric framework for learning from labeled and unlabeled examples we propose a family of learning algorithms based on a new form of regularization that allows us to exploit the geometry of the marginal distribution we focus on a semi supervised framework that incorporates labeled and unlabeled data in a general purpose learner some transductive graph learning algorithms and standard methods including support vector machines and regularized least squares can be obtained as special cases we use properties of reproducing kernel hilbert spaces to prove new representer theorems that provide theoretical basis for the algorithms as a result in contrast to purely graph based approaches we obtain a natural out of sample extension to novel examples and so are able to handle both transductive and truly semi supervised settings we present experimental evidence suggesting that our semi supervised algorithms are able to use unlabeled data effectively finally we have a brief discussion of unsupervised and fully supervised learning within our general framework

structural and functional discovery in dynamic networks with non negative matrix factorization time series of graphs are increasingly prevalent in modern data and pose unique challenges to visual exploration and pattern extraction this paper describes the development and application of matrix factorizations for exploration and time varying community detection in time evolving graph sequences the matrix factorization model allows the user to home in on and display interesting underlying structure and its evolution over time the methods are scalable to weighted networks with a large number of time points or nodes and can accommodate sudden changes to graph topology our techniques are demonstrated with several dynamic graph series from both synthetic and real world data including citation and trade networks these examples illustrate how users can steer the techniques and combine them with existing methods to discover and display meaningful patterns in sizable graphs over many time points

ensemble manifold rank preserving for acceleration based human activity recognition with the rapid development of mobile devices and pervasive computing technologies acceleration based human activity recognition a difficult yet essential problem in mobile apps has received intensive attention recently different acceleration signals for representing different activities or even a same activity have different attributes which causes troubles in normalizing the signals we thus cannot directly compare these signals with each other because they are embedded in a nonmetric space therefore we present a nonmetric scheme that retains discriminative and robust frequency domain information by developing a novel ensemble manifold rank preserving emrp algorithm emrp simultaneously considers three aspects 1 it encodes the local geometry using the ranking order information of intraclass samples distributed on local patches 2 it keeps the discriminative information by maximizing the margin between samples of different classes and 3 it finds the optimal linear combination of the alignment matrices to approximate the intrinsic manifold lied in the data experiments are conducted on the south china university of technology naturalistic 3 d acceleration based activity dataset and the naturalistic mobile devices based human activity dataset to demonstrate the robustness and effectiveness of the new nonmetric scheme for acceleration based human activity recognition

partially shared latent factor learning with multiview data multiview representations reveal the fundamental attributes of the studied instances from different perspectives some common perspectives are reviewed by multiple views simultaneously while some specific ones are reflected by individual views that is there are two kinds of properties embedded in the multiview data 1 consistency and 2 complementarity different from most multiview learning approaches only focusing on either consistency or complementarity this paper proposes a novel semisupervised multiview learning algorithm called partially shared latent factor pslf learning which jointly exploits both consistent and complementary information among multiple views in pslf a nonnegative matrix factorization nmf based formulation is adopted to learn a compact and comprehensive partially shared latent representation which is composed of common latent factors shared by multiple views and some specific latent factors to each view with the learned representations of multiview data we introduce a robust sparse regression model to predict the cluster labels of labeled data by integrating the nmf based model and the regression model we obtain a unified formulation and propose a multiplicative based alternative algorithm for optimization in addition pslf can learn the weights of different views adaptively according to the reconstruction precisions of data matrices our experimental study indicates different multiview data that contains consistent and complementary information in different degrees in addition the encouraging results of the proposed algorithm are achieved in comparison with the state of the art algorithms on real world data sets

graph regularized restricted boltzmann machine the restricted boltzmann machine rbm has received an increasing amount of interest in recent years it determines good mapping weights that capture useful latent features in an unsupervised manner the rbm and its generalizations have been successfully applied to a variety of image classification and speech recognition tasks however most of the existing rbm based models disregard the preservation of the data manifold structure in many real applications the data generally reside on a low dimensional manifold embedded in high dimensional ambient space in this brief we propose a novel graph regularized rbm to capture features and learning representations explicitly considering the local manifold structure of the data by imposing manifold based locality that preserves constraints on the hidden layer of the rbm the model ultimately learns sparse and discriminative representations the representations can reflect data distributions while simultaneously preserving the local manifold structure of data we test our model using several benchmark image data sets for unsupervised clustering and supervised classification problem the results demonstrate that the performance of our method exceeds the state of the art alternatives

normalized cuts and image segmentation we propose a novel approach for solving the perceptual grouping problem in vision rather than focusing on local features and their consistencies in the image data our approach aims at extracting the global impression of an image we treat image segmentation as a graph partitioning problem and propose a novel global criterion the normalized cut for segmenting the graph the normalized cut criterion measures both the total dissimilarity between the different groups as well as the total similarity within the groups we show that an efficient computational technique based on a generalized eigenvalue problem can be used to optimize this criterion we applied this approach to segmenting static images as well as motion sequences and found the results to be very encouraging

constrained clustering with nonnegative matrix factorization nonnegative matrix factorization nmf and symmetric nmf symnmf have been shown to be effective for clustering linearly separable data and nonlinearly separable data respectively nevertheless many practical applications demand constrained algorithms in which a small number of constraints in the form of must link and cannot link are available in this paper we propose an nmf based constrained clustering framework in which the similarity between two points on a must link is enforced to approximate 1 and the similarity between two points on a cannot link is enforced to approximate 0 we then formulate the framework using nmf and symnmf to deal with clustering of linearly separable data and nonlinearly separable data respectively furthermore we present multiplicative update rules to solve them and show the correctness and convergence experimental results on various text data sets university of california irvine uci data sets and gene expression data sets demonstrate the superiority of our algorithms over existing constrained clustering algorithms

two fast vector wise update algorithms for orthogonal nonnegative matrix factorization with sparsity constraint abstract recently orthogonal nonnegative matrix factorization onmf has been introduced and shown to work remarkably well for clustering tasks because of the nonnegativity and the orthogonality constraints the orthogonal factor matrix of onmf is naturally sparse based on this fact by introducing sparsity constraints on the orthogonal coefficient matrix we propose two vector wise algorithms based on hierarchical alternating least squares hals and block prox linear bpl methods to the approximately sparse orthogonal nonnegative matrix factorization sonmf some global convergence results are established under the mild conditions numerical results including synthetic and real world datasets are given to illustrate that the proposed algorithms compute highly accurate values and perform better than the other testing onmf methods

enhancing low rank subspace clustering by manifold regularization recently low rank representation lrr method has achieved great success in subspace clustering sc which aims to cluster the data points that lie in a union of low dimensional subspace given a set of data points lrr seeks the lowest rank representation among the many possible linear combinations of the bases in a given dictionary or in terms of the data itself however lrr only considers the global euclidean structure while the local manifold structure which is often important for many real applications is ignored in this paper to exploit the local manifold structure of the data a manifold regularization characterized by a laplacian graph has been incorporated into lrr leading to our proposed laplacian regularized lrr laplrr an efficient optimization procedure which is based on alternating direction method of multipliers admm is developed for laplrr experimental results on synthetic and real data sets are presented to demonstrate that the performance of lrr has been enhanced by using the manifold regularization

snmfca supervised nmf based image classification and annotation in this paper we propose a novel supervised nonnegative matrix factorization based framework for both image classification and annotation the framework consists of two phases training and prediction in the training phase two supervised nonnegative matrix factorizations for image descriptors and annotation terms are combined to identify the latent image bases and to represent the training images in the bases space these latent bases can capture the representation of the images in terms of both descriptors and annotation terms based on the new representation of training images classifiers can be learnt and built in the prediction phase a test image is first represented by the latent bases via solving a linear least squares problem and then its class label and annotation can be predicted via the trained classifiers and the proposed annotation mapping model in the algorithm we develop a three block proximal alternating nonnegative least squares algorithm to determine the latent image bases and show its convergent property extensive experiments on real world image data sets suggest that the proposed framework is able to predict the label and annotation for testing images successfully experimental results have also shown that our algorithm is computationally efficient and effective for image classification and annotation

non negative matrix factorization with sparseness constraints non negative matrix factorization nmf is a recently developed technique for finding parts based linear representations of non negative data although it has successfully been applied in several applications it does not always result in parts based representations in this paper we show how explicitly incorporating the notion of sparseness improves the found decompositions additionally we provide complete matlab code both for standard nmf and for our extension our hope is that this will further the application of these methods to solving novel data analysis problems

multiview alignment hashing for efficient image search hashing is a popular and efficient method for nearest neighbor search in large scale data spaces by embedding high dimensional feature descriptors into a similarity preserving hamming space with a low dimension for most hashing methods the performance of retrieval heavily depends on the choice of the high dimensional feature descriptor furthermore a single type of feature cannot be descriptive enough for different images when it is used for hashing thus how to combine multiple representations for learning effective hashing functions is an imminent task in this paper we present a novel unsupervised multiview alignment hashing approach based on regularized kernel nonnegative matrix factorization which can find a compact representation uncovering the hidden semantics and simultaneously respecting the joint probability distribution of data in particular we aim to seek a matrix factorization to effectively fuse the multiple information sources meanwhile discarding the feature redundancy since the raised problem is regarded as nonconvex and discrete our objective function is then optimized via an alternate way with relaxation and converges to a locally optimal solution after finding the low dimensional representation the hashing functions are finally obtained through multivariable logistic regression the proposed method is systematically evaluated on three data sets 1 caltech 256 2 cifar 10 and 3 cifar 20 and the results show that our method significantly outperforms the state of the art multiview hashing techniques

laplacian regularized low rank representation and its applications low rank representation lrr has recently attracted a great deal of attention due to its pleasing efficacy in exploring low dimensional subspace structures embedded in data for a given set of observed data corrupted with sparse errors lrr aims at learning a lowest rank representation of all data jointly lrr has broad applications in pattern recognition computer vision and signal processing in the real world data often reside on low dimensional manifolds embedded in a high dimensional ambient space however the lrr method does not take into account the non linear geometric structures within data thus the locality and similarity information among data may be missing in the learning process to improve lrr in this regard we propose a general laplacian regularized low rank representation framework for data representation where a hypergraph laplacian regularizer can be readily introduced into i e a non negative sparse hyper laplacian regularized lrr model nshlrr by taking advantage of the graph regularizer our proposed method not only can represent the global low dimensional structures but also capture the intrinsic non linear geometric information in data the extensive experimental results on image clustering semi supervised image classification and dimensionality reduction tasks demonstrate the effectiveness of the proposed method

graph regularized nonnegative matrix factorization for data representation matrix factorization techniques have been frequently applied in information retrieval computer vision and pattern recognition among them nonnegative matrix factorization nmf has received considerable attention due to its psychological and physiological interpretation of naturally occurring data whose representation may be parts based in the human brain on the other hand from the geometric perspective the data is usually sampled from a low dimensional manifold embedded in a high dimensional ambient space one then hopes to find a compact representation which uncovers the hidden semantics and simultaneously respects the intrinsic geometric structure in this paper we propose a novel algorithm called graph regularized nonnegative matrix factorization gnmf for this purpose in gnmf an affinity graph is constructed to encode the geometrical information and we seek a matrix factorization which respects the graph structure our empirical study shows encouraging results of the proposed algorithm in comparison to the state of the art algorithms on real world problems

lasso screening rules via dual polytope projection lasso is a widely used regression technique to find sparse representations when the dimension of the feature space and the number of samples are extremely large solving the lasso problem remains challenging to improve the efficiency of solving large scale lasso problems el ghaoui and his colleagues have proposed the safe rules which are able to quickly identify the inactive predictors i e predictors that have 0 components in the solution vector then the inactive predictors or features can be removed from the optimization problem to reduce its scale by transforming the standard lasso to its dual form it can be shown that the inactive predictors include the set of inactive constraints on the optimal dual solution in this paper we propose an efficient and effective screening rule via dual polytope projections dpp which is mainly based on the uniqueness and nonexpansiveness of the optimal dual solution due to the fact that the feasible set in the dual space is a convex and closed polytope moreover we show that our screening rule can be extended to identify inactive groups in group lasso to the best of our knowledge there is currently no exact screening rule for group lasso we have evaluated our screening rule using synthetic and real data sets results show that our rule is more effective in identifying inactive predictors than existing state of the art screening rules for lasso

low rank matrix factorization with multiple hypergraph regularizer this paper presents a novel low rank matrix factorization method named multihmmf which incorporates multiple hypergraph manifold regularization to the low rank matrix factorization in order to effectively exploit high order information among the data samples the hypergraph is introduced to model the local structure of the intrinsic manifold specifically multiple hypergraph regularization terms are separately constructed to consider the local invariance the optimal intrinsic manifold is constructed by linearly combining multiple hypergraph manifolds then the regularization term is incorporated into a truncated singular value decomposition framework resulting in a unified objective function so that matrix factorization is changed into an optimization problem alternating optimization is used to solve the optimization problem with the result that the low dimensional representation of data space is obtained the experimental results of image clustering demonstrate that the proposed method outperforms state of the art data representation methods we propose a novel low rank matrix factorization method we incorporate multiple hypergraph manifold regularization to the matrix factorization we adopt alternating optimization to solve the optimization problem

on the convergence of multiplicative update algorithms for nonnegative matrix factorization nonnegative matrix factorization nmf is useful to find basis information of nonnegative data currently multiplicative updates are a simple and popular way to find the factorization however for the common nmf approach of minimizing the euclidean distance between approximate and true values no proof has shown that multiplicative updates converge to a stationary point of the nmf optimization problem stationarity is important as it is a necessary condition of a local minimum this paper discusses the difficulty of proving the convergence we propose slight modifications of existing updates and prove their convergence techniques invented in this paper may be applied to prove the convergence for other bound constrained optimization problems

on reduced rank nonnegative matrix factorization for symmetric nonnegative matrices abstract let v r m n be a nonnegative matrix the nonnegative matrix factorization nnmf problem consists of finding nonnegative matrix factors w r m r and h r r n such that v wh lee and seung proposed two algorithms one of which finds nonnegative w and h such that v wh f is minimized after examining the case in which r 1 about which a complete characterization of the solution is possible we consider the case in which m n and v is symmetric we focus on questions concerning when the best approximate factorization results in the product wh being symmetric and on cases in which the best approximation cannot be a symmetric matrix finally we show that the class of positive semidefinite symmetric nonnegative matrices v generated via a soules basis admit for every 1 r rank v a nonnegative factorization wh which coincides with the best approximation in the frobenius norm to v in r n n of rank not exceeding r an example of applications in which nnmf factorizations for nonnegative symmetric matrices v arise is video and other media summarization technology where v is obtained from a distance matrix

truncated cauchy non negative matrix factorization non negative matrix factorization nmf minimizes the euclidean distance between the data matrix and its low rank approximation and it fails when applied to corrupted data because the loss function is sensitive to outliers in this paper we propose a truncated cauchynmf loss that handle outliers by truncating large errors and develop a truncated cauchynmf to robustly learn the subspace on noisy datasets contaminated by outliers we theoretically analyze the robustness of truncated cauchynmf comparing with the competing models and theoretically prove that truncated cauchynmf has a generalization bound which converges at a rate of order o sqrt ln n n where n is the sample size we evaluate truncated cauchynmf by image clustering on both simulated and real datasets the experimental results on the datasets containing gross corruptions validate the effectiveness and robustness of truncated cauchynmf for learning robust subspaces

spectral unmixing via data guided sparsity hyperspectral unmixing the process of estimating a common set of spectral bases and their corresponding composite percentages at each pixel is an important task for hyperspectral analysis visualization and understanding from an unsupervised learning perspective this problem is very challenging both the spectral bases and their composite percentages are unknown making the solution space too large to reduce the solution space many approaches have been proposed by exploiting various priors in practice these priors would easily lead to some unsuitable solution this is because they are achieved by applying an identical strength of constraints to all the factors which does not hold in practice to overcome this limitation we propose a novel sparsity based method by learning a data guided map dgmap to describe the individual mixed level of each pixel through this dgmap the ell p left 0 constraint is applied in an adaptive manner such implementation not only meets the practical situation but also guides the spectral bases toward the pixels under highly sparse constraint what is more an elegant optimization scheme as well as its convergence proof have been provided in this paper extensive experiments on several datasets also demonstrate that the dgmap is feasible and high quality unmixing results could be obtained by our method

robust semi nonnegative graph embedding nonnegative matrix factorization nmf has received considerable attention in image processing computer vision and patter recognition an important variant of nmf is nonnegative graph embedding nge which encodes the statistical or geometric information of data in the process of matrix factorization the nge offers a general framework for unsupervised supervised settings however nge like algorithms often suffer from noisy data unreliable graphs and noisy labels which are commonly encountered in real world applications to address these issues in this paper we first propose a robust nonnegative graph embedding rnge framework where the joint sparsity in both graph embedding and data reconstruction endues robustness to undesirable noises next we present a robust seminonnegative graph embedding rsnge framework which only constrains the coefficient matrix to be nonnegative while places no constraint on the base matrix this extends the applicable range of rnge to data which are not nonnegative and endows more discriminative power of the learnt base matrix the rnge rsnge provides a general formulation such that all the algorithms unified within the graph embedding framework can be easily extended to obtain their robust nonnegative seminonnegative solutions further we develop elegant multiplicative updating solutions that can solve rnge rsnge efficiently and offer a rigorous convergence analysis we conduct extensive experiments on four real world data sets and compare the proposed rnge rsnge to other representative nmf variants and data factorization methods the experimental results demonstrate the robustness and effectiveness of the proposed approaches

sce a manifold regularized set covering method for data partitioning cluster analysis plays a very important role in data analysis in these years cluster ensemble as a cluster analysis tool has drawn much attention for its robustness stability and accuracy many efforts have been done to combine different initial clustering results into a single clustering solution with better performance however they neglect the structure information of the raw data in performing the cluster ensemble in this paper we propose a structural cluster ensemble sce algorithm for data partitioning formulated as a set covering problem in particular we construct a laplacian regularized objective function to capture the structure information among clusters moreover considering the importance of the discriminative information underlying in the initial clustering results we add a discriminative constraint into our proposed objective function finally we verify the performance of the sce algorithm on both synthetic and real data sets the experimental results show the effectiveness of our proposed method sce algorithm

mr ntd manifold regularization nonnegative tucker decomposition for tensor data dimension reduction and representation with the advancement of data acquisition techniques tensor multidimensional data objects are increasingly accumulated and generated for example multichannel electroencephalographies multiview images and videos in these applications the tensor objects are usually nonnegative since the physical signals are recorded as the dimensionality of tensor objects is often very high a dimension reduction technique becomes an important research topic of tensor data from the perspective of geometry high dimensional objects often reside in a low dimensional submanifold of the ambient space in this paper we propose a new approach to perform the dimension reduction for nonnegative tensor objects our idea is to use nonnegative tucker decomposition ntd to obtain a set of core tensors of smaller sizes by finding a common set of projection matrices for tensor objects to preserve geometric information in tensor data we employ a manifold regularization term for the core tensors constructed in the tucker decomposition an algorithm called manifold regularization ntd mr ntd is developed to solve the common projection matrices and core tensors in an alternating least squares manner the convergence of the proposed algorithm is shown and the computational complexity of the proposed method scales linearly with respect to the number of tensor objects and the size of the tensor objects respectively these theoretical results show that the proposed algorithm can be efficient extensive experimental results have been provided to further demonstrate the effectiveness and efficiency of the proposed mr ntd algorithm

on the convergence of constrained parallel variable distribution algorithms we consider the parallel variable distribution pvd approach proposed by ferris and mangasarian siam j optim 4 1994 pp 815 832 for solving optimization problems the problem variables are distributed among p processors with each processor having the primary responsibility for updating its block of variables while allowing the remaining secondary variables to change in a restricted fashion along some easily computable directions for constrained nonlinear programs convergence in m c ferris and o l mangasarian siam j optim 4 1994 pp 815 832 was established in the special case of convex block separable constraints for general inseparable constraints it was suggested that a dual differentiable exact penalty function reformulation of the problem be used we propose to apply the pvd approach to problems with general convex constraints directly and show that the algorithm converges provided certain conditions are imposed on the change of secondary variables these conditions are both natural and practically implementable we also show that the original requirement of exact global solution of the parallel subproblems can be replaced by a less stringent sufficient descent condition the first rate of convergence result for the class of constrained pvd algorithms is also given

semi supervised concept factorization for document clustering nonnegative matrix factorization nmf and concept factorization cf are two popular methods for finding the low rank approximation of nonnegative matrix different from nmf cf can be applied not only to the matrix containing negative values but also to the kernel space based on nmf and cf many methods such as graph regularized nonnegative matrix factorization gnmf and locally consistent clustering factorization lccf can significantly improve the performance of clustering unfortunately these are unsupervised learning methods in order to enhance the clustering performance with the supervisory information a semi supervised concept factorization sscf is proposed in this paper by incorporating the pairwise constraints into cf as the reward and penalty terms which can guarantee that the data points belonging to a cluster in the original space are still in the same cluster in the transformed space by comparing with the state of the arts algorithms km nmf cf lccf gnmf pccf experimental results on document clustering show that the proposed algorithm has better performance in terms of accuracy and mutual information

projective robust nonnegative factorization nonnegative matrix factorization nmf has been successfully used in many fields as a low dimensional representation method projective nonnegative matrix factorization pnmf is a variant of nmf that was proposed to learn a subspace for feature extraction however both original nmf and pnmf are sensitive to noise and are unsuitable for feature extraction if data is grossly corrupted in order to improve the robustness of nmf a framework named projective robust nonnegative factorization prnf is proposed in this paper for robust image feature extraction and classification since learned projections can weaken noise disturbances prnf is more suitable for classification and feature extraction in order to preserve the geometrical structure of original data prnf introduces a graph regularization term which encodes geometrical structure in the prnf framework three algorithms are proposed that add a sparsity constraint on the noise matrix based on l1 2 norm l1 norm and l2 1 norm respectively robustness and classification performance of the three proposed algorithms are verified with experiments on four face image databases and results are compared with state of the art robust nmf based algorithms experimental results demonstrate the robustness and effectiveness of the algorithms for image classification and feature extraction

global discriminative based nonnegative spectral clustering based on spectral graph theory spectral clustering is an optimal graph partition problem it has been proven that the spectral clustering is equivalent to nonnegative matrix factorization nmf under certain conditions based on the equivalence some spectral clustering methods are proposed but the global discriminative information of the dataset is neglected in this paper based on the equivalence between spectral clustering and nmf we simultaneously maximize the between class scatter matrix and minimize the within class scatter matrix to enhance the discriminating power we integrate the geometrical structure and discriminative structure in a joint framework with a global discriminative regularization term added into the nonnegative matrix factorization framework we propose two novel spectral clustering methods named global discriminative based nonnegative and spectral clustering gdbnsc ncut and gdbnsc rcut these new spectral clustering algorithms can preserve both the global geometrical structure and global discriminative structure the intrinsic geometrical information of the dataset is detected and clustering quality is improved with enhanced discriminating power in addition the proposed algorithms also have very good abilities of handling out of sample data experimental results on real word data demonstrate that the proposed algorithms outperform some state of the art methods with good clustering qualities we maximize the between class scatter matrix meanwhile we minimize the within class scatter matrix we integrate the geometrical structure and discriminative structure in a framework a global discriminative regularization term added into the objective functions the new algorithms can preserve the global geometrical and discriminative structure

learning fast low rank projection for image classification rooted in a basic hypothesis that a data matrix is strictly drawn from some independent subspaces the low rank representation lrr model and its variations have been successfully applied in various image classification tasks however this hypothesis is very strict to the lrr model as it cannot always be guaranteed in real images moreover the hypothesis also prevents the sub dictionaries of different subspaces from collaboratively representing an image fortunately in supervised image classification low rank signal can be extracted from the independent label subspaces ils instead of the independent image subspaces iis therefore this paper proposes a projective low rank representation plr model by directly training a projective function to approximate the lrr derived from the labels to the best of our knowledge plr is the first attempt to use the ils hypothesis to relax the rigorous iis hypothesis in the lrr models we further prove a low rank effect that the representations learned by plr have high intraclass similarities and large interclass differences which are beneficial to the classification tasks the effectiveness of our proposed approach is validated by the experimental results on three databases

local coordinate concept factorization for image representation learning sparse representation of high dimensional data is a state of the art method for modeling data matrix factorization based techniques such as nonnegative matrix factorization and concept factorization cf have shown great advantages in this area especially useful for image representation both of them are linear learning problems and lead to a sparse representation of the images however the sparsity obtained by these methods does not always satisfy locality conditions for example the learned new basis vectors may be relatively far away from the original data thus we may not be able to achieve the optimal performance when using the new representation for other learning tasks such as classification and clustering in this paper we introduce a locality constraint into the traditional cf by requiring the concepts basis vectors to be as close to the original data points as possible each datum can be represented by a linear combination of only a few basis concepts thus our method is able to achieve sparsity and locality simultaneously we analyze the complexity of our novel algorithm and demonstrate the effectiveness in comparison with the state of the art approaches through a set of evaluations based on real world applications

learning descriptive visual representation for image classification and annotation abstract this paper presents a novel semantic regularized matrix factorization method for learning descriptive visual bag of words bow representation although very influential in image classification the traditional visual bow representation has one distinct drawback that is for efficiency purposes this visual representation is often generated by directly clustering the low level visual feature vectors extracted from local keypoints or regions without considering the high level semantics of images in other words it still suffers from the semantic gap and may lead to significant performance degradation in more challenging tasks e g image classification over social collections with large intra class variations to learn descriptive visual bow representation for such image classification task we develop a semantic regularized matrix factorization method by adding laplacian regularization defined with the tags easy to access of social images into matrix factorization moreover given that image annotation only provides the tags of training images in advance while the tags of all social images are available we can readily apply the proposed method to image annotation by first running a round of image annotation to predict the tags maybe incorrect of test images and thus obtaining the tags of all images experimental results show the promising performance of the proposed method