nonsmooth nonnegative matrix factorization nsnmf we propose a novel nonnegative matrix factorization model that aims at finding localized part based representations of nonnegative multivariate data items unlike the classical nonnegative matrix factorization nmf technique this new model denoted nonsmooth nonnegative matrix factorization nsnmf corresponds to the optimization of an unambiguous cost function designed to explicitly represent sparseness in the form of nonsmoothness which is controlled by a single parameter in general this method produces a set of basis and encoding vectors that are not only capable of representing the original data but they also extract highly focalized patterns which generally lend themselves to improved interpretability the properties of this new method are illustrated with several data sets comparisons to previously published methods show that the new nsnmf method has some advantages in keeping faithfulness to the data in the achieving a high degree of sparseness for both the estimated basis and the encoding vectors and in better interpretability of the factors

on the geometric interpretation of the nonnegative rank the nonnegative rank of a nonnegative matrix is the minimum number of nonnegative rank one factors needed to reconstruct it exactly the problem of determining this rank and computing the corresponding nonnegative factors is difficult however it has many potential applications e g in data mining graph theory and computational geometry in particular it can be used to characterize the minimal size of any extended reformulation of a given combinatorial optimization program in this paper we introduce and study a related quantity called the restricted nonnegative rank we show that computing this quantity is equivalent to a problem in polyhedral combinatorics and fully characterize its computational complexity this in turn sheds new light on the nonnegative rank problem and in particular allows us to provide new improved lower bounds based on its geometric interpretation we apply these results to slack matrices and linear euclidean distance matrices and obtain counter examples to two conjectures of beasly and laffey namely we show that the nonnegative rank of linear euclidean distance matrices is not necessarily equal to their dimension and that the rank of a matrix is not always greater than the nonnegative rank of its square

on the estimation of a convex set with corners in robotic vision using laser radar measurements noisy data on convex sets with corners are derived in terms of the set s support function the corners represent abutting edges of manufactured items and convey important information about the items shape however simple methods for set estimation for example based on fitting random polygons or smooth sets either add additional corners as an artifact of the algorithm or approximate corners by smooth curves it might be argued however that corners have special significance in the interpretation of a set and should not be introduced as an artifact of the estimation procedure in this paper we suggest a corner diagnostic approach in the form of a three step algorithm which a identifies the number and positions of corners b fits smooth curves between corners and c splices together the smooth curves and the corners to produce an over all estimate of the convex set the corner finding step is parametric in character and although it is based on detecting change points in high order derivatives of the support function it produces root n consistent estimators of the locations of corners on the other hand the smooth curve fitting step is entirely nonparametric the splicing step marries these two disparate approaches into a single practical method

nonlinear non negative component analysis algorithms in this paper general solutions for nonlinear non negative component analysis for data representation and recognition are proposed motivated by a combination of the non negative matrix factorization nmf algorithm and kernel theory which has lead to a recently proposed nmf algorithm in a polynomial feature space we propose a general framework where one can build a nonlinear non negative component analysis method using kernels the so called projected gradient kernel non negative matrix factorization pgknmf in the proposed approach arbitrary positive definite kernels can be adopted while at the same time it is ensured that the limit point of the procedure is a stationary point of the optimization problem moreover we propose fixed point algorithms for the special case of gaussian radial basis function rbf kernels we demonstrate the power of the proposed methods in face and facial expression recognition applications

numerical experiments in 2 d ihcp on bounded domains part i the interior cube problem abstract different space marching implementations of the mollification method are introduced to numerically recover the temperature and heat flux histories at interior points of bounded subdomains of a finite two dimensional rectangular body when the temperature and heat flux functions are approximately measured at one boundary side

a method for approximating the solution set of a system of convex inequalities by polytopes abstract in this note a method for computing approximating by polytopes of the solution set q of a system of convex inequalities is presented it is shown that such approximations can be determined by an algorithm which converges in finitely many steps when the solution set of the given system of inequalities is bounded in this case the algorithm generates inner and outer approximations having the hausdorff distance to each other and to the set q no greater than an a priori fixed and having their extreme points in q and in the relative exterior of q respectively

adaptive method for nonsmooth nonnegative matrix factorization nonnegative matrix factorization nmf is an emerging tool for meaningful low rank matrix representation in nmf explicit constraints are usually required such that nmf generates desired products or factorizations especially when the products have significant sparseness features it is known that the ability of nmf in learning sparse representation can be improved by embedding a smoothness factor between the products motivated by this result we propose an adaptive nonsmooth nmf ans nmf method in this paper in our method the embedded factor is obtained by using a data related approach so it matches well with the underlying products implying a superior faithfulness of the representations besides due to the usage of an adaptive selection scheme to this factor the sparseness of the products can be separately constrained leading to wider applicability and interpretability furthermore since the adaptive selection scheme is processed through solving a series of typical linear programming problems it can be easily implemented simulations using computer generated data and real world data show the advantages of the proposed ans nmf method over the state of the art methods

a quasi likelihood approach to nonnegative matrix factorization a unified approach to nonnegative matrix factorization based on the theory of generalized linear models is proposed this approach embeds a variety of statistical models including the exponential family within a single theoretical framework and provides a unified view of such factorizations from the perspective of quasi likelihood using this framework a family of algorithms for handling signal dependent noise is developed and its convergence proved using the expectation maximization algorithm in addition a measure to evaluate the goodness of fit of the resulting factorization is described the proposed methods allow modeling of nonlinear effects using appropriate link functions and are illustrated using an application in biomedical signal processing

multiplicative update rules for concurrent nonnegative matrix factorization and maximum margin classification the state of the art classification methods which employ nonnegative matrix factorization nmf employ two consecutive independent steps the first one performs data transformation dimensionality reduction and the second one classifies the transformed data using classification methods such as nearest neighbor centroid or support vector machines svms in the following we focus on using nmf factorization followed by svm classification typically the parameters of these two steps e g the nmf bases coefficients and the support vectors are optimized independently thus leading to suboptimal classification performance in this paper we merge these two steps into one by incorporating maximum margin classification constraints into the standard nmf optimization the notion behind the proposed framework is to perform nmf while ensuring that the margin between the projected data of the two classes is maximal the concurrent nmf factorization and support vector optimization are performed through a set of multiplicative update rules in the same context the maximum margin classification constraints are imposed on the nmf problem with additional discriminant constraints and respective multiplicative update rules are extracted the impact of the maximum margin classification constraints on the nmf factorization problem is addressed in section vi experimental results in several databases indicate that the incorporation of the maximum margin classification constraints into the nmf and discriminant nmf objective functions improves the accuracy of the classification

maximum contrast analysis for nonnegative blind source separation in this paper we propose a maximum contrast analysis mca method for nonnegative blind source separation where both the mixing matrix and the source signals are nonnegative we first show that the contrast degree of the source signals is greater than that of the mixed signals motivated by this observation we propose an mca based cost function it is further shown that the separation matrix can be obtained by maximizing the proposed cost function then we derive an iterative determinant maximization algorithm for estimating the separation matrix in the case of two sources a closed form solution exists and is derived unlike most existing blind source separation methods the proposed mca method needs neither the independence assumption nor the sparseness requirement of the sources the effectiveness of the new method is illustrated by experiments using x ray images remote sensing images infrared spectral images and real world fluorescence microscopy images

population dynamics of spiking neurons fast transients asynchronous states and locking an integral equation describing the time evolution of the population activity in a homogeneous pool of spiking neurons of the integrate and fire type is discussed it is analytically shown that transients from a state of incoherent firing can be immediate the stability of incoherent firing is analyzed in terms of the noise level and transmission delay and a bifurcation diagram is derived the response of a population of noisy integrate and fire neurons to an input current of small amplitude is calculated and characterized by a linear filter l the stability of perfectly synchronized locked solutions is analyzed

extracting non negative basis images using pixel dispersion penalty non negativity matrix factorization nmf and its variants have been explored in the last decade and are still attractive due to its ability of extracting non negative basis images however most existing nmf based methods are not ready for encoding higher order data information one reason is that they do not directly explicitly model structured data information during learning and therefore the extracted basis images may not completely describe the parts in an image 1 very well in order to solve this problem the structured sparse nmf has been recently proposed in order to learn structured basis images it however depends on some special prior knowledge i e one needs to exhaustively define a set of structured patterns in advance in this paper we wish to perform structured sparsity learning as automatically as possible to that end we propose a pixel dispersion penalty pdp which effectively describes the spatial dispersion of pixels in an image without using any manually predefined structured patterns as constraints in pdp we consider each part based feature pattern of an image as a cluster of non zero pixels that is the non zero pixels of a local pattern should be spatially close to each other furthermore by incorporating the proposed pdp we develop a spatial non negative matrix factorization spatial nmf and a spatial non negative component analysis spatial nca in spatial nca the non negativity constraint is only imposed on basis images and such constraint on coefficients is released so both subtractive and additive combinations of non negative basis images are allowed for reconstructing any images extensive experiments are conducted to validate the effectiveness of the proposed pixel dispersion penalty we also experimentally show that spatial nca is more flexible for extracting non negative basis images and obtains better and more stable performance

sparse and unique nonnegative matrix factorization through data preprocessing nonnegative matrix factorization nmf has become a very popular technique in machine learning because it automatically extracts meaningful features through a sparse and part based representation however nmf has the drawback of being highly ill posed that is there typically exist many different but equivalent factorizations in this paper we introduce a completely new way to obtaining more well posed nmf problems whose solutions are sparser our technique is based on the preprocessing of the nonnegative input data matrix and relies on the theory of m matrices and the geometric interpretation of nmf this approach provably leads to optimal and sparse solutions under the separability assumption of donoho and stodden 2003 and for rank three matrices makes the number of exact factorizations finite we illustrate the effectiveness of our technique on several image data sets

an optimal modified method for a two dimensional inverse heat conduction problem we consider a two dimensional inverse heat conduction problem which is severely ill posed i e the solution if it exists does not depend continuously on the data from the frequency domain we propose an optimal modified method to solve the problem in the presence of noisy data we give and prove the optimal convergence estimate which shows that the regularized solution is dependent continuously on the data and is an approximation of the exact solution of the two dimensional inverse heat conduction problem

new families of skewed higher order kernel estimators to solve the bss ica problem for multimodal sources mixtures this letter suggests two new types of asymmetrical higher order kernels hok that are generated using the orthogonal polynomials laguerre positive or right skew and bessel negative or left skew these skewed hok are implemented in the blind source separation independent component analysis bss ica algorithm the tests for these proposed hok are accomplished using three scenarios to simulate a real environment using actual sound sources an environment of mixtures of multimodal fast changing probability density function pdf sources that represent a challenge to the symmetrical hok and an environment of an adverse case near gaussian the separation is performed by minimizing the mutual information mi among the mixed sources the performance of the skewed kernels is compared to the performance of the standard kernels such as epanechnikov bisquare trisquare and gaussian and the performance of the symmetrical hok generated using the polynomials chebyshev1 chebyshev2 gegenbauer jacobi and legendre to the tenth order the gaussian hok are generated using the hermite polynomial and the wand and schucany procedure the comparison among the 96 kernels is based on the average intersymbol interference ratio aisir and the time needed to complete the separation in terms of aisir the skewed kernels performance is better than that of the standard kernels and rivals most of the symmetrical kernels performance the importance of these new skewed hok is manifested in the environment of the multimodal pdf mixtures in such an environment the skewed hok come in first place compared with the symmetrical hok these new families can substitute for symmetrical hoks in such applications

uniqueness of nonnegative tensor approximations we show that for a nonnegative tensor a best nonnegative rank r approximation is almost always unique its best rank one approximation may always be chosen to be a best nonnegative rank one approximation and the set of nonnegative tensors with nonunique best rank one approximations forms an algebraic hypersurface we show that the last part holds true more generally for real tensors and thereby determine a polynomial equation so that a real or nonnegative tensor that does not satisfy this equation is guaranteed to have a unique best rank one approximation we also establish an analogue for real or nonnegative symmetric tensors in addition we prove a singular vector variant of the perron frobenius theorem for positive tensors and apply it to show that a best nonnegative rank r approximation of a positive tensor can never be obtained by deflation as an aside we verify that the euclidean distance ed discriminants of the segre variety and the veronese variety are hypersurfaces and give defining equations of these ed discriminants

population coding and decoding in a neural field a computational study this study uses a neural field model to investigate computational aspects of population coding and decoding when the stimulus is a single variable a general prototype model for the encoding process is proposed in which neural responses are correlated with strength specified by a gaussian function of their difference in preferred stimuli based on the model we study the effect of correlation on the fisher information compare the performances of three decoding methods that differ in the amount of encoding information being used and investigate the implementation of the three methods by using a recurrent network this study not only rediscovers main results in existing literatures in a unified way but also reveals important new features especially when the neural correlation is strong as the neural correlation of firing becomes larger the fisher information decreases drastically we confirm that as the width of correlation increases the fisher information saturates and no longer increases in proportion to the number of neurons however we prove that as the width increases further wider than 2 times the effective width of the turning function the fisher information increases again and it increases without limit in proportion to the number of neurons furthermore we clarify the asymptotic efficiency of the maximum likelihood inference mli type of decoding methods for correlated neural signals it shows that when the correlation covers a nonlocal range of population excepting the uniform correlation and when the noise is extremely small the mli type of method whose decoding error satisfies the cauchy type distribution is not asymptotically efficient this implies that the variance is no longer adequate to measure decoding accuracy

quadratic nonnegative matrix factorization in nonnegative matrix factorization nmf a nonnegative matrix is approximated by a product of lower rank factorizing matrices most nmf methods assume that each factorizing matrix appears only once in the approximation thus the approximation is linear in the factorizing matrices we present a new class of approximative nmf methods called quadratic nonnegative matrix factorization qnmf where some factorizing matrices occur twice in the approximation we demonstrate qnmf solutions to four potential pattern recognition problems in graph partitioning two way clustering estimating hidden markov chains and graph matching we derive multiplicative algorithms that monotonically decrease the approximation error under a variety of measures we also present extensions in which one of the factorizing matrices is constrained to be orthogonal or stochastic empirical studies show that for certain application scenarios qnmf is more advantageous than other existing nonnegative matrix factorization methods

non negative matrix factorization with sparseness constraints non negative matrix factorization nmf is a recently developed technique for finding parts based linear representations of non negative data although it has successfully been applied in several applications it does not always result in parts based representations in this paper we show how explicitly incorporating the notion of sparseness improves the found decompositions additionally we provide complete matlab code both for standard nmf and for our extension our hope is that this will further the application of these methods to solving novel data analysis problems

the spherical convex floating body abstract for a convex body on the euclidean unit sphere the spherical convex floating body is introduced the asymptotic behavior of the volume difference of a spherical convex body and its spherical floating body is investigated this gives rise to a new spherical area measure the floating area remarkably this floating area turns out to be a spherical analogue to the classical affine surface area from affine differential geometry several properties of the floating area are established

non negative matrix factorization revisited uniqueness and algorithm for symmetric decomposition non negative matrix factorization nmf has found numerous applications due to its ability to provide interpretable decompositions perhaps surprisingly existing results regarding its uniqueness properties are rather limited and there is much room for improvement in terms of algorithms as well uniqueness aspects of nmf are revisited here from a geometrical point of view both symmetric and asymmetric nmf are considered the former being tantamount to element wise non negative square root factorization of positive semidefinite matrices new uniqueness results are derived e g it is shown that a sufficient condition for uniqueness is that the conic hull of the latent factors is a superset of a particular second order cone checking this condition is shown to be np complete yet this and other results offer insights on the role of latent sparsity in this context on the computational side a new algorithm for symmetric nmf is proposed which is very different from existing ones it alternates between procrustes rotation and projection onto the non negative orthant to find a non negative matrix close to the span of the dominant subspace simulation results show promising performance with respect to the state of art finally the new algorithm is applied to a clustering problem for co authorship data yielding meaningful and interpretable results

1d lda vs 2d lda when is vector based linear discriminant analysis better than matrix based recent advances have shown that algorithms with 2d matrix based representation perform better than the traditional 1d vector based ones in particular 2d lda has been widely reported to outperform 1d lda however would the matrix based linear discriminant analysis be always superior and when would 1d lda be better in this paper we investigate into these questions and have a comprehensive comparison between 1d lda and 2d lda in theory and in experiments we analyze the heteroscedastic problem in 2d lda and formulate mathematical equalities to explore the relationship between 1d lda and 2d lda then we point out potential problems in 2d lda it is shown that 2d lda has eliminated the information contained in the covariance information between different local geometric structures such as the rows or the columns which is useful for discriminant feature extraction whereas 1d lda could preserve such information interestingly this new finding indicates that 1d lda is able to gain higher fisher score than 2d lda in some extreme case furthermore sufficient conditions on which 2d lda would be bayes optimal for two class classification problem are derived and comparison with 1d lda in this aspect is also analyzed this could help understand how 2d lda is expected to achieve at its best further discover its relationship with 1d lda and well support other findings after the theoretical analysis comprehensive experimental results are reported by fairly and extensively comparing 1d lda with 2d lda in contrast to the existing view that some 2d lda based algorithms would perform better than 1d lda when the number of training samples for each class is small or when the number of discriminant features used is small we show that it is not always true and show that some standard 1d lda based algorithms could perform better in those cases on some challenging data sets

rank functions of tropical matrices abstract we consider rank functions important in tropical linear algebra the tropical rank equal to the topological dimension of the tropical linear span of the columns of a given matrix the factor rank equal to the smallest number of vectors containing these columns in their span the kapranov rank related to problems of tropical algebraic geometry and the determinantal and gondran minoux ranks which generalize the classical linear algebraic notion of rank we are interested in studying the arithmetic properties of the rank functions their mutual behavior and the related computational problems we discuss different open problems related to rank functions of tropical matrices and illustrate them by the number of examples

symmetric nonnegative matrix factorization algorithms and applications to probabilistic clustering nonnegative matrix factorization nmf is an unsupervised learning method useful in various applications including image processing and semantic analysis of documents this paper focuses on symmetric nmf snmf which is a special case of nmf decomposition three parallel multiplicative update algorithms using level 3 basic linear algebra subprograms directly are developed for this problem first by minimizing the euclidean distance a multiplicative update algorithm is proposed and its convergence under mild conditions is proved based on it we further propose another two fast parallel methods snmf and snmf algorithms all of them are easy to implement these algorithms are applied to probabilistic clustering we demonstrate their effectiveness for facial image clustering document categorization and pattern clustering in gene expression

maximum likelihood decoding of neuronal inputs from an interspike interval distribution an expression for the probability distribution of the interspike interval of a leaky integrate and fire lif model neuron is rigorously derived based on recent theoretical developments in the theory of stochastic processes this enables us to find for the first time a way of developing maximum likelihood estimates mle of the input information e g afferent rate and variance for an lif neuron from a set of recorded spike trains dynamic inputs to pools of lif neurons both with and without interactions are efficiently and reliably decoded by applying the mle even within time windows as short as 25 msec

low dimensional polytope approximation and its applications to nonnegative matrix factorization in this study nonnegative matrix factorization is recast as the problem of approximating a polytope on the probability simplex by another polytope with fewer facets working on the probability simplex has the advantage that data are limited to a compact set with a known boundary making it easier to trace the approximation procedure in particular the supporting hyperplane that separates a point from a disjoint polytope a fact asserted by the hahn banach theorem can be calculated in finitely many steps this approach leads to a convenient way of computing the proximity map which in contrast to most existing algorithms where only an approximate map is used finds the unique and global minimum per iteration this paper sets up a theoretical framework outlines a numerical algorithm and suggests an effective implementation testing results strongly evidence that this approach obtains a better low rank nonnegative matrix approximation in fewer steps than conventional methods

discriminant nonnegative tensor factorization algorithms nonnegative matrix factorization nmf has proven to be very successful for image analysis especially for object representation and recognition nmf requires the object tensor with valence more than one to be vectorized this procedure may result in information loss since the local object structure is lost due to vectorization recently in order to remedy this disadvantage of nmf methods nonnegative tensor factorizations ntf algorithms that can be applied directly to the tensor representation of object collections have been introduced in this paper we propose a series of unsupervised and supervised ntf methods that is we extend several nmf methods using arbitrary valence tensors moreover by incorporating discriminant constraints inside the ntf decompositions we present a series of discriminant ntf methods the proposed approaches are tested for face verification and facial expression recognition where it is shown that they outperform other popular subspace approaches

suprathreshold stochastic resonance in neural processing tuned by correlation suprathreshold stochastic resonance ssr is examined in the context of integrate and fire neurons with an emphasis on the role of correlation in the neuronal firing we employed a model based on a network of spiking neurons which received synaptic inputs modeled by poisson processes stimulated by a stepped input signal the smoothed ensemble firing rate provided an output signal and the mutual information between this signal and the input was calculated for networks with different noise levels and different numbers of neurons it was found that an ssr effect was present in this context we then examined a more biophysically plausible scenario where the noise was not controlled directly but instead was tuned by the correlation between the inputs the ssr effect remained present in this scenario with nonzero noise providing improved information transmission and it was found that negative correlation between the inputs was optimal finally an examination of ssr in the context of this model revealed its connection with more traditional stochastic resonance and showed a trade off between supratheshold and subthreshold components we discuss these results in the context of existing empirical evidence concerning correlations in neuronal firing

incremental subspace learning via non negative matrix factorization in this paper we introduce an incremental non negative matrix factorization inmf scheme in order to overcome the difficulties that conventional nmf has in online processing of large data sets the proposed scheme enables incrementally updating its factors by reflecting the influence of each observation on the factorization appropriately this is achieved via a weighted cost function which also allows controlling the memorylessness of the factorization unlike conventional nmf with its incremental nature and weighted cost function the inmf scheme successfully utilizes adaptability to dynamic data content changes with a lower computational complexity test results reported for two video applications namely background modeling in video surveillance and clustering demonstrate that inmf is capable of online representing data content while reducing dimension significantly

decoding spikes in a spiking neuronal network we investigate how to reliably decode the input information from the output of a spiking neuronal network a maximum likelihood estimator of the input signal together with its fisher information is rigorously calculated the advantage of the maximum likelihood estimation over the brute force rate coding estimate is clearly demonstrated it is pointed out that the ergodic assumption in neuroscience i e a temporal average is equivalent to an ensemble average is in general not true averaging over an ensemble of neurons usually gives a biased estimate of the input information a method on how to compensate for the bias is proposed reconstruction of dynamical input signals with a group of spiking neurons is extensively studied and our results show that less than a spike is sufficient to accurately decode dynamical inputs

nonnegative matrix factorization algorithms for link prediction in temporal networks using graph communicability abstract networks derived from many disciplines such as social relations web contents and cancer progression are temporal and incomplete link prediction in temporal networks is of theoretical interest and practical significance because spurious links are critical for investigating evolving mechanisms in this study we address the temporal link prediction problem in networks i e predicting links at time t 1 based on a given temporal network from time 1 to t to address the relationships among matrix decomposition based algorithms we prove the equivalence between the eigendecomposition and nonnegative matrix factorization nmf algorithms which serves as the theoretical foundation for designing nmf based algorithms for temporal link prediction a novel nmf based algorithm is proposed based on such equivalence the algorithm factorizes each network to obtain features using graph communicability and then collapses the feature matrices to predict temporal links compared with state of the art methods the proposed algorithm exhibits significantly improved accuracy by avoiding the collapse of temporal networks experimental results of a number of artificial and real temporal networks illustrate that the proposed method is not only more accurate but also more robust than state of the art approaches

volume approximation of convex bodies by inscribed polytopes a convex body c in d dimensional euclidean space e is a compact convex subset of le a with non empty interior we say that c is of differentiability class k if its boundary considered as a manifold is of class k the symmetric difference metric 6 s also called nikodym metric on the space of convex bodies is defined for any two convex bodies as the volume of their symmetric difference given a convex body c let c n d l d 2 denote the family of convex polytopes p having at most n vertices and being inscribed into c that is all vertices are on the boundary bdc of c it is well known that for each n there is a polytope p such that s c p s c inf s c p p

on the nonnegative rank of euclidean distance matrices abstract the euclidean distance matrix for n distinct points in r r is generically of rank r 2 it is shown in this paper via a geometric argument that its nonnegative rank for the case r 1 is generically n

solving an ill posed cauchy problem for a two dimensional parabolic pde with variable coefficients using a preconditioned gmres method the sideways parabolic equation spe is a model of the problem of determining the temperature on the surface of a body from the interior measurements mathematically it can be formulated as a noncharacteristic cauchy problem for a parabolic partial differential equation this problem is severely ill posed in an l 2 setting we use a preconditioned generalized minimum residual method gmres to solve a two dimensional spe with variable coefficients the preconditioner is singular and chosen in a way that allows efficient implementation using the fft the preconditioner is a stabilized solver for a nearby problem with constant coefficients and it reduces the number of iterations in the gmres algorithm significantly numerical experiments are performed that demonstrate the performance of the proposed method