low rank doubly stochastic matrix decomposition for cluster analysis cluster analysis by nonnegative low rank approximations has experienced a remarkable progress in the past decade however the majority of such approximation approaches are still restricted to nonnegative matrix factorization nmf and suffer from the following two drawbacks 1 they are unable to produce balanced partitions for large scale manifold data which are common in real world clustering tasks 2 most existing nmf type clustering methods cannot automatically determine the number of clusters we propose a new low rank learning method to address these two problems which is beyond matrix factorization our method approximately decomposes a sparse input similarity in a normalized way and its objective can be used to learn both cluster assignments and the number of clusters for efficient optimization we use a relaxed formulation based on data cluster data random walk which is also shown to be equivalent to low rank factorization of the doubly stochastically normalized cluster incidence matrix the probabilistic cluster assignments can thus be learned with a multiplicative majorization minimization algorithm experimental results show that the new method is more accurate both in terms of clustering large scale manifold data sets and of selecting the number of clusters

finding imaging patterns of structural covariance via non negative matrix factorization in this paper we investigate the use of non negative matrix factorization nnmf for the analysis of structural neuroimaging data the goal is to identify the brain regions that co vary across individuals in a consistent way hence potentially being part of underlying brain networks or otherwise influenced by underlying common mechanisms such as genetics and pathologies nnmf offers a directly data driven way of extracting relatively localized co varying structural regions thereby transcending limitations of principal component analysis pca independent component analysis ica and other related methods that tend to produce dispersed components of positive and negative loadings in particular leveraging upon the well known ability of nnmf to produce parts based representations of image data we derive decompositions that partition the brain into regions that vary in consistent ways across individuals importantly these decompositions achieve dimensionality reduction via highly interpretable ways and generalize well to new data as shown via split sample experiments we empirically validate nnmf in two data sets i a diffusion tensor dt mouse brain development study and ii a structural magnetic resonance smr study of human brain aging we demonstrate the ability of nnmf to produce sparse parts based representations of the data at various resolutions these representations seem to follow what we know about the underlying functional organization of the brain and also capture some pathological processes moreover we show that these low dimensional representations favorably compare to descriptions obtained with more commonly used matrix factorization methods like pca and ica

online nonnegative matrix factorization with robust stochastic approximation nonnegative matrix factorization nmf has become a popular dimension reduction method and has been widely applied to image processing and pattern recognition problems however conventional nmf learning methods require the entire dataset to reside in the memory and thus cannot be applied to large scale or streaming datasets in this paper we propose an efficient online rsa nmf algorithm or nmf that learns nmf in an incremental fashion and thus solves this problem in particular or nmf receives one sample or a chunk of samples per step and updates the bases via robust stochastic approximation benefitting from the smartly chosen learning rate and averaging technique or nmf converges at the rate of in each update of the bases furthermore we prove that or nmf almost surely converges to a local optimal solution by using the quasi martingale by using a buffering strategy we keep both the time and space complexities of one step of the or nmf constant and make or nmf suitable for large scale or streaming datasets preliminary experimental results on real world datasets show that or nmf outperforms the existing online nmf onmf algorithms in terms of efficiency experimental results of face recognition and image annotation on public datasets confirm the effectiveness of or nmf compared with the existing onmf algorithms

learning the information divergence information divergence that measures the difference between two nonnegative matrices or tensors has found its use in a variety of machine learning problems examples are nonnegative matrix tensor factorization stochastic neighbor embedding topic models and bayesian network optimization the success of such a learning task depends heavily on a suitable divergence a large variety of divergences have been suggested and analyzed but very few results are available for an objective choice of the optimal divergence for a given task here we present a framework that facilitates automatic selection of the best divergence among a given family based on standard maximum likelihood estimation we first propose an approximated tweedie distribution for the beta divergence family selecting the best beta then becomes a machine learning problem solved by maximum likelihood next we reformulate alpha divergence in terms of beta divergence which enables automatic selection of alpha by maximum likelihood with reuse of the learning principle for beta divergence furthermore we show the connections between gamma and beta divergences as well as renyi and alpha divergences such that our automatic selection framework is extended to non separable divergences experiments on both synthetic and real world data demonstrate that our method can quite accurately select information divergence across different learning problems and various divergence families

low rank matrix factorization with adaptive graph regularizer in this paper we present a novel low rank matrix factorization algorithm with adaptive graph regularizer lmfagr we extend the recently proposed low rank matrix with manifold regularization mmf method with an adaptive regularizer different from mmf which constructs an affinity graph in advance lmfagr can simultaneously seek graph weight matrix and low dimensional representations of data that is graph construction and low rank matrix factorization are incorporated into a unified framework which results in an automatically updated graph rather than a predefined one the experimental results on some data sets demonstrate that the proposed algorithm outperforms the state of the art low rank matrix factorization methods

large cone nonnegative matrix factorization nonnegative matrix factorization nmf has been greatly popularized by its parts based interpretation and the effective multiplicative updating rule for searching local solutions in this paper we study the problem of how to obtain an attractive local solution for nmf which not only fits the given training data well but also generalizes well on the unseen test data based on the geometric interpretation of nmf we introduce two large cone penalties for nmf and propose large cone nmf lcnmf algorithms compared with nmf lcnmf will obtain bases comprising a larger simplicial cone and therefore has three advantages 1 the empirical reconstruction error of lcnmf could mostly be smaller 2 the generalization ability of the proposed algorithm is much more powerful and 3 the obtained bases of lcnmf have a low overlapping property which enables the bases to be sparse and makes the proposed algorithms very robust experiments on synthetic and real world data sets confirm the efficiency of lcnmf

finding community structure in networks using the eigenvectors of matrices we consider the problem of detecting communities or modules in networks groups of vertices with a higher than average density of edges connecting them previous work indicates that a robust approach to this problem is the maximization of the benefit function known as modularity over possible divisions of a network here we show that this maximization process can be written in terms of the eigenspectrum of a matrix we call the modularity matrix which plays a role in community detection similar to that played by the graph laplacian in graph partitioning calculations this result leads us to a number of possible algorithms for detecting community structure as well as several other results including a spectral measure of bipartite structure in networks and a centrality measure that identifies vertices that occupy central positions within the communities to which they belong the algorithms and measures proposed are illustrated with applications to a variety of real world complex networks

unified development of multiplicative algorithms for linear and quadratic nonnegative matrix factorization multiplicative updates have been widely used in approximative nonnegative matrix factorization nmf optimization because they are convenient to deploy their convergence proof is usually based on the minimization of an auxiliary upper bounding function the construction of which however remains specific and only available for limited types of dissimilarity measures here we make significant progress in developing convergent multiplicative algorithms for nmf first we propose a general approach to derive the auxiliary function for a wide variety of nmf problems as long as the approximation objective can be expressed as a finite sum of monomials with real exponents multiplicative algorithms with theoretical guarantee of monotonically decreasing objective function sequence can thus be obtained the solutions of nmf based on most commonly used dissimilarity measures such as and divergence as well as many other more comprehensive divergences can be derived by the new unified principle second our method is extended to a nonseparable case that includes e g divergence and renyi divergence third we develop multiplicative algorithms for nmf using second order approximative factorizations in which each factorizing matrix may appear twice preliminary numerical experiments demonstrate that the multiplicative algorithms developed using the proposed procedure can achieve satisfactory karush kuhn tucker optimality we also demonstrate nmf problems where algorithms by the conventional method fail to guarantee descent at each iteration but those by our principle are immune to such violation

linear and nonlinear projective nonnegative matrix factorization a variant of nonnegative matrix factorization nmf which was proposed earlier is analyzed here it is called projective nonnegative matrix factorization pnmf the new method approximately factorizes a projection matrix minimizing the reconstruction error into a positive low rank matrix and its transpose the dissimilarity between the original data matrix and its approximation can be measured by the frobenius matrix norm or the modified kullback leibler divergence both measures are minimized by multiplicative update rules whose convergence is proven for the first time enforcing orthonormality to the basic objective is shown to lead to an even more efficient update rule which is also readily extended to nonlinear cases the formulation of the pnmf objective is shown to be connected to a variety of existing nmf methods and clustering approaches in addition the derivation using lagrangian multipliers reveals the relation between reconstruction and sparseness for kernel principal component analysis pca with the binary constraint useful in graph partitioning problems the nonlinear kernel pnmf provides a good approximation which outperforms an existing discretization approach empirical study on three real world databases shows that pnmf can achieve the best or close to the best in clustering the proposed algorithm runs more efficiently than the compared nmf methods especially for high dimensional data moreover contrary to the basic nmf the trained projection matrix can be readily used for newly coming samples and demonstrates good generalization

normalized cuts and image segmentation we propose a novel approach for solving the perceptual grouping problem in vision rather than focusing on local features and their consistencies in the image data our approach aims at extracting the global impression of an image we treat image segmentation as a graph partitioning problem and propose a novel global criterion the normalized cut for segmenting the graph the normalized cut criterion measures both the total dissimilarity between the different groups as well as the total similarity within the groups we show that an efficient computational technique based on a generalized eigenvalue problem can be used to optimize this criterion we applied this approach to segmenting static images as well as motion sequences and found the results to be very encouraging

convex and semi nonnegative matrix factorizations we present several new variations on the theme of nonnegative matrix factorization nmf considering factorizations of the form x fgt we focus on algorithms in which g is restricted to containing nonnegative entries but allowing the data matrix x to have mixed signs thus extending the applicable range of nmf methods we also consider algorithms in which the basis vectors of f are constrained to be convex combinations of the data points this is used for a kernel extension of nmf we provide algorithms for computing these new factorizations and we provide supporting theoretical analysis we also analyze the relationships between our algorithms and clustering algorithms and consider the implications for sparseness of solutions finally we present experimental results that explore the properties of these new methods

concept factorization with adaptive neighbors for document clustering in this paper a novel concept factorization cf method called cf with adaptive neighbors cfans is proposed the idea of cfan is to integrate an ans regularization constraint into the cf decomposition the goal of cfan is to extract the representation space that maintains geometrical neighborhood structure of the data similar to the existing graph regularized cf cfan builds a neighbor graph weights matrix the key difference is that the cfan performs dimensionality reduction and finds the neighbor graph weights matrix simultaneously an efficient algorithm is also derived to solve the proposed problem we apply the proposed method to the problem of document clustering on the 20 newsgroups reuters 21578 and tdt2 document data sets our experiments demonstrate the effectiveness of the method

decision theoretic foundations of qualitative possibility theory abstract this paper presents a justification of two qualitative counterparts of the expected utility criterion for decision under uncertainty which only require bounded linearly ordered valuation sets for expressing uncertainty and preferences this is carried out in the style of savage starting with a set of acts equipped with a complete preordering relation conditions on acts are given that imply a possibilistic representation of the decision maker uncertainty in this framework pessimistic i e uncertainty averse as well as optimistic attitudes can be explicitly captured the approach thus proposes an operationally testable description of possibility theory

exponentiated gradient versus gradient descent for linear predictors we consider two algorithm for on line prediction based on a linear model the algorithms are the well known gradient descent gd algorithm and a new algorithm which we call eg they both maintain a weight vector using simple updates for the gd algorithm the update is based on subtracting the gradient of the squared error made on a prediction the eg algorithm uses the components of the gradient in the exponents of factors that are used in updating the weight vector multiplicatively we present worst case loss bounds for eg and compare them to previously known bounds for the gd algorithm the bounds suggest that the losses of the algorithms are in general incomparable but eg has a much smaller loss if only a few components of the input are relevant for the predictions we have performed experiments which show that our worst case upper bounds are quite tight already on simple artificial data

least squares quantization in pcm it has long been realized that in pulse code modulation pcm with a given ensemble of signals to handle the quantum values should be spaced more closely in the voltage regions where the signal amplitude is more likely to fall it has been shown by panter and dite that in the limit as the number of quanta becomes infinite the asymptotic fractional density of quanta per unit voltage should vary as the one third power of the probability density per unit voltage of signal amplitudes in this paper the corresponding result for any finite number of quanta is derived that is necessary conditions are found that the quanta and associated quantization intervals of an optimum finite quantization scheme must satisfy the optimization criterion used is that the average quantization noise power be a minimum it is shown that the result obtained here goes over into the panter and dite result as the number of quanta become large the optimum quautization schemes for 2 b quanta b 1 2 cdots 7 are given numerically for gaussian and for laplacian distribution of signal amplitudes

black box multigrid one major problem with the multigrid method has been that each new grid configuration has required a major programming effort to develop a code that specifically handles that grid configuration such a penalty is not required for methods like sor iccg etc in these methods one need only specify the matrix problem no matter what the grid configuration in this paper we investigate such a situation for the multigrid method the end result is code boxmg in which one need only specify the logically rectangular positive definite matrix problem boxmg does everything else necessary to set up the auxilliary coarser problems to achieve a multigrid solution

decision making on the sole basis of statistical likelihood this paper presents a new axiomatic decision theory for choice under uncertainty unlike bayesian decision theory where uncertainty is represented by a probability function in our theory uncertainty is given in the form of a likelihood function extracted from statistical evidence the likelihood principle in statistics stipulates that likelihood functions encode all relevant information obtainable from experimental data in particular we do not assume any knowledge of prior probabilities consequently a bayesian conversion of likelihoods to posterior probabilities is not possible in our setting we make an assumption that defines the likelihood of a set of hypotheses as the maximum likelihood over the elements of the set we justify an axiomatic system similar to that used by von neumann and morgenstern for choice under risk our main result is a representation theorem using the new concept of binary utility we also discuss how ambiguity attitudes are handled applied to the statistical inference problem our theory suggests a novel solution the results in this paper could be useful for probabilistic model selection

algebraic multilevel method with application to the maxwell equations a multilevel method for the iterative solution of large sparse linear systems is introduced the method is defined in terms of the coefficient matrix alone no underlying pde or mesh is assumed the method is applied to the nonhermitian complex maxwell equations in 3d when supplemented with outer acceleration the v 1 5 cycle yields the convergence factor 0 4

an axiomatic treatment of three qualitative decision criteria the need for computationally efficient decision making techniques together with the desire to simplify the processes of knowledge acquisition and agent specification have led various researchers in artificial intelligence to examine qualitative decision tools however the adequacy of such tools is not clear this paper investigates the foundations of maximin minmax regret and competitive ratio three central qualitative decision criteria by characterizing those behaviors that could result from their use this characterizaton provides two important insights 1 under what conditions can we employ an agent model based on these basic qualitative decision criteria and 2 how rational are these decision procedures for the competitive ratio criterion in particular this latter issue is of central importance to our understanding of current work on on line algorithms our main result is a constructive representation theorem that uses two choice axioms to characterize maximin minmax regret and competitive ratio

qualitative decision theory from savage s axioms to nonmonotonic reasoning this paper investigates to what extent a purely symbolic approach to decision making under uncertainty is possible in the scope of artificial intelligence contrary to classical approaches to decision theory we try to rank acts without resorting to any numerical representation of utility or uncertainty and without using any scale on which both uncertainty and preference could be mapped our approach is a variant of savage s where the setting is finite and the strict preference on acts is a partial order it is shown that although many axioms of savage theory are preserved and despite the intuitive appeal of the ordinal method for constructing a preference over acts the approach is inconsistent with a probabilistic representation of uncertainty the latter leads to the kind of paradoxes encountered in the theory of voting it is shown that the assumption of ordinal invariance enforces a qualitative decision procedure that presupposes a comparative possibility representation of uncertainty originally due to lewis and usual in nonmonotonic reasoning our axiomatic investigation thus provides decision theoretic foundations to the preferential inference of lehmann and colleagues however the obtained decision rules are sometimes either not very decisive or may lead to overconfident decisions although their basic principles look sound this paper points out some limitations of purely ordinal approaches to savage like decision making under uncertainty in perfect analogy with similar difficulties in voting theory

automatic relevance determination in nonnegative matrix factorization with the beta divergence this paper addresses the estimation of the latent dimensionality in nonnegative matrix factorization nmf with the divergence the divergence is a family of cost functions that includes the squared euclidean distance kullback leibler kl and itakura saito is divergences as special cases learning the model order is important as it is necessary to strike the right balance between data fidelity and overfitting we propose a bayesian model based on automatic relevance determination ard in which the columns of the dictionary matrix and the rows of the activation matrix are tied together through a common scale parameter in their prior a family of majorization minimization mm algorithms is proposed for maximum a posteriori map estimation a subset of scale parameters is driven to a small lower bound in the course of inference with the effect of pruning the corresponding spurious components we demonstrate the efficacy and robustness of our algorithms by performing extensive experiments on synthetic data the swimmer dataset a music decomposition example and a stock price prediction task

lexicographic probabilities and equilibrium refinements this paper develops a decision theoretic approach to normal form refinements of nash equilibrium and provides characterizations of normal form perfect equilibrium and proper equilibrium the approach relies on a theory of single person decision making that is a non archimedean version of subjective expected utility theory copyright 1991 by the econometric society

two axiomatic approaches to decision making using possibility theory this paper proposes a utility theory for decision making under uncertainty that is described by possibility theory we show that our approach is a natural generalization of the two axiomatic systems that correspond to pessimistic and optimistic decision criteria proposed by dubois et al the generalization is achieved by removing axioms that are supposed to reflect attitudes toward uncertainty namely pessimism and optimism in their place we adopt an axiom that imposes an order on a class of canonical lotteries that realize either in the best or in the worst prize we prove an expected utility theorem for the generalized axiomatic system based on the newly introduced concept of binary utility

the computational complexity of probabilistic inference using bayesian belief networks research note abstract bayesian belief networks provide a natural efficient method for representing probabilistic dependencies among a set of variables for these reasons numerous researchers are exploring the use of belief networks as a knowledge representation in artificial intelligence algorithms have been developed previously for efficient probabilistic inference using special classes of belief networks more general classes of belief networks however have eluded efforts to develop efficient inference algorithms we show that probabilistic inference using belief networks is np hard therefore it seems unlikely that an exact algorithm can be developed to perform probabilistic inference efficiently over all classes of belief networks this result suggests that research should be directed away from the search for a general efficient probabilistic inference algorithm and toward the design of efficient special case average case and approximation algorithms

a general framework for explaining the results of a multi attribute preference model the automatic generation of an explanation of the prescription made by a multi attribute decision model is crucial in many applications such as recommender systems this task is complex since the quantitative models are not designed to be easily explainable the major limitation of the previous research is that there is no formal justification of the arguments that are selected in the explanation the goal of this paper is to define a general framework to justify which arguments shall be selected in the case where the decision model is based on weights assigned to the attributes due to the complexity of explaining a preference model based on utility theory several explanation reasonings are necessary to cover all cases ranging from situations where the prescription is trivial to situations where the prescription is much more tight the set of selected arguments is in this framework a non dominated element of a combinatorial structure in the sense of an order relation our general approach is instantiated precisely on three models the probabilistic expected utility model the qualitative pessimistic minmax model and the concordance rule which are all constructed from a weight vector

an algebraic multilevel parallelizable preconditioner for large scale cfd problems abstract an efficient parallelizable preconditioner for solving large scale cfd problems is presented it is adapted to coarse grain parallelism and can be used for both shared and distributed memory parallel computers the proposed preconditioner consists of two independent approximations of the system matrix the first one is a block diagonal fully parallelizable approximation of the given system the second matrix is coarser than the original one and is built using algebraic multi grid methods the preconditioner is used to compute the steady solution of the compressible navier stokes equations for subsonic laminar flows on shared memory computers for a moderate number of processors the coupled two level preconditioner is robust and has a large potential for parallelism interesting savings in computational time for parallel computations are obtained when comparing the two level preconditioner with the well known incomplete gaussian factorization

quadratic nonnegative matrix factorization in nonnegative matrix factorization nmf a nonnegative matrix is approximated by a product of lower rank factorizing matrices most nmf methods assume that each factorizing matrix appears only once in the approximation thus the approximation is linear in the factorizing matrices we present a new class of approximative nmf methods called quadratic nonnegative matrix factorization qnmf where some factorizing matrices occur twice in the approximation we demonstrate qnmf solutions to four potential pattern recognition problems in graph partitioning two way clustering estimating hidden markov chains and graph matching we derive multiplicative algorithms that monotonically decrease the approximation error under a variety of measures we also present extensions in which one of the factorizing matrices is constrained to be orthogonal or stochastic empirical studies show that for certain application scenarios qnmf is more advantageous than other existing nonnegative matrix factorization methods

non negative matrix factorization with sparseness constraints non negative matrix factorization nmf is a recently developed technique for finding parts based linear representations of non negative data although it has successfully been applied in several applications it does not always result in parts based representations in this paper we show how explicitly incorporating the notion of sparseness improves the found decompositions additionally we provide complete matlab code both for standard nmf and for our extension our hope is that this will further the application of these methods to solving novel data analysis problems

an algebraic two level preconditioner for asymmetric positive definite systems a two level linear algebraic solver for asymmetric positive definite systems is developed using matrices arising from stabilized finite element formulations to motivate the approach supported by an analysis of a representative smoother the parent space is divided into oscillatory and smooth subspaces according to the eigenvectors of the associated normal system using a mesh based aggregation technique which relies only on information contained in the matrix a restriction prolongation operator is constructed various numerical examples on both structured and unstructured meshes are performed using the two level cycle as the basis for a preconditioner results demonstrate the complementarity between the smoother and the coarse level correction as well as convergence rates that are nearly independent of the problem size copyright 2001 john wiley sons ltd

a fast algorithm for nonnegative matrix factorization and its convergence nonnegative matrix factorization nmf has recently become a very popular unsupervised learning method because of its representational properties of factors and simple multiplicative update algorithms for solving the nmf however for the common nmf approach of minimizing the euclidean distance between approximate and true values the convergence of multiplicative update algorithms has not been well resolved this paper first discusses the convergence of existing multiplicative update algorithms we then propose a new multiplicative update algorithm for minimizing the euclidean distance between approximate and true values based on the optimization principle and the auxiliary function method we prove that our new algorithm not only converges to a stationary point but also does faster than existing ones to verify our theoretical results the experiments on three data sets have been conducted by comparing our proposed algorithm with other existing methods

fuzzy sets as a basis for a theory of possibility the theory of possibility described in this paper is related to the theory of fuzzy sets by defining the concept of a possibility distribution as a fuzzy restriction which acts as an elastic constraint on the values that may be assigned to a variable more specifically if f is a fuzzy subset of a universe of discourse u u which is characterized by its membership function f then a proposition of the form x is f where x is a variable taking values in u induces a possibility distribution tx which equates the possibility of x taking the value u to f u the compatibility of u with f in this way x becomes a fuzzy variable which is associated with the possibility distribution tx in much the same way as a random variable is associated with a probability distribution in general a variable may be associated both with a possibility distribution and a probability distribution with the weak connection between the two expressed as the possibility probability consistency principle a thesis advanced in this paper is that the imprecision that is intrinsic in natural languages is in the main possibilistic rather than probabilistic in nature thus by employing the concept of a possibility distribution a proposition p in a natural language may be translated into a procedure which computes the probability distribution of a set of attributes which are implied by p several types of conditional translation rules are discussed and in particular a translation rule for propositions of the form x is f is possible where is a number in the interval 0 1 is formulated and illustrated by examples

preference structures the problem of the consistency of qualitative judgements about the difference of attractiveness between alternatives is studied our proposal deals with smooth t conorms to be used for aggregation of elementary judgements

finding maps for belief networks is np hard given a probabilistic world model an important problem is to find the maximum a posteriori probability map instantiation of all the random variables given the evidence numerous researchers using such models employ some graph representation for the distributions such as a bayesian belief network this representation simplifies the complexity of specifying the distributions from exponential in n the number of variables in the model to linear in n in many interesting cases we show however that finding the map is np hard in the general case when these representations are used even if the size of the representation happens to be linear in n furthermore minor modifications to the proof show that the problem remains np hard for various restrictions of the topology of the graphs the same technique can be applied to the results of a related paper by cooper to further restrict belief network topology in the proof that probabilistic inference is np hard

symmetric nonnegative matrix factorization algorithms and applications to probabilistic clustering nonnegative matrix factorization nmf is an unsupervised learning method useful in various applications including image processing and semantic analysis of documents this paper focuses on symmetric nmf snmf which is a special case of nmf decomposition three parallel multiplicative update algorithms using level 3 basic linear algebra subprograms directly are developed for this problem first by minimizing the euclidean distance a multiplicative update algorithm is proposed and its convergence under mild conditions is proved based on it we further propose another two fast parallel methods snmf and snmf algorithms all of them are easy to implement these algorithms are applied to probabilistic clustering we demonstrate their effectiveness for facial image clustering document categorization and pattern clustering in gene expression

lexicographic probabilities and choice under uncertainty conventional bayesian theory of choice under uncertainty subjective expected utility theory fails to satisfy the properties of admissibility and existence of well defined conditional probabilities weakly dominated acts may be chosen and the usual definition of conditional probabilities applies only to nonnull events this paper develops a non archimedean variant of subjective expected utility where decisionmakers have lexicographic beliefs this generalization can be made to satisfy admissibility and yield well defined conditional probabilities and at the same time allow for null events copyright 1991 by the econometric society

research note a representation theorem for minmax regret policies decision making under uncertainty is one of the central tasks of artificial agents due to their simplicity and ease of specification qualitative decision tools are popular in artificial intelligence brafman and tennenholtz r i brafman m tennenholtz an axiomatic treatment of three qualitative decision criteria j acm 47 3 2000 452 482 model an agent s uncertain knowledge as her local state which consists of states of the world that she deems possible a policy determines for each local state a total preorder of the set of actions which represents the agent s preference over these actions it is known that a policy is maximin representable if and only if it is closed under unions and satisfies a certain acyclicity condition in this paper we show that the above conditions although necessary are insufficient for minmax regret and competitive ratio policies a complete characterization of these policies is obtained by introducing the best equally strictness

minimum volume constrained nonnegative matrix factorization enhanced ability of learning parts nonnegative matrix factorization nmf with minimum volume constraint mvc is exploited in this paper our results show that mvc can actually improve the sparseness of the results of nmf this sparseness is l0 norm oriented and can give desirable results even in very weak sparseness situations thereby leading to the significantly enhanced ability of learning parts of nmf the close relation between nmf sparse nmf and the mvc nmf is discussed first then two algorithms are proposed to solve the mvc nmf model one is called quadratic programming mvc nmf qp mvc nmf which is based on quadratic programming and the other is called negative glow mvc nmf ng mvc nmf because it uses multiplicative updates incorporating natural gradient ingeniously the qp mvc nmf algorithm is quite efficient for small scale problems and the ng mvc nmf algorithm is more suitable for large scale problems simulations show the efficiency and validity of the proposed methods in applications of blind source separation and human face images analysis

qualitative decision theory with preference relations and comparative uncertainty an axiomatic approach this paper investigates a purely qualitative approach to decision making under uncertainty since the pioneering work of savage most models of decision under uncertainty rely on a numerical representation where utility and uncertainty are commensurate giving up this tradition we relax this assumption and introduce an axiom of ordinal invariance requiring that the decision maker s preference between two acts only depends on the relative position of their consequences for each state within this qualitative framework we determine the only possible form of the corresponding decision rule then assuming the transitivity of the strict preference the underlying partial confidence relations are those at work in non monotonic inference and thus satisfy one of the main properties of possibility theory the satisfaction of additional postulates of unanimity and anonymity enforces the use of a necessity measure unique up to a monotonic transformation for encoding the relative likelihood of events

a collective neurodynamic optimization approach to nonnegative matrix factorization nonnegative matrix factorization nmf is an advanced method for nonnegative feature extraction with widespread applications however the nmf solution often entails to solve a global optimization problem with a nonconvex objective function and nonnegativity constraints this paper presents a collective neurodynamic optimization cno approach to this challenging problem the proposed collective neurodynamic system consists of a population of recurrent neural networks rnns at the lower level and a particle swarm optimization pso algorithm with wavelet mutation at the upper level the rnns act as search agents carrying out precise local searches according to their neurodynamics and initial conditions the pso algorithm coordinates and guides the rnns with updated initial states toward global optimal solution s a wavelet mutation operator is added to enhance pso exploration diversity through iterative interaction and improvement of the locally best solutions of rnns and global best positions of the whole population the population based neurodynamic systems are almost sure able to achieve the global optimality for the nmf problem it is proved that the convergence of the group best state to the global optimal solution with probability one the experimental results substantiate the efficacy and superiority of the cno approach to bound constrained global optimization with several benchmark nonconvex functions and nmf based clustering with benchmark data sets in comparison with the state of the art algorithms

improving non negative matrix factorizations through structured initialization in this paper we explore a recent iterative compression technique called non negative matrix factorization nmf several special properties are obtained as a result of the constrained optimization problem of nmf for facial images the additive nature of nmf results in a basis of features such as eyes noses and lips we explore various methods for efficiently computing nmf placing particular emphasis on the initialization of current algorithms we propose using spherical k means clustering to produce a structured initialization for nmf we demonstrate some of the properties that result from this initialization and develop an efficient way of choosing the rank of the low dimensional nmf representation

adaptive subset kernel principal component analysis for time varying patterns kernel principal component analysis kpca and its online learning algorithms have been proposed and widely used since kpca uses training samples for bases of the operator its online learning algorithms require the preparation of all training samples beforehand subset kpca subkpca which uses a subset of samples for the basis set has been proposed and has demonstrated better performance with less computational complexity in this paper we extend subkpca to an online version and propose methods to add and exchange a sample in the basis set since the proposed method uses the basis set we do not need to prepare all training samples beforehand therefore the proposed method can be applied to time varying patterns in contrast to existing online kpca algorithms experimental results demonstrate the advantages of the proposed method

subclass discriminant nonnegative matrix factorization for facial image analysis nonnegative matrix factorization nmf is among the most popular subspace methods widely used in a variety of image processing problems recently a discriminant nmf method that incorporates linear discriminant analysis inspired criteria has been proposed which achieves an efficient decomposition of the provided data to its discriminant parts thus enhancing classification performance however this approach possesses certain limitations since it assumes that the underlying data distribution is unimodal which is often unrealistic to remedy this limitation we regard that data inside each class have a multimodal distribution thus forming clusters and use criteria inspired by clustering based discriminant analysis the proposed method incorporates appropriate discriminant constraints in the nmf decomposition cost function in order to address the problem of finding discriminant projections that enhance class separability in the reduced dimensional projection space while taking into account subclass information the developed algorithm has been applied for both facial expression and face recognition on three popular databases experimental results verified that it successfully identified discriminant facial parts thus enhancing recognition performance

algebraic multigrid for complex symmetric matrices and applications this paper is concerned with the numerical study of an algebraic multigrid preconditioner for complex symmetric system matrices we use several different krylov subspace methods as an outer iteration namely the qmr method proposed by freund and nachtigal the bicgcr method of clemens and the csym method of bunse gerstner and stoever in addition we compare the results with the standard jacobi preconditioner for complex symmetric problems we test our approach on the numerical simulation of high voltage insulators

joint sparse representation and embedding propagation learning a framework for graph based semisupervised learning in this paper we propose a novel graph based semisupervised learning framework called joint sparse representation and embedding propagation learning jsrepl the idea of jsrepl is to join epl with sparse representation to perform label propagation like most of graph based semisupervised propagation learning algorithms jsrepl also constructs weights graph matrix from given data different from classical approaches which build weights graph matrix and estimate the labels of unlabeled data in sequence jsrepl simultaneously builds weights graph matrix and estimates the labels of unlabeled data we also propose an efficient algorithm to solve the proposed problem the proposed method is applied to the problem of semisupervised image clustering using the orl yale pie and yaleb data sets our experiments demonstrate the effectiveness of our proposed algorithm

projective robust nonnegative factorization nonnegative matrix factorization nmf has been successfully used in many fields as a low dimensional representation method projective nonnegative matrix factorization pnmf is a variant of nmf that was proposed to learn a subspace for feature extraction however both original nmf and pnmf are sensitive to noise and are unsuitable for feature extraction if data is grossly corrupted in order to improve the robustness of nmf a framework named projective robust nonnegative factorization prnf is proposed in this paper for robust image feature extraction and classification since learned projections can weaken noise disturbances prnf is more suitable for classification and feature extraction in order to preserve the geometrical structure of original data prnf introduces a graph regularization term which encodes geometrical structure in the prnf framework three algorithms are proposed that add a sparsity constraint on the noise matrix based on l1 2 norm l1 norm and l2 1 norm respectively robustness and classification performance of the three proposed algorithms are verified with experiments on four face image databases and results are compared with state of the art robust nmf based algorithms experimental results demonstrate the robustness and effectiveness of the algorithms for image classification and feature extraction

an algebraic interpretation of multigrid methods the main objective of this paper is to treat the correction cycle of multigrid as a newton like method and to analyze it together with relaxation via a natural decomposition of the grid function space the purpose is to provide a simplified view of multigrid and motivate some general principles for algorithm design

projected gradient methods for nonnegative matrix factorization nonnegative matrix factorization nmf can be formulated as a minimization problem with bound constraints although bound constrained optimization has been studied extensively in both theory and practice so far no study has formally applied its techniques to nmf in this letter we propose two projected gradient methods for nmf both of which exhibit strong optimization properties we discuss efficient implementations and demonstrate that one of the proposed methods converges faster than the popular multiplicative update approach a simple matlab code is also provided