nonsmooth nonnegative matrix factorization nsnmf we propose a novel nonnegative matrix factorization model that aims at finding localized part based representations of nonnegative multivariate data items unlike the classical nonnegative matrix factorization nmf technique this new model denoted nonsmooth nonnegative matrix factorization nsnmf corresponds to the optimization of an unambiguous cost function designed to explicitly represent sparseness in the form of nonsmoothness which is controlled by a single parameter in general this method produces a set of basis and encoding vectors that are not only capable of representing the original data but they also extract highly focalized patterns which generally lend themselves to improved interpretability the properties of this new method are illustrated with several data sets comparisons to previously published methods show that the new nsnmf method has some advantages in keeping faithfulness to the data in the achieving a high degree of sparseness for both the estimated basis and the encoding vectors and in better interpretability of the factors

locality regularized linear regression discriminant analysis for feature extraction abstract locality regularized linear regression classification llrc is an effective classifier that shows great potential for face recognition however the original feature space cannot guarantee the classification efficiency of llrc to alleviate this problem we propose a novel dimensionality reduction method called locality regularized linear regression discriminant analysis llrda for feature extraction the proposed llrda is developed according to the decision rule of llrc and seeks to generate a subspace that is discriminant for llrc specifically the intra class and inter class local reconstruction scatters are first defined to characterize the compactness and separability of samples respectively then the objective function for llrda is derived by maximizing the inter class local reconstruction scatter and simultaneously minimizing the intra class local reconstruction scatter extensive experimental results on cmu pie orl feret and yale b face databases validate the effectiveness of our proposed method

the entire regularization path for the support vector machine the support vector machine svm is a widely used tool for classification many efficient implementations exist for fitting a two class svm model the user has to supply values for the tuning parameters the regularization cost parameter and the kernel parameters it seems a common practice is to use a default value for the cost parameter often leading to the least restrictive model in this paper we argue that the choice of the cost parameter can be critical we then derive an algorithm that can fit the entire path of svm solutions for every value of the cost parameter with essentially the same computational cost as fitting one svm model we illustrate our algorithm on some examples and use our representation to give further insight into the range of svm solutions

action recognition using 3d histograms of texture and a multi class boosting classifier human action recognition is an important yet challenging task this paper presents a low cost descriptor called 3d histograms of texture 3dhots to extract discriminant features from a sequence of depth maps 3dhots are derived from projecting depth frames onto three orthogonal cartesian planes i e the frontal side and top planes and thus compactly characterize the salient information of a specific action on which texture features are calculated to represent the action besides this fast feature descriptor a new multi class boosting classifier mbc is also proposed to efficiently exploit different kinds of features in a unified framework for action classification compared with the existing boosting frameworks we add a new multi class constraint into the objective function which helps to maintain a better margin distribution by maximizing the mean of margin whereas still minimizing the variance of margin experiments on the msraction3d msrgesture3d msractivity3d and utd mhad data sets demonstrate that the proposed system combining 3dhots and mbc is superior to the state of the art

expressing infinite matrices as products of involutions abstract we take into consideration ut k the group of upper triangular infinite matrices whose entries lying on the main diagonal are equal to either 1 or 1 we show that for any field k every element of this group can be expressed as a product of at most five involutions moreover we prove that if k is a field of characteristic different from 2 then four involutions suffice

collapsing words permutation conditions and coherent colorings of trees given a word w over a finite alphabet s and a finite deterministic automaton a the inequality d q w q n means that under the natural action of the word w the image of the state set q is reduced by at least n states a word w is n collapsing if this inequality holds for any deterministic finite automaton that satisfies such an inequality for at least one word in this paper we prove that the problem of recognizing n collapsing words is generally co np complete while restricted to 2 collapsing words over 2 element alphabet it belongs to p this is connected with introducing a new approach to collapsing words which is shown to be much more effective in solving various problems in the area it leads to interesting connections with combinatorial problems concerning solving systems of permutation conditions on one hand and coloring trees with distinguished nodes on the other hand

sparse regression with output correlation for cardiac ejection fraction estimation traditional regression methods minimize the sum of errors of samples with various regularization terms such as the 1 norm and 2 norm for the diagnosis of cardiovascular diseases the cardiac ejection fraction ef represents an essential measure however existing regularization terms do not consider the output correlation the correlation between ground truth volumes and estimated volumes w r t each subject which is beneficial in estimating the cardiac ef in this paper we first propose a sparse regression with two regularization terms of the 1 norm and output correlation sroc then we propose a one dimensional solution path algorithm for quickly finding two good regulation parameters in the formulation of sroc the solution path algorithm can effectively handle singularities and infinities in the key matrix finally we conduct experiments on a clinical cardiac image dataset with 100 subjects the experimental results show that our method produces competitive and strong results for estimating the cardiac ef based on quantitative and qualitative analyses

an r squared measure of goodness of fit for some common nonlinear regression models abstract for regression models other than the linear model r squared type goodness of fit summary statistics have been constructed for particular models using a variety of methods we propose an r squared measure of goodness of fit for the class of exponential family regression models which includes logit probit poisson geometric gamma and exponential this r squared is defined as the proportionate reduction in uncertainty measured by kullback leibler divergence due to the inclusion of regressors under further conditions concerning the conditional mean function it can also be interpreted as the fraction of uncertainty explained by the fitted model

supervised dimension reduction of intrinsically low dimensional data high dimensional data generated by a system with limited degrees of freedom are often constrained in low dimensional manifolds in the original space in this article we investigate dimension reduction methods for such intrinsically low dimensional data through linear projections that preserve the manifold structure of the data for intrinsically one dimensional data this implies projecting to a curve on the plane with as few intersections as possible we are proposing a supervised projection pursuit method that can be regarded as an extension of the single index model for nonparametric regression we show results from a toy and two robotic applications

nonlinear embedding preserving multiple local linearities locally linear embedding lle is one of the effective and efficient algorithms for nonlinear dimensionality reduction this paper discusses the stability of lle focusing on the optimal weights for extracting local linearity behind the considered manifold it is proven that there are multiple sets of weights that are approximately optimal and can be used to improve the stability of lle a new algorithm using multiple weights is then proposed together with techniques for constructing multiple weights this algorithm is called as nonlinear embedding preserving multiple local linearities neml neml improves the preservation of local linearity and is more stable than lle a short analysis for neml is also given for isometric manifolds neml is compared with the local tangent space alignment ltsa in methodology since both of them adopt multiple local constraints numerical examples are given to show the improvement and efficiency of neml

new support vector algorithms we propose a new class of support vector algorithms for regression and classification in these algorithms a parameter lets one effectively control the number of support vectors while this can be useful in its own right the parameterization has the additional benefit of enabling us to eliminate one of the other free parameters of the algorithm the accuracy parameter epsilon in the regression case and the regularization constant c in the classification case we describe the algorithms give some theoretical results concerning the meaning and the choice of and report experimental results

conjugacy in groups of upper triangular matrices abstract we show that in the group u n f 2 a sylow 2 subgroup of gl n f 2 there are elements not conjugate to their inverses if n 13 it follows that there are irreducible characters of this group that are not real valued and that a conjecture of kirillov is not correct as stated we give a partial explanation of why these group elements do not exist if n 12 and an example to show that analogous phenomena can occur for odd primes

laplacian eigenmaps for dimensionality reduction and data representation one of the central problems in machine learning and pattern recognition is to develop appropriate representations for complex data we consider the problem of constructing a representation for data lying on a low dimensional manifold embedded in a high dimensional space drawing on the correspondence between the graph laplacian the laplace beltrami operator on the manifold and the connections to the heat equation we propose a geometrically motivated algorithm for representing the high dimensional data the algorithm provides a computationally efficient approach to nonlinear dimensionality reduction that has locality preserving properties and a natural connection to clustering some potential applications and illustrative examples are discussed

a robust regularization path algorithm for nu support vector classification the nu support vector classification has the advantage of using a regularization parameter nu to control the number of support vectors and margin errors recently a regularization path algorithm for nu support vector classification nu svcpath suffers exceptions and singularities in some special cases in this brief we first present a new equivalent dual formulation for nu svc and then propose a robust nu svcpath based on lower upper decomposition with partial pivoting theoretical analysis and experimental results verify that our proposed robust regularization path algorithm can avoid the exceptions completely handle the singularities in the key matrix and fit the entire solution path in a finite number of steps experimental results also show that our proposed algorithm fits the entire solution path with fewer steps and less running time than original one does

robust support vector regression for uncertain input and output data in this paper a robust support vector regression rsvr method with uncertain input and output data is studied first the data uncertainties are investigated under a stochastic framework and two linear robust formulations are derived linear formulations robust to ellipsoidal uncertainties are also considered from a geometric perspective second kernelized rsvr formulations are established for nonlinear regression problems both linear and nonlinear formulations are converted to second order cone programming problems which can be solved efficiently by the interior point method simulation demonstrates that the proposed method outperforms existing rsvrs in the presence of both input and output data uncertainties

support vector ordinal regression in this letter we propose two new support vector approaches for ordinal regression which optimize multiple thresholds to define parallel discriminant hyperplanes for the ordinal scales both approaches guarantee that the thresholds are properly ordered at the optimal solution the size of these optimization problems is linear in the number of training samples the sequential minimal optimization algorithm is adapted for the resulting optimization problems it is extremely easy to implement and scales efficiently as a quadratic function of the number of examples the results of numerical experiments on some benchmark and real world data sets including applications of ordinal regression to information retrieval verify the usefulness of these approaches

on nonnegative matrix factorization algorithms for signal dependent noise with application to electromyography data nonnegative matrix factorization nmf by the multiplicative updates algorithm is a powerful machine learning method for decomposing a high dimensional nonnegative matrix v into two nonnegative matrices w and h where it has been successfully applied in the analysis and interpretation of large scale data arising in neuroscience computational biology and natural language processing among other areas a distinctive feature of nmf is its nonnegativity constraints that allow only additive linear combinations of the data thus enabling it to learn parts that have distinct physical representations in reality in this letter we describe an information theoretic approach to nmf for signal dependent noise based on the generalized inverse gaussian model specifically we propose three novel algorithms in this setting each based on multiplicative updates and prove monotonicity of updates using the em algorithm in addition we develop algorithm specific measures to evaluate their goodness of fit on data our methods are demonstrated using experimental data from electromyography studies as well as simulated data in the extraction of muscle synergies and compared with existing algorithms for signal dependent noise

non negative patch alignment framework in this paper we present a non negative patch alignment framework npaf to unify popular non negative matrix factorization nmf related dimension reduction algorithms it offers a new viewpoint to better understand the common property of different nmf algorithms although multiplicative update rule mur can solve npaf and is easy to implement it converges slowly thus we propose a fast gradient descent fgd to overcome the aforementioned problem fgd uses the newton method to search the optimal step size and thus converges faster than mur experiments on synthetic and real world datasets confirm the efficiency of fgd compared with mur for optimizing npaf based on npaf we develop non negative discriminative locality alignment ndla experiments on face image and handwritten datasets suggest the effectiveness of ndla in classification tasks and its robustness to image occlusions compared with representative nmf related dimension reduction algorithms

principal manifolds and nonlinear dimensionality reduction via tangent space alignment we present a new algorithm for manifold learning and nonlinear dimensionality reduction based on a set of unorganized data points sampled with noise from a parameterized manifold the local geometry of the manifold is learned by constructing an approximation for the tangent space at each data point and those tangent spaces are then aligned to give the global coordinates of the data points with respect to the underlying manifold we also present an error analysis of our algorithm showing that reconstruction errors can be quite small in some cases we illustrate our algorithm using curves and surfaces both in two dimensional three dimensional 2d 3d euclidean spaces and in higher dimensional euclidean spaces we also address several theoretical and algorithmic issues for further research and improvements

global discriminative based nonnegative spectral clustering based on spectral graph theory spectral clustering is an optimal graph partition problem it has been proven that the spectral clustering is equivalent to nonnegative matrix factorization nmf under certain conditions based on the equivalence some spectral clustering methods are proposed but the global discriminative information of the dataset is neglected in this paper based on the equivalence between spectral clustering and nmf we simultaneously maximize the between class scatter matrix and minimize the within class scatter matrix to enhance the discriminating power we integrate the geometrical structure and discriminative structure in a joint framework with a global discriminative regularization term added into the nonnegative matrix factorization framework we propose two novel spectral clustering methods named global discriminative based nonnegative and spectral clustering gdbnsc ncut and gdbnsc rcut these new spectral clustering algorithms can preserve both the global geometrical structure and global discriminative structure the intrinsic geometrical information of the dataset is detected and clustering quality is improved with enhanced discriminating power in addition the proposed algorithms also have very good abilities of handling out of sample data experimental results on real word data demonstrate that the proposed algorithms outperform some state of the art methods with good clustering qualities we maximize the between class scatter matrix meanwhile we minimize the within class scatter matrix we integrate the geometrical structure and discriminative structure in a framework a global discriminative regularization term added into the objective functions the new algorithms can preserve the global geometrical and discriminative structure

involutions and characters of upper triangular matrix groups we study the realizability over r of representations of the group u n of upper triangular n x n matrices over f 2 we prove that all the representations of u n are realizable over r if n 12 but that if n 13 u n has representations not realizable over r this theorem is a variation on a result that can be obtained by combining work of j arregi and a vera lopez and of the authors but the proof of the theorem in this paper is much more natural

learning eigenfunctions links spectral embedding and kernel pca in this letter we show a direct relation between spectral embedding methods and kernel principal components analysis and how both are special cases of a more general learning problem learning the principal eigenfunctions of an operator defined from a kernel and the unknown data generating density whereas spectral embedding methods provided only coordinates for the training points the analysis justifies a simple extension to out of sample examples the nystrom formula for multidimensional scaling mds spectral clustering laplacian eigenmaps locally linear embedding lle and isomap the analysis provides for all such spectral embedding methods the definition of a loss function whose empirical average is minimized by the traditional algorithms the asymptotic expected value of that loss defines a generalization performance and clarifies what these algorithms are trying to learn experiments with lle isomap spectral clustering and mds show that this out of sample embedding formula generalizes well with a level of error comparable to the effect of small perturbations of the training set on the embedding

learning robust and discriminative low rank representations for face recognition with occlusion for robust face recognition tasks we particularly focus on the ubiquitous scenarios where both training and testing images are corrupted due to occlusions previous low rank based methods stacked each error image into a vector and then used l1 or l2 norm to measure the error matrix however in the stacking step the structure information of the error image can be lost depart from the previous methods in this paper we propose a novel method by exploiting the low rankness of both the data representation and each occlusion induced error image simultaneously by which the global structure of data together with the error images can be well captured in order to learn more discriminative low rank representations we formulate our objective such that the learned representations are optimal for classification with the available supervised information and close to an ideal code regularization term with strong structure information preserving and discrimination capabilities the learned robust and discriminative low rank representation rdlrr works very well on face recognition problems especially with face images corrupted by continuous occlusions together with a simple linear classifier the proposed approach is shown to outperform several other state of the art face recognition methods on databases with a variety of face variations we focus on face recognition scenarios where both training and testing image are corrupted duo to occlusions we propose to learn robust and discriminative representation based on low rank matrix recovery model we solve the proposed model by using alm and the complexity analysis and convergence analysis are also provided experimental results demonstrate the effectiveness of our method

learning visual spatial pooling by strong pca dimension reduction in visual modeling invariance properties of visual cells are often explained by a pooling mechanism in which outputs of neurons with similar selectivities to some stimulus parameters are integrated so as to gain some extent of invariance to other parameters for example the classical energy model of phase invariant v1 complex cells pools model simple cells preferring similar orientation but different phases prior studies such as independent subspace analysis have shown that phase invariance properties of v1 complex cells can be learned from spatial statistics of natural inputs however those previous approaches assumed a squaring nonlinearity on the neural outputs to capture energy correlation such nonlinearity is arguably unnatural from a neurobiological viewpoint but hard to change due to its tight integration into their formalisms moreover they used somewhat complicated objective functions requiring expensive computations for optimization in this study we show that visual spatial pooling can be learned in a much simpler way using strong dimension reduction based on principal component analysis this approach learns to ignore a large part of detailed spatial structure of the input and thereby estimates a linear pooling matrix using this framework we demonstrate that pooling of model v1 simple cells learned in this way even with nonlinearities other than squaring can reproduce standard tuning properties of v1 complex cells for further understanding we analyze several variants of the pooling model and argue that a reasonable pooling can generally be obtained from any kind of linear transformation that retains several of the first principal components and suppresses the remaining ones in particular we show how the classic wiener filtering theory leads to one such variant

proximity based frameworks for generating embeddings from multi output data this paper is about supervised and semi supervised dimensionality reduction dr by generating spectral embeddings from multi output data based on the pairwise proximity information two flexible and generic frameworks are proposed to achieve supervised dr sdr for multilabel classification one is able to extend any existing single label sdr to multilabel via sample duplication referred to as mesd the other is a multilabel design framework that tackles the sdr problem by computing weight proximity matrices based on simultaneous feature and label information referred to as mope as a generalization of many current techniques a diverse set of different schemes for label based proximity calculation as well as a mechanism for combining label based and feature based weight information by considering information importance and prioritization are proposed for mope additionally we summarize many current spectral methods for unsupervised dr udr single multilabel sdr and semi supervised dr ssdr and express them under a common template representation as a general guide to researchers in the field we also propose a general framework for achieving ssdr by combining existing sdr and udr models and also a procedure of reducing the computational cost via learning with a target set of relation features the effectiveness of our proposed methodologies is demonstrated with experiments with document collections for multilabel text categorization from the natural language processing domain

a generalized divergence measure for nonnegative matrix factorization this letter presents a general parametric divergence measure the metric includes as special cases quadratic error and kullback leibler divergence a parametric generalization of the two different multiplicative update rules for nonnegative matrix factorization by lee and seung 2001 is shown to lead to locally optimal solutions of the nonnegative matrix factorization problem with this new cost function numeric simulations demonstrate that the new update rule may improve the quadratic distance convergence speed a proof of convergence is given that as in lee and seung uses an auxiliary function known from the expectation maximization theoretical framework

learning to classify ordinal data the data replication method classification of ordinal data is one of the most important tasks of relation learning this paper introduces a new machine learning paradigm specifically intended for classification problems where the classes have a natural order the technique reduces the problem of classifying ordered classes to the standard two class problem the introduced method is then mapped into support vector machines and neural networks generalization bounds of the proposed ordinal classifier are also provided an experimental study with artificial and real data sets including an application to gene expression analysis verifies the usefulness of the proposed approach

manifold regularized discriminative nonnegative matrix factorization with fast gradient descent nonnegative matrix factorization nmf has become a popular data representation method and has been widely used in image processing and pattern recognition problems this is because the learned bases can be interpreted as a natural parts based representation of data and this interpretation is consistent with the psychological intuition of combining parts to form a whole for practical classification tasks however nmf ignores both the local geometry of data and the discriminative information of different classes in addition existing research results show that the learned basis is unnecessarily parts based because there is neither explicit nor implicit constraint to ensure the representation parts based in this paper we introduce the manifold regularization and the margin maximization to nmf and obtain the manifold regularized discriminative nmf md nmf to overcome the aforementioned problems the multiplicative update rule mur can be applied to optimizing md nmf but it converges slowly in this paper we propose a fast gradient descent fgd to optimize md nmf fgd contains a newton method that searches the optimal step length and thus fgd converges much faster than mur in addition fgd includes mur as a special case and can be applied to optimizing nmf and its variants for a problem with 165 samples in r1600 fgd converges in 28 s while mur requires 282 s we also apply fgd in a variant of md nmf and experimental results confirm its efficiency experimental results on several face image datasets suggest the effectiveness of md nmf

conjugacy classes in sylow p subgroups of gl n q taking s to consist of lower unitriangular matrices higman notes that for i 3 ic g j

linear dimensionality reduction survey insights and generalizations linear dimensionality reduction methods are a cornerstone of analyzing high dimensional data due to their simple geometric interpretations and typically attractive computational properties these methods capture many data features of interest such as covariance dynamical structure correlation between data sets input output relationships and margin between data classes methods have been developed with a variety of names and motivations in many fields and perhaps as a result the connections between all these methods have not been highlighted here we survey methods from this disparate literature as optimization programs over matrix manifolds we discuss principal component analysis factor analysis linear multidimensional scaling fisher s linear discriminant analysis canonical correlations analysis maximum autocorrelation factors slow feature analysis sufficient dimensionality reduction undercomplete independent component analysis linear regression distance metric learning and more this optimization framework gives insight to some rarely discussed shortcomings of well known methods such as the suboptimality of certain eigenvector solutions modern techniques for optimization over matrix manifolds enable a generic linear dimensionality reduction solver which accepts as input data and an objective to be optimized and returns as output an optimal low dimensional projection of the data this simple optimization framework further allows straightforward generalizations and novel variants of classical methods which we demonstrate here by creating an orthogonal projection canonical correlations analysis more broadly this survey and generic solver suggest that linear dimensionality reduction can move toward becoming a blackbox objective agnostic numerical technology

web image search re ranking with click based similarity and typicality in image search re ranking besides the well known semantic gap intent gap which is the gap between the representation of users query demand and the real intent of the users is becoming a major problem restricting the development of image retrieval to reduce human effects in this paper we use image click through data which can be viewed as the implicit feedback from users to help overcome the intention gap and further improve the image search performance generally the hypothesis visually similar images should be close in a ranking list and the strategy images with higher relevance should be ranked higher than others are widely accepted to obtain satisfying search results thus image similarity and the level of relevance typicality are determinate factors correspondingly however when measuring image similarity and typicality conventional re ranking approaches only consider visual information and initial ranks of images while overlooking the influence of click through data this paper presents a novel re ranking approach named spectral clustering re ranking with click based similarity and typicality first to learn an appropriate similarity measurement we propose click based multi feature similarity learning algorithm which conducts metric learning based on click based triplets selection and integrates multiple features into a unified similarity space via multiple kernel learning then based on the learnt click based image similarity measure we conduct spectral clustering to group visually and semantically similar images into same clusters and get the final re rank list by calculating click based clusters typicality and within clusters click based image typicality in descending order our experiments conducted on two real world query image data sets with diverse representative queries show that our proposed re ranking approach can significantly improve initial search results and outperform several existing re ranking approaches

algorithms for nonnegative matrix factorization with the β divergence this letter describes algorithms for nonnegative matrix factorization nmf with the divergence nmf the divergence is a family of cost functions parameterized by a single shape parameter that takes the euclidean distance the kullback leibler divergence and the itakura saito divergence as special cases 2 1 0 respectively the proposed algorithms are based on a surrogate auxiliary function a local majorization of the criterion function we first describe a majorization minimization algorithm that leads to multiplicative updates which differ from standard heuristic multiplicative updates by a dependent power exponent the monotonicity of the heuristic algorithm can however be proven for 0 1 using the proposed auxiliary function then we introduce the concept of the majorization equalization me algorithm which produces updates that move along constant level sets of the auxiliary function and lead to larger steps than mm simulations on synthetic and real data illustrate the faster convergence of the me approach the letter also describes how the proposed algorithms can be adapted to two common variants of nmf penalized nmf when a penalty function of the factors is added to the criterion function and convex nmf when the dictionary is assumed to belong to a known subspace

factor analysis and aic the information criterion aic was introduced to extend the method of maximum likelihood to the multimodel situation it was obtained by relating the successful experience of the order determination of an autoregressive model to the determination of the number of factors in the maximum likelihood factor analysis the use of the aic criterion in the factor analysis is particularly interesting when it is viewed as the choice of a bayesian model this observation shows that the area of application of aic can be much wider than the conventional i i d type models on which the original derivation of the criterion was based the observation of the bayesian structure of the factor analysis model leads us to the handling of the problem of improper solution by introducing a natural prior distribution of factor loadings

cluster ensembles a knowledge reuse framework for combining multiple partitions this paper introduces the problem of combining multiple partitionings of a set of objects into a single consolidated clustering without accessing the features or algorithms that determined these partitionings we first identify several application scenarios for the resultant knowledge reuse framework that we call cluster ensembles the cluster ensemble problem is then formalized as a combinatorial optimization problem in terms of shared mutual information in addition to a direct maximization approach we propose three effective and efficient techniques for obtaining high quality combiners consensus functions the first combiner induces a similarity measure from the partitionings and then reclusters the objects the second combiner is based on hypergraph partitioning the third one collapses groups of clusters into meta clusters which then compete for each object to determine the combined clustering due to the low computational costs of our techniques it is quite feasible to use a supra consensus function that evaluates all three approaches against the objective function and picks the best solution for a given situation we evaluate the effectiveness of cluster ensembles in three qualitatively different application scenarios i where the original clusters were formed based on non identical sets of features ii where the original clustering algorithms worked on non identical sets of objects and iii where a common data set is used and the main purpose of combining multiple clusterings is to improve the quality and robustness of the solution promising results are obtained in all three situations for synthetic as well as real data sets

soft margin multiple kernel learning multiple kernel learning mkl has been proposed for kernel methods by learning the optimal kernel from a set of predefined base kernels however the traditional l 1 rm mkl method often achieves worse results than the simplest method using the average of base kernels i e average kernel in some practical applications in order to improve the effectiveness of mkl this paper presents a novel soft margin perspective for mkl specifically we introduce an additional slack variable called kernel slack variable to each quadratic constraint of mkl which corresponds to one support vector machine model using a single base kernel we first show that l 1 rm mkl can be deemed as hard margin mkl and then we propose a novel soft margin framework for mkl three commonly used loss functions including the hinge loss the square hinge loss and the square loss can be readily incorporated into this framework leading to the new soft margin mkl objective functions many existing mkl methods can be shown as special cases under our soft margin framework for example the hinge loss soft margin mkl leads to a new box constraint for kernel combination coefficients using different hyper parameter values for this formulation we can inherently bridge the method using average kernel l 1 rm mkl and the hinge loss soft margin mkl the square hinge loss soft margin mkl unifies the family of elastic net constraint regularizer based approaches and the square loss soft margin mkl incorporates l 2 rm mkl naturally moreover we also develop efficient algorithms for solving both the hinge loss and square hinge loss soft margin mkl comprehensive experimental studies for various mkl algorithms on several benchmark data sets and two real world applications including video action recognition and event recognition demonstrate that our proposed algorithms can efficiently achieve an effective yet sparse solution for mkl

constrained nonnegative matrix factorization for image representation nonnegative matrix factorization nmf is a popular technique for finding parts based linear representations of nonnegative data it has been successfully applied in a wide range of applications such as pattern recognition information retrieval and computer vision however nmf is essentially an unsupervised method and cannot make use of label information in this paper we propose a novel semi supervised matrix decomposition method called constrained nonnegative matrix factorization cnmf which incorporates the label information as additional constraints specifically we show how explicitly combining label information improves the discriminating power of the resulting matrix decomposition we explore the proposed cnmf method with two cost function formulations and provide the corresponding update solutions for the optimization problems empirical experiments demonstrate the effectiveness of our novel algorithm in comparison to the state of the art approaches through a set of evaluations based on real world applications

pairwise sparsity preserving embedding for unsupervised subspace learning and classification two novel unsupervised dimensionality reduction techniques termed sparse distance preserving embedding sdpe and sparse proximity preserving embedding sppe are proposed for feature extraction and classification sdpe and sppe perform in the clean data space recovered by sparse representation and enhanced euclidean distances over noise removed data are employed to measure pairwise similarities of points in extracting informative features sdpe and sppe aim at preserving pairwise similarities between data points in addition to preserving the sparse characteristics this paper calculates the sparsest representation of all vectors jointly by a convex optimization the sparsest codes enable certain local information of data to be preserved and can endow sdpe and sppe a natural discriminating power adaptive neighborhood and robust characteristic against noise and errors in delivering low dimensional embeddings we also mathematically show sdpe and sppe can be effectively extended for discriminant learning in a supervised manner the validity of sdpe and sppe is examined by extensive simulations comparison with other related state of the art unsupervised algorithms show that promising results are delivered by our techniques

a novel framework for the nmf methods with experiments to unmixing signals and feature representation abstract non negative matrix factorization nmf can be used in clustering feature representation or blind source separation many nmf methods have been developed including least squares ls error kullback leibler kl divergence itakura saito is divergence bregman divergence divergence divergence divergence convex constrained graph regularized nmfs the main contribution of this paper is to develop a framework to generalize the existing nmf methods and also provide new nmf methods this paper constructs a general optimization model and develops a generic updating rule with a simple structure using a surrogate function which possesses similar properties as the standard nmf methods the experimental results obtained using several standard databases demonstrate the power of the work in which some new methods provide performance superior to that of the other existing methods

stallings foldings and subgroups of free groups we re cast in a more combinatorial and computational form the topological approach of j stallings to the study of subgroups of free groups

decentralized clustering by finding loose and distributed density cores abstract centroid based clustering approaches fail to recognize extremely complex patterns that are non isotropic we analyze the underlying causes and find some inherent flaws in these approaches including shape loss false distances and false peaks which typically cause centroid based approaches to fail when applied to complex patterns as an alternative to current methods we propose a hybrid decentralized approach named dcore which is based on finding density cores instead of centroids to overcome these flaws the underlying idea is that we consider each cluster to have a shrunken density core that roughly retains the shape of the cluster each such core consists of a set of loosely connected local density peaks of higher density than their surroundings borders edges and outliers are distributed around the outsides of these cores in a hierarchical structure experiments demonstrate that the promise of dcore lies in its power to recognize extremely complex patterns and its high performance in real applications for example image segmentation and face clustering regardless of the dimensionality of the space in which the data are embedded

learning the dynamical system behind sensory data this letter looks at the physics behind sensory data by identifying the parameters that govern the physical system and estimating them from sensory observations we extend takens s delay embedding theorem to the dynamical systems controlled by parameters an embedding of the product space of the phase and the parameter spaces of the dynamical system can be obtained by the delay embedding map provided that the parameter of the dynamical system changes slowly the reconstruction error is bounded for slowly varying parameters a manifold learning technique is applied to the embedding obtained to extract a low dimensional global coordinate system representing the product space the phase space of the deterministic dynamics can be contracted by using the adjacency relationship in time which enables recovery of only the parameter space as examples the manifolds of synthetic and real world vowels with time varying fundamental frequency f0 are analyzed and the f0 contours are extracted by an unsupervised algorithm experimental results show that the proposed method leads to robust performance under various noise conditions and rapid changes of f0 compared with the current state of the art f0 estimation algorithms

a quasi likelihood approach to nonnegative matrix factorization a unified approach to nonnegative matrix factorization based on the theory of generalized linear models is proposed this approach embeds a variety of statistical models including the exponential family within a single theoretical framework and provides a unified view of such factorizations from the perspective of quasi likelihood using this framework a family of algorithms for handling signal dependent noise is developed and its convergence proved using the expectation maximization algorithm in addition a measure to evaluate the goodness of fit of the resulting factorization is described the proposed methods allow modeling of nonlinear effects using appropriate link functions and are illustrated using an application in biomedical signal processing

riemannian manifold learning recently manifold learning has been widely exploited in pattern recognition data analysis and machine learning this paper presents a novel framework called riemannian manifold learning rml based on the assumption that the input high dimensional data lie on an intrinsically low dimensional riemannian manifold the main idea is to formulate the dimensionality reduction problem as a classical problem in riemannian geometry that is how to construct coordinate charts for a given riemannian manifold we implement the riemannian normal coordinate chart which has been the most widely used in riemannian geometry for a set of unorganized data points first two input parameters the neighborhood size k and the intrinsic dimension d are estimated based on an efficient simplicial reconstruction of the underlying manifold then the normal coordinates are computed to map the input high dimensional data into a low dimensional space experiments on synthetic data as well as real world images demonstrate that our algorithm can learn intrinsic geometric structures of the data preserve radial geodesic distances and yield regular embeddings

clustering documents with labeled and unlabeled documents using fuzzy semi kmeans while focusing on document clustering this work presents a fuzzy semi supervised clustering algorithm called fuzzy semi kmeans the fuzzy semi kmeans is an extension of k means clustering model and it is inspired by an em algorithm and a gaussian mixture model additionally the fuzzy semi kmeans provides the flexibility to employ different fuzzy membership functions to measure the distance between data this work employs gaussian weighting function to conduct experiments but cosine similarity function can be used as well this work conducts experiments on three data sets and compares fuzzy semi kmeans with several methods the experimental results indicate that fuzzy semi kmeans can generally outperform the other methods

data based robust optimal control of continuous time affine nonlinear systems with matched uncertainties in this paper the robust optimal control of continuous time affine nonlinear systems with matched uncertainties is investigated by using a data based integral policy iteration approach it is a natural extension of the traditional optimal control design under the framework of adaptive dynamic programming adp method to robust optimal control of nonlinear systems with matched uncertainties in theoretical aspect by increasing a feedback gain to the optimal controller of the nominal system the robust controller of the matched uncertain system is obtained which also achieves optimality with a newly well defined cost function when regarding the implementation the data based integral policy iteration algorithm is used to solve the hamilton jacobi bellman equation corresponding to the nominal system with completely unknown dynamics information then the actor critic technique based on neural networks and least squares implementation method are employed to facilitate deriving the optimal control law iteratively so that the closed form expression of the robust optimal controller is available additionally two simulation examples with application backgrounds are presented to illustrate the effectiveness of the established robust optimal control scheme in summary it is important to note that the result developed in this paper broadens the application scope of adp based optimal control approach to more general nonlinear systems possessing dynamical uncertainties

an algorithm for recognition of n collapsing words a word w over a finite alphabet s is n collapsing if for an arbitrary deterministic finite automaton a the inequality d q w q n holds provided that d q u q n for some word email protected email protected depending on a we prove that the property of n collapsing is algorithmically recognizable for any given positive integer n we also prove that the language of all n collapsing words is context sensitive

characterizing the functional connectivity diversity of the insula cortex subregions diversity curves and behavior the connectivity of the insula cortex is diverse we present new models to characterize the resting state connectional diversity of the human insula cortex and perform model selection using high quality fmri data from the human connectome project we first attempt to parcellate the insula into distinct subregions using traditional clustering methods but find that the resulting subregions are not homogeneous and that the optimal number of subregions is substantially influenced by data smoothness we then introduce the concept of a diversity curve which we use to continuously parameterize the insula s laplacian eigenmap with respect to streamlines propagated through the eigenmap s gradient field to perform model selection we compare the insula s diversity curve to benchmark diversity curves for i two distinct regions ii a continuum of gradual change and iii an absence of any connectional diversity i e homogenous region of the three benchmarks tested we find that the insula s connectional diversity is most parsimoniously modeled as continuum of gradual change from dorsal posterior to ventral anterior we find that individuals who score high on measures of positive affect self efficacy emotion recognition motor dexterity and gustation show greater diversity within the anterior insula our findings are replicated using data from a second fmri session we conclude that the functional connectivity diversity of the insula can be characterized parsimoniously as a continuum avoiding the vexed task of determining an optimal number of insula subregions and that inter individual variation in this continuum can explain significant variation in behavior

image completion by diffusion maps and spectral relaxation we present a framework for image inpainting that utilizes the diffusion framework approach to spectral dimensionality reduction we show that on formulating the inpainting problem in the embedding domain the domain to be inpainted is smoother in general particularly for the textured images thus the textured images can be inpainted through simple exemplar based and variational methods we discuss the properties of the induced smoothness and relate it to the underlying assumptions used in contemporary inpainting schemes as the diffusion embedding is nonlinear and noninvertible we propose a novel computational approach to approximate the inverse mapping from the inpainted embedding space to the image domain we formulate the mapping as a discrete optimization problem solved through spectral relaxation the effectiveness of the presented method is exemplified by inpainting real images where it is shown to compare favorably with contemporary state of the art schemes

expressing infinite matrices over rings as products of involutions abstract let k be an arbitrary field and r be an arbitrary associative ring with identity 1 slowik in 12 proved that each matrix of ut k the group of upper triangular infinite matrices whose entries lying on the main diagonal are equal to either 1 or 1 can be expressed as a product of at most five involutions in this article we extend the investigate to an arbitrary associative ring r with identity 1 our conclusion is that every element of ut r can be expressed as a product of at most four involutions we also prove that for the complex field every element of t c the group of upper triangular infinite matrices whose entries lying on the main diagonal satisfy a a 1 can be expressed as a product of at most three coninvolutions matrices satisfying a a i

nonnegative matrix factorization with rank regularization and hard constraint nonnegative matrix factorization nmf is well known to be an effective tool for dimensionality reduction in problems involving big data for this reason it frequently appears in many areas of scientific and engineering literature this letter proposes a novel semisupervised nmf algorithm for overcoming a variety of problems associated with nmf algorithms including poor use of prior information negative impact on manifold structure of the sparse constraint and inaccurate graph construction our proposed algorithm nonnegative matrix factorization with rank regularization and hard constraint nmfrc incorporates label information into data representation as a hard constraint which makes full use of prior information nmfrc also measures pairwise similarity according to geodesic distance rather than euclidean distance this results in more accurate measurement of pairwise relationships resulting in more effective manifold information furthermore nmfrc adopts rank constraint instead of norm constraints for regularization to balance the sparseness and smoothness of data in this way the new data representation is more representative and has better interpretability experiments on real data sets suggest that nmfrc outperforms four other state of the art algorithms in terms of clustering accuracy

manifold regularized cross modal embedding for zero shot learning zero shot learning zsl aims at classifying previously unseen class samples and has gained its popularity in applications where samples of some categories are scarce for training the basic idea to address this issue is transferring knowledge from the seen classes to the unseen classes through mapping the visual feature to an embedding space spanned by class semantic information the class semantic information can be obtained from human labeled attributes or text corpus in an unsupervised fashion therefore the embedding function from visual space to the embedding space is extremely important however the existing embedding approaches to zsl mainly focus on aligning pairwise semantic consistency from heterogeneous spaces but ignore the intrinsic structure of the locally homogeneous isomorph in order to preserve the locally visual structure in the embedding process this paper proposes a manifold regularized cross modal embedding mcme approach for zsl by formulating the manifold constraint for intrinsic structure of the visual features as well as aligning pairwise consistency the linear closed form solution makes mcme efficient to compute furthermore rather than applying the embedding function learned from the seen classes directly we also propose a new domain adaptation strategy to overcome the domain shift problem during the knowledge transfer process the mcme with the domain adaptation method is called mcme da extensive experiments on the benchmark datasets of awa and cub validate the superiority and promise of mcme and mcme da

approximate orthogonal sparse embedding for dimensionality reduction locally linear embedding lle is one of the most well known manifold learning methods as the representative linear extension of lle orthogonal neighborhood preserving projection onpp has attracted widespread attention in the field of dimensionality reduction in this paper a unified sparse learning framework is proposed by introducing the sparsity or l 1 norm learning which further extends the lle based methods to sparse cases theoretical connections between the onpp and the proposed sparse linear embedding are discovered the optimal sparse embeddings derived from the proposed framework can be computed by iterating the modified elastic net and singular value decomposition we also show that the proposed model can be viewed as a general model for sparse linear and nonlinear kernel subspace learning based on this general model sparse kernel embedding is also proposed for nonlinear sparse feature extraction extensive experiments on five databases demonstrate that the proposed sparse learning framework performs better than the existing subspace learning algorithm particularly in the cases of small sample sizes

online ranking by projecting we discuss the problem of ranking instances in our framework each instance is associated with a rank or a rating which is an integer in 1 to k our goal is to find a rank prediction rule that assigns each instance a rank that is as close as possible to the instance s true rank we discuss a group of closely related online algorithms analyze their performance in the mistake bound model and prove their correctness we describe two sets of experiments with synthetic data and with the eachmovie data set for collaborative filtering in the experiments we performed our algorithms outperform online algorithms for regression and classification applied to ranking

robust locally linear embedding in the past few years some nonlinear dimensionality reduction nldr or nonlinear manifold learning methods have aroused a great deal of interest in the machine learning community these methods are promising in that they can automatically discover the low dimensional nonlinear manifold in a high dimensional data space and then embed the data points into a low dimensional embedding space using tractable linear algebraic techniques that are easy to implement and are not prone to local minima despite their appealing properties these nldr methods are not robust against outliers in the data yet so far very little has been done to address the robustness problem in this paper we address this problem in the context of an nldr method called locally linear embedding lle based on robust estimation techniques we propose an approach to make lle more robust we refer to this approach as robust locally linear embedding rlle we also present several specific methods for realizing this general rlle approach experimental results on both synthetic and real world data show that rlle is very robust against outliers

image reducing words and subgroups of free groups a word w over a finite alphabet is said to be n collapsing if for an arbitrary finite automaton a q the inequality q w q n holds provided that q u q n for some word u depending on a we give an algorithm to test whether a word is 2 collapsing to this aim we associate to every word w a finite family of finitely generated subgroups in finitely generated free groups and prove that the property of being 2 collapsing reflects in the property that each of these subgroups has index at most 2 in the corresponding free group we also find a similar characterization for the closely related class of so called 2 synchronizing words

a hybrid binary particle swarm optimization for the obnoxious p median problem abstract the obnoxious p median problem can be formulated as a constrained binary linear program it is np hard and has a lot of real world applications in this paper a hybrid binary particle swarm optimization is proposed to solve the obnoxious p median problem a new position updating rule is presented to inherit the good structure of previous high quality solutions furthermore two tabu based mutation operators are used to avoid the premature convergence and guide the search to a promising area a greedy repair procedure is developed to repair infeasible solutions in addition an iterated greedy local search procedure is utilized to enhance the exploitation ability extensive experiments are done on a set of 72 benchmark instances from the literature experimental results and comparisons with some existing algorithms demonstrate the effectiveness of the proposed algorithm in particular the proposed algorithm finds new best solutions for 15 instances compared with existing algorithms the proposed algorithm is able to find better average objective function value in a short average computing time

recognizing 3 collapsing words over a binary alphabet a finite deterministic automaton a q is k compressible if there is a word w such that the image of the state set q under the natural action of w is reduced by at least k states in such case w is called a k compressing word for a it is known that for any alphabet and any k 2 there exist words that are k compressing for each k compressible automaton with the input alphabet such words are called k collapsing it has been proved that recognizing 2 collapsing words over a 2 element alphabet may be done in polynomial time while recognizing 2 collapsing words over an alphabet of size n 3 is co np complete the computational complexity of recognizing k collapsing words with k 3 over an n element alphabet has remained open in this paper we prove that this remaining problem is co np complete in the simplest case with k 3 and n 2

incremental support vector learning for ordinal regression support vector ordinal regression svor is a popular method to tackle ordinal regression problems however until now there were no effective algorithms proposed to address incremental svor learning due to the complicated formulations of svor recently an interesting accurate on line algorithm was proposed for training nu support vector classification nu svc which can handle a quadratic formulation with a pair of equality constraints in this paper we first present a modified svor formulation based on a sum of margins strategy the formulation has multiple constraints and each constraint includes a mixture of an equality and an inequality then we extend the accurate on line nu svc algorithm to the modified formulation and propose an effective incremental svor algorithm the algorithm can handle a quadratic formulation with multiple constraints where each constraint is constituted of an equality and an inequality more importantly it tackles the conflicts between the equality and inequality constraints we also provide the finite convergence analysis for the algorithm numerical experiments on the several benchmark and real world data sets show that the incremental algorithm can converge to the optimal solution in a finite number of steps and is faster than the existing batch and incremental svor algorithms meanwhile the modified formulation has better accuracy than the existing incremental svor algorithm and is as accurate as the sum of margins based formulation of shashua and levin

verbal subgroups in the group of triangular matrices over field of characteristic 0 in this paper we describe the verbal subgroups of groups utn k and tn k which are the groups of respectively unitriangular and triangular matrices over field k of characteristic 0 we prove that every verbal subgroup of utn k coincides with a term of the lower central series of the group and that every verbal subgroup v tn k either is of the form v k utn k or coincides with a verbal subgroup of utn k we also describe the width of these verbal subgroups and make a few remarks regarding the groups of infinite triangular matrices

spectral embedded clustering a framework for in sample and out of sample spectral clustering spectral clustering sc methods have been successfully applied to many real world applications the success of these sc methods is largely based on the manifold assumption namely that two nearby data points in the high density region of a low dimensional data manifold have the same cluster label however such an assumption might not always hold on high dimensional data when the data do not exhibit a clear low dimensional manifold structure e g high dimensional and sparse data the clustering performance of sc will be degraded and become even worse than k means clustering in this paper motivated by the observation that the true cluster assignment matrix for high dimensional data can be always embedded in a linear space spanned by the data we propose the spectral embedded clustering sec framework in which a linearity regularization is explicitly added into the objective function of sc methods more importantly the proposed sec framework can naturally deal with out of sample data we also present a new laplacian matrix constructed from a local regression of each pattern and incorporate it into our sec framework to capture both local and global discriminative information for clustering comprehensive experiments on eight real world high dimensional datasets demonstrate the effectiveness and advantages of our sec framework over existing sc methods and k means based clustering methods our sec framework significantly outperforms sc using the nystrom algorithm on unseen data

structural minimax probability machine minimax probability machine mpm is an interesting discriminative classifier based on generative prior knowledge it can directly estimate the probabilistic accuracy bound by minimizing the maximum probability of misclassification the structural information of data is an effective way to represent prior knowledge and has been found to be vital for designing classifiers in real world problems however mpm only considers the prior probability distribution of each class with a given mean and covariance matrix which does not efficiently exploit the structural information of data in this paper we use two finite mixture models to capture the structural information of the data from binary classification for each subdistribution in a finite mixture model only its mean and covariance matrix are assumed to be known based on the finite mixture models we propose a structural mpm smpm smpm can be solved effectively by a sequence of the second order cone programming problems moreover we extend a linear model of smpm to a nonlinear model by exploiting kernelization techniques we also show that the smpm can be interpreted as a large margin classifier and can be transformed to support vector machine and maxi min margin machine under certain special conditions experimental results on both synthetic and real world data sets demonstrate the effectiveness of smpm

reduction for characters of finite algebra groups abstract let j be a finite dimensional nilpotent algebra over a finite field f q we formulate a procedure for analysing characters of the group 1 j in particular we study characters of the group u n q of unipotent triangular n n matrices over f q using our procedure we compute the number of irreducible characters of u n q of each degree for n 13 also we explain and generalise a phenomenon concerning the group u 13 2 discovered by isaacs and karagueuzian

feasibility and finite convergence analysis for accurate on line nu support vector machine the support vector machine svm for classification has the advantage of using a parameter on controlling the number of support vectors and margin errors recently an interesting accurate on line algorithm accurate on line svm algorithm aonsvm is proposed for training svm aonsvm can be viewed as a special case of parametric quadratic programming techniques it is demonstrated that aonsvm avoids the infeasible updating path as far as possible and successfully converges to the optimal solution based on experimental analysis however because of the differences between aonsvm and classical parametric quadratic programming techniques there is no theoretical justification for these conclusions in this paper we prove the feasibility and finite convergence of aonsvm under two assumptions the main results of feasibility analysis include 1 the inverses of the two key matrices in aonsvm always exist 2 the rules for updating the two key inverse matrices are reliable 3 the variable can control the adjustment of the sum of all the weights efficiently and 4 a sample cannot migrate back and forth in successive adjustment steps among the set of margin support vectors the set of error support vectors and the set of the remaining vectors moreover the analyses of aonsvm also provide the proofs of the feasibility and finite convergence for accurate on line c svm learning directly

nonlinear component analysis as a kernel eigenvalue problem a new method for performing a nonlinear form of principal component analysis is proposed by the use of integral operator kernel functions one can efficiently compute principal components in high dimensional feature spaces related to input space by some nonlinear map for instance the space of all possible five pixel products in 16 16 images we give the derivation of the method and present experimental results on polynomial feature extraction for pattern recognition

toward optimal manifold hashing via discrete locally linear embedding binary code learning also known as hashing has received increasing attention in large scale visual search by transforming high dimensional features to binary codes the original euclidean distance is approximated via hamming distance more recently it is advocated that it is the manifold distance rather than the euclidean distance that should be preserved in the hamming space however it retains as an open problem to directly preserve the manifold structure by hashing in particular it first needs to build the local linear embedding in the original feature space and then quantize such embedding to binary codes such a two step coding is problematic and less optimized besides the off line learning is extremely time and memory consuming which needs to calculate the similarity matrix of the original data in this paper we propose a novel hashing algorithm termed discrete locality linear embedding hashing dllh which well addresses the above challenges the dllh directly reconstructs the manifold structure in the hamming space which learns optimal hash codes to maintain the local linear relationship of data points to learn discrete locally linear embedding codes we further propose a discrete optimization algorithm with an iterative parameters updating scheme moreover an anchor based acceleration scheme termed anchor dllh is further introduced which approximates the large similarity matrix by the product of two low rank matrices experimental results on three widely used benchmark data sets i e cifar10 nus wide and youtube face have shown superior performance of the proposed dllh over the state of the art approaches

reversible data hiding with contrast enhancement and tamper localization for medical images digital transmission of medical images often involves hiding crucial information in some parts of the images which should be later extracted to authenticate ownership and identity in this paper a new reversible data hiding rdh algorithm for medical images is proposed the primary objective of the algorithm is to achieve contrast enhancement of the region of interest roi without introducing distortion and achieve tamper localization against attacks on the roi first the background and roi of the medical image are segmented using otsu s automatic optimal thresholding method to reduce the visual distortion an improved scheme for preprocessing is applied to reduce the number of disordered pixels by expanding the peak pairs of the roi histogram data embedding along with distortion less contrast enhancement of the roi is achieved the feature bit matrix generated from the roi is embedded into the least significant bits lsbs of the background pixels at the receiving end the tampered contents from the roi of the detected image can be located using a difference matrix between the feature bit matrix generated from the roi and that extracted from the background in the absence of tampering the original roi can be completely restored after the embedded data is extracted experimental results demonstrate that in comparison with some state of the art rdh algorithms the proposed algorithm achieves better performance in terms of contrast enhancement of roi preserving visual quality of the background and tamper localization

non negative matrix factorization with sparseness constraints non negative matrix factorization nmf is a recently developed technique for finding parts based linear representations of non negative data although it has successfully been applied in several applications it does not always result in parts based representations in this paper we show how explicitly incorporating the notion of sparseness improves the found decompositions additionally we provide complete matlab code both for standard nmf and for our extension our hope is that this will further the application of these methods to solving novel data analysis problems

the global convergence of a modified bfgs method for nonconvex functions abstract the standard bfgs method plays an important role among the quasi newton algorithms for constrained un constrained optimization problems however dai 2003 constructed a counterexample to demonstrate that this method may fail for non convex functions with inexact wolfe line searches and mascarenhas 2004 showed the nonconvergence of this method and other methods in the broyden family even with exact line search techniques these works motivate us to try to find another way to obtain another quasi newton method with better convergence based on the standard bfgs formula in this paper four approaches are used in the designed algorithm 1 a modified weak wolfe powell line search technique is introduced 2 if one defined condition is satisfied then the search direction and its associated step length are accepted and the next point is designed 3 otherwise a parabolic will be presented and it is regarded as the projection surface to avoid using the failed direction and the next point x k 1 is generated by a projection technique and 4 to easily obtain the global convergence of the proposed algorithm the projection point is not used in the current bfgs update but rather for all the next iterations the new algorithm possesses global convergence for general functions under the inexact modified weak wolfe powell line search technique and it is shown that other methods in the broyden class also have this property numerical results are reported to show the performance of the given algorithm and other similar methods

a least squares framework for component analysis over the last century component analysis ca methods such as principal component analysis pca linear discriminant analysis lda canonical correlation analysis cca locality preserving projections lpp and spectral clustering sc have been extensively used as a feature extraction step for modeling classification visualization and clustering ca techniques are appealing because many can be formulated as eigen problems offering great potential for learning linear and nonlinear representations of data in closed form however the eigen formulation often conceals important analytic and computational drawbacks of ca techniques such as solving generalized eigen problems with rank deficient matrices e g small sample size problem lacking intuitive interpretation of normalization factors and understanding commonalities and differences between ca methods this paper proposes a unified least squares framework to formulate many ca methods we show how pca lda cca lpp sc and its kernel and regularized extensions correspond to a particular instance of least squares weighted kernel reduced rank regression ls wkrrr the ls wkrrr formulation of ca methods has several benefits 1 provides a clean connection between many ca techniques and an intuitive framework to understand normalization factors 2 yields efficient numerical schemes to solve ca techniques 3 overcomes the small sample size problem 4 provides a framework to easily extend ca methods we derive weighted generalizations of pca lda sc and cca and several new ca techniques

graph regularized nonnegative matrix factorization for data representation matrix factorization techniques have been frequently applied in information retrieval computer vision and pattern recognition among them nonnegative matrix factorization nmf has received considerable attention due to its psychological and physiological interpretation of naturally occurring data whose representation may be parts based in the human brain on the other hand from the geometric perspective the data is usually sampled from a low dimensional manifold embedded in a high dimensional ambient space one then hopes to find a compact representation which uncovers the hidden semantics and simultaneously respects the intrinsic geometric structure in this paper we propose a novel algorithm called graph regularized nonnegative matrix factorization gnmf for this purpose in gnmf an affinity graph is constructed to encode the geometrical information and we seek a matrix factorization which respects the graph structure our empirical study shows encouraging results of the proposed algorithm in comparison to the state of the art algorithms on real world problems

transductive ordinal regression ordinal regression is commonly formulated as a multiclass problem with ordinal constraints the challenge of designing accurate classifiers for ordinal regression generally increases with the number of classes involved due to the large number of labeled patterns that are needed the availability of ordinal class labels however is often costly to calibrate or difficult to obtain unlabeled patterns on the other hand often exist in much greater abundance and are freely available to take benefits from the abundance of unlabeled patterns we present a novel transductive learning paradigm for ordinal regression in this paper namely transductive ordinal regression tor the key challenge of this paper lies in the precise estimation of both the ordinal class label of the unlabeled data and the decision functions of the ordinal classes simultaneously the core elements of the proposed tor include an objective function that caters to several commonly used loss functions casted in transductive settings for general ordinal regression a label swapping scheme that facilitates a strictly monotonic decrease in the objective function value is also introduced extensive numerical studies on commonly used benchmark datasets including the real world sentiment prediction problem are then presented to showcase the characteristics and efficacies of the proposed tor further comparisons to recent state of the art ordinal regression methods demonstrate the introduced transductive learning paradigm for ordinal regression led to the robust and improved performance

objective quality assessment of image retargeting by incorporating fidelity measures and inconsistency detection the tremendous growth in mobile devices has resulted in huge generation and usage of digital images image quality assessment is thus an important issue for mobile media applications in this paper we focus on the quality evaluation of images generated by content aware image retargeting in which the reference and the distorted images are of different sizes through retargeting many types of deformation inconsistency lead to shape distortion deformation artifacts and content information loss worsening its perceptual quality the deformation inconsistency occurs on different levels of the retargeted images limited by the accuracy of the alignment between the original and retargeted images previous methods only focus on pixel level and patch level fidelity analyses and fail to detect deformation inconsistency in this paper we improve the alignment algorithm and propose a three level representation of the retargeting process based on the analysis of this three level representation both fidelity measures and inconsistency detection are combined to determine the final retargeting quality the proposed algorithm is validated on the public data sets retargetme and cuhk experimental results demonstrate that inconsistency detection contributes to accurately assessing the image retargeting perceptual quality this inspires us to investigate more about deformation inconsistency to formulate the objective quality of image retargeting

topology of finite graphs this paper derives from a course in group theory which i gave at berkeley in 1982 i wanted to prove the standard theorems on free groups and discovered that after a few preliminaries the notion of locally injective map or immersion of graphs was very useful this enables one to see in an effective easy algorithmic way just what happens with finitely generated free groups one can understand in this way 1 howson s theorem that if a and b are finitely generated subgroups of a free group then a b is finitely generated and 2 m hall s theorem that finitely generated subgroups of free groups are closed in the profinite topology during this course s m gersten came up with a simple proof of h neumann s inequality on the ranks in howson s theorem one of the ideas in gersten s proof was to use core graphs graphs with no trees hanging on subsequently i found that some consequences of a paper of greenberg s could be proved using core graphs and covering translations of immersions the most striking such result is that if a and b are finitely generated subgroups of a free group and if a c b is of finite index in both a and b then a b is of finite index in a v b the subgroup generated by a ub

pattern vectors from algebraic graph theory graph structures have proven computationally cumbersome for pattern analysis the reason for this is that before graphs can be converted to pattern vectors correspondences must be established between the nodes of structures which are potentially of different size to overcome this problem in this paper we turn to the spectral decomposition of the laplacian matrix we show how the elements of the spectral matrix for the laplacian can be used to construct symmetric polynomials that are permutation invariants the coefficients of these polynomials can be used as graph features which can be encoded in a vectorial manner we extend this representation to graphs in which there are unary attributes on the nodes and binary attributes on the edges by using the spectral decomposition of a hermitian property matrix that can be viewed as a complex analogue of the laplacian to embed the graphs in a pattern space we explore whether the vectors of invariants can be embedded in a low dimensional space using a number of alternative strategies including principal components analysis pca multidimensional scaling mds and locality preserving projection lpp experimentally we demonstrate that the embeddings result in well defined graph clusters our experiments with the spectral representation involve both synthetic and real world data the experiments with synthetic data demonstrate that the distances between spectral feature vectors can be used to discriminate between graphs on the basis of their structure the real world experiments show that the method can be used to locate clusters of graphs

online learning of complex prediction problems using simultaneous projections we describe and analyze an algorithmic framework for online classification where each online trial consists of multiple prediction tasks that are tied together we tackle the problem of updating the online predictor by defining a projection problem in which each prediction task corresponds to a single linear constraint these constraints are tied together through a single slack parameter we then introduce a general method for approximately solving the problem by projecting simultaneously and independently on each constraint which corresponds to a prediction sub problem and then averaging the individual solutions we show that this approach constitutes a feasible albeit not necessarily optimal solution of the original projection problem we derive concrete simultaneous projection schemes and analyze them in the mistake bound model we demonstrate the power of the proposed algorithm in experiments with synthetic data and with multiclass text categorization tasks

iterative character constructions for algebra groups abstract we construct a family of orthogonal characters of an algebra group which decompose the supercharacters defined by diaconis and isaacs 2008 6 like supercharacters these characters are given by nonnegative integer linear combinations of kirillov functions and are induced from linear supercharacters of certain algebra subgroups we derive a formula for these characters and give a condition for their irreducibility generalizing a theorem of otto 2010 20 we also show that each such character has the same number of kirillov functions and irreducible characters as constituents in proving these results we observe as an application how a recent computation by evseev 2010 7 implies that every irreducible character of the unitriangular group ut n q of unipotent n n upper triangular matrices over a finite field with q elements is a kirillov function if and only if n 12 as a further application we discuss some more general conditions showing that kirillov functions are characters and describe some results related to counting the irreducible constituents of supercharacters

graph spectral image smoothing using the heat kernel a new method for smoothing both gray scale and color images is presented that relies on the heat diffusion equation on a graph we represent the image pixel lattice using a weighted undirected graph the edge weights of the graph are determined by the gaussian weighted distances between local neighboring windows we then compute the associated laplacian matrix the degree matrix minus the adjacency matrix anisotropic diffusion across this weighted graph structure with time is captured by the heat equation and the solution i e the heat kernel is found by exponentiating the laplacian eigensystem with time image smoothing is accomplished by convolving the heat kernel with the image and its numerical implementation is realized by using the krylov subspace technique the method has the effect of smoothing within regions but does not blur region boundaries we also demonstrate the relationship between our method standard diffusion based pdes fourier domain signal processing and spectral clustering experiments and comparisons on standard images illustrate the effectiveness of the method