on minimum class locality preserving variance support vector machine in this paper a so called minimum class locality preserving variance support machine mclpv svm algorithm is presented by introducing the basic idea of the locality preserving projections lpp which can be seen as a modified class of support machine svm and or minimum class variance support machine mcvsvm mclpv svm in contrast to svm and mcvsvm takes the intrinsic manifold structure of the data space into full consideration and inherits the characteristics of svm and mcvsvm we discuss in the paper the linear case the small sample size case and the nonlinear case of the mclpv svm similar to mcvsvm the mclpv svm optimization problem in the small sample size case is solved by using dimensionality reduction through principal component analysis pca and one in the nonlinear case is transformed into an equivalent linear mclpv svm problem under kernel pca kpca experimental results on real datasets indicate the effectiveness of the mclpv svm by comparing it with svm and mcvsvm

eigenfaces vs fisherfaces recognition using class specific linear projection we develop a face recognition algorithm which is insensitive to large variation in lighting direction and facial expression taking a pattern classification approach we consider each pixel in an image as a coordinate in a high dimensional space we take advantage of the observation that the images of a particular face under varying illumination but fixed pose lie in a 3d linear subspace of the high dimensional image space if the face is a lambertian surface without shadowing however since faces are not truly lambertian surfaces and do indeed produce self shadowing images will deviate from this linear subspace rather than explicitly modeling this deviation we linearly project the image into a subspace in a manner which discounts those regions of the face with large deviation our projection method is based on fisher s linear discriminant and produces well separated classes in a low dimensional subspace even under severe variation in lighting and facial expressions the eigenface technique another method based on linearly projecting the image space to a low dimensional subspace has similar computational requirements yet extensive experimental results demonstrate that the proposed fisherface method has error rates that are lower than those of the eigenface technique for tests on the harvard and yale face databases

nonsmooth nonnegative matrix factorization nsnmf we propose a novel nonnegative matrix factorization model that aims at finding localized part based representations of nonnegative multivariate data items unlike the classical nonnegative matrix factorization nmf technique this new model denoted nonsmooth nonnegative matrix factorization nsnmf corresponds to the optimization of an unambiguous cost function designed to explicitly represent sparseness in the form of nonsmoothness which is controlled by a single parameter in general this method produces a set of basis and encoding vectors that are not only capable of representing the original data but they also extract highly focalized patterns which generally lend themselves to improved interpretability the properties of this new method are illustrated with several data sets comparisons to previously published methods show that the new nsnmf method has some advantages in keeping faithfulness to the data in the achieving a high degree of sparseness for both the estimated basis and the encoding vectors and in better interpretability of the factors

least squares recursive projection twin support vector machine for classification in this paper we formulate a least squares version of the recently proposed projection twin support vector machine ptsvm for binary classification this formulation leads to extremely simple and fast algorithm called least squares projection twin support vector machine lsptsvm for generating binary classifiers different from ptsvm we add a regularization term ensuring the optimization problems in our lsptsvm are positive definite and resulting better generalization ability instead of usually solving two dual problems we solve two modified primal problems by solving two systems of linear equations whereas ptsvm need to solve two quadratic programming problems along with two systems of linear equations our experiments on publicly available datasets indicate that our lsptsvm has comparable classification accuracy to that of ptsvm but with remarkably less computational time

improved discriminate analysis for high dimensional data and its application to face recognition many pattern recognition applications involve the treatment of high dimensional data and the small sample size problem principal component analysis pca is a common used dimension reduction technique linear discriminate analysis lda is often employed for classification pca plus lda is a famous framework for discriminant analysis in high dimensional space and singular cases in this paper we examine the theory of this framework and find out that even if there is no small sample size problem the pca dimension reduction cannot guarantee the subsequent successful application of lda we thus develop an improved discriminate analysis method by introducing an inverse fisher criterion and adding a constrain in pca procedure so that the singularity phenomenon will not occur experiment results on face recognition suggest that this new approach works well and can be applied even when the number of training samples is one per class

low rank doubly stochastic matrix decomposition for cluster analysis cluster analysis by nonnegative low rank approximations has experienced a remarkable progress in the past decade however the majority of such approximation approaches are still restricted to nonnegative matrix factorization nmf and suffer from the following two drawbacks 1 they are unable to produce balanced partitions for large scale manifold data which are common in real world clustering tasks 2 most existing nmf type clustering methods cannot automatically determine the number of clusters we propose a new low rank learning method to address these two problems which is beyond matrix factorization our method approximately decomposes a sparse input similarity in a normalized way and its objective can be used to learn both cluster assignments and the number of clusters for efficient optimization we use a relaxed formulation based on data cluster data random walk which is also shown to be equivalent to low rank factorization of the doubly stochastically normalized cluster incidence matrix the probabilistic cluster assignments can thus be learned with a multiplicative majorization minimization algorithm experimental results show that the new method is more accurate both in terms of clustering large scale manifold data sets and of selecting the number of clusters

generalizing discriminant analysis using the generalized singular value decomposition discriminant analysis has been used for decades to extract features that preserve class separability it is commonly defined as an optimization problem involving covariance matrices that represent the scatter within and between clusters the requirement that one of these matrices be nonsingular limits its application to data sets with certain relative dimensions we examine a number of optimization criteria and extend their applicability by using the generalized singular value decomposition to circumvent the nonsingularity requirement the result is a generalization of discriminant analysis that can be applied even when the sample size is smaller than the dimension of the sample data we use classification results from the reduced representation to compare the effectiveness of this approach with some alternatives and conclude with a discussion of their relative merits

joint distance and similarity measure learning based on triplet based constraints a novel model is proposed to learn combined distance and similarity measure cdsm based on triplet based constraints our cdsm can be kernelized for learning nonlinear measures two effective strategies are suggested to speed up the training and testing of kernelized cdsm extensive experiments validate the superiority of cdsm and kernelized cdsm distance and similarity measures usually are complementary to pattern classification with pairwise constraints several approaches have been proposed to combine distance and similarity measures however it remains less investigated to use triplets of samples for joint learning of distance and similarity measures moreover the kernel extension of triplet based model is also nontrivial and computationally expensive in this paper we propose a novel method to learn a combined distance and similarity measure cdsm by incorporating with the max margin model we suggest a triplet based cdsm learning model with a unified regularizer of the frobenius norm a support vector machine svm based algorithm is then adopted to solve the optimization problem furthermore we extend cdsm for learning nonlinear measures via the kernel trick two effective strategies are adopted to speed up training and testing of kernelized cdsm experiments on the uci handwritten digits and person re identification datasets demonstrate that cdsm and kernelized cdsm outperform several state of the art metric learning methods

recursive projection twin support vector machine via within class variance minimization in this paper a novel binary classifier coined projection twin support vector machine ptsvm is proposed the idea is to seek two projection directions one for each class such that the projected samples of one class are well separated from those of the other class in its respective subspace in order to further boost performance a recursive algorithm for ptsvm is proposed to generate more than one projection axis for each class to overcome the singularity problem principal component analysis pca is utilized to transform the data in the original space into a low dimensional subspace wherein the optimization problem of ptsvm is convex and can be solved efficiently the experimental results on several uci benchmark data sets and usps digit database show the feasibility and effectiveness of the proposed method

learning the information divergence information divergence that measures the difference between two nonnegative matrices or tensors has found its use in a variety of machine learning problems examples are nonnegative matrix tensor factorization stochastic neighbor embedding topic models and bayesian network optimization the success of such a learning task depends heavily on a suitable divergence a large variety of divergences have been suggested and analyzed but very few results are available for an objective choice of the optimal divergence for a given task here we present a framework that facilitates automatic selection of the best divergence among a given family based on standard maximum likelihood estimation we first propose an approximated tweedie distribution for the beta divergence family selecting the best beta then becomes a machine learning problem solved by maximum likelihood next we reformulate alpha divergence in terms of beta divergence which enables automatic selection of alpha by maximum likelihood with reuse of the learning principle for beta divergence furthermore we show the connections between gamma and beta divergences as well as renyi and alpha divergences such that our automatic selection framework is extended to non separable divergences experiments on both synthetic and real world data demonstrate that our method can quite accurately select information divergence across different learning problems and various divergence families

face recognition using scale adaptive directional and textural features a novel approach to face recognition problem using directional and texture information from face images is proposed in this paper in order to capture the directionality specially designed using local polynomial approximation technique scale adaptive digital filters are used for texture features extraction a low dimensional and computationally effective local descriptor is utilized textural and directional features are captured at the holistic and part based levels resulting in a robust face descriptor the proposed method is tested on a number of standard test face datasets orl xm2vts extended yale cmu pie ar and feret for different scenarios and its performance is compared with several state of the art techniques

comments on globally maximizing locally minimizing unsupervised discriminant projection with application to face and palm biometrics this paper develops an unsupervised discriminant projection udp technique for dimensionality reduction of high dimensional data in small sample size cases udp can be seen as a linear approximation of a multimanifolds based learning framework which takes into account both the local and nonlocal quantities udp characterizes the local scatter as well as the nonlocal scatter seeking to find a projection that simultaneously maximizes the nonlocal scatter and minimizes the local scatter this characteristic makes udp more intuitive and more powerful than the most up to date method locality preserving projection lpp which considers only the local scatter for clustering or classification tasks the proposed method is applied to face and palm biometrics and is examined using the yale feret and ar face image databases and the polyu palmprint database the experimental results show that udp consistently outperforms lpp and pca and outperforms lda when the training sample size per class is small this demonstrates that udp is a good choice for real world biometrics applications

mblda a novel multiple between class linear discriminant analysis abstract linear discriminant analysis lda with its extensions is a group of classical methods in dimensionality reduction for supervised learning however when some classes are far away from the others it may be difficult to find the optimal direction by lda because of the average between class scatter moreover ldas are always time consuming for high dimensional problem since the involved generalized eigenvalue problem is needed to be solved in this paper a multiple between class linear discriminant analysis mblda is proposed for dimensionality reduction mblda finds the transformation directions by approximating the solution to a min max programming problem leading to well separability in the reduced space with a fast learning speed on the high dimensional problem it is proved theoretically that the proposed method can deal with the special generalized eigenvalue problem by solving a underdetermined homogeneous system of linear equations experimental results on the artificial and benchmark datasets show that mblda can not only reduce the dimension to a powerful linear discriminant level but also have a fast learning speed

unified development of multiplicative algorithms for linear and quadratic nonnegative matrix factorization multiplicative updates have been widely used in approximative nonnegative matrix factorization nmf optimization because they are convenient to deploy their convergence proof is usually based on the minimization of an auxiliary upper bounding function the construction of which however remains specific and only available for limited types of dissimilarity measures here we make significant progress in developing convergent multiplicative algorithms for nmf first we propose a general approach to derive the auxiliary function for a wide variety of nmf problems as long as the approximation objective can be expressed as a finite sum of monomials with real exponents multiplicative algorithms with theoretical guarantee of monotonically decreasing objective function sequence can thus be obtained the solutions of nmf based on most commonly used dissimilarity measures such as and divergence as well as many other more comprehensive divergences can be derived by the new unified principle second our method is extended to a nonseparable case that includes e g divergence and renyi divergence third we develop multiplicative algorithms for nmf using second order approximative factorizations in which each factorizing matrix may appear twice preliminary numerical experiments demonstrate that the multiplicative algorithms developed using the proposed procedure can achieve satisfactory karush kuhn tucker optimality we also demonstrate nmf problems where algorithms by the conventional method fail to guarantee descent at each iteration but those by our principle are immune to such violation

a multi manifold discriminant analysis method for image feature extraction in this paper we propose a multi manifold discriminant analysis mmda method for an image feature extraction and pattern recognition based on graph embedded learning and under the fisher discriminant analysis framework in an mmda the within class graph and between class graph are respectively designed to characterize the within class compactness and the between class separability seeking for the discriminant matrix to simultaneously maximize the between class scatter and minimize the within class scatter in addition in an mmda the within class graph can represent the sub manifold information while the between class graph can represent the multi manifold information the proposed mmda is extensively examined by using the feret ar and orl face databases and the polyu finger knuckle print databases the experimental results demonstrate that an mmda is effective in feature extraction leading to promising image recognition performance

incremental linear discriminant analysis a fast algorithm and comparisons it has always been a challenging task to develop a fast and an efficient incremental linear discriminant analysis ilda algorithm for this purpose we conduct a new study for linear discriminant analysis lda in this paper and develop a new ilda algorithm we propose a new batch lda algorithm called lda qr lda qr is a simple and fast lda algorithm which is obtained by computing the economic qr factorization of the data matrix followed by solving a lower triangular linear system the relationship between lda qr and uncorrelated lda ulda is also revealed based on lda qr we develop a new incremental lda algorithm called ilda qr the main features of our ilda qr include that 1 it can easily handle the update from one new sample or a chunk of new samples 2 it has efficient computational complexity and space complexity and 3 it is very fast and always achieves competitive classification accuracy compared with ulda algorithm and existing ilda algorithms numerical experiments based on some real world data sets demonstrate that our ilda qr is very efficient and competitive with the state of the art ilda algorithms in terms of classification accuracy computational complexity and space complexity

linear regression for face recognition in this paper we present a novel approach of face identification by formulating the pattern recognition problem in terms of linear regression using a fundamental concept that patterns from a single object class lie on a linear subspace we develop a linear model representing a probe image as a linear combination of class specific galleries the inverse problem is solved using the least squares method and the decision is ruled in favor of the class with the minimum reconstruction error the proposed linear regression classification lrc algorithm falls in the category of nearest subspace classification the algorithm is extensively evaluated on several standard databases under a number of exemplary evaluation protocols reported in the face recognition literature a comparative study with state of the art algorithms clearly reflects the efficacy of the proposed approach for the problem of contiguous occlusion we propose a modular lrc approach introducing a novel distance based evidence fusion def algorithm the proposed methodology achieves the best results ever reported for the challenging problem of scarf occlusion

graph embedded nonparametric mutual information for supervised dimensionality reduction in this paper we propose a novel algorithm for dimensionality reduction that uses as a criterion the mutual information mi between the transformed data and their corresponding class labels the mi is a powerful criterion that can be used as a proxy to the bayes error rate furthermore recent quadratic nonparametric implementations of mi are computationally efficient and do not require any prior assumptions about the class densities we show that the quadratic nonparametric mi can be formulated as a kernel objective in the graph embedding framework moreover we propose its linear equivalent as a novel linear dimensionality reduction algorithm the derived methods are compared against the state of the art dimensionality reduction algorithms with various classifiers and on various benchmark and real life datasets the experimental results show that nonparametric mi as an optimization objective for dimensionality reduction gives comparable and in most of the cases better results compared with other dimensionality reduction methods

subspace projection a unified framework for a class of partition based dimension reduction techniques similarity search in high dimensional space is a nontrivial problem due to the so called curse of dimensionality recent techniques such as piecewise aggregate approximation paa segmented means smean and mean standard deviation ms prove to be very effective in reducing data dimensionality by partitioning dimensions into subsets and extracting aggregate values from each dimension subset these partition based techniques have many advantages including very efficient multi phased approximation while being simple to implement they however are not adaptive to the different characteristics of data in diverse applications we propose subspace projection ssp as a unified framework for these partition based techniques ssp projects data onto subspaces and computes a fixed number of salient features with respect to a reference vector a study of the relationships between query selectivity and the corresponding space partitioning schemes uncovers indicators that can be used to predict the performance of the partitioning configuration accordingly we design a greedy algorithm to efficiently determine a good partitioning of the data dimensions the results of our extensive experiments indicate that the proposed method consistently outperforms state of the art techniques

perturbation lda learning the difference between the class empirical mean and its expectation fisher s linear discriminant analysis lda is popular for dimension reduction and extraction of discriminant features in many pattern recognition applications especially biometric learning in deriving the fisher s lda formulation there is an assumption that the class empirical mean is equal to its expectation however this assumption may not be valid in practice in this paper from the perturbation perspective we develop a new algorithm called perturbation lda p lda in which perturbation random vectors are introduced to learn the effect of the difference between the class empirical mean and its expectation in fisher criterion this perturbation learning in fisher criterion would yield new forms of within class and between class covariance matrices integrated with some perturbation factors moreover a method is proposed for estimation of the covariance matrices of perturbation random vectors for practical implementation the proposed p lda is evaluated on both synthetic data sets and real face image data sets experimental results show that p lda outperforms the popular fisher s lda based algorithms in the undersampled case

normalized cuts and image segmentation we propose a novel approach for solving the perceptual grouping problem in vision rather than focusing on local features and their consistencies in the image data our approach aims at extracting the global impression of an image we treat image segmentation as a graph partitioning problem and propose a novel global criterion the normalized cut for segmenting the graph the normalized cut criterion measures both the total dissimilarity between the different groups as well as the total similarity within the groups we show that an efficient computational technique based on a generalized eigenvalue problem can be used to optimize this criterion we applied this approach to segmenting static images as well as motion sequences and found the results to be very encouraging

why can lda be performed in pca transformed space abstract pca plus lda is a popular framework for linear discriminant analysis lda in high dimensional and singular case in this paper we focus on building a theoretical foundation for this framework moreover we point out the weakness of the previous lda based methods and suggest a complete pca plus lda algorithm experimental results on orl face image database indicate that the proposed method is more effective than the previous ones

convex and semi nonnegative matrix factorizations we present several new variations on the theme of nonnegative matrix factorization nmf considering factorizations of the form x fgt we focus on algorithms in which g is restricted to containing nonnegative entries but allowing the data matrix x to have mixed signs thus extending the applicable range of nmf methods we also consider algorithms in which the basis vectors of f are constrained to be convex combinations of the data points this is used for a kernel extension of nmf we provide algorithms for computing these new factorizations and we provide supporting theoretical analysis we also analyze the relationships between our algorithms and clustering algorithms and consider the implications for sparseness of solutions finally we present experimental results that explore the properties of these new methods

linear and nonlinear projective nonnegative matrix factorization a variant of nonnegative matrix factorization nmf which was proposed earlier is analyzed here it is called projective nonnegative matrix factorization pnmf the new method approximately factorizes a projection matrix minimizing the reconstruction error into a positive low rank matrix and its transpose the dissimilarity between the original data matrix and its approximation can be measured by the frobenius matrix norm or the modified kullback leibler divergence both measures are minimized by multiplicative update rules whose convergence is proven for the first time enforcing orthonormality to the basic objective is shown to lead to an even more efficient update rule which is also readily extended to nonlinear cases the formulation of the pnmf objective is shown to be connected to a variety of existing nmf methods and clustering approaches in addition the derivation using lagrangian multipliers reveals the relation between reconstruction and sparseness for kernel principal component analysis pca with the binary constraint useful in graph partitioning problems the nonlinear kernel pnmf provides a good approximation which outperforms an existing discretization approach empirical study on three real world databases shows that pnmf can achieve the best or close to the best in clustering the proposed algorithm runs more efficiently than the compared nmf methods especially for high dimensional data moreover contrary to the basic nmf the trained projection matrix can be readily used for newly coming samples and demonstrates good generalization

multiclass classifiers based on dimension reduction with generalized lda linear discriminant analysis lda has been widely used for dimension reduction of data sets with multiple classes the lda has been recently extended to various generalized lda methods that are applicable regardless of the relative sizes between the data dimension and the number of data items in this paper we propose several multiclass classifiers based on generalized lda glda algorithms taking advantage of the dimension reducing transformation matrix without requiring additional training or parameter optimization a marginal linear discriminant classifier mldc a bayesian linear discriminant classifier bldc and a one dimensional bldc are introduced for multiclass classification our experimental results illustrate that these classifiers produce higher ten fold cross validation accuracy than knn and centroid based classifiers in the reduced dimensional space obtained from glda

multi granularity distance metric learning via neighborhood granule margin maximization learning a distance metric from training samples is often a crucial step in machine learning and pattern recognition locality compactness and consistency are considered as the key principles in distance metric learning however the existing metric learning methods just consider one or two of them in this paper we develop a multi granularity distance learning technique first a new index neighborhood granule margin which simultaneously considers locality compactness and consistency of neighborhood is introduced to evaluate a distance metric by maximizing neighborhood granule margin we formulate the distance metric learning problem as a sample pair classification problem which can be solved by standard support vector machine solvers then a set of distance metrics are learned in different granular spaces the weights of the granular spaces are learned through optimizing the margin distribution finally the decisions from different granular spaces are combined with weighted voting experiments on uci datasets gender classification and object categorization tasks show that the proposed method is superior to the state of the art distance metric learning algorithms

3d discrete rotations using hinge angles in this paper we study 3d rotations on grid points computed by using only integers for that purpose we investigate the intersection between the 3d half grid and the rotation plane from this intersection we define 3d hinge angles which determine a transit of a grid point from a voxel to its adjacent voxel during the rotation then we give a method to sort all 3d hinge angles with integer computations the study of 3d hinge angles allows us to design a 3d discrete rotation and to estimate the rotation between a pair of digital images in correspondence

computing upper and lower bounds of rotation angles from digital images rotations in the discrete plane are important for many applications such as image matching or construction of mosaic images we suppose that a digital image a is transformed to another digital image b by a rotation in the discrete plane there are many angles giving the rotation from a to b which we call admissible rotation angles from a to b for such a set of admissible rotation angles there exist two angles that achieve the lower and the upper bounds to find those lower and upper bounds we use hinge angles as used in nouvel and remila incremental and transitive discrete rotations in r reulke u eckardt b flash u knauer k polthier eds combinatorial image analysis lecture notes in computer science vol 4040 springer berlin 2006 pp 199 213 a sequence of hinge angles is a set of particular angles determined by a digital image in the sense that any angle between two consecutive hinge angles gives the identical rotation of the digital image we propose a method for obtaining the lower and the upper bounds of admissible rotation angles using hinge angles from a given euclidean angle or from a pair of corresponding digital images

median null sw based method for face feature recognition with the progress of science and technology artificial intelligence is being paid more and more attention people want to use computers to deal with complex practical problems so linear discriminant analysis lda is widely used as a dimensionality reduction technique in image and text recognition classification tasks however a weakness of lda model is that the class average vector in the formula completely depends on class sample average under special circumstances such as noise bright light some outliers will appear in the practical input databases therefore by employing several given practical samples the class sample average is not enough to estimate the class average accurately so the recognition performance of lda model will decline compared to human intelligence computers are far short of necessary fundamental knowledge of judgment which people normally acquire during the formative years of their lives in order to solve the problem and also to render lda model more robust we propose a within class scatter matrix null space median method m n s w which first transforms the original space by employing a basis of within class scatter matrix null space and then in the transformed space the maximum of between class scatter matrix is pursued in the second stage within class median vector is used in the traditional lda model experiments on orl feret and yale face data sets are performed to test and evaluate the effectiveness of the proposed method

linear dimensionality reduction by maximizing the chernoff distance in the transformed space linear dimensionality reduction ldr techniques are quite important in pattern recognition due to their linear time complexity and simplicity in this paper we present a novel ldr technique which though linear aims to maximize the chernoff distance in the transformed space thus augmenting the class separability in such a space we present the corresponding criterion which is maximized via a gradient based algorithm and provide convergence and initialization proofs we have performed a comprehensive performance analysis of our method combined with two well known classifiers linear and quadratic on synthetic and real life data and compared it with other ldr techniques the results on synthetic and standard real life data sets show that the proposed criterion outperforms the latter when combined with both linear and quadratic classifiers

random walks for image segmentation a novel method is proposed for performing multilabel interactive image segmentation given a small number of pixels with user defined or predefined labels one can analytically and quickly determine the probability that a random walker starting at each unlabeled pixel will first reach one of the prelabeled pixels by assigning each pixel to the label for which the greatest probability is calculated a high quality image segmentation may be obtained theoretical properties of this algorithm are developed along with the corresponding connections to discrete potential theory and electrical circuits this algorithm is formulated in discrete space i e on a graph using combinatorial analogues of standard operators and principles from continuous potential theory allowing it to be applied in arbitrary dimension on arbitrary graphs

1d correlation filter based class dependence feature analysis for face recognition in this paper a novel one dimensional correlation filter based class dependence feature analysis 1d cfa method is presented for robust face recognition compared with original cfa that works in the two dimensional 2d image space 1d cfa encodes the image data as vectors in 1d cfa a new correlation filter called optimal extra class origin output tradeoff filter oeotf which is designed in the low dimensional principal component analysis pca subspace is proposed for effective feature extraction experimental results on benchmark face databases such as feret ar and frgc show that oeotf based 1d cfa consistently outperforms other state of the art face recognition methods this demonstrates the effectiveness and robustness of the novel method

uncorrelated linear discriminant analysis based on weighted pairwise fisher criterion in this paper we propose a novel uncorrelated weighted linear discriminant analysis uwlda method for feature extraction and recognition the uwlda first introduces a weighting function to restrain the dominant role of the classes with larger distance and then searches the optimal discriminant vectors under the conjugative orthogonal constrains in the null space of the within class scatter matrix and its conjugative orthogonal complement space respectively as a result the proposed technique not only derive the optimal and lossless discriminative information but also guarantee that all extracted features are statistically uncorrelated experiments on feret face database and ar face database are performed to test and evaluate the proposed algorithm the results demonstrate the effectiveness of uwlda

a generalized divergence measure for nonnegative matrix factorization this letter presents a general parametric divergence measure the metric includes as special cases quadratic error and kullback leibler divergence a parametric generalization of the two different multiplicative update rules for nonnegative matrix factorization by lee and seung 2001 is shown to lead to locally optimal solutions of the nonnegative matrix factorization problem with this new cost function numeric simulations demonstrate that the new update rule may improve the quadratic distance convergence speed a proof of convergence is given that as in lee and seung uses an auxiliary function known from the expectation maximization theoretical framework

scatter balance an angle based supervised dimensionality reduction subspace selection is widely applied in data classification clustering and visualization the samples projected into subspace can be processed efficiently in this paper we research the linear discriminant analysis lda and maximum margin criterion mmc algorithms intensively and analyze the effects of scatters to subspace selection meanwhile we point out the boundaries of scatters in lda and mmc algorithms to illustrate the differences and similarities of subspace selection in different circumstances besides the effects of outlier classes on subspace selection are also analyzed according to the above analysis we propose a new subspace selection method called angle linear discriminant embedding alde on the basis of angle measurement alde utilizes the cosine of the angle to get new within class and between class scatter matrices and avoids the small sample size problem simultaneously to deal with high dimensional data we extend alde to a two stage alde ts alde the synthetic data experiments indicate that alde can balance the within class and between class scatters and be robust to outlier classes the experimental results based on uci machine learning repository and image databases show that ts alde has a lower time complexity than alde while processing high dimensional data

consistency analysis on orientation features for fast and accurate palmprint identification a typical palmprint identification system identifies a query palmprint by matching it with the templates in the database and comparing the similarity score with a pre defined threshold however continual database expansion would make conventional brute force search method inefficient for large scale palmprint identification in this paper we propose a consistent orientation pattern cop hashing method to enforce fast search using the orientation and response features extracted by steerable filter we first give an analysis on the consistency of orientation features and then introduce a method to construct cop using the consistent features because the cop is very stable across the samples of the same subject the cop hashing method can find the target template quickly and thus can lead to early termination of the searching process experiments have been carried out on several real and synthetic databases and the results show that the cop hashing method can accelerate the identification process by more than an order of magnitude and can achieve comparable identification accuracy while compared with the state of the art approaches

an eigendecomposition approach to weighted graph matching problems an approximate solution to the weighted graph matching problem is discussed for both undirected and directed graphs the weighted graph matching problem is that of finding the optimum matching between two weighted graphs which are graphs with weights at each arc the proposed method uses an analytic instead of a combinatorial or iterative approach to the optimum matching problem using the eigendecompositions of the adjacency matrices in the case of the undirected graph matching problem or hermitian matrices derived from the adjacency matrices in the case of the directed graph matching problem a matching close to the optimum can be found efficiently when the graphs are sufficiently close to each other simulation results are given to evaluate the performance of the proposed method

from classifiers to discriminators a nearest neighbor rule induced discriminant analysis the current discriminant analysis method design is generally independent of classifiers thus the connection between discriminant analysis methods and classifiers is loose this paper provides a way to design discriminant analysis methods that are bound with classifiers we begin with a local mean based nearest neighbor lm nn classifier and use its decision rule to supervise the design of a discriminator therefore the derived discriminator called local mean based nearest neighbor discriminant analysis lm nnda matches the lm nn classifier optimally in theory in contrast to that lm nnda is a nn classifier induced discriminant analysis method we further show that the classical fisher linear discriminant analysis flda is a minimum distance classifier i e nearest class mean classifier induced discriminant analysis method the proposed lm nnda method is evaluated using the cenparmi handwritten numeral database the nust603 handwritten chinese character database the eth80 object category database and the feret face image database the experimental results demonstrate the performance advantage of lm nnda over other feature extraction methods with respect to the lm nn or nn classifier

jordan lie super algebra and jordan lie triple system abstract we introduce notions of jordan lie super algebras and jordan lie triple systems as well as doubly graded lie super algebras they are intimately related to both lie and jordan super algebras as well as antiassociative algebra

image clustering using local discriminant models and global integration in this paper we propose a new image clustering algorithm referred to as clustering using local discriminant models and global integration ldmgi to deal with the data points sampled from a nonlinear manifold for each data point we construct a local clique comprising this data point and its neighboring data points inspired by the fisher criterion we use a local discriminant model for each local clique to evaluate the clustering performance of samples within the local clique to obtain the clustering result we further propose a unified objective function to globally integrate the local models of all the local cliques with the unified objective function spectral relaxation and spectral rotation are used to obtain the binary cluster indicator matrix for all the samples we show that ldmgi shares a similar objective function with the spectral clustering sc algorithms e g normalized cut ncut in contrast to ncut in which the laplacian matrix is directly calculated based upon a gaussian function a new laplacian matrix is learnt in ldmgi by exploiting both manifold structure and local discriminant information we also prove that k means and discriminative k means diskmeans are both special cases of ldmgi extensive experiments on several benchmark image datasets demonstrate the effectiveness of ldmgi we observe in the experiments that ldmgi is more robust to algorithmic parameter when compared with ncut thus ldmgi is more appealing for the real image clustering applications in which the ground truth is generally not available for tuning algorithmic parameters

quadratic nonnegative matrix factorization in nonnegative matrix factorization nmf a nonnegative matrix is approximated by a product of lower rank factorizing matrices most nmf methods assume that each factorizing matrix appears only once in the approximation thus the approximation is linear in the factorizing matrices we present a new class of approximative nmf methods called quadratic nonnegative matrix factorization qnmf where some factorizing matrices occur twice in the approximation we demonstrate qnmf solutions to four potential pattern recognition problems in graph partitioning two way clustering estimating hidden markov chains and graph matching we derive multiplicative algorithms that monotonically decrease the approximation error under a variety of measures we also present extensions in which one of the factorizing matrices is constrained to be orthogonal or stochastic empirical studies show that for certain application scenarios qnmf is more advantageous than other existing nonnegative matrix factorization methods

non negative matrix factorization with sparseness constraints non negative matrix factorization nmf is a recently developed technique for finding parts based linear representations of non negative data although it has successfully been applied in several applications it does not always result in parts based representations in this paper we show how explicitly incorporating the notion of sparseness improves the found decompositions additionally we provide complete matlab code both for standard nmf and for our extension our hope is that this will further the application of these methods to solving novel data analysis problems

recursive concave convex fisher linear discriminant with applications to face handwritten digit and terrain recognition in classification previous studies have shown that an eigenvalue based technique can be cast as an related svm type problem and that by solving this svm type problem the performance can be improved significantly in this paper we develop a recursive concave convex fisher linear discriminant dr rpfld for dimension reduction technique of high dimensional data to extract as many meaningful features as possible which incorporates the fundamental idea behind fisher linear discriminant and casts the fisher linear discriminant as a concave convex programming problem based on the hinge loss the solution of our method follows from solving the related svm type optimization problems iteratively which means the proposed method can be viewed as the combination of multiple related svm type problems the special formulation of our method provides convenience for constructing sparse multi class fisher linear discriminant directly due to use of a recursive procedure the number of features available from rpfld is independent of the number of classes meaning that in contrast to the original fisher linear discriminant the number of features available from our method has no upper bound we evaluate our algorithm on the yale and orl face image databases handwritten digit database and terrain image dataset experimental results show that rpfld outperforms other fisher linear discriminant algorithms

an efficient approach for face recognition based on common eigenvalues in this paper a simple technique is proposed for face recognition among many human faces it is based on the polynomial coefficients covariance matrix and algorithm on common eigenvalues the main advantage of the proposed approach is that the identification of similarity between human faces is carried out without computing actual eigenvalues and eigenvectors a symmetric matrix is calculated using the polynomial coefficients based companion matrices of two compared images the nullity of a calculated symmetric matrix is used as similarity measure for face recognition the value of nullity is very small for dissimilar images and distinctly large for similar face images the feasibility of the propose approach is demonstrated on three face databases i e the orl database the yale database b and the feret database experimental results have shown the effectiveness of the proposed approach for feature extraction and classification of the face images having large variation in pose and illumination

1d lda vs 2d lda when is vector based linear discriminant analysis better than matrix based recent advances have shown that algorithms with 2d matrix based representation perform better than the traditional 1d vector based ones in particular 2d lda has been widely reported to outperform 1d lda however would the matrix based linear discriminant analysis be always superior and when would 1d lda be better in this paper we investigate into these questions and have a comprehensive comparison between 1d lda and 2d lda in theory and in experiments we analyze the heteroscedastic problem in 2d lda and formulate mathematical equalities to explore the relationship between 1d lda and 2d lda then we point out potential problems in 2d lda it is shown that 2d lda has eliminated the information contained in the covariance information between different local geometric structures such as the rows or the columns which is useful for discriminant feature extraction whereas 1d lda could preserve such information interestingly this new finding indicates that 1d lda is able to gain higher fisher score than 2d lda in some extreme case furthermore sufficient conditions on which 2d lda would be bayes optimal for two class classification problem are derived and comparison with 1d lda in this aspect is also analyzed this could help understand how 2d lda is expected to achieve at its best further discover its relationship with 1d lda and well support other findings after the theoretical analysis comprehensive experimental results are reported by fairly and extensively comparing 1d lda with 2d lda in contrast to the existing view that some 2d lda based algorithms would perform better than 1d lda when the number of training samples for each class is small or when the number of discriminant features used is small we show that it is not always true and show that some standard 1d lda based algorithms could perform better in those cases on some challenging data sets

feature extraction using two dimensional maximum embedding difference in this paper we propose a novel method combining graph embedding and difference criterion techniques for image feature extraction namely two dimensional maximum embedding difference 2dmed this method directly extracts the optimal projective vectors from 2d image matrices by simultaneously considering characteristic that is the intraclass compactness graph the margin graph and inter class separability graph respectively in this method it is not necessary to convert the image matrix into high dimensional image vector so that much computational time would be saved in addition the proposed method preserves the manifold reconstruction relationships in the low dimensional subspace experimental results on the orl yale face and usps database show the effectiveness of the proposed method

feature extraction using a fast null space based linear discriminant analysis algorithm the small sample size problem is often encountered in pattern recognition several algorithms for null space based linear discriminant analysis nlda have been developed to solve the problem however these algorithms for nlda have high computational cost in this paper we simplify the recently proposed algorithm for nlda in chu and thye 2010 5 with the assumption that all the training data vectors are linearly independent and propose a new and fast algorithm for nlda our main observation is that two steps of economic qr decomposition with column pivoting can be replaced by one step of economic qr decomposition without column pivoting if the related matrix is of full column rank the main features of our algorithm for nlda include i our nlda algorithm is carried out by only one step of economic qr decomposition and does not compute any singular value decomposition svd when all the training data vectors are linearly independent ii the main cost of our method is the cost of an economic qr decomposition of an mx n 1 matrix here m is the dimension of the training data and n is the number of samples our method is a fast one experimental studies on orl feret and pie face databases demonstrate the effectiveness of our new algorithm for nlda

kpca plus lda a complete kernel fisher discriminant framework for feature extraction and recognition this paper examines the theory of kernel fisher discriminant analysis kfd in a hilbert space and develops a two phase kfd framework i e kernel principal component analysis kpca plus fisher linear discriminant analysis lda this framework provides novel insights into the nature of kfd based on this framework the authors propose a complete kernel fisher discriminant analysis ckfd algorithm ckfd can be used to carry out discriminant analysis in double discriminant subspaces the fact that it can make full use of two kinds of discriminant information regular and irregular makes ckfd a more powerful discriminator the proposed algorithm was tested and evaluated using the feret face database and the cenparmi handwritten numeral database the experimental results show that ckfd outperforms other kfd algorithms

hyperplane navigation a method to set individual scores in fmri group datasets abstract recent studies have demonstrated that spatial patterns of fmri bold activity distribution over the brain may be used to classify different groups or mental states these studies are based on the application of advanced pattern recognition approaches and multivariate statistical classifiers most published articles in this field are focused on improving the accuracy rates and many approaches have been proposed to accomplish this task nevertheless a point inherent to most machine learning methods and still relatively unexplored in neuroimaging is how the discriminative information can be used to characterize groups and their differences in this work we introduce the maximum uncertainty linear discrimination analysis mlda and show how it can be applied to infer groups patterns by discriminant hyperplane navigation in addition we show that it naturally defines a behavioral score i e an index quantifying the distance between the states of a subject from predefined groups we validate and illustrate this approach using a motor block design fmri experiment data with 35 subjects

a collaborative representation based projections method for feature extraction in graph embedding based methods we usually need to manually choose the nearest neighbors and then compute the edge weights using the nearest neighbors via l2 norm e g lle it is difficult and unstable to manually choose the nearest neighbors in high dimensional space so how to automatically construct a graph is very important in this paper first we give a l2 graph like l1 graph l2 graph calculates the edge weights using the total samples avoiding manually choosing the nearest neighbors second a l2 graph based feature extraction method is presented called collaborative representation based projections crp like spp crp aims to preserve the collaborative representation based reconstruction relationship of data crp utilizes a l2 norm graph to characterize the local compactness information crp maximizes the ratio between the total separability information and the local compactness information to seek the optimal projection matrix crp is much faster than spp since crp calculates the objective function with l2 norm while spp calculate the objective function with l1 norm experimental results on feret ar yale face databases and the polyu finger knuckle print database demonstrate that crp works well in feature extraction and leads to a good recognition performance we give a l2 norm graph based on collaborative representation we propose a collaborative representation based projections crp for feature extraction crp is a rayleigh quotient form and can be calculated via generalized eigenvalue decomposition

graph regularized nonnegative matrix factorization for data representation matrix factorization techniques have been frequently applied in information retrieval computer vision and pattern recognition among them nonnegative matrix factorization nmf has received considerable attention due to its psychological and physiological interpretation of naturally occurring data whose representation may be parts based in the human brain on the other hand from the geometric perspective the data is usually sampled from a low dimensional manifold embedded in a high dimensional ambient space one then hopes to find a compact representation which uncovers the hidden semantics and simultaneously respects the intrinsic geometric structure in this paper we propose a novel algorithm called graph regularized nonnegative matrix factorization gnmf for this purpose in gnmf an affinity graph is constructed to encode the geometrical information and we seek a matrix factorization which respects the graph structure our empirical study shows encouraging results of the proposed algorithm in comparison to the state of the art algorithms on real world problems

nonnegative matrix factorization with the itakura saito divergence with application to music analysis this letter presents theoretical algorithmic and experimental results about nonnegative matrix factorization nmf with the itakura saito is divergence we describe how is nmf is underlaid by a well defined statistical model of superimposed gaussian components and is equivalent to maximum likelihood estimation of variance parameters this setting can accommodate regularization constraints on the factors through bayesian priors in particular inverse gamma and gamma markov chain priors are considered in this work estimation can be carried out using a space alternating generalized expectation maximization sage algorithm this leads to a novel type of nmf algorithm whose convergence to a stationary point of the is cost function is guaranteed we also discuss the links between the is divergence and other cost functions used in nmf in particular the euclidean distance and the generalized kullback leibler kl divergence as such we describe how is nmf can also be performed using a gradient multiplicative algorithm a standard algorithm structure in nmf whose convergence is observed in practice though not proven finally we report a furnished experimental comparative study of euclidean nmf kl nmf and is nmf algorithms applied to the power spectrogram of a short piano sequence recorded in real conditions with various initializations and model orders then we show how is nmf can successfully be employed for denoising and upmix mono to stereo conversion of an original piece of early jazz music these experiments indicate that is nmf correctly captures the semantics of audio and is better suited to the representation of music signals than nmf with the usual euclidean and kl costs

face recognition using a kernel fractional step discriminant analysis algorithm feature extraction is among the most important problems in face recognition systems in this paper we propose an enhanced kernel discriminant analysis kda algorithm called kernel fractional step discriminant analysis kfda for nonlinear feature extraction and dimensionality reduction not only can this new algorithm like other kernel methods deal with nonlinearity required for many face recognition tasks it can also outperform traditional kda algorithms in resisting the adverse effects due to outlier classes moreover to further strengthen the overall performance of kda algorithms for face recognition we propose two new kernel functions cosine fractional power polynomial kernel and non normal gaussian rbf kernel we perform extensive comparative studies based on the yaleb and feret face databases experimental results show that our kfda algorithm outperforms traditional kernel principal component analysis kpca and kda algorithms moreover further improvement can be obtained when the two new kernel functions are used

bagging null space locality preserving discriminant classifiers for face recognition in this paper we propose a novel bagging null space locality preserving discriminant analysis bagnlpda method for facial feature extraction and recognition the bagnlpda method first projects all the training samples into the range space of a so called locality preserving total scatter matrix without losing any discriminative information the projected training samples are then randomly sampled using bagging to generate a set of bootstrap replicates null space discriminant analysis is performed in each replicate and the results of them are combined using majority voting as a result the proposed method aggregates a set of complementary null space locality preserving discriminant classifiers experiments on feret and pie subsets demonstrate the effectiveness of bagnlpda

feature extraction based on laplacian bidirectional maximum margin criterion maximum margin criterion mmc based feature extraction is more efficient than linear discriminant analysis lda for calculating the discriminant vectors since it does not need to calculate the inverse within class scatter matrix however mmc ignores the discriminative information within the local structures of samples and the structural information embedding in the images in this paper we develop a novel criterion namely laplacian bidirectional maximum margin criterion lbmmc to address the issue we formulate the image total laplacian matrix image within class laplacian matrix and image between class laplacian matrix using the sample similar weight that is widely used in machine learning the proposed lbmmc based feature extraction computes the discriminant vectors by maximizing the difference between image between class laplacian matrix and image within class laplacian matrix in both row and column directions experiments on the feret and yale face databases show the effectiveness of the proposed lbmmc based feature extraction method

complete large margin linear discriminant analysis using mathematical programming approach in this paper we develop a novel dimensionality reduction dr framework coined complete large margin linear discriminant analysis clmlda inspired by several recently proposed dr methods clmlda constructs two mathematical programming models by maximizing the minimum distance between each class center and the total class center respectively in the null space of within class scatter matrix and its orthogonal complementary space in this way clmlda not only makes full use of the discriminative information contained in the whole feature space but also overcome the weakness of linear discriminant analysis lda in dealing with the class separation problem the solutions of clmlda follow from solving two nonconvex optimization problems each of which is transformed to a series of convex quadratic programming problems by using the constrained concave convex procedure first and then solved by off the shelf optimization toolbox experiments on both toy and several publicly available databases demonstrate its feasibility and effectiveness

a new lda based face recognition system which can solve the small sample size problem a new lda based face recognition system is presented in this paper linear discriminant analysis lda is one of the most popular linear projection techniques for feature extraction the major drawback of applying lda is that it may encounter the small sample size problem in this paper we propose a new lda based technique which can solve the small sample size problem we also prove that the most expressive vectors derived in the null space of the within class scatter matrix using principal component analysis pca are equal to the optimal discriminant vectors derived in the original space using lda the experimental results show that the new lda process improves the performance of a face recognition system signi cantly 2000 pattern recognition society published by elsevier science ltd all rights reserved

k local hyperplane distance nearest neighbor classifier oriented local discriminant analysis k local hyperplane distance nearest neighbor hknn classifier is an improved k nearest neighbor knn algorithm that has been successfully applied to pattern classification this paper embeds the decision rule of hknn classifier into the discriminant analysis model to develop a new feature extractor the obtained feature extractor is called k local hyperplane distance nearest neighbor classifier oriented local discriminant analysis holda in which a regularization item is imposed on the original hknn algorithm to obtain a more reliable distance metric based on this distance metric the homo class and hetero class local scatters are characterized in holda by maximizing the ratio of the hetero class local scatter to the homo class local scatter we obtain a subspace which is suitable for feature extraction and classification in general this paper provides a framework for building a feature extractor from the decision rule of a classifier by this means the feature extractor and classifier can be seamlessly integrated experimental results on four databases demonstrate that the integrated pattern recognition system is effective

multi class pairwise linear dimensionality reduction using heteroscedastic schemes linear dimensionality reduction ldr techniques have been increasingly important in pattern recognition pr due to the fact that they permit a relatively simple mapping of the problem onto a lower dimensional subspace leading to simple and computationally efficient classification strategies although the field has been well developed for the two class problem the corresponding issues encountered when dealing with multiple classes are far from trivial in this paper we argue that as opposed to the traditional ldr multi class schemes if we are dealing with multiple classes it is not expedient to treat it as a multi class problem per se rather we shall show that it is better to treat it as an ensemble of chernoff based two class reductions onto different subspaces whence the overall solution is achieved by resorting to either voting weighting or to a decision tree strategy the experimental results obtained on benchmark datasets demonstrate that the proposed methods are not only efficient but that they also yield accuracies comparable to that obtained by the optimal bayes classifier

on the relation between discriminant analysis and mutual information for supervised linear feature extraction this paper provides a unifying view of three discriminant linear feature extraction methods linear discriminant analysis heteroscedastic discriminant analysis and maximization of mutual information we propose a model independent reformulation of the criteria related to these three methods that stresses their similarities and elucidates their differences based on assumptions for the probability distribution of the classification data we obtain sufficient conditions under which two or more of the above criteria coincide it is shown that these conditions also suffice for bayes optimality of the criteria our approach results in an information theoretic derivation of linear discriminant analysis and heteroscedastic discriminant analysis finally regarding linear discriminant analysis we discuss its relation to multidimensional independent component analysis and derive suboptimality bounds based on information theory

a collective neurodynamic optimization approach to nonnegative matrix factorization nonnegative matrix factorization nmf is an advanced method for nonnegative feature extraction with widespread applications however the nmf solution often entails to solve a global optimization problem with a nonconvex objective function and nonnegativity constraints this paper presents a collective neurodynamic optimization cno approach to this challenging problem the proposed collective neurodynamic system consists of a population of recurrent neural networks rnns at the lower level and a particle swarm optimization pso algorithm with wavelet mutation at the upper level the rnns act as search agents carrying out precise local searches according to their neurodynamics and initial conditions the pso algorithm coordinates and guides the rnns with updated initial states toward global optimal solution s a wavelet mutation operator is added to enhance pso exploration diversity through iterative interaction and improvement of the locally best solutions of rnns and global best positions of the whole population the population based neurodynamic systems are almost sure able to achieve the global optimality for the nmf problem it is proved that the convergence of the group best state to the global optimal solution with probability one the experimental results substantiate the efficacy and superiority of the cno approach to bound constrained global optimization with several benchmark nonconvex functions and nmf based clustering with benchmark data sets in comparison with the state of the art algorithms

efficient and robust feature extraction by maximum margin criterion in pattern recognition feature extraction techniques are widely employed to reduce the dimensionality of data and to enhance the discriminatory information principal component analysis pca and linear discriminant analysis lda are the two most popular linear dimensionality reduction methods however pca is not very effective for the extraction of the most discriminant features and lda is not stable due to the small sample size problem in this paper we propose some new linear and nonlinear feature extractors based on maximum margin criterion mmc geometrically feature extractors based on mmc maximize the average margin between classes after dimensionality reduction it is shown that mmc can represent class separability better than pca as a connection to lda we may also derive lda from mmc by incorporating some constraints by using some other constraints we establish a new linear feature extractor that does not suffer from the small sample size problem which is known to cause serious stability problems for lda the kernelized nonlinear counterpart of this linear feature extractor is also established in the paper our extensive experiments demonstrate that the new feature extractors are effective stable and efficient

a comparison of generalized linear discriminant analysis algorithms linear discriminant analysis lda is a dimension reduction method which finds an optimal linear transformation that maximizes the class separability however in undersampled problems where the number of data samples is smaller than the dimension of data space it is difficult to apply lda due to the singularity of scatter matrices caused by high dimensionality in order to make lda applicable several generalizations of lda have been proposed recently in this paper we present theoretical and algorithmic relationships among several generalized lda algorithms and compare their computational complexities and performances in text classification and face recognition towards a practical dimension reduction method for high dimensional data an efficient algorithm is proposed which reduces the computational complexity greatly while achieving competitive prediction accuracies we also present nonlinear extensions of these lda algorithms based on kernel methods it is shown that a generalized eigenvalue problem can be formulated in the kernel based feature space and generalized lda algorithms are applied to solve the generalized eigenvalue problem resulting in nonlinear discriminant analysis performances of these linear and nonlinear discriminant analysis algorithms are compared extensively

trace ratio criterion based generalized discriminative learning for semi supervised dimensionality reduction dealing with high dimensional data has always been a major problem in many pattern recognition and machine learning applications trace ratio criterion is a criterion that can be applicable to many dimensionality reduction methods as it directly reflects euclidean distance between data points of within or between classes in this paper we analyze the trace ratio problem and propose a new efficient algorithm to find the optimal solution based on the proposed algorithm we are able to derive an orthogonal constrained semi supervised learning framework the new algorithm incorporates unlabeled data into training procedure so that it is able to preserve the discriminative structure as well as geometrical structure embedded in the original dataset under such a framework many existing semi supervised dimensionality reduction methods such as sda lap lda ssdr ssmmc can be improved using our proposed framework which can also be used to formulate a corresponding kernel framework for handling nonlinear problems theoretical analysis indicates that there are certain relationships between linear and nonlinear methods finally extensive simulations on synthetic dataset and real world dataset are presented to show the effectiveness of our algorithms the results demonstrate that our proposed algorithm can achieve great superiority to other state of art algorithms

nuclear norm based 2 dpca for extracting features from images the 2 d principal component analysis 2 dpca is a widely used method for image feature extraction however it can be equivalently implemented via image row based principal component analysis this paper presents a structured 2 d method called nuclear norm based 2 dpca n 2 dpca which uses a nuclear norm based reconstruction error criterion the nuclear norm is a matrix norm which can provide a structured 2 d characterization for the reconstruction error image the reconstruction error criterion is minimized by converting the nuclear norm based optimization problem into a series of f norm based optimization problems in addition n 2 dpca is extended to a bilateral projection based n 2 dpca n b2 dpca the virtue of n b2 dpca over n 2 dpca is that an image can be represented with fewer coefficients n 2 dpca and n b2 dpca are applied to face recognition and reconstruction and evaluated using the extended yale b cmu pie frgc and ar databases experimental results demonstrate the effectiveness of the proposed methods

discriminant waveletfaces and nearest feature classifiers for face recognition feature extraction discriminant analysis and classification rules are three crucial issues for face recognition we present hybrid approaches to handle three issues together for feature extraction we apply the multiresolution wavelet transform to extract the waveletface we also perform the linear discriminant analysis on waveletfaces to reinforce discriminant power during classification the nearest feature plane nfp and nearest feature space nfs classifiers are explored for robust decisions in presence of wide facial variations their relationships to conventional nearest neighbor and nearest feature line classifiers are demonstrated in the experiments the discriminant waveletface incorporated with the nfs classifier achieves the best face recognition performance

non negativity constraints on the pre image for pattern recognition with kernel machines rules of physics in many real life problems force some constraints to be satisfied this paper deals with nonlinear pattern recognition under non negativity constraints while kernel principal component analysis can be applied for feature extraction or data denoising in a feature space associated to the considered kernel function a pre image technique is required to go back to the input space e g representing a feature in the space of input signals the main purpose of this paper is to study a constrained pre image problem with non negativity constraints we provide new theoretical results on the pre image problem including the weighted combination form of the pre image and demonstrate sufficient conditions for the convexity of the problem the constrained problem is considered with the non negativity either on the pre image itself or on the weights we propose a simple iterative scheme to incorporate both constraints a fortuitous side effect of our method is the sparsity in the representation a property investigated in this paper experimental results are conducted on artificial and real datasets where many properties are investigated including the sparsity property and compared to other methods from the literature the relevance of the proposed method is demonstrated with experimentations on artificial data and on two types of real datasets in signal and image processing

a nonconvex splitting method for symmetric nonnegative matrix factorization convergence analysis and optimality symmetric nonnegative matrix factorization symnmf has important applications in data analytics problems such as document clustering community detection and image segmentation in this paper we propose a novel nonconvex variable splitting method for solving symnmf the proposed algorithm is guaranteed to converge to the set of karush kuhn tucker kkt points of the nonconvex symnmf problem furthermore it achieves a global sublinear convergence rate we also show that the algorithm can be efficiently implemented in parallel further sufficient conditions are provided that guarantee the global and local optimality of the obtained solutions extensive numerical results performed on both synthetic and real datasets suggest that the proposed algorithm converges quickly to a local minimum solution

rapid and brief communication generalizing relevance weighted lda in this paper a new variant on linear discriminant analysis lda that we refer to as generalizing relevance weighted lda or grw lda is proposed grw lda extends the applicability to cases that lda cannot handle by combining the advantages of two recent lda enhancements namely generalized singular value decomposition based lda and relevance weighted lda experimental results on feret face database demonstrate the effectiveness of the proposed method

the number of n point digital discs a digital disc is the set of all integer points inside some given disc let dn be the number of different digital discs consisting of n points different up to translation the upper bound d n o n2 was shown recently no corresponding lower bound is known in this paper we refine the upper bound to dn o n which seems to be the true order of magnitude and we show that the average dn d1 d2 dn n has upper and lower bounds which are of polynomial growth in n

minimum class variance support vector machines in this paper a modified class of support vector machines svms inspired from the optimization of fisher s discriminant ratio is presented the so called minimum class variance svms mcvsvms the mcvsvms optimization problem is solved in cases in which the training set contains less samples that the dimensionality of the training vectors using dimensionality reduction through principal component analysis pca afterward the mcvsvms are extended in order to find nonlinear decision surfaces by solving the optimization problem in arbitrary hilbert spaces defined by mercer s kernels in that case it is shown that under kernel pca the nonlinear optimization problem is transformed into an equivalent linear mcvsvms problem the effectiveness of the proposed approach is demonstrated by comparing it with the standard svms and other classifiers like kernel fisher discriminant analysis in facial image characterization problems like gender determination eyeglass and neutral facial expression detection

multimodal graph based reranking for web image search this paper introduces a web image search reranking approach that explores multiple modalities in a graph based learning scheme different from the conventional methods that usually adopt a single modality or integrate multiple modalities into a long feature vector our approach can effectively integrate the learning of relevance scores weights of modalities and the distance metric and its scaling for each modality into a unified scheme in this way the effects of different modalities can be adaptively modulated and better reranking performance can be achieved we conduct experiments on a large dataset that contains more than 1000 queries and 1 million images to evaluate our approach experimental results demonstrate that the proposed reranking approach is more robust than using each individual modality and it also performs better than many existing methods

efficient and non convex coordinate descent for symmetric nonnegative matrix factorization given a symmetric nonnegative matrix a symmetric nonnegative matrix factorization symnmf is the problem of finding a nonnegative matrix h usually with much fewer columns than a such that a approx hh t symnmf can be used for data analysis and in particular for various clustering tasks unlike standard nmf which is traditionally solved by a series of quadratic convex subproblems we propose to solve symnmf by directly solving the nonconvex problem namely minimize vert a hh t vert 2 which is a fourth order nonconvex problem in this paper we propose simple and very efficient coordinate descent schemes which solve a series of fourth order univariate subproblems exactly we also derive convergence guarantees for our methods and show that they perform favorably compared to recent state of the art methods on synthetic and real world datasets especially on large and sparse input matrices

projected gradient methods for nonnegative matrix factorization nonnegative matrix factorization nmf can be formulated as a minimization problem with bound constraints although bound constrained optimization has been studied extensively in both theory and practice so far no study has formally applied its techniques to nmf in this letter we propose two projected gradient methods for nmf both of which exhibit strong optimization properties we discuss efficient implementations and demonstrate that one of the proposed methods converges faster than the popular multiplicative update approach a simple matlab code is also provided