pairwise sparsity preserving embedding for unsupervised subspace learning and classification two novel unsupervised dimensionality reduction techniques termed sparse distance preserving embedding sdpe and sparse proximity preserving embedding sppe are proposed for feature extraction and classification sdpe and sppe perform in the clean data space recovered by sparse representation and enhanced euclidean distances over noise removed data are employed to measure pairwise similarities of points in extracting informative features sdpe and sppe aim at preserving pairwise similarities between data points in addition to preserving the sparse characteristics this paper calculates the sparsest representation of all vectors jointly by a convex optimization the sparsest codes enable certain local information of data to be preserved and can endow sdpe and sppe a natural discriminating power adaptive neighborhood and robust characteristic against noise and errors in delivering low dimensional embeddings we also mathematically show sdpe and sppe can be effectively extended for discriminant learning in a supervised manner the validity of sdpe and sppe is examined by extensive simulations comparison with other related state of the art unsupervised algorithms show that promising results are delivered by our techniques

worst case linear discriminant analysis as scalable semidefinite feasibility problems in this paper we propose an efficient semidefinite programming sdp approach to worst case linear discriminant analysis wlda compared with the traditional lda wlda considers the dimensionality reduction problem from the worst case viewpoint which is in general more robust for classification however the original problem of wlda is non convex and difficult to optimize in this paper we reformulate the optimization problem of wlda into a sequence of semidefinite feasibility problems to efficiently solve the semidefinite feasibility problems we design a new scalable optimization method with a quasi newton method and eigen decomposition being the core components the proposed method is orders of magnitude faster than standard interior point sdp solvers experiments on a variety of classification problems demonstrate that our approach achieves better performance than standard lda our method is also much faster and more scalable than standard interior point sdp solvers based wlda the computational complexity for an sdp with m constraints and matrices of size d by d is roughly reduced from mathcal o m 3 md 3 m 2 d 2 to mathcal o d 3 m d in our case

scaling for the critical percolation backbone we study the backbone connecting two given sites of a two dimensional lattice separated by an arbitrary distance r in a system of size l at the percolation threshold we find a scaling form for the average backbone mass approximately l db g r l where g can be well approximated by a power law for 0 approximately l db psi r psi for the entire range 0 r l we also propose a scaling form for the probability distribution p m b of backbone mass for a given r for r approximately l p mb is peaked around l db whereas for r l p m b decreases as a power law m taub b with taub approximately 1 20 0 03 the exponents psi and taub satisfy the relation psi db taub 1 and psi is the codimension of the backbone psi d db

eigenfaces vs fisherfaces recognition using class specific linear projection we develop a face recognition algorithm which is insensitive to large variation in lighting direction and facial expression taking a pattern classification approach we consider each pixel in an image as a coordinate in a high dimensional space we take advantage of the observation that the images of a particular face under varying illumination but fixed pose lie in a 3d linear subspace of the high dimensional image space if the face is a lambertian surface without shadowing however since faces are not truly lambertian surfaces and do indeed produce self shadowing images will deviate from this linear subspace rather than explicitly modeling this deviation we linearly project the image into a subspace in a manner which discounts those regions of the face with large deviation our projection method is based on fisher s linear discriminant and produces well separated classes in a low dimensional subspace even under severe variation in lighting and facial expressions the eigenface technique another method based on linearly projecting the image space to a low dimensional subspace has similar computational requirements yet extensive experimental results demonstrate that the proposed fisherface method has error rates that are lower than those of the eigenface technique for tests on the harvard and yale face databases

modified principal component analysis an integration of multiple similarity subspace models we modify the conventional principal component analysis pca and propose a novel subspace learning framework modified pca mpca using multiple similarity measurements mpca computes three similarity matrices exploiting the similarity measurements 1 mutual information 2 angle information and 3 gaussian kernel similarity we employ the eigenvectors of similarity matrices to produce new subspaces referred to as similarity subspaces a new integrated similarity subspace is then generated using a novel feature selection approach this approach needs to construct a kind of vector set termed weak machine cell wmc which contains an appropriate number of the eigenvectors spanning the similarity subspaces combining the wrapper method and the forward selection scheme mpca selects a wmc at a time that has a powerful discriminative capability to classify samples mpca is very suitable for the application scenarios in which the number of the training samples is less than the data dimensionality mpca outperforms the other state of the art pca based methods in terms of both classification accuracy and clustering result in addition mpca can be applied to face image reconstruction mpca can use other types of similarity measurements extensive experiments on many popular real world data sets such as face databases show that mpca achieves desirable classification results as well as has a powerful capability to represent data

improved discriminate analysis for high dimensional data and its application to face recognition many pattern recognition applications involve the treatment of high dimensional data and the small sample size problem principal component analysis pca is a common used dimension reduction technique linear discriminate analysis lda is often employed for classification pca plus lda is a famous framework for discriminant analysis in high dimensional space and singular cases in this paper we examine the theory of this framework and find out that even if there is no small sample size problem the pca dimension reduction cannot guarantee the subsequent successful application of lda we thus develop an improved discriminate analysis method by introducing an inverse fisher criterion and adding a constrain in pca procedure so that the singularity phenomenon will not occur experiment results on face recognition suggest that this new approach works well and can be applied even when the number of training samples is one per class

probabilistic visual learning for object representation we present an unsupervised technique for visual learning which is based on density estimation in high dimensional spaces using an eigenspace decomposition two types of density estimates are derived for modeling the training data a multivariate gaussian for unimodal distributions and a mixture of gaussians model for multimodal distributions those probability densities are then used to formulate a maximum likelihood estimation framework for visual search and target detection for automatic object recognition and coding our learning technique is applied to the probabilistic visual modeling detection recognition and coding of human faces and nonrigid objects such as hands

approximate information discriminant analysis a computationally simple heteroscedastic feature extraction technique in this article we develop a novel linear dimensionality reduction technique for classification the technique utilizes the first two statistical moments of data and retains the computational simplicity characteristic of second order techniques such as linear discriminant analysis formally the technique maximizes a criterion that belongs to the class of probability dependence measures and is naturally defined for multiple classes the criterion is based on an approximation of an information theoretic measure and is capable of handling heteroscedastic data the performance of our method along with similar feature extraction approaches is demonstrated based on experimental results with real world datasets our method compares favorably to similar second order linear dimensionality techniques

rapid and brief communication inverse fisher discriminate criteria for small sample size problem and its application to face recognition this paper addresses the small sample size problem in linear discriminant analysis which occurs in face recognition applications belhumeur et al ieee trans pattern anal mach intell 19 7 1997 711 720 proposed the fisherface method we find out that the fisherface method might fail since after the pca transform the corresponding within class covariance matrix can still be singular this phenomenon is verified with the yale face database hence we propose to use an inverse fisher criteria our method works when the number of training images per class is one experiment results suggest that this new approach performs well

computing hopf bifurcations ii three examples from neurophysiology in guckenheimer myers and sturmfels siam j numer anal 34 1997 we present algorithms for detecting hopf bifurcations in two parameter families of vector fields based on classical algebraic constructions in addition to their utility as augmented systems for use with standard newton type continuation methods they are shown to be particularly well adapted for solution by computer algebra techniques for vector fields of small or moderate dimension the present study examines the performance of these methods on test problems selected from models of current research interest in neurophysiology implementation issues are examined and the numerical properties of the proposed methods are compared with several alternative algorithms for hopf pathfollowing that appear in the literature

comments on globally maximizing locally minimizing unsupervised discriminant projection with application to face and palm biometrics this paper develops an unsupervised discriminant projection udp technique for dimensionality reduction of high dimensional data in small sample size cases udp can be seen as a linear approximation of a multimanifolds based learning framework which takes into account both the local and nonlocal quantities udp characterizes the local scatter as well as the nonlocal scatter seeking to find a projection that simultaneously maximizes the nonlocal scatter and minimizes the local scatter this characteristic makes udp more intuitive and more powerful than the most up to date method locality preserving projection lpp which considers only the local scatter for clustering or classification tasks the proposed method is applied to face and palm biometrics and is examined using the yale feret and ar face image databases and the polyu palmprint database the experimental results show that udp consistently outperforms lpp and pca and outperforms lda when the training sample size per class is small this demonstrates that udp is a good choice for real world biometrics applications

face recognition using scale adaptive directional and textural features a novel approach to face recognition problem using directional and texture information from face images is proposed in this paper in order to capture the directionality specially designed using local polynomial approximation technique scale adaptive digital filters are used for texture features extraction a low dimensional and computationally effective local descriptor is utilized textural and directional features are captured at the holistic and part based levels resulting in a robust face descriptor the proposed method is tested on a number of standard test face datasets orl xm2vts extended yale cmu pie ar and feret for different scenarios and its performance is compared with several state of the art techniques

solving the small sample size problem in face recognition using generalized discriminant analysis the goal of face recognition is to distinguish persons via their facial images each person s images form a cluster and a new image is recognized by assigning it to the correct cluster since the images are very high dimensional it is necessary to reduce their dimension linear discriminant analysis lda has been shown to be effective at dimension reduction while preserving the cluster structure of the data it is classically defined as an optimization problem involving covariance matrices that represent the scatter within and between clusters the requirement that one of these matrices be nonsingular restricts its application to datasets in which the dimension of the data does not exceed the sample size for face recognition however the dimension typically exceeds the number of images in the database resulting in what is referred to as the small sample size problem recently the applicability of lda has been extended by using the generalized singular value decomposition gsvd to circumvent the nonsingularity requirement thus making lda directly applicable to face recognition data our experiments confirm that lda gsvd solves the small sample size problem very effectively as compared with other current methods

sparse orthogonal linear discriminant analysis in this paper sparse orthogonal linear discriminant analysis olda is studied the main contributions of the present work include the following i all minimum frobenius norm dimension solutions of the optimization problem used for establishing olda are characterized explicitly and ii this explicit characterization leads to two numerical algorithms for computing a sparse linear transformation for olda the first is based on the gradient flow approach while the second is a sequential linear bregman method we experiment with real world datasets to illustrate that the sequential linear bregman method is much better than the gradient flow approach the sequential linear bregman method always achieves comparable classification accuracy with the normal olda satisfactory sparsity and orthogonality and acceptable cpu times

algebraic feature extraction for image recognition based on an optimal discriminant criterion abstract a novel algebraic feature extraction method for image recognition is presented the main difference between the present method and any other existing algebraic feature extraction method is that the present method uses an optimal discriminant criterion to extract algebraic features for the given training image samples a set of optimal discriminant projection vectors is calculated according to a generalized fisher criterion function on the basis of the optimal discriminant projection vector the projective feature vectors of an image i e algebraic features are extracted by projecting the image onto all optimal discriminant projection vectors the method of calculating the set of optimal discriminant projection vectors is discussed a minimum distance classifier is also proposed for classifying projective feature vectors experimental results showed that the present method has good recognition performance an important conclusion about the present method is that the foley sammon optimal set of discriminant vectors is a special case of the set of optimal discriminant projection vectors

multi class fukunaga koontz discriminant analysis for enhanced face recognition linear subspace learning methods such as fisher s linear discriminant analysis lda unsupervised discriminant projection udp and locality preserving projections lpp have been widely used in face recognition applications as a tool to capture low dimensional discriminant information however when these methods are applied in the context of face recognition they often encounter the small sample size problem in order to overcome this problem a separate principal component analysis pca step is usually adopted to reduce the dimensionality of the data however such a step may discard dimensions that contain important discriminative information that can aid classification performance in this work we propose a new idea which we named multi class fukunaga koontz discriminant analysis fkda by incorporating the fukunaga koontz transform within the optimization for maximizing class separation criteria in lda udp and lpp in contrast to traditional lda udp and lpp our approach can work with very high dimensional data as input without requiring a separate dimensionality reduction step to make the scatter matrices full rank in addition the fkda formulation seeks optimal projection direction vectors that are orthogonal which the existing methods cannot guarantee and it has the capability of finding the exact solutions to the trace ratio objective in discriminant analysis problems while traditional methods can only deal with a relaxed and inexact ratio trace objective we have shown using six face database in the context of large scale unconstrained face recognition face recognition with occlusions and illumination invariant face recognition under closed set semi open set and open set recognition scenarios that our proposed fkda significantly outperforms traditional linear discriminant subspace learning methods as well as five other competing algorithms highlightssolve small sample size problem in lda udp lpp using fkt formulation can work with high dimensional data without inverting any scatter matrices finds optimal projection direction vectors that are orthogonal finds exact solutions to the objective in the form of trace ratio improvement in unconstrained face recognition scenarios

perturbation lda learning the difference between the class empirical mean and its expectation fisher s linear discriminant analysis lda is popular for dimension reduction and extraction of discriminant features in many pattern recognition applications especially biometric learning in deriving the fisher s lda formulation there is an assumption that the class empirical mean is equal to its expectation however this assumption may not be valid in practice in this paper from the perturbation perspective we develop a new algorithm called perturbation lda p lda in which perturbation random vectors are introduced to learn the effect of the difference between the class empirical mean and its expectation in fisher criterion this perturbation learning in fisher criterion would yield new forms of within class and between class covariance matrices integrated with some perturbation factors moreover a method is proposed for estimation of the covariance matrices of perturbation random vectors for practical implementation the proposed p lda is evaluated on both synthetic data sets and real face image data sets experimental results show that p lda outperforms the popular fisher s lda based algorithms in the undersampled case

multiclass classifiers based on dimension reduction with generalized lda linear discriminant analysis lda has been widely used for dimension reduction of data sets with multiple classes the lda has been recently extended to various generalized lda methods that are applicable regardless of the relative sizes between the data dimension and the number of data items in this paper we propose several multiclass classifiers based on generalized lda glda algorithms taking advantage of the dimension reducing transformation matrix without requiring additional training or parameter optimization a marginal linear discriminant classifier mldc a bayesian linear discriminant classifier bldc and a one dimensional bldc are introduced for multiclass classification our experimental results illustrate that these classifiers produce higher ten fold cross validation accuracy than knn and centroid based classifiers in the reduced dimensional space obtained from glda

wavelet based discriminant analysis for face recognition the linear fisher discriminant analysis lda is a well known and popular statistical method in pattern recognition and classification when applied to face recognition problem the small sample size problem occurs we investigate the nature of this phenomenon and use wavelet transform for dimension reduction moreover we propose a regularized scheme based face recognition system comparisons are made with the tikhonov regularization method and the infinity fisher index method we find out that when the small sample size problem appears optimizing the fisher index does not lead to good results

incremental learning of bidirectional principal components for face recognition recently bidirectional principal component analysis bdpca has been proven to be an efficient tool for pattern recognition and image analysis encouraging experimental results have been reported and discussed in the literature however bdpca has to be performed in batch mode it means that all the training data has to be ready before we calculate the projection matrices if there are additional samples need to be incorporated into an existing system it has to be retrained with the whole updated training set moreover the scatter matrices of bdpca are formulated as the sum of k samples size image covariance matrices this leads to the incremental learning directly on the scatters impossible thus it presents new challenge for on line training in fact there are two major reasons for building incremental algorithms the first reason is that in some cases when the number of training images is very large the batch algorithm cannot process the entire training set due to large computational or space requirements of the batch approach the second reason is when the learning algorithm is supposed to operate in a dynamical settings that all the training data is not given in advance and new training samples may arrive at any time and they have to be processed in an on line manner through matricizations of third order tensor we successfully transfer the eigenvalue decomposition problem of scatters to the singular value decomposition svd of corresponding unfolded matrices followed by complexity and memory analysis on the novel algorithm a theoretical clue for selecting suitable dimensionality parameters without losing classification information is also presented in this paper experimental results on feret and cmu pie pose illumination and expression databases show that the ibdpca algorithm gives a close approximation to the bdpca method but using less time

discriminative common vectors for face recognition in face recognition tasks the dimension of the sample space is typically larger than the number of the samples in the training set as a consequence the within class scatter matrix is singular and the linear discriminant analysis lda method cannot be applied directly this problem is known as the small sample size problem in this paper we propose a new face recognition method called the discriminative common vector method based on a variation of fisher s linear discriminant analysis for the small sample size case two different algorithms are given to extract the discriminative common vectors representing each person in the training set of the face database one algorithm uses the within class scatter matrix of the samples in the training set while the other uses the subspace methods and the gram schmidt orthogonalization procedure to obtain the discriminative common vectors then the discriminative common vectors are used for classification of new faces the proposed method yields an optimal solution for maximizing the modified fisher s linear discriminant criterion given in the paper our test results show that the discriminative common vector method is superior to other methods in terms of recognition accuracy efficiency and numerical stability

fisher regularized support vector machine this paper proposes fisher regularized support vector machine fishersvm fishersvm is a graph based supervised learning method fishersvm has two regularization terms maximum margin and fisher regularizations fishersvm aims to maximize the margin and minimize the within class scatter support vector machine svm and fisher discriminant analysis fda are two commonly used methods in machine learning and pattern recognition a combined method of the linear svm and fda called svm lda linear discriminant analysis has been proposed only for the linear case this paper generalizes this combined method to the nonlinear case from the view of regularization a fisher regularization is defined and incorporated into svm to obtain a fisher regularized support vector machine fishersvm in fishersvm there are two regularization terms the maximum margin regularization and fisher regularization which allow fishersvm to maximize the classification margin and minimize the within class scatter roughly speaking fishersvm can approximatively fulfill the fisher criterion and obtain good statistical separability this paper also discusses the connections of fishersvm to graph based regularization methods and the mathematical programming method to kernel fisher discriminant analysis kfda it shows that fishersvm can be thought of as a graph based supervised learning method or a robust kfda experimental results on artificial and real world data show that fishersvm has a promising generalization performance

computing hopf bifurcations i this paper addresses the problems of detecting hopf bifurcations in systems of ordinary differential equations and following curves of hopf points in two parameter families of vector fields the established approach to this problem relies upon augmenting the equilibrium condition so that a hopf bifurcation occurs at an isolated regular point of the extended system we propose two new methods of this type based on classical algebraic results regarding the roots of polynomial equations and properties of kronecker products for matrices in addition to their utility as augmented systems for use with standard newton type continuation methods they are also particularly well adapted for solution by computer algebra techniques for vector fields of small or moderate dimension

a new and fast implementation for null space based linear discriminant analysis in this paper we present a new implementation for the null space based linear discriminant analysis the main features of our implementation include i the optimal transformation matrix is obtained easily by only orthogonal transformations without computing any eigendecomposition and singular value decomposition svd consequently our new implementation is eigendecomposition free and svd free ii its main computational complexity is from a economic qr factorization of the data matrix and a economic qr factorization of a nxn matrix with column pivoting here n is the sample size thus our new implementation is a fast one the effectiveness of our new implementation is demonstrated by some real world data sets

eigenfeature regularization and extraction in face recognition this work proposes a subspace approach that regularizes and extracts eigenfeatures from the face image eigenspace of the within class scatter matrix is decomposed into three subspaces a reliable subspace spanned mainly by the facial variation an unstable subspace due to noise and finite number of training samples and a null subspace eigenfeatures are regularized differently in these three subspaces based on an eigenspectrum model to alleviate problems of instability overfitting or poor generalization this also enables the discriminant evaluation performed in the whole space feature extraction or dimensionality reduction occurs only at the final stage after the discriminant assessment these efforts facilitate a discriminative and a stable low dimensional feature representation of the face image experiments comparing the proposed approach with some other popular subspace methods on the feret orl ar and gt databases show that our method consistently outperforms others

optimal discriminant plane for a small number of samples and design method of classifier on the plane abstract in a previous work zi quan hong and jing yu yang minimum distance classifier on the optimal discriminant plane we suggested and derived a method for constructing classifier on the optimal discriminant plane using minimum distance criteria in this paper the problem of solving optimal discriminant plane for a small number of samples is discussed which is based on the above paper by the same authors in the case of a small number of samples generalized eigenequation ax bx established for a large number of samples usually has no solution because within class scatter matrix is singular to obtain the solution of the generalized eigenequation a new method is suggested in which the singular value perturbation is added to the within class scatter matrix such that the matrix becomes a nonsingular matrix therefore the generalized eigenequation can be solved using existing algorithms we proved that the generalized eigenequations on the optimal discriminant plane are stable in respect of eigenvalues and the generalized eigenvectors are indeed the optimal discriminant directions if the perturbation is subject to some certain conditions the experimental results have shown that our method works well and the minimum distance classifier on the optimal discriminant plane can be constructed with high performance even in the case of a small number of samples

robust classification using l 2 1 norm based regression model a novel classification method using 2 1 norm based regression is proposed in this paper the 2 1 norm based loss function is robust to outliers or large variations distributed in the given data and the 2 1 norm regularization term selects correlated samples across the whole training set with grouped sparsity a probabilistic interpretation under the multiple task learning framework presents theoretical foundation for the optimal solution complexity analysis of our proposed classification algorithm is also presented several benchmark data sets including facial images and gene expression data are used for evaluating the effectiveness of the new proposed algorithm and the results show competitive performance particularly better than those using dummy matrix as the response variables this result is very useful since it is important for selecting appropriate response variables in classification oriented regression models

a direct method for the computation of hopf bifurcation points we present a method for the numerical computation of a hopf bifurcation point of an evolution equation an augmented time independent system of equations has to be solved for which the hopf bifurcation point is an isolated solution in the case of a system of ordinary differential equations we propose an efficient implementation of newton s method for the solution of the resulting system of algebraic equations if the method is applied directly to a partial differential system with one space variable a two point boundary value problem is obtained we also compare this technique with a related method and results are given for two systems of partial differential equations describing chemical reacting systems

sweeping algebraic curves for singular solutions many problems give rise to polynomial systems these systems often have several parameters and we are interested to study how the solutions vary when we change the values for the parameters using predictor corrector methods we track the solution paths a point along a solution path is critical when the jacobian matrix is rank deficient the simplest case of quadratic turning points is well understood but these methods no longer work for general types of singularities in order not to miss any singular solutions along a path we propose to monitor the determinant of the jacobian matrix we examine the operation range of deflation and relate the effectiveness of deflation to the winding number computational experiments on systems coming from different application fields are presented

lyapunov inverse iteration for identifying hopf bifurcations in models of incompressible flow the identification of instability in large scale dynamical systems caused by hopf bifurcation is difficult because of the problem of identifying the rightmost pair of complex eigenvalues of large sparse generalized eigenvalue problems a new method developed in k meerbergen and a spence siam j matrix anal appl 31 2010 pp 1982 1999 avoids this computation instead performing an inverse iteration for a certain set of real eigenvalues that requires the solution of a large scale lyapunov equation at each iteration in this study we refine the lyapunov inverse iteration method to make it more robust and efficient and we examine its performance on challenging test problems arising from fluid dynamics various implementation issues are discussed including the use of inexact inner iterations and the impact of the choice of iterative solution for the lyapunov equations and the effect of eigenvalue distribution on performance numerical experiments demonstrate the robustness of the algorithm

face recognition using discriminant locality preserving projections based on maximum margin criterion in this paper we propose a new discriminant locality preserving projections based on maximum margin criterion dlpp mmc dlpp mmc seeks to maximize the difference rather than the ratio between the locality preserving between class scatter and locality preserving within class scatter dlpp mmc is theoretically elegant and can derive its discriminant vectors from both the range of the locality preserving between class scatter and the range space of locality preserving within class scatter dlpp mmc can also derive its discriminant vectors from the null space of locality preserving within class scatter when the parameter of dlpp mmc approaches experiments on the orl yale feret and pie face databases show the effectiveness of the proposed dlpp mmc

postbreakthrough behavior in flow through porous media we numerically simulate the traveling time of a tracer in convective flow between two points injection and extraction separated by a distance r in a model of porous media d 2 percolation we calculate and analyze the traveling time probability density function for two values of the fraction of connecting bonds p the homogeneous case p 1 and the inhomogeneous critical threshold case p p c we analyze both constant current and constant pressure conditions at p p c the homogeneous p 1 case serves as a comparison base for the more complicated p p c situation we find several regions in the probability density of the traveling times for the homogeneous case p 1 and also for the critical case p p c for both constant pressure and constant current conditions for constant pressure the first region i p corresponds to the short times before the flow breakthrough occurs when the probability distribution is strictly zero the second region ii p corresponds to numerous fast flow lines reaching the extraction point with the probability distribution reaching its maximum the third region iii p corresponds to intermediate times and is characterized by a power law decay the fourth region iv p corresponds to very long traveling times and is characterized by a different power law decaying tail the power law characterizing region iv p is related to the multifractal properties of flow in percolation and an expression for its dependence on the system size l is presented the constant current behavior is different from the constant pressure behavior and can be related analytically to the constant pressure case we present theoretical arguments for the values of the exponents characterizing each region and crossover times our results are summarized in two scaling assumptions for the traveling time probability density one for constant pressure and one for constant current we also present the production curve associated with the probability of traveling times which is of interest to oil recovery

local discriminant wavelet packet coordinates for face recognition face recognition is a challenging problem due to variations in pose illumination and expression techniques that can provide effective feature representation with enhanced discriminability are crucial wavelets have played an important role in image processing for its ability to capture localized spatial frequency information of images in this paper we propose a novel local discriminant coordinates method based on wavelet packet for face recognition to compensate for these variations traditional wavelet based methods for face recognition select or operate on the most discriminant subband and neglect the scattered characteristic of discriminant features the proposed method selects the most discriminant coordinates uniformly from all spatial frequency subbands to overcome the deficiency of traditional wavelet based methods to measure the discriminability of coordinates a new dilation invariant entropy and a maximum a posterior logistic model are put forward moreover a new triangle square ratio criterion is used to improve classification using the euclidean distance and the cosine criterion experimental results show that the proposed method is robust for face recognition under variations in illumination pose and expression

an efficient discriminant based solution for small sample size problem classification of high dimensional statistical data is usually not amenable to standard pattern recognition techniques because of an underlying small sample size problem to address the problem of high dimensional data classification in the face of a limited number of samples a novel principal component analysis pca based feature extraction classification scheme is proposed the proposed method yields a piecewise linear feature subspace and is particularly well suited to difficult recognition problems where achievable classification rates are intrinsically low such problems are often encountered in cases where classes are highly overlapped or in cases where a prominent curvature in data renders a projection onto a single linear subspace inadequate the proposed feature extraction classification method uses class dependent pca in conjunction with linear discriminant feature extraction and performs well on a variety of real world datasets ranging from digit recognition to classification of high dimensional bioinformatics and brain imaging data

optimal fisher discriminant analysis using the rank decomposition abstract the fisher optimal discriminant vector is a very efficient means in high dimensional pattern analysis in the case of a small number of samples the within class scatter matrix s w is singular therefore the calculation of the fisher optimal discriminant vector becomes very important in this paper the conception of the rank decomposition of matrices is first introduced and a new method for calculating the fisher optimal discriminant vector is presented which is particularly well suited to the case of a small number of samples in the sense that the scatter matrices are rank deficient the new method is compared with the pseudoinverse and perturbation methods in tian j opt soc am a 5 1670 1672 1988 and hong and yang pattern recognition 24 317 324 1991 an important conclusion is proved in the three methods our method is the best one for calculating the fisher optimal discriminant vectors in the cases of both a large and a small number of samples the experimental results have also shown that our method can give the highest recognition rate in the case of a small number of samples among the three methods

a linpack style condition estimator for the equation ax xb t c estimation of the condition number of ax xb t c requires an approximation of the smallest l 2 singular value of the linear transformation x rightarrow ax xb t the approximation is obtained from a step of inverse iteration applied to a heuristically chosen starting matrix in the style of linpack the cost of the estimator is kept down by using the same matrix factorizatlons to make the estimate as are used to solve ax xb t c computational experiments verify its reliability

ga based optimal selection of pzmi features for face recognition one of the key problems in automated face recognition system is that of handling the face image variation in terms of scale rotation in plane and translation one approach is fixing mentioned problems in recognition processes by extracting one linear transformation invariant feature this paper presents a novel method for face recognition pseudo zernike moment invariant pzmi which has linear transformation invariance properties and is robust in the presence of noise utilized to produce feature vectors for decreasing computational complexity of feature extraction step we use genetic algorithm ga to select the optimal feature set which contains optimal pzmi orders and corresponding repetitions in addition we have investigated the effect of pzmi orders on recognition rate in noisy images proposed scheme has been tested on the feret database experimental results prove the advantages of the proposed method when compared with other pzmi based face recognition systems

a direct electrifying algorithm for backbone identification this paper proposes a new algorithm for identifying backbones in the application of percolation theory this algorithm is based on the current carrying definition of backbone and is carried out on the predetermined spanning cluster it is fairly easy to implement and further parallelize the efficiency is enhanced by the fact that the conductivity of a percolating system can be obtained in the same processing of backbone identification the critical exponents of backbone mass red bonds sites and conductivity obtained by this algorithm are in very good agreement with the existing results

defining functions for multiple hopf bifurcations let a u alpha f u u alpha u in r n alpha in r k be a family of real n times n matrices arising as the jacobian matrices of equilibrium solutions to the dynamical system stackrel u f u alpha an equilibrium point is called a hopf point if a has a conjugate pair of pure imaginary eigenvalues pm i omega omega 0 it is called a double hopf point if there are two such pairs pm i omega 1 pm i omega 2 and a 1 1 resonant double hopf point if in addition omega 1 omega 2 defining functions are obtained for the numerical detection computation and continuation of hopf double hopf and 1 1 resonant double hopf points they are based on a combination of matrix biproduct and bordered matrix methods example computations are done in a fairly realistic and complicated neural model problem with n 13 and k 29 however to make the methods applicable to large scale problems e g discretized boundary value problems we reduce the state space to a subspace that essentially contains the generalized eigenspaces of the eigenvalues with largest real part

analytical search for bifurcation surfaces in parameter space the method of resultants can be used to compute hopf bifurcations in ode systems we discuss this method from the applicant s point of view furthermore we show by theory and examples that the method may be extended to cover other bifurcation situations as well among them are the real hopf situation which plays an important role in the transition to shil nikov chaos as well as some higher codimension bifurcations such as takens bogdanov gavrilov guckenheimer and double hopf bifurcations the method yields an analytical test function that can be solved analytically or by computer algebra systems in contrast to common analytical techniques based on eigenvalue computation which can only be applied to systems of size n 4 the method is applicable for systems of intermediate size n 10 we illustrate the power of the method by discussing examples from different disciplines of science a lorenz like oscillator two coupled oscillators and a five species food chain

regularized discriminant analysis and its application to face recognition the linear fisher discriminant analysis lda is a well known and popular statistical method in pattern recognition and classi5cation the basic idea is to optimize the discriminant criteria in which the ratio between the interand the intra class distance is maximized this approach is theoretically sound and a number of papers have shown its superior performance when applying in pattern recognition applications 1 5 however this approach su ers from the small size problem this problem occurs when the sample size is small compared with the size of feature vector which always appears in the face recognition applications this is because the typical image dimension for face image is greater than 32 32 and usually only 2 6 images are used for training thus the within class covariance estimation cw will be singular in this case the eigenvalue problem is ill posed the fisher index in t will be reached at the null space of cw and will always result in in5nity index in t a number of methods have been proposed in the last decade to overcome the limitation of lda on small sample size these methods in applying to face recognition can be roughly grouped into three categories the 5rst approach applied a dimension reduction method such as principal component analysis pca to extract important components for lda this approach is straightforward but the dimension reduction method may remove some useful information for recognition the second approach modi5es the fisher s optimization criteria theoretically this approach can solve the problem but the constraints 1 3 will limit the

1d lda vs 2d lda when is vector based linear discriminant analysis better than matrix based recent advances have shown that algorithms with 2d matrix based representation perform better than the traditional 1d vector based ones in particular 2d lda has been widely reported to outperform 1d lda however would the matrix based linear discriminant analysis be always superior and when would 1d lda be better in this paper we investigate into these questions and have a comprehensive comparison between 1d lda and 2d lda in theory and in experiments we analyze the heteroscedastic problem in 2d lda and formulate mathematical equalities to explore the relationship between 1d lda and 2d lda then we point out potential problems in 2d lda it is shown that 2d lda has eliminated the information contained in the covariance information between different local geometric structures such as the rows or the columns which is useful for discriminant feature extraction whereas 1d lda could preserve such information interestingly this new finding indicates that 1d lda is able to gain higher fisher score than 2d lda in some extreme case furthermore sufficient conditions on which 2d lda would be bayes optimal for two class classification problem are derived and comparison with 1d lda in this aspect is also analyzed this could help understand how 2d lda is expected to achieve at its best further discover its relationship with 1d lda and well support other findings after the theoretical analysis comprehensive experimental results are reported by fairly and extensively comparing 1d lda with 2d lda in contrast to the existing view that some 2d lda based algorithms would perform better than 1d lda when the number of training samples for each class is small or when the number of discriminant features used is small we show that it is not always true and show that some standard 1d lda based algorithms could perform better in those cases on some challenging data sets

the method of resultants for computing real solutions of polynomial systems a new method for determining the real solutions to a set of polynomial equations is presented it is based on the theory of multiresultants the inherently unstable calculation of the determinant is replaced by a stable minimization procedure which is able to take advantage of the sparseness of the resultant matrix two numerical examples illustrate the method the paper contains preliminary work which demonstrates the feasibility of the given approach

the hopf bifurcation theorem for quasilinear differential algebraic equations we prove a generalization of the hopf bifurcation theorem for quasilinear differential equations daes i e equations of the form a g where the matrix a has constant but not full rank and hence the system cannot be made into an explicit ode the paper includes an appendix by j ernsthausen addressing the numerical calculation of the hopf points in the dae setting

a collaborative representation based projections method for feature extraction in graph embedding based methods we usually need to manually choose the nearest neighbors and then compute the edge weights using the nearest neighbors via l2 norm e g lle it is difficult and unstable to manually choose the nearest neighbors in high dimensional space so how to automatically construct a graph is very important in this paper first we give a l2 graph like l1 graph l2 graph calculates the edge weights using the total samples avoiding manually choosing the nearest neighbors second a l2 graph based feature extraction method is presented called collaborative representation based projections crp like spp crp aims to preserve the collaborative representation based reconstruction relationship of data crp utilizes a l2 norm graph to characterize the local compactness information crp maximizes the ratio between the total separability information and the local compactness information to seek the optimal projection matrix crp is much faster than spp since crp calculates the objective function with l2 norm while spp calculate the objective function with l1 norm experimental results on feret ar yale face databases and the polyu finger knuckle print database demonstrate that crp works well in feature extraction and leads to a good recognition performance we give a l2 norm graph based on collaborative representation we propose a collaborative representation based projections crp for feature extraction crp is a rayleigh quotient form and can be calculated via generalized eigenvalue decomposition

hyperplane navigation a method to set individual scores in fmri group datasets abstract recent studies have demonstrated that spatial patterns of fmri bold activity distribution over the brain may be used to classify different groups or mental states these studies are based on the application of advanced pattern recognition approaches and multivariate statistical classifiers most published articles in this field are focused on improving the accuracy rates and many approaches have been proposed to accomplish this task nevertheless a point inherent to most machine learning methods and still relatively unexplored in neuroimaging is how the discriminative information can be used to characterize groups and their differences in this work we introduce the maximum uncertainty linear discrimination analysis mlda and show how it can be applied to infer groups patterns by discriminant hyperplane navigation in addition we show that it naturally defines a behavioral score i e an index quantifying the distance between the states of a subject from predefined groups we validate and illustrate this approach using a motor block design fmri experiment data with 35 subjects

feature extraction based on laplacian bidirectional maximum margin criterion maximum margin criterion mmc based feature extraction is more efficient than linear discriminant analysis lda for calculating the discriminant vectors since it does not need to calculate the inverse within class scatter matrix however mmc ignores the discriminative information within the local structures of samples and the structural information embedding in the images in this paper we develop a novel criterion namely laplacian bidirectional maximum margin criterion lbmmc to address the issue we formulate the image total laplacian matrix image within class laplacian matrix and image between class laplacian matrix using the sample similar weight that is widely used in machine learning the proposed lbmmc based feature extraction computes the discriminant vectors by maximizing the difference between image between class laplacian matrix and image within class laplacian matrix in both row and column directions experiments on the feret and yale face databases show the effectiveness of the proposed lbmmc based feature extraction method

optimal subset division based discrimination and its kernelization for face and palmprint recognition discriminant analysis is effective in extracting discriminative features and reducing dimensionality in this paper we propose an optimal subset division based discrimination osdd approach to enhance the classification performance of discriminant analysis technique osdd first divides the sample set into several subsets by using an improved stability criterion and k means algorithm we separately calculate the optimal discriminant vectors from each subset then we construct the projection transformation by combining the discriminant vectors derived from all subsets furthermore we provide a nonlinear extension of osdd that is the optimal subset division based kernel discrimination oskd approach it employs the kernel k means algorithm to divide the sample set in the kernel space and obtains the nonlinear projection transformation the proposed approaches are applied to face and palmprint recognition and are examined using the ar and feret face databases and the polyu palmprint database the experimental results demonstrate that the proposed approaches outperform several related linear and nonlinear discriminant analysis methods

classifiability based discriminatory projection pursuit fisher s linear discriminant fld is one of the most widely used linear feature extraction method especially in many visual computation tasks based on the analysis on several limitations of the traditional fld this paper attempts to propose a new computational paradigm for discriminative linear feature extraction named classifiability based discriminatory projection pursuit cdpp which is different from the traditional fld and its variants there are two steps in the proposed cdpp one is the construction of a candidate projection set cps and the other is the pursuit of discriminatory projections specifically in the former step candidate projections are generated by using the nearest between class boundary samples while the latter is efficiently achieved by classifiability based adaboost learning from the cps we show that the new projection pursuit paradigm not only does not suffer from the limitations of the traditional fld but also inherits good generalizability from the boundary attribute of candidate projections extensive experiments on both synthetic and real datasets validate the effectiveness of cdpp for discriminative linear feature extraction

image covariance based subspace method for face recognition this paper proposes a new subspace method that is based on image covariance obtained from windowed features of images a windowed input feature consists of a number of pixels and the dimension of input space is determined by the number of windowed features each element of an image covariance matrix can be obtained from the inner product of two windowed features the 2d pca and 2d lda methods are then obtained from principal component analysis and linear discriminant analysis respectively using the image covariance matrix in the case of 2d lda there is no need for pca preprocessing and the dimension of subspace can be greater than the number of classes because the within class and between class image covariance matrices have full ranks comparative experiments are performed using the feret cmu and orl databases of facial images the experimental results show that the proposed 2d lda provides the best recognition rate among several subspace methods in all of the tests

an algorithm for the computation of hopf bifurcation points in comparison with other methods we present an algorithm for the computation of a hopf bifurcation point based on a direct method i e an augmented time independent system is solved the algorithm is designed to deal efficiently with problems representing a finite difference discretization of a partial differential system in this case the bandstructure of the jacobian matrix is exploited we also compare this technique with some other methods in regard to its efficiency and reliability

an optimization criterion for generalized discriminant analysis on undersampled problems an optimization criterion is presented for discriminant analysis the criterion extends the optimization criteria of the classical linear discriminant analysis lda through the use of the pseudoinverse when the scatter matrices are singular it is applicable regardless of the relative sizes of the data dimension and sample size overcoming a limitation of classical lda the optimization problem can be solved analytically by applying the generalized singular value decomposition gsvd technique the pseudoinverse has been suggested and used for undersampled problems in the past where the data dimension exceeds the number of data points the criterion proposed in this paper provides a theoretical justification for this procedure an approximation algorithm for the gsvd based approach is also presented it reduces the computational complexity by finding subclusters of each cluster and uses their centroids to capture the structure of each cluster this reduced problem yields much smaller matrices to which the gsvd can be applied efficiently experiments on text data with up to 7 000 dimensions show that the approximation algorithm produces results that are close to those produced by the exact algorithm

a new lda based face recognition system which can solve the small sample size problem a new lda based face recognition system is presented in this paper linear discriminant analysis lda is one of the most popular linear projection techniques for feature extraction the major drawback of applying lda is that it may encounter the small sample size problem in this paper we propose a new lda based technique which can solve the small sample size problem we also prove that the most expressive vectors derived in the null space of the within class scatter matrix using principal component analysis pca are equal to the optimal discriminant vectors derived in the original space using lda the experimental results show that the new lda process improves the performance of a face recognition system signi cantly 2000 pattern recognition society published by elsevier science ltd all rights reserved

regression analysis of locality preserving projections via sparse penalty recent studies have shown that linear subspace algorithms such as principal component analysis linear discriminant analysis and locality preserving projections have attracted tremendous attention in many fields of information processing however the projection results obtained by these algorithms are linear combination of the original features which is difficult to be interpreted psychologically and physiologically motivated by compressive sensing theory we formulate the generalized eigenvalue problem under cs framework which then allows us to apply a sparsity penalty and minimization procedure to locality preserving projections the proposed algorithm is called sparse locality preserving projections which performs locality preserving projections in the lasso regression framework that dimensionality reduction feature selection and classification are merged into one analysis the method is also extended to its regularized form to improve its generalization the proposed algorithm is a combination of locality preserving with sparse penalty additionally the algorithm can be performed in either supervised or unsupervised tasks experimental results on toy and real data sets show that our methods are effective and demonstrate much higher performance

characterization of a family of algorithms for generalized discriminant analysis on undersampled problems a generalized discriminant analysis based on a new optimization criterion is presented the criterion extends the optimization criteria of the classical linear discriminant analysis lda when the scatter matrices are singular an efficient algorithm for the new optimization problem is presented the solutions to the proposed criterion form a family of algorithms for generalized lda which can be characterized in a closed form we study two specific algorithms namely uncorrelated lda ulda and orthogonal lda olda ulda was previously proposed for feature extraction and dimension reduction whereas olda is a novel algorithm proposed in this paper the features in the reduced space of ulda are uncorrelated while the discriminant vectors of olda are orthogonal to each other we have conducted a comparative study on a variety of real world data sets to evaluate ulda and olda in terms of classification accuracy

regularized orthogonal linear discriminant analysis in this paper the regularized orthogonal linear discriminant analysis rolda is studied the major issue of the regularized linear discriminant analysis is to choose an appropriate regularization parameter in existing regularized linear discriminant analysis methods they all select the best regularization parameter from a given parameter candidate set by using cross validation for classification an obvious limitation of such regularized linear discriminant analysis methods is that it is not clear how to choose an appropriate candidate set therefore up to now there is no concrete mathematical theory available in selecting an appropriate regularization parameter in practical applications of the regularized linear discriminant analysis the present work is to fill this gap here we derive the mathematical relationship between orthogonal linear discriminant analysis and the regularized orthogonal linear discriminant analysis first and then by means of this relationship we find a mathematical criterion for selecting the regularization parameter in rolda and consequently we develop a new regularized orthogonal linear discriminant analysis method in which no candidate set of regularization parameter is needed the effectiveness of our proposed regularized orthogonal linear discriminant analysis is illustrated by some real world data sets

novel fisher discriminant classifiers at the present several applications need to classify high dimensional points belonging to highly unbalanced classes unfortunately when the training set cardinality is small compared to the data dimensionality small sample size problem the classification performance of several well known classifiers strongly decreases similarly the classification accuracy of several discriminative methods decreases when non linearly separable and unbalanced classes are treated in this paper we firstly survey state of the art methods that employ improved versions of linear discriminant analysis lda to deal with the above mentioned problems secondly we propose a family of classifiers based on the fisher subspace estimation which efficiently deal with the small sample size problem non linearly separable classes and unbalanced classes the promising results obtained by the proposed techniques on benchmark datasets and the comparison with state of the art predictors show the efficacy of the proposed techniques

efficient and robust feature extraction by maximum margin criterion in pattern recognition feature extraction techniques are widely employed to reduce the dimensionality of data and to enhance the discriminatory information principal component analysis pca and linear discriminant analysis lda are the two most popular linear dimensionality reduction methods however pca is not very effective for the extraction of the most discriminant features and lda is not stable due to the small sample size problem in this paper we propose some new linear and nonlinear feature extractors based on maximum margin criterion mmc geometrically feature extractors based on mmc maximize the average margin between classes after dimensionality reduction it is shown that mmc can represent class separability better than pca as a connection to lda we may also derive lda from mmc by incorporating some constraints by using some other constraints we establish a new linear feature extractor that does not suffer from the small sample size problem which is known to cause serious stability problems for lda the kernelized nonlinear counterpart of this linear feature extractor is also established in the paper our extensive experiments demonstrate that the new feature extractors are effective stable and efficient

scaling of cluster mass between two lines in 3d percolation abstract we consider the cluster mass distribution between two lines of arbitrary orientations and lengths in porous media in three dimensions and model the porous media by bond percolation at the percolation threshold p c we observe that for many geometrical configurations the mass probability distribution presents power law behavior we determine how the characteristic mass of the distribution scales with such geometrical parameters as the line length w the minimal distance between lines r and the angle between the lines the fractal dimensions of the cluster mass are independent of w r and the slope of the power law regime of the cluster mass is unaffected by changes in these three variables however the characteristic mass of the cluster depends upon we propose new scaling functions that reproduce the dependence of the characteristic mass found in the simulations

local linear discriminant analysis framework using sample neighbors the linear discriminant analysis lda is a very popular linear feature extraction approach the algorithms of lda usually perform well under the following two assumptions the first assumption is that the global data structure is consistent with the local data structure the second assumption is that the input data classes are gaussian distributions however in real world applications these assumptions are not always satisfied in this paper we propose an improved lda framework the local lda llda which can perform well without needing to satisfy the above two assumptions our llda framework can effectively capture the local structure of samples according to different types of local data structure our llda framework incorporates several different forms of linear feature extraction approaches such as the classical lda and principal component analysis the proposed framework includes two llda algorithms a vector based llda algorithm and a matrix based llda mllda algorithm mllda is directly applicable to image recognition such as face recognition our algorithms need to train only a small portion of the whole training set before testing a sample they are suitable for learning large scale databases especially when the input data dimensions are very high and can achieve high classification accuracy extensive experiments show that the proposed algorithms can obtain good classification results

face recognition using intrinsicfaces in this paper we propose a novel face model called intrinsic face model under this model each face image is divided into three components i e facial commonness difference individuality difference and intrapersonal difference to characterize some certain differences conveyed by this image then a new supervised dimensionality reduction technique coined intrinsic discriminant analysis ida is developed intrinsic discriminant analysis tries to best classify different face images by maximizing the individuality difference while minimizing the intrapersonal difference by using perturbation technique to tackle the singularity problem of ida which occurs frequently in face recognition we obtain a new appearance based face recognition method called intrinsicfaces a series of experiments to compare our proposed approach with other dimensionality reduction methods are tested on three well known face databases experimental results demonstrate the efficacy of the proposed intrinsicfaces approach in face recognition

kernel quadratic discriminant analysis for small sample size problem it is generally believed that quadratic discriminant analysis qda can better fit the data in practical pattern recognition applications compared to linear discriminant analysis lda method this is due to the fact that qda relaxes the assumption made by lda based methods that the covariance matrix for each class is identical however it still assumes that the class conditional distribution is gaussian which is usually not the case in many real world applications in this paper a novel kernel based qda method is proposed to further relax the gaussian assumption by using the kernel machine technique the proposed method solves the complex pattern recognition problem by combining the qda solution and the kernel machine technique and at the same time tackles the so called small sample size problem through a regularized estimation of the covariance matrix extensive experimental results indicate that the proposed method is a more sophisticated solution outperforming many traditional kernel based learning algorithms

inexact implementation using krylov subspace methods for large scale exponential discriminant analysis with applications to high dimensionality reduction problems abstract exponential discriminant analysis eda is a generalized discriminant analysis method based on matrix exponential it can essentially overcome the intrinsic difficulty of small sample size problem that exists in the classical linear discriminant analysis lda however for data with high dimensionality one has to solve a large matrix exponential eigenproblem in this method and the time complexity is dominated by the computation of exponential of large matrices in this paper we propose two inexact krylov subspace algorithms for solving the large matrix exponential eigenproblem efficiently the contribution of this work is threefold first we consider how to compute matrix exponential vector products efficiently which is the key step in the krylov subspace method second we compare the discriminant analysis criterion of eda and that of lda from a theoretical point of view third we establish the relationship between the accuracy of the approximate eigenvectors and the distance to nearest neighbor classifier and show why the matrix exponential eigenproblem can be solved approximately in practice numerical experiments on some real world databases show the superiority of our new algorithms over their original counterpart for face recognition

matrix manifolds and the jordan structure of the bialternate matrix product the bialternate product of matrices was introduced at the end of the 19th century and recently revived as a computational tool in problems where real matrices with conjugate pairs of pure imaginary eigenvalues are important i e in stability theory and hopf bifurcation problems we give a complete description of the jordan structure of the bialternate product 2a in of an n n matrix a thus extending several partial results in the literature we use these results to obtain regular local defining systems for some manifolds of matrices which occur naturally in applications in particular for manifolds with resonant conjugate pairs of pure imaginary eigenvalues such defining systems can be used analytically to obtain local parameterizations of the manifolds or numerically to set up newton systems with local quadratic convergence we give references to explicit numerical applications and implementations in software we expect that the analysis provided in this paper can be used to further improve such implementations

formulating robust linear regression estimation as a one class lda criterion discriminative hat matrix linear discriminant analysis such as fisher s criterion is a statistical learning tool traditionally devoted to separating a training dataset into two or even several classes by the way of linear decision boundaries in this paper we show that this tool can formalize the robust linear regression problem as a robust estimator will do more precisely we develop a one class fischer s criterion in which the maximization provides both the regression parameters and the separation of the data in two classes typical data and atypical data or outliers this new criterion is built on the statistical properties of the subspace decomposition of the hat matrix from this angle we improve the discriminative properties of the hat matrix which is traditionally used as outlier diagnostic measure in linear regression naturally we call this new approach discriminative hat matrix the proposed algorithm is fully nonsupervised and needs only the initialization of one parameter synthetic and real datasets are used to study the performance both in terms of regression and classification of the proposed approach we also illustrate its potential application to image recognition and fundamental matrix estimation in computer vision

enhanced fisher discriminant criterion for image recognition many previous studies have shown that image recognition can be significantly improved by fisher linear discriminant analysis flda technique however flda ignores the variation of data points from the same class which characterizes the most important modes of variability of patterns and helps to improve the generalization capability of flda thus the performance of flda on testing data is not good enough in this paper we propose an enhanced fisher discriminant criterion efdc efdc explicitly considers the intra class variation and incorporates the intra class variation into the fisher discriminant criterion to build a robust and efficient dimensionality reduction function efdc obtains a subspace which best detects the discriminant structure and simultaneously preserves the modes of variability of patterns which will result in stable intraclass representation experimental results on four image database show a clear improvement over the results of flda based methods

a generalized optimal set of discriminant vectors abstract a generalized optimal set of discriminant vectors for linear feature extraction is presented first the criteria of selecting the generalized optimal discriminant vectors are introduced and then a unified solving method is derived to solve the vectors of the generalized optimal set in both cases of a large number of samples and a small number of samples the experimental results show that the present method is superior to the foley sammon method foley and sammon ieee trans comput 24 281 289 1975 the positive pseudoinverse method tian et al opt engng 25 7 834 839 1986 the perturbation method hong and yang pattern recognition 24 317 324 1991 and the matrix rank decomposition method cheng et al pattern recognition 25 101 111 1992 in terms of correct classification rate