pairwise sparsity preserving embedding for unsupervised subspace learning and classification two novel unsupervised dimensionality reduction techniques termed sparse distance preserving embedding sdpe and sparse proximity preserving embedding sppe are proposed for feature extraction and classification sdpe and sppe perform in the clean data space recovered by sparse representation and enhanced euclidean distances over noise removed data are employed to measure pairwise similarities of points in extracting informative features sdpe and sppe aim at preserving pairwise similarities between data points in addition to preserving the sparse characteristics this paper calculates the sparsest representation of all vectors jointly by a convex optimization the sparsest codes enable certain local information of data to be preserved and can endow sdpe and sppe a natural discriminating power adaptive neighborhood and robust characteristic against noise and errors in delivering low dimensional embeddings we also mathematically show sdpe and sppe can be effectively extended for discriminant learning in a supervised manner the validity of sdpe and sppe is examined by extensive simulations comparison with other related state of the art unsupervised algorithms show that promising results are delivered by our techniques

convex inner approximations of nonconvex semialgebraic sets applied to fixed order controller design we describe an elementary algorithm to build convex inner approximations of nonconvex sets both input and output sets are basic semialgebraic sets given as lists of defining multivariate polynomials even though no optimality guarantees can be given e g in terms of volume maximisation for bounded sets the algorithm is designed to preserve convex boundaries as much as possible while removing regions with concave boundaries in particular the algorithm leaves invariant a given convex set the algorithm is based on gloptipoly 3 a public domain matlab package solving nonconvex polynomial optimisation problems with the help of convex semidefinite programming optimisation over linear matrix inequalities or lmis we illustrate how the algorithm can be used to design fixed order controllers for linear systems following a polynomial approach

a uniformization theorem for complete non compact kahler surfaces with positive bisectional curvature in this paper by combining techniques from ricci flow and algebraic geometry we prove the following generalization of the classical uniformization theorem of riemann surfaces given a complete non compact complex two dimensional kahler manifold m of positive and bounded holomorphic bisectional curvature suppose its geodesic balls have maximal volume growth then m is biholomorphic to c2 this gives a partial affirmative answer to the well known conjecture of yau 41 on uniformization theorem during the proof we also verify an interesting gap phenomenon predicted by yau in 42 which says that a kahler manifold as above automatically has quadratic curvature decay at infinity in the average sense

worst case linear discriminant analysis as scalable semidefinite feasibility problems in this paper we propose an efficient semidefinite programming sdp approach to worst case linear discriminant analysis wlda compared with the traditional lda wlda considers the dimensionality reduction problem from the worst case viewpoint which is in general more robust for classification however the original problem of wlda is non convex and difficult to optimize in this paper we reformulate the optimization problem of wlda into a sequence of semidefinite feasibility problems to efficiently solve the semidefinite feasibility problems we design a new scalable optimization method with a quasi newton method and eigen decomposition being the core components the proposed method is orders of magnitude faster than standard interior point sdp solvers experiments on a variety of classification problems demonstrate that our approach achieves better performance than standard lda our method is also much faster and more scalable than standard interior point sdp solvers based wlda the computational complexity for an sdp with m constraints and matrices of size d by d is roughly reduced from mathcal o m 3 md 3 m 2 d 2 to mathcal o d 3 m d in our case

eigenfaces vs fisherfaces recognition using class specific linear projection we develop a face recognition algorithm which is insensitive to large variation in lighting direction and facial expression taking a pattern classification approach we consider each pixel in an image as a coordinate in a high dimensional space we take advantage of the observation that the images of a particular face under varying illumination but fixed pose lie in a 3d linear subspace of the high dimensional image space if the face is a lambertian surface without shadowing however since faces are not truly lambertian surfaces and do indeed produce self shadowing images will deviate from this linear subspace rather than explicitly modeling this deviation we linearly project the image into a subspace in a manner which discounts those regions of the face with large deviation our projection method is based on fisher s linear discriminant and produces well separated classes in a low dimensional subspace even under severe variation in lighting and facial expressions the eigenface technique another method based on linearly projecting the image space to a low dimensional subspace has similar computational requirements yet extensive experimental results demonstrate that the proposed fisherface method has error rates that are lower than those of the eigenface technique for tests on the harvard and yale face databases

simple approximations of semialgebraic sets and their applications to control many uncertainty sets encountered in control systems analysis and design can be expressed in terms of semialgebraic sets that is as the intersection of sets described by means of polynomial inequalities important examples are for instance the solution set of linear matrix inequalities or the schur hurwitz stability domains these sets often have very complicated shapes non convex and even non connected which renders very difficult their manipulation it is therefore of considerable importance to find simple enough approximations of these sets able to capture their main characteristics while maintaining a low level of complexity for these reasons in the past years several convex approximations based for instance on hyperrect angles polytopes or ellipsoids have been proposed in this work we move a step further and propose possibly non convex approximations based on a small volume polynomial superlevel set of a single positive polynomial of given degree we show how these sets can be easily approximated by minimizing the l1 norm of the polynomial over the semialgebraic set subject to positivity constraints intuitively this corresponds to the trace minimization heuristic commonly encounter in minimum volume ellipsoid problems from a computational viewpoint we design a hierarchy of linear matrix inequality problems to generate these approximations and we provide theoretically rigorous convergence results in the sense that the hierarchy of outer approximations converges in volume or equivalently almost everywhere and almost uniformly to the original set two main applications of the proposed approach are considered the first one aims at reconstruction approximation of sets from a finite number of samples in the second one we show how the concept of polynomial superlevel set can be used to generate samples uniformly distributed on a given semialgebraic set the efficiency of the proposed approach is demonstrated by different numerical examples

improved discriminate analysis for high dimensional data and its application to face recognition many pattern recognition applications involve the treatment of high dimensional data and the small sample size problem principal component analysis pca is a common used dimension reduction technique linear discriminate analysis lda is often employed for classification pca plus lda is a famous framework for discriminant analysis in high dimensional space and singular cases in this paper we examine the theory of this framework and find out that even if there is no small sample size problem the pca dimension reduction cannot guarantee the subsequent successful application of lda we thus develop an improved discriminate analysis method by introducing an inverse fisher criterion and adding a constrain in pca procedure so that the singularity phenomenon will not occur experiment results on face recognition suggest that this new approach works well and can be applied even when the number of training samples is one per class

polytope volume computation a combinatorial form of gram s relation for convex polytopes can be adapted for use in computing polytope volume we present an algorithm for volume computation based on this observation this algorithm is useful in finding the volume of a polytope given as the solution set of a system of linear inequalities p x e r ax b as an illustration we compute a formula for the volume of a projective image of the n cube from this formula we deduce that when a and b have rational entries so that the volume of p is also a rational number the number of binary digits in the denominator of the volume cannot be bounded by a polynomial in the total number of digits in the numerators and denominators of entries of a and b this settles a question posed by dyer and frieze

on a problem of yau regarding a higher dimensional generalization of the cohn vossen inequality we show that a problem asked by yau open problems in geometry chern a great geometer of the twentieth century pp 275 319 1992 cannot be true in general the counterexamples are constructed based on the recent work of wu and zheng examples of positively curved complete kahler manifolds geometry and analysis vol 17 pp 517 542 2010

modified principal component analysis an integration of multiple similarity subspace models we modify the conventional principal component analysis pca and propose a novel subspace learning framework modified pca mpca using multiple similarity measurements mpca computes three similarity matrices exploiting the similarity measurements 1 mutual information 2 angle information and 3 gaussian kernel similarity we employ the eigenvectors of similarity matrices to produce new subspaces referred to as similarity subspaces a new integrated similarity subspace is then generated using a novel feature selection approach this approach needs to construct a kind of vector set termed weak machine cell wmc which contains an appropriate number of the eigenvectors spanning the similarity subspaces combining the wrapper method and the forward selection scheme mpca selects a wmc at a time that has a powerful discriminative capability to classify samples mpca is very suitable for the application scenarios in which the number of the training samples is less than the data dimensionality mpca outperforms the other state of the art pca based methods in terms of both classification accuracy and clustering result in addition mpca can be applied to face image reconstruction mpca can use other types of similarity measurements extensive experiments on many popular real world data sets such as face databases show that mpca achieves desirable classification results as well as has a powerful capability to represent data

approximate information discriminant analysis a computationally simple heteroscedastic feature extraction technique in this article we develop a novel linear dimensionality reduction technique for classification the technique utilizes the first two statistical moments of data and retains the computational simplicity characteristic of second order techniques such as linear discriminant analysis formally the technique maximizes a criterion that belongs to the class of probability dependence measures and is naturally defined for multiple classes the criterion is based on an approximation of an information theoretic measure and is capable of handling heteroscedastic data the performance of our method along with similar feature extraction approaches is demonstrated based on experimental results with real world datasets our method compares favorably to similar second order linear dimensionality techniques

on the complexity of computing the volume of a polyhedron we show that computing the volume of a polyhedron given either as a list of facets or as a list of vertices is as hard as computing the permanent of a matrix

probabilistic visual learning for object representation we present an unsupervised technique for visual learning which is based on density estimation in high dimensional spaces using an eigenspace decomposition two types of density estimates are derived for modeling the training data a multivariate gaussian for unimodal distributions and a mixture of gaussians model for multimodal distributions those probability densities are then used to formulate a maximum likelihood estimation framework for visual search and target detection for automatic object recognition and coding our learning technique is applied to the probabilistic visual modeling detection recognition and coding of human faces and nonrigid objects such as hands

comments on globally maximizing locally minimizing unsupervised discriminant projection with application to face and palm biometrics this paper develops an unsupervised discriminant projection udp technique for dimensionality reduction of high dimensional data in small sample size cases udp can be seen as a linear approximation of a multimanifolds based learning framework which takes into account both the local and nonlocal quantities udp characterizes the local scatter as well as the nonlocal scatter seeking to find a projection that simultaneously maximizes the nonlocal scatter and minimizes the local scatter this characteristic makes udp more intuitive and more powerful than the most up to date method locality preserving projection lpp which considers only the local scatter for clustering or classification tasks the proposed method is applied to face and palm biometrics and is examined using the yale feret and ar face image databases and the polyu palmprint database the experimental results show that udp consistently outperforms lpp and pca and outperforms lda when the training sample size per class is small this demonstrates that udp is a good choice for real world biometrics applications

face recognition using scale adaptive directional and textural features a novel approach to face recognition problem using directional and texture information from face images is proposed in this paper in order to capture the directionality specially designed using local polynomial approximation technique scale adaptive digital filters are used for texture features extraction a low dimensional and computationally effective local descriptor is utilized textural and directional features are captured at the holistic and part based levels resulting in a robust face descriptor the proposed method is tested on a number of standard test face datasets orl xm2vts extended yale cmu pie ar and feret for different scenarios and its performance is compared with several state of the art techniques

is gauss quadrature better than clenshaw curtis we compare the convergence behavior of gauss quadrature with that of its younger brother clenshaw curtis seven line matlab codes are presented that implement both methods and experiments show that the supposed factor of 2 advantage of gauss quadrature is rarely realized theorems are given to explain this effect first following o hara and smith in the 1960s the phenomenon is explained as a consequence of aliasing of coefficients in chebyshev expansions then another explanation is offered based on the interpretation of a quadrature formula as a rational approximation of log z 1 z 1 in the complex plane gauss quadrature corresponds to pade approximation at z infty clenshaw curtis quadrature corresponds to an approximation whose order of accuracy at z infty is only half as high but which is nevertheless equally accurate near 1 1

algebraic feature extraction for image recognition based on an optimal discriminant criterion abstract a novel algebraic feature extraction method for image recognition is presented the main difference between the present method and any other existing algebraic feature extraction method is that the present method uses an optimal discriminant criterion to extract algebraic features for the given training image samples a set of optimal discriminant projection vectors is calculated according to a generalized fisher criterion function on the basis of the optimal discriminant projection vector the projective feature vectors of an image i e algebraic features are extracted by projecting the image onto all optimal discriminant projection vectors the method of calculating the set of optimal discriminant projection vectors is discussed a minimum distance classifier is also proposed for classifying projective feature vectors experimental results showed that the present method has good recognition performance an important conclusion about the present method is that the foley sammon optimal set of discriminant vectors is a special case of the set of optimal discriminant projection vectors

information discriminant analysis feature extraction with an information theoretic objective using elementary information theoretic tools we develop a novel technique for linear transformation from the space of observations into a low dimensional feature subspace for the purpose of classification the technique is based on a numerical optimization of an information theoretic objective function which can be computed analytically the advantages of the proposed method over several other techniques are discussed and the conditions under which the method reduces to linear discriminant analysis are given we show that the novel objective function enjoys many of the properties of the mutual information and the bayes error and we give sufficient conditions for the method to be bayes optimal since the objective function is maximized numerically we show how the calculations can be accelerated to yield feasible solutions the performance of the method compares favorably to other linear discriminant based feature extraction methods on a number of simulated and real world data sets

computing volumes of polyhedra in this note we give two simple methods for calculating the volume of any closed bounded polyhedron in r having an orientable boundary which is triangulated into a set of n 1 dimensional simplices the formulas given require only coordinates of the vertices of the polyhedron 1 introduction the purpose of this note is to give two simple methods for calculating the volume of any closed bounded polyhedron p in r having an orientable boundary ap which is triangulated into a set t of n 1 dimensional simplices following hadwiger 2 we define a polyhedron to be the union of pairwise disjoint convex polyhedra each of which is the convex hull of a finite number of points in 1 we have described an algorithm for obtaining a piecewise linear manifold which closely approximates an implicitly defined manifold if p has been given in such a way then the affine pieces of ap are in general easy to triangulate with an inherited orientation for polyhedra p which are determined by a given system of inequalities methods and programs for triangulating p have been given in 5 6 for such polyhedra a triangulation of the boundary is not easily available so our method is inappropriate of course our approach would also be unnecessary for computing the volume of a parallelotope practical applications of the methods given here may be made to the approxima tion of an area bounded by an implicitly defined curve or to approximation of the volume of a solid which is bounded by an implicitly defined surface although some formulas for the volume of convex polyhedra in rn appear in the literature e g 3 these formulas generally require the computation of the n 1 volume of the facets and additionally they involve extra computations of certain distances the volume formulas we give here involve only the coordinates of the vertices of p

random sampling for subspace face recognition subspace face recognition often suffers from two problems 1 the training sample set is small compared with the high dimensional feature vector 2 the performance is sensitive to the subspace dimension instead of pursuing a single optimal subspace we develop an ensemble learning framework based on random sampling on all three key components of a classification system the feature space training samples and subspace parameters fisherface and null space lda n lda are two conventional approaches to address the small sample size problem but in many cases these lda classifiers are overfitted to the training set and discard some useful discriminative information by analyzing different overfitting problems for the two kinds of lda classifiers we use random subspace and bagging to improve them respectively by random sampling on feature vectors and training samples multiple stabilized fisherface and n lda classifiers are constructed and the two groups of complementary classifiers are integrated using a fusion rule so nearly all the discriminative information is preserved in addition we further apply random sampling on parameter selection in order to overcome the difficulty of selecting optimal parameters in our algorithms then we use the developed random sampling framework for the integration of multiple features a robust random sampling face recognition system integrating shape texture and gabor responses is finally constructed

perturbation lda learning the difference between the class empirical mean and its expectation fisher s linear discriminant analysis lda is popular for dimension reduction and extraction of discriminant features in many pattern recognition applications especially biometric learning in deriving the fisher s lda formulation there is an assumption that the class empirical mean is equal to its expectation however this assumption may not be valid in practice in this paper from the perturbation perspective we develop a new algorithm called perturbation lda p lda in which perturbation random vectors are introduced to learn the effect of the difference between the class empirical mean and its expectation in fisher criterion this perturbation learning in fisher criterion would yield new forms of within class and between class covariance matrices integrated with some perturbation factors moreover a method is proposed for estimation of the covariance matrices of perturbation random vectors for practical implementation the proposed p lda is evaluated on both synthetic data sets and real face image data sets experimental results show that p lda outperforms the popular fisher s lda based algorithms in the undersampled case

convex computation of the region of attraction of polynomial control systems we address the long standing problem of computing the region of attraction roa of a target set e g a neighborhood of an equilibrium point of a controlled nonlinear system with polynomial dynamics and semialgebraic state and input constraints we show that the roa can be computed by solving an infinite dimensional convex linear programming lp problem over the space of measures in turn this problem can be solved approximately via a classical converging hierarchy of convex finite dimensional linear matrix inequalities lmis our approach is genuinely primal in the sense that convexity of the problem of computing the roa is an outcome of optimizing directly over system trajectories the dual infinite dimensional lp on nonnegative continuous functions approximated by polynomial sum of squares allows us to generate a hierarchy of semialgebraic outer approximations of the roa at the price of solving a sequence of lmi problems with asymptotically vanishing conservatism this sharply contrasts with the existing literature which follows an exclusively dual lyapunov approach yielding either nonconvex bilinear matrix inequalities or conservative lmi conditions the approach is simple and readily applicable as the outer approximations are the outcome of a single semidefinite program with no additional data required besides the problem description the approach is demonstrated on several numerical examples

multiclass classifiers based on dimension reduction with generalized lda linear discriminant analysis lda has been widely used for dimension reduction of data sets with multiple classes the lda has been recently extended to various generalized lda methods that are applicable regardless of the relative sizes between the data dimension and the number of data items in this paper we propose several multiclass classifiers based on generalized lda glda algorithms taking advantage of the dimension reducing transformation matrix without requiring additional training or parameter optimization a marginal linear discriminant classifier mldc a bayesian linear discriminant classifier bldc and a one dimensional bldc are introduced for multiclass classification our experimental results illustrate that these classifiers produce higher ten fold cross validation accuracy than knn and centroid based classifiers in the reduced dimensional space obtained from glda

wavelet based discriminant analysis for face recognition the linear fisher discriminant analysis lda is a well known and popular statistical method in pattern recognition and classification when applied to face recognition problem the small sample size problem occurs we investigate the nature of this phenomenon and use wavelet transform for dimension reduction moreover we propose a regularized scheme based face recognition system comparisons are made with the tikhonov regularization method and the infinity fisher index method we find out that when the small sample size problem appears optimizing the fisher index does not lead to good results

a computationally efficient scheme for feature extraction with kernel discriminant analysis the kernel discriminant analysis kda an extension of the linear discriminant analysis lda and null space based lda into the kernel space generally provides good pattern recognition pr performance for both small sample size sss and non sss pr problems due to the eigen decomposition technique adopted however the original scheme for the feature extraction with the kda suffers from a high complexity burden in this paper we derive a transformation of the kda into a linear equation problem and propose a novel scheme for the feature extraction with the kda the proposed scheme is shown to provide us with a reduction of complexity without degradation of pr performance in addition to enhance the pr performance further we address the incorporation of regularization into the proposed scheme highlightswe propose a complexity reduced scheme for the kernel discriminant analysis the proposed scheme does not cause degradation of recognition accuracy performance the core is the transformation of the original problem into a linear equation problem we further address the incorporation of regularization into the proposed scheme

complexity reduced scheme for feature extraction with linear discriminant analysis owing to the singularity of the within class scatter linear discriminant analysis lda becomes ill posed for small sample size sss problems null space based lda nlda which is an extension of lda provides good discriminant performances for sss problems yet as the original scheme for the feature extractor fe of nlda suffers from a complexity burden a few modified schemes have since been proposed for complexity reduction in this brief by transforming the problem of finding the fe of nlda into a linear equation problem a novel scheme is derived offering a further reduction of the complexity

graph embedding and extensions a general framework for dimensionality reduction a large family of algorithms supervised or unsupervised stemming from statistics or geometry theory has been designed to provide different solutions to the problem of dimensionality reduction despite the different motivations of these algorithms we present in this paper a general formulation known as graph embedding to unify them within a common framework in graph embedding each algorithm can be considered as the direct graph embedding or its linear kernel tensor extension of a specific intrinsic graph that describes certain desired statistical or geometric properties of a data set with constraints from scale normalization or a penalty graph that characterizes a statistical or geometric property that should be avoided furthermore the graph embedding framework can be used as a general platform for developing new dimensionality reduction algorithms by utilizing this framework as a tool we propose a new supervised dimensionality reduction algorithm called marginal fisher analysis in which the intrinsic graph characterizes the intraclass compactness and connects each data point with its neighboring points of the same class while the penalty graph connects the marginal points and characterizes the interclass separability we show that mfa effectively overcomes the limitations of the traditional linear discriminant analysis algorithm due to data distribution assumptions and available projection directions real face recognition experiments show the superiority of our proposed mfa in comparison to lda also for corresponding kernel and tensor extensions

fisher regularized support vector machine this paper proposes fisher regularized support vector machine fishersvm fishersvm is a graph based supervised learning method fishersvm has two regularization terms maximum margin and fisher regularizations fishersvm aims to maximize the margin and minimize the within class scatter support vector machine svm and fisher discriminant analysis fda are two commonly used methods in machine learning and pattern recognition a combined method of the linear svm and fda called svm lda linear discriminant analysis has been proposed only for the linear case this paper generalizes this combined method to the nonlinear case from the view of regularization a fisher regularization is defined and incorporated into svm to obtain a fisher regularized support vector machine fishersvm in fishersvm there are two regularization terms the maximum margin regularization and fisher regularization which allow fishersvm to maximize the classification margin and minimize the within class scatter roughly speaking fishersvm can approximatively fulfill the fisher criterion and obtain good statistical separability this paper also discusses the connections of fishersvm to graph based regularization methods and the mathematical programming method to kernel fisher discriminant analysis kfda it shows that fishersvm can be thought of as a graph based supervised learning method or a robust kfda experimental results on artificial and real world data show that fishersvm has a promising generalization performance

feature space locality constraint for kernel based nonlinear discriminant analysis subspace learning is an important approach in pattern recognition nonlinear discriminant analysis nda due to its capability of describing nonlinear manifold structure of samples is considered to be more powerful to undertake classification tasks in image related problems in kernel based nda representation there are three spaces involved i e original data space implicitly mapped high dimension feature space and the target low dimension subspace existing methods mainly focus on the information in original data space to find the most discriminant low dimension subspace the implicit high dimension feature space plays a role that connects the original space and the target subspace to realize the nonlinear dimension reduction but the sample geometric structure information in feature space is not involved in this work we try to utilize and explore this information specifically the locality information of samples in feature space is modeled and integrated into the traditional kernel based nda methods in this way both the sample distributions in original data space and the mapped high dimension feature space are modeled and more information is expected to be explored to improve the discriminative ability of the subspace two algorithms named fslc kda and fslc ksr are presented extensive experiments on orl extended yaleb pie multi pie and frgc databases validate the efficacy of the proposed method

inner approximations for polynomial matrix inequalities and robust stability regions following a polynomial approach many robust fixed order controller design problems can be formulated as optimization problems whose set of feasible solutions is modeled by parametrized polynomial matrix inequalities pmis these feasibility sets are typically nonconvex given a parametrized pmi set we provide a hierarchy of linear matrix inequality lmi problems whose optimal solutions generate inner approximations modeled by a single polynomial superlevel set those inner approximations converge in a well defined analytic sense to the nonconvex original feasible set with asymptotically vanishing conservatism one may also impose the hierarchy of inner approximations to be nested or convex in the latter case they do not converge any more to the feasible set but they can be used in a convex optimization framework at the price of some conservatism finally we show that the specific geometry of nonconvex polynomial stability regions can be exploited to improve convergence of the hierarchy of inner approximations

optimization of polynomials on compact semialgebraic sets a basic closed semialgebraic subset s of r n is defined by simultaneous polynomial inequalities g 1 ge 0 dotsc g m ge 0 we give a short introduction to lasserre s method for minimizing a polynomial f on a compact set s of this kind it consists of successively solving tighter and tighter convex relaxations of this problem which can be formulated as semidefinite programs we give a new short proof for the convergence of the optimal values of these relaxations to the infimum f ast of f on s which is constructive and elementary in the case where f possesses a unique minimizer x ast we prove that every sequence of nearly optimal solutions of the successive relaxations gives rise to a sequence of points in r n converging to x ast

uncorrelated linear discriminant analysis based on weighted pairwise fisher criterion in this paper we propose a novel uncorrelated weighted linear discriminant analysis uwlda method for feature extraction and recognition the uwlda first introduces a weighting function to restrain the dominant role of the classes with larger distance and then searches the optimal discriminant vectors under the conjugative orthogonal constrains in the null space of the within class scatter matrix and its conjugative orthogonal complement space respectively as a result the proposed technique not only derive the optimal and lossless discriminative information but also guarantee that all extracted features are statistically uncorrelated experiments on feret face database and ar face database are performed to test and evaluate the proposed algorithm the results demonstrate the effectiveness of uwlda

scatter balance an angle based supervised dimensionality reduction subspace selection is widely applied in data classification clustering and visualization the samples projected into subspace can be processed efficiently in this paper we research the linear discriminant analysis lda and maximum margin criterion mmc algorithms intensively and analyze the effects of scatters to subspace selection meanwhile we point out the boundaries of scatters in lda and mmc algorithms to illustrate the differences and similarities of subspace selection in different circumstances besides the effects of outlier classes on subspace selection are also analyzed according to the above analysis we propose a new subspace selection method called angle linear discriminant embedding alde on the basis of angle measurement alde utilizes the cosine of the angle to get new within class and between class scatter matrices and avoids the small sample size problem simultaneously to deal with high dimensional data we extend alde to a two stage alde ts alde the synthetic data experiments indicate that alde can balance the within class and between class scatters and be robust to outlier classes the experimental results based on uci machine learning repository and image databases show that ts alde has a lower time complexity than alde while processing high dimensional data

optimal discriminant plane for a small number of samples and design method of classifier on the plane abstract in a previous work zi quan hong and jing yu yang minimum distance classifier on the optimal discriminant plane we suggested and derived a method for constructing classifier on the optimal discriminant plane using minimum distance criteria in this paper the problem of solving optimal discriminant plane for a small number of samples is discussed which is based on the above paper by the same authors in the case of a small number of samples generalized eigenequation ax bx established for a large number of samples usually has no solution because within class scatter matrix is singular to obtain the solution of the generalized eigenequation a new method is suggested in which the singular value perturbation is added to the within class scatter matrix such that the matrix becomes a nonsingular matrix therefore the generalized eigenequation can be solved using existing algorithms we proved that the generalized eigenequations on the optimal discriminant plane are stable in respect of eigenvalues and the generalized eigenvectors are indeed the optimal discriminant directions if the perturbation is subject to some certain conditions the experimental results have shown that our method works well and the minimum distance classifier on the optimal discriminant plane can be constructed with high performance even in the case of a small number of samples

nonlinear component analysis as a kernel eigenvalue problem a new method for performing a nonlinear form of principal component analysis is proposed by the use of integral operator kernel functions one can efficiently compute principal components in high dimensional feature spaces related to input space by some nonlinear map for instance the space of all possible five pixel products in 16 16 images we give the derivation of the method and present experimental results on polynomial feature extraction for pattern recognition

face recognition using discriminant locality preserving projections based on maximum margin criterion in this paper we propose a new discriminant locality preserving projections based on maximum margin criterion dlpp mmc dlpp mmc seeks to maximize the difference rather than the ratio between the locality preserving between class scatter and locality preserving within class scatter dlpp mmc is theoretically elegant and can derive its discriminant vectors from both the range of the locality preserving between class scatter and the range space of locality preserving within class scatter dlpp mmc can also derive its discriminant vectors from the null space of locality preserving within class scatter when the parameter of dlpp mmc approaches experiments on the orl yale feret and pie face databases show the effectiveness of the proposed dlpp mmc

trace ratio problem revisited dimensionality reduction is an important issue in many machine learning and pattern recognition applications and the trace ratio tr problem is an optimization problem involved in many dimensionality reduction algorithms conventionally the solution is approximated via generalized eigenvalue decomposition due to the difficulty of the original problem however prior works have indicated that it is more reasonable to solve it directly than via the conventional way in this brief we propose a theoretical overview of the global optimum solution to the tr problem via the equivalent trace difference problem eigenvalue perturbation theory is introduced to derive an efficient algorithm based on the newton raphson method theoretical issues on the convergence and efficiency of our algorithm compared with prior literature are proposed and are further supported by extensive empirical results

an efficient discriminant based solution for small sample size problem classification of high dimensional statistical data is usually not amenable to standard pattern recognition techniques because of an underlying small sample size problem to address the problem of high dimensional data classification in the face of a limited number of samples a novel principal component analysis pca based feature extraction classification scheme is proposed the proposed method yields a piecewise linear feature subspace and is particularly well suited to difficult recognition problems where achievable classification rates are intrinsically low such problems are often encountered in cases where classes are highly overlapped or in cases where a prominent curvature in data renders a projection onto a single linear subspace inadequate the proposed feature extraction classification method uses class dependent pca in conjunction with linear discriminant feature extraction and performs well on a variety of real world datasets ranging from digit recognition to classification of high dimensional bioinformatics and brain imaging data

optimal fisher discriminant analysis using the rank decomposition abstract the fisher optimal discriminant vector is a very efficient means in high dimensional pattern analysis in the case of a small number of samples the within class scatter matrix s w is singular therefore the calculation of the fisher optimal discriminant vector becomes very important in this paper the conception of the rank decomposition of matrices is first introduced and a new method for calculating the fisher optimal discriminant vector is presented which is particularly well suited to the case of a small number of samples in the sense that the scatter matrices are rank deficient the new method is compared with the pseudoinverse and perturbation methods in tian j opt soc am a 5 1670 1672 1988 and hong and yang pattern recognition 24 317 324 1991 an important conclusion is proved in the three methods our method is the best one for calculating the fisher optimal discriminant vectors in the cases of both a large and a small number of samples the experimental results have also shown that our method can give the highest recognition rate in the case of a small number of samples among the three methods

locally linear discriminant embedding an efficient method for face recognition in this paper an efficient feature extraction method named as locally linear discriminant embedding llde is proposed for face recognition it is well known that a point can be linearly reconstructed by its neighbors and the reconstruction weights are under the sum to one constraint in the classical locally linear embedding lle so the constrained weights obey an important symmetry for any particular data point they are invariant to rotations rescalings and translations the latter two are introduced to the proposed method to strengthen the classification ability of the original lle the data with different class labels are translated by the corresponding vectors and those belonging to the same class are translated by the same vector in order to cluster the data with the same label closer they are also rescaled to some extent so after translation and rescaling the discriminability of the data will be improved significantly the proposed method is compared with some related feature extraction methods such as maximum margin criterion mmc as well as other supervised manifold learning based approaches for example ensemble unified lle and linear discriminant analysis en ullelda locally linear discriminant analysis llda experimental results on yale and cmu pie face databases convince us that the proposed method provides a better representation of the class information and obtains much higher recognition accuracies

ga based optimal selection of pzmi features for face recognition one of the key problems in automated face recognition system is that of handling the face image variation in terms of scale rotation in plane and translation one approach is fixing mentioned problems in recognition processes by extracting one linear transformation invariant feature this paper presents a novel method for face recognition pseudo zernike moment invariant pzmi which has linear transformation invariance properties and is robust in the presence of noise utilized to produce feature vectors for decreasing computational complexity of feature extraction step we use genetic algorithm ga to select the optimal feature set which contains optimal pzmi orders and corresponding repetitions in addition we have investigated the effect of pzmi orders on recognition rate in noisy images proposed scheme has been tested on the feret database experimental results prove the advantages of the proposed method when compared with other pzmi based face recognition systems

a comparative study of feature extraction methods and their application to p rbf nns in face recognition problem this study is concerned with a design of a face recognition algorithm realized based on feature extraction with the aid of the 2 directional 2 dimensional linear discriminant analysis referred to as 2d 2lda the 2dlda algorithm yields high accuracy but comes with an unresolved problem of handling large feature matrices present in face recognition problems the proposed 2d 2lda algorithm is computationally more efficient and produces more powerful discrimination decision the proposed p rbf nns is used as a recognition module the architecture of this module consists of three functional components the coefficients of the p rbf nns model are obtained by fuzzy inference method forming the inference part of fuzzy rules the essential design parameters including learning rate momentum fuzzification coefficient and the feature selection mechanism of the networks are optimized by means of differential evolution de the experimental results reported for benchmark face datasets the yale orl database and ic ci dataset demonstrate the effectiveness and efficiency of 2d 2lda algorithm compared with other pre processing approaches such as lpp 2d pca 2d lpp sr pca 2d 2pca and fusion of pca lda the experimental results show that 2d 2lda based p rbf nns achieves higher performance than being reported by other methods

convex computation of the maximum controlled invariant set for polynomial control systems we characterize the maximum controlled invariant mci set for discrete as well as continuous time nonlinear dynamical systems as the solution of an infinite dimensional linear programming problem for systems with polynomial dynamics and compact semialgebraic state and control constraints we describe a hierarchy of finite dimensional linear matrix inequality lmi relaxations whose optimal values converge to the volume of the mci set dual to these lmi relaxations are sum of squares sos problems providing a converging sequence of outer approximations to the mci set the approach is simple and readily applicable in the sense that the approximations are the outcome of a single semidefinite program with no additional input apart from the problem description a number of numerical examples illustrate the approach

approximate volume and integration for basic semialgebraic sets given a basic compact semialgebraic set mathbf k subset mathbb r n we introduce a methodology that generates a sequence converging to the volume of mathbf k this sequence is obtained from optimal values of a hierarchy of either semidefinite or linear programs not only the volume but also every finite vector of moments of the probability measure that is uniformly distributed on mathbf k can be approximated as closely as desired which permits the approximation of the integral on mathbf k of any given polynomial the extension to integration against some weight functions is also provided finally some numerical issues associated with the algorithms involved are briefly discussed

plurisubharmonic functions and the structure of complete kahler manifolds with nonnegative curvature in this paper we study global properties of continuous plurisubharmonic functions on complete noncompact kahler manifolds with nonnegative bisectional curvature and their applications to the structure of such manifolds we prove that continuous plurisubharmonic functions with reasonable growth rate on such manifolds can be approximated by smooth plurisubharmonic functions through the heat flow deformation optimal liouville type theorem for the plurisubharmonic functions as well as a splitting theorem in terms of harmonic functions and holomorphic functions are established the results are then applied to prove several structure theorems on complete noncompact kahler manifolds with nonnegative bisectional or sectional curvature

face recognition using kernel direct discriminant analysis algorithms techniques that can introduce low dimensional feature representation with enhanced discriminatory power is of paramount importance in face recognition fr systems it is well known that the distribution of face images under a perceivable variation in viewpoint illumination or facial expression is highly nonlinear and complex it is therefore not surprising that linear techniques such as those based on principle component analysis pca or linear discriminant analysis lda cannot provide reliable and robust solutions to those fr problems with complex face variations in this paper we propose a kernel machine based discriminant analysis method which deals with the nonlinearity of the face patterns distribution the proposed method also effectively solves the so called small sample size sss problem which exists in most fr tasks the new algorithm has been tested in terms of classification error rate performance on the multiview umist face database results indicate that the proposed methodology is able to achieve excellent performance with only a very small set of features being used and its error rate is approximately 34 and 48 of those of two other commonly used kernel fr approaches the kernel pca kpca and the generalized discriminant analysis gda respectively

hyperplane navigation a method to set individual scores in fmri group datasets abstract recent studies have demonstrated that spatial patterns of fmri bold activity distribution over the brain may be used to classify different groups or mental states these studies are based on the application of advanced pattern recognition approaches and multivariate statistical classifiers most published articles in this field are focused on improving the accuracy rates and many approaches have been proposed to accomplish this task nevertheless a point inherent to most machine learning methods and still relatively unexplored in neuroimaging is how the discriminative information can be used to characterize groups and their differences in this work we introduce the maximum uncertainty linear discrimination analysis mlda and show how it can be applied to infer groups patterns by discriminant hyperplane navigation in addition we show that it naturally defines a behavioral score i e an index quantifying the distance between the states of a subject from predefined groups we validate and illustrate this approach using a motor block design fmri experiment data with 35 subjects

the moment problem on compact semi algebraic sets in this chapter we begin the study of the multidimensional moment problem the passage to dimensions d 2 brings new difficulties and unexpected phenomena in sect 3 2 we derived solvability criteria of the moment problem on intervals in terms of positivity conditions it seems to be natural to look for similar characterizations in higher dimensions as well this leads us immediately into the realm of real algebraic geometry and to descriptions of positive polynomials on semi algebraic sets in this chapter we treat this approach for basic closed compact semi algebraic subsets of mathbb r d it turns out that for such sets there is a close interaction between the moment problem and positivstellensatze for strictly positive polynomials

dimensionality reduction of multimodal labeled data by local fisher discriminant analysis reducing the dimensionality of data without losing intrinsic information is an important preprocessing step in high dimensional data analysis fisher discriminant analysis fda is a traditional technique for supervised dimensionality reduction but it tends to give undesired results if samples in a class are multimodal an unsupervised dimensionality reduction method called locality preserving projection lpp can work well with multimodal data due to its locality preserving property however since lpp does not take the label information into account it is not necessarily useful in supervised learning scenarios in this paper we propose a new linear supervised dimensionality reduction method called local fisher discriminant analysis lfda which effectively combines the ideas of fda and lpp lfda has an analytic form of the embedding transformation and the solution can be easily computed just by solving a generalized eigenvalue problem we demonstrate the practical usefulness and high scalability of the lfda method in data visualization and classification tasks through extensive simulation studies we also show that lfda can be extended to non linear dimensionality reduction scenarios by applying the kernel trick

face recognition using laplacianfaces we propose an appearance based face recognition method called the laplacianface approach by using locality preserving projections lpp the face images are mapped into a face subspace for analysis different from principal component analysis pca and linear discriminant analysis lda which effectively see only the euclidean structure of face space lpp finds an embedding that preserves local information and obtains a face subspace that best detects the essential face manifold structure the laplacianfaces are the optimal linear approximations to the eigenfunctions of the laplace beltrami operator on the face manifold in this way the unwanted variations resulting from changes in lighting facial expression and pose may be eliminated or reduced theoretical analysis shows that pca lda and lpp can be obtained from different graph models we compare the proposed laplacianface approach with eigenface and fisherface methods on three different face data sets experimental results suggest that the proposed laplacianface approach provides a better representation and achieves lower error rates in face recognition

image covariance based subspace method for face recognition this paper proposes a new subspace method that is based on image covariance obtained from windowed features of images a windowed input feature consists of a number of pixels and the dimension of input space is determined by the number of windowed features each element of an image covariance matrix can be obtained from the inner product of two windowed features the 2d pca and 2d lda methods are then obtained from principal component analysis and linear discriminant analysis respectively using the image covariance matrix in the case of 2d lda there is no need for pca preprocessing and the dimension of subspace can be greater than the number of classes because the within class and between class image covariance matrices have full ranks comparative experiments are performed using the feret cmu and orl databases of facial images the experimental results show that the proposed 2d lda provides the best recognition rate among several subspace methods in all of the tests

classifiability based discriminatory projection pursuit fisher s linear discriminant fld is one of the most widely used linear feature extraction method especially in many visual computation tasks based on the analysis on several limitations of the traditional fld this paper attempts to propose a new computational paradigm for discriminative linear feature extraction named classifiability based discriminatory projection pursuit cdpp which is different from the traditional fld and its variants there are two steps in the proposed cdpp one is the construction of a candidate projection set cps and the other is the pursuit of discriminatory projections specifically in the former step candidate projections are generated by using the nearest between class boundary samples while the latter is efficiently achieved by classifiability based adaboost learning from the cps we show that the new projection pursuit paradigm not only does not suffer from the limitations of the traditional fld but also inherits good generalizability from the boundary attribute of candidate projections extensive experiments on both synthetic and real datasets validate the effectiveness of cdpp for discriminative linear feature extraction

a random polynomial time algorithm for approximating the volume of convex bodies a randomized polynomial time algorithm for approximating the volume of a convex body k in n dimensional euclidean space is presented the proof of correctness of the algorithm relies on recent theory of rapidly mixing markov chains and isoperimetric inequalities to show that a certain random walk can be used to sample nearly uniformly from within k

efficient monte carlo procedures for generating points uniformly distributed over bounded regions we consider the monte carlo problem of generating points uniformly distributed within an arbitrary bounded measurable region the class of markovian methods considered generate points asymptotically uniformly distributed within the region computational experience suggests the methods are potentially superior to conventional rejection techniques for large dimensional regions

a new lda based face recognition system which can solve the small sample size problem a new lda based face recognition system is presented in this paper linear discriminant analysis lda is one of the most popular linear projection techniques for feature extraction the major drawback of applying lda is that it may encounter the small sample size problem in this paper we propose a new lda based technique which can solve the small sample size problem we also prove that the most expressive vectors derived in the null space of the within class scatter matrix using principal component analysis pca are equal to the optimal discriminant vectors derived in the original space using lda the experimental results show that the new lda process improves the performance of a face recognition system signi cantly 2000 pattern recognition society published by elsevier science ltd all rights reserved

fast adaptive algorithms and networks for class separability features abstract in this article we introduce accelerated algorithms for linear discriminant analysis lda and feature extraction from unimodal multiclass gaussian data current adaptive methods based on the gradient descent optimization technique use a fixed or a monotonically decreasing step size in each iteration which results in a slow convergence rate here we use a variable step size optimally computed in each iteration using the steepest descent method in order to accelerate the convergence of the algorithm based on the new adaptive algorithm we present a self organizing neural network for adaptive computation of the square root of the inverse covariance matrix 1 2 and use it i in a network for optimal feature extraction from gaussian data and ii in cascaded form with a principal component analysis network for lda experimental results demonstrate fast convergence and high stability of the algorithm and justify its advantages for on line pattern recognition applications with stationary and non stationary input data

novel fisher discriminant classifiers at the present several applications need to classify high dimensional points belonging to highly unbalanced classes unfortunately when the training set cardinality is small compared to the data dimensionality small sample size problem the classification performance of several well known classifiers strongly decreases similarly the classification accuracy of several discriminative methods decreases when non linearly separable and unbalanced classes are treated in this paper we firstly survey state of the art methods that employ improved versions of linear discriminant analysis lda to deal with the above mentioned problems secondly we propose a family of classifiers based on the fisher subspace estimation which efficiently deal with the small sample size problem non linearly separable classes and unbalanced classes the promising results obtained by the proposed techniques on benchmark datasets and the comparison with state of the art predictors show the efficacy of the proposed techniques

a monotonicity formula on complete kahler manifolds with nonnegative bisectional curvature in y yau proposed to study the uniformization of complete kahler manifolds with nonnegative curvature in particular one wishes to determine whether or not a complete kahler manifold m with positive bisectional curvature is biholomorphic to cm see also gw si for this sake it was further asked in y whether or not the ring of the holomorphic functions with polynomial growth which we denote by op m is finitely generated and whether or not the dimension of the spaces of holomorphic functions of polynomial growth is bounded from above by the dimension of the corresponding spaces of polynomials on cm this paper addresses the latter questions we denote by 0 m the space of holomorphic functions of polynomial growth with degree d see section 3 for the precise definition then op m jd 0 od m in this paper we show that

regularized class specific subspace classifier in this paper we mainly focus on how to achieve the translated subspace representation for each class which could simultaneously indicate the distribution of the associated class and the differences from its complementary classes by virtue of the reconstruction problem the class specific subspace classifier cssc problem could be represented as a series of biobjective optimization problems which minimize and maximize the reconstruction errors of the related class and its complementary classes respectively besides the regularization term is specifically introduced to ensure the whole system s stability accordingly a regularized class specific subspace classifier rcssc method can be further proposed based on solving a general quadratic ratio problem the proposed rcssc method consistently converges to the global optimal subspace and translation under the variations of the regularization parameter furthermore the proposed rcssc method could be extended to the unregularized case which is known as unregularized cssc ucssc method via orthogonal decomposition technique as a result the effectiveness and the superiority of both proposed rcssc and ucssc methods can be verified analytically and experimentally

global optimization with polynomials and the problem of moments we consider the problem of finding the unconstrained global minimum of a real valued polynomial p x mathbb r n to mathbb r as well as the global minimum of p x in a compact set k defined by polynomial inequalities it is shown that this problem reduces to solving an often finite sequence of convex linear matrix inequality lmi problems a notion of karush kuhn tucker polynomials is introduced in a global optimality condition some illustrative examples are provided

efficient and robust feature extraction by maximum margin criterion in pattern recognition feature extraction techniques are widely employed to reduce the dimensionality of data and to enhance the discriminatory information principal component analysis pca and linear discriminant analysis lda are the two most popular linear dimensionality reduction methods however pca is not very effective for the extraction of the most discriminant features and lda is not stable due to the small sample size problem in this paper we propose some new linear and nonlinear feature extractors based on maximum margin criterion mmc geometrically feature extractors based on mmc maximize the average margin between classes after dimensionality reduction it is shown that mmc can represent class separability better than pca as a connection to lda we may also derive lda from mmc by incorporating some constraints by using some other constraints we establish a new linear feature extractor that does not suffer from the small sample size problem which is known to cause serious stability problems for lda the kernelized nonlinear counterpart of this linear feature extractor is also established in the paper our extensive experiments demonstrate that the new feature extractors are effective stable and efficient

face recognition using kernel scatter difference based discriminant analysis there are two fundamental problems with the fisher linear discriminant analysis for face recognition one is the singularity problem of the within class scatter matrix due to small training sample size the other is that it cannot efficiently describe complex nonlinear variations of face images because of its linear property in this letter a kernel scatter difference based discriminant analysis is proposed to overcome these two problems we first use the nonlinear kernel trick to map the input data into an implicit feature space f then a scatter difference based discriminant rule is defined to analyze the data in f the proposed method can not only produce nonlinear discriminant features but also avoid the singularity problem of the within class scatter matrix extensive experiments show encouraging recognition performance of the new algorithm

on complete noncompact kahler manifolds with positive bisectional curvature we prove that a complete noncompact kahler manifold m n of positive bisectional curvature satisfying suitable growth conditions is biholomorphic to a pseudoconvex domain of c n and we show that the manifold is topologically r 2 n in particular when m n is a kahler surface of positive bisectional curvature satisfying certain natural geometric growth conditions it is biholomorphic to c 2

kernel quadratic discriminant analysis for small sample size problem it is generally believed that quadratic discriminant analysis qda can better fit the data in practical pattern recognition applications compared to linear discriminant analysis lda method this is due to the fact that qda relaxes the assumption made by lda based methods that the covariance matrix for each class is identical however it still assumes that the class conditional distribution is gaussian which is usually not the case in many real world applications in this paper a novel kernel based qda method is proposed to further relax the gaussian assumption by using the kernel machine technique the proposed method solves the complex pattern recognition problem by combining the qda solution and the kernel machine technique and at the same time tackles the so called small sample size problem through a regularized estimation of the covariance matrix extensive experimental results indicate that the proposed method is a more sophisticated solution outperforming many traditional kernel based learning algorithms

an extension of matlab to continuous functions and operators an object oriented matlab system is described for performing numerical linear algebra on continuous functions and operators rather than the usual discrete vectors and matrices about eighty matlab functions from plot and sum to svd and cond have been overloaded so that one can work with our chebfun objects using almost exactly the usual matlab syntax all functions live on 1 1 and are represented by values at sufficiently many chebyshev points for the polynomial interpolant to be accurate to close to machine precision each of our overloaded operations raises questions about the proper generalization of familiar notions to the continuous context and about appropriate methods of interpolation differentiation integration zerofinding or transforms applications in approximation theory and numerical analysis are explored and possible extensions for more substantial problems of scientific computing are mentioned

inexact implementation using krylov subspace methods for large scale exponential discriminant analysis with applications to high dimensionality reduction problems abstract exponential discriminant analysis eda is a generalized discriminant analysis method based on matrix exponential it can essentially overcome the intrinsic difficulty of small sample size problem that exists in the classical linear discriminant analysis lda however for data with high dimensionality one has to solve a large matrix exponential eigenproblem in this method and the time complexity is dominated by the computation of exponential of large matrices in this paper we propose two inexact krylov subspace algorithms for solving the large matrix exponential eigenproblem efficiently the contribution of this work is threefold first we consider how to compute matrix exponential vector products efficiently which is the key step in the krylov subspace method second we compare the discriminant analysis criterion of eda and that of lda from a theoretical point of view third we establish the relationship between the accuracy of the approximate eigenvectors and the distance to nearest neighbor classifier and show why the matrix exponential eigenproblem can be solved approximately in practice numerical experiments on some real world databases show the superiority of our new algorithms over their original counterpart for face recognition

a laplace transform algorithm for the volume of a convex polytope we provide two algorithms for computing the volume of the convex polytope x n ax b for a m n b n the computational complexity of both algorithms is essentially described by nm which makes them especially attractive for large n and relatively small m when the other methods with o mn complexity fail the methodology which differs from previous existing methods uses a laplace transform technique that is well suited to the half space representation of

hit and run algorithms for generating multivariate distributions we introduce a general class of hit and run algorithms for generating essentially arbitrary absolutely continuous distributions on rd they include the hypersphere directions algorithm and the coordinate directions algorithm that have been proposed for identifying nonredundant linear constraints and for generating uniform distributions over subsets of rd given a bounded open set s in rd an absolutely continuous probability distribution on s the target distribution and an arbitrary probability distribution i on the boundary of the d dimensional unit sphere centered at the origin the direction distribution the i hit and run algorithm produces a sequence of iteration points as follows given the nth iteration point x choose a direction i according to the distribution i and then choose the n 1st iteration point according to the conditionalization of the distribution along the line defined by x and x i under mild conditions we show that this sequence of points is a harris recurrent reversible markov chain converging in total variation to the target distribution

semidefinite programming vs lp relaxations for polynomial programming we consider the global minimization of a multivariate polynomial on a semi algebraic set o defined with polynomial inequalities we then compare two hierarchies of relaxations namely lp relaxations based on products of the original constraints in the spirit of the rlt procedure of sherali and adams 1990 and recent semidefinite programming sdp relaxations introduced by the author the comparison is analyzed in light of recent results in real algebraic geometry on various representations of polynomials positive on a compact semi algebraic set

formulating robust linear regression estimation as a one class lda criterion discriminative hat matrix linear discriminant analysis such as fisher s criterion is a statistical learning tool traditionally devoted to separating a training dataset into two or even several classes by the way of linear decision boundaries in this paper we show that this tool can formalize the robust linear regression problem as a robust estimator will do more precisely we develop a one class fischer s criterion in which the maximization provides both the regression parameters and the separation of the data in two classes typical data and atypical data or outliers this new criterion is built on the statistical properties of the subspace decomposition of the hat matrix from this angle we improve the discriminative properties of the hat matrix which is traditionally used as outlier diagnostic measure in linear regression naturally we call this new approach discriminative hat matrix the proposed algorithm is fully nonsupervised and needs only the initialization of one parameter synthetic and real datasets are used to study the performance both in terms of regression and classification of the proposed approach we also illustrate its potential application to image recognition and fundamental matrix estimation in computer vision

enhanced fisher discriminant criterion for image recognition many previous studies have shown that image recognition can be significantly improved by fisher linear discriminant analysis flda technique however flda ignores the variation of data points from the same class which characterizes the most important modes of variability of patterns and helps to improve the generalization capability of flda thus the performance of flda on testing data is not good enough in this paper we propose an enhanced fisher discriminant criterion efdc efdc explicitly considers the intra class variation and incorporates the intra class variation into the fisher discriminant criterion to build a robust and efficient dimensionality reduction function efdc obtains a subspace which best detects the discriminant structure and simultaneously preserves the modes of variability of patterns which will result in stable intraclass representation experimental results on four image database show a clear improvement over the results of flda based methods

a semidefinite programming approach to the generalized problem of moments we consider the generalized problem of moments gpm from a computational point of view and provide a hierarchy of semidefinite programming relaxations whose sequence of optimal values converges to the optimal value of the gpm we then investigate in detail various examples of applications in optimization probability financial economics and optimal control which all can be viewed as particular instances of the gpm

discriminative common vector method with kernels in some pattern recognition tasks the dimension of the sample space is larger than the number of samples in the training set this is known as the small sample size problem linear discriminant analysis lda techniques cannot be applied directly to the small sample size case the small sample size problem is also encountered when kernel approaches are used for recognition in this paper we attempt to answer the question of how should one choose the optimal projection vectors for feature extraction in the small sample size case based on our findings we propose a new method called the kernel discriminative common vector method in this method we first nonlinearly map the original input space to an implicit higher dimensional feature space in which the data are hoped to be linearly separable then the optimal projection vectors are computed in this transformed space the proposed method yields an optimal solution for maximizing a modified fisher s linear discriminant criterion discussed in the paper thus under certain conditions a 100 recognition rate is guaranteed for the training set samples experiments on test data also show that in many situations the generalization performance of the proposed method compares favorably with other kernel approaches

convergence rates of moment sum of squares hierarchies for optimal control problems we study the convergence rate of moment sum of squares hierarchies of semidefinite programs for optimal control problems with polynomial data it is known that these hierarchies generate polynomial under approximations to the value function of the optimal control problem and that these under approximations converge in the l1 norm to the value function as their degree d tends to infinity we show that the rate of this convergence is o 1 log log d we treat in detail the continuous time infinite horizon discounted problem and describe in brief how the same rate can be obtained for the finite horizon continuous time problem and for the discrete time counterparts of both problems

a generalized optimal set of discriminant vectors abstract a generalized optimal set of discriminant vectors for linear feature extraction is presented first the criteria of selecting the generalized optimal discriminant vectors are introduced and then a unified solving method is derived to solve the vectors of the generalized optimal set in both cases of a large number of samples and a small number of samples the experimental results show that the present method is superior to the foley sammon method foley and sammon ieee trans comput 24 281 289 1975 the positive pseudoinverse method tian et al opt engng 25 7 834 839 1986 the perturbation method hong and yang pattern recognition 24 317 324 1991 and the matrix rank decomposition method cheng et al pattern recognition 25 101 111 1992 in terms of correct classification rate