a multivariate two sample test based on the number of nearest neighbor type coincidences for independent d variate random samples x 1 cdots x n 1 i i d f x y 1 cdots y n 2 i i d g x where the densities f and g are assumed to be continuous a e consider the number t of all k nearest neighbor comparisons in which observations and their neighbors belong to the same sample we show that if f g a e the limiting normal distribution of t as min n 1 n 2 rightarrow infty n 1 n 1 n 2 rightarrow tau 0 tau 1 does not depend on f an omnibus procedure for testing the hypothesis h 0 f g a e is obtained by rejecting h 0 for large values of t the result applies to a general distance generated by a norm on mathbb r d for determining nearest neighbors and it generalizes to the multisample situation

eigenfaces vs fisherfaces recognition using class specific linear projection we develop a face recognition algorithm which is insensitive to large variation in lighting direction and facial expression taking a pattern classification approach we consider each pixel in an image as a coordinate in a high dimensional space we take advantage of the observation that the images of a particular face under varying illumination but fixed pose lie in a 3d linear subspace of the high dimensional image space if the face is a lambertian surface without shadowing however since faces are not truly lambertian surfaces and do indeed produce self shadowing images will deviate from this linear subspace rather than explicitly modeling this deviation we linearly project the image into a subspace in a manner which discounts those regions of the face with large deviation our projection method is based on fisher s linear discriminant and produces well separated classes in a low dimensional subspace even under severe variation in lighting and facial expressions the eigenface technique another method based on linearly projecting the image space to a low dimensional subspace has similar computational requirements yet extensive experimental results demonstrate that the proposed fisherface method has error rates that are lower than those of the eigenface technique for tests on the harvard and yale face databases

generalizing discriminant analysis using the generalized singular value decomposition discriminant analysis has been used for decades to extract features that preserve class separability it is commonly defined as an optimization problem involving covariance matrices that represent the scatter within and between clusters the requirement that one of these matrices be nonsingular limits its application to data sets with certain relative dimensions we examine a number of optimization criteria and extend their applicability by using the generalized singular value decomposition to circumvent the nonsingularity requirement the result is a generalization of discriminant analysis that can be applied even when the sample size is smaller than the dimension of the sample data we use classification results from the reduced representation to compare the effectiveness of this approach with some alternatives and conclude with a discussion of their relative merits

towards a generalized singular value decomposition we suggest a form for and give a constructive derivation of the generalized singular value decomposition of any two matrices having the same number of columns we outline its desirable characteristics and compare it to an earlier suggestion by van loan siam j numer anal 13 1976 pp 76 83 the present form largely follows from the work of van loan but is slightly more general and computationally more amenable than that in the paper cited we also prove a useful extension of a theorem of stewart siam rev 19 1977 pp 634 662 on unitary decompositions of submatrices of a unitary matrix

mblda a novel multiple between class linear discriminant analysis abstract linear discriminant analysis lda with its extensions is a group of classical methods in dimensionality reduction for supervised learning however when some classes are far away from the others it may be difficult to find the optimal direction by lda because of the average between class scatter moreover ldas are always time consuming for high dimensional problem since the involved generalized eigenvalue problem is needed to be solved in this paper a multiple between class linear discriminant analysis mblda is proposed for dimensionality reduction mblda finds the transformation directions by approximating the solution to a min max programming problem leading to well separability in the reduced space with a fast learning speed on the high dimensional problem it is proved theoretically that the proposed method can deal with the special generalized eigenvalue problem by solving a underdetermined homogeneous system of linear equations experimental results on the artificial and benchmark datasets show that mblda can not only reduce the dimension to a powerful linear discriminant level but also have a fast learning speed

computable elastic distances between shapes we dene distances between geometric curves by the square root of the minimal energy required to transform one curve into the other the energy is formally dened from a left invariant riemannian distance on an innite dimensional group acting on the curves which can be explicitly computed the obtained distance boils down to a variational problem for which an optimal matching between the curves has to be computed an analysis of the distance when the curves are polygonal leads to a numerical procedure for the solution of the variational problem which can eciently be implemented as illustrated by experiments 1 1 generalities the problem of matching two objects together is very impor tant in computer vision and shape recognition in many cases recognition is based on shapes outlines with the help of some suitably designed distance a general principle is to associate with any pair o1 o 2 of objects to be compared a measure of discrepancy d o1 o 2 the recognition of an observed object o may be done by nding from a dictionary of templates the previously recorded object otemp for which d o otemp is minimal clearly the denition of the distance is the crucial step of the method and much research has been done in this direction we shall not try here to provide a review of the huge literature existing on the subject see for example 17 but rather focus on methods related to deformable templates with which we are directly concerned instead of basing recognition on a nite collection of points of interest primitives taken from the outline of an object corners inflexion points etc which is a popular way of handling the problem our purpose is to base the comparison on the whole outline considered as a plane curve the distance we shall dene incorporates some deformation energy between the curves the approach as we will see turns out to be intrinsic and robust to usual euclidean transformations the method is related to the wide literature on snakes 14 7 21 etc in the way that our distance corresponds to some continuous process of deformation of one curve into another it is also related to papers on elastic matching such as 8 however we provide an elastic matching algorithm which is based on a true distance between intrinsic properties of the shapes taking into account possible invariance to scaling or euclidean transformations in the case they are required from this point of view our results are indebted to the seminal work of grenander on group theory applied to pattern recognition cf 10 and 11 in particular see also 1 2 12 another source of inspiration may come from mathematical physics since we are going to look to the path process of lowest energy which deforms one object into

shape analysis of elastic curves in euclidean spaces this paper introduces a square root velocity srv representation for analyzing shapes of curves in euclidean spaces under an elastic metric in this srv representation the elastic metric simplifies to the il2 metric the reparameterization group acts by isometries and the space of unit length curves becomes the unit sphere the shape space of closed curves is the quotient space of a submanifold of the unit sphere modulo rotation and reparameterization groups and we find geodesics in that space using a path straightening approach these geodesics and geodesic distances provide a framework for optimally matching deforming and comparing shapes these ideas are demonstrated using 1 shape analysis of cylindrical helices for studying protein structure 2 shape analysis of facial curves for recognizing faces 3 a wrapped probability distribution for capturing shapes of planar closed curves and 4 parallel transport of deformations for predicting shapes from novel poses

a computationally efficient scheme for feature extraction with kernel discriminant analysis the kernel discriminant analysis kda an extension of the linear discriminant analysis lda and null space based lda into the kernel space generally provides good pattern recognition pr performance for both small sample size sss and non sss pr problems due to the eigen decomposition technique adopted however the original scheme for the feature extraction with the kda suffers from a high complexity burden in this paper we derive a transformation of the kda into a linear equation problem and propose a novel scheme for the feature extraction with the kda the proposed scheme is shown to provide us with a reduction of complexity without degradation of pr performance in addition to enhance the pr performance further we address the incorporation of regularization into the proposed scheme highlightswe propose a complexity reduced scheme for the kernel discriminant analysis the proposed scheme does not cause degradation of recognition accuracy performance the core is the transformation of the original problem into a linear equation problem we further address the incorporation of regularization into the proposed scheme

why can lda be performed in pca transformed space abstract pca plus lda is a popular framework for linear discriminant analysis lda in high dimensional and singular case in this paper we focus on building a theoretical foundation for this framework moreover we point out the weakness of the previous lda based methods and suggest a complete pca plus lda algorithm experimental results on orl face image database indicate that the proposed method is more effective than the previous ones

regularized discriminant analysis abstract linear and quadratic discriminant analysis are considered in the small sample high dimensional setting alternatives to the usual maximum likelihood plug in estimates for the covariance matrices are proposed these alternatives are characterized by two parameters the values of which are customized to individual situations by jointly minimizing a sample based estimate of future misclassification risk computationally fast implementations are presented and the efficacy of the approach is examined through simulation studies and application to data these studies indicate that in many circumstances dramatic gains in classification accuracy can be achieved

w1 p quasiconvexity and variational problems for multiple integrals abstract variational problems for the multiple integral i u g u x dx where r m and u r n are studied a new condition on g called w1 p quasiconvexity is introduced which generalizes in a natural way the quasiconvexity condition of c b morrey it being shown in particular to be necessary for sequential weak lower semicontinuity of i in w 1 p r n and for the existence of minimizers for certain related integrals counterexamples are given concerning the weak continuity properties of jacobians in w 1 p r n p n m an existence theorem for nonlinear elastostatics is proved under optimal growth hypotheses

sign consistency for the linear programming discriminant rule abstract linear discriminant analysis lda is an important conventional model for data classification classical theory shows that lda is bayes consistent for a fixed data dimensionality p and a large training sample size n however in high dimensional settings when p n lda is difficult due to the inconsistent estimation of the covariance matrix and the mean vectors of populations recently a linear programming discriminant lpd rule was proposed for high dimensional linear discriminant analysis based on the sparsity assumption over the discriminant function it is shown that the lpd rule is bayes consistent in high dimensional settings in this paper we further show that the lpd rule is sign consistent under the sparsity assumption such sign consistency ensures the lpd rule to select the optimal discriminative features for high dimensional data classification problems evaluations on both synthetic and real data validate our result on the sign consistency of the lpd rule

an efficiently computable metric for comparing polygonal shapes a method for comparing polygons that is a metric invariant under translation rotation and change of scale reasonably easy to compute and intuitive is presented the method is based on the l sub 2 distance between the turning functions of the two polygons it works for both convex and nonconvex polygons and runs in time o mn log mn where m is the number of vertices in one polygon and n is the number of vertices in the other some examples showing that the method produces answers that are intuitively reasonable are presented

nonlinear component analysis as a kernel eigenvalue problem a new method for performing a nonlinear form of principal component analysis is proposed by the use of integral operator kernel functions one can efficiently compute principal components in high dimensional feature spaces related to input space by some nonlinear map for instance the space of all possible five pixel products in 16 16 images we give the derivation of the method and present experimental results on polynomial feature extraction for pattern recognition

face recognition based on the uncorrelated discriminant transformation abstract the extraction of discriminant features is the most fundamental and important problem in face recognition this paper presents a method to extract optimal discriminant features for face images by using the uncorrelated discriminant transformation and kl expansion experiments on the orl database and the nust603 database have been performed experimental results show that the uncorrelated discriminant transformation is superior to the foley sammon discriminant transformation and the new method to extract uncorrelated discriminant features for face images is very effective an error rate of 2 5 is obtained with the experiments on the orl database an average error rate of 1 2 is obtained with the experiments on the nust603 database experiments show that by extracting uncorrelated discriminant features face recognition could be performed with higher accuracy on lower than 16 16 resolution mosaic images it is suggested that for the uncorrelated discriminant transformation the optimal face image resolution can be regarded as the resolution m n which makes the dimensionality n mn of the original image vector space be larger and closer to the number of known face classes

fast computation of pca bases of image subspace using its inner product subspace computation of principal component analysis pca basis for image subspace requires more memory and takes comparatively more time for computation in this paper a simple approach to obtain the pca basis of the image subspace using its inner product space is proposed experimental results depicting the reduction of computation time using the proposed technique is reported

an efficient algorithm to solve the small sample size problem for lda in this paper we present an efficient algorithm to solve the most discriminant vectors of lda for high dimensional data set the experiments on orl face database confirm the effectiveness of the proposed method

a new lda based face recognition system which can solve the small sample size problem a new lda based face recognition system is presented in this paper linear discriminant analysis lda is one of the most popular linear projection techniques for feature extraction the major drawback of applying lda is that it may encounter the small sample size problem in this paper we propose a new lda based technique which can solve the small sample size problem we also prove that the most expressive vectors derived in the null space of the within class scatter matrix using principal component analysis pca are equal to the optimal discriminant vectors derived in the original space using lda the experimental results show that the new lda process improves the performance of a face recognition system signi cantly 2000 pattern recognition society published by elsevier science ltd all rights reserved

a comparison of generalized linear discriminant analysis algorithms linear discriminant analysis lda is a dimension reduction method which finds an optimal linear transformation that maximizes the class separability however in undersampled problems where the number of data samples is smaller than the dimension of data space it is difficult to apply lda due to the singularity of scatter matrices caused by high dimensionality in order to make lda applicable several generalizations of lda have been proposed recently in this paper we present theoretical and algorithmic relationships among several generalized lda algorithms and compare their computational complexities and performances in text classification and face recognition towards a practical dimension reduction method for high dimensional data an efficient algorithm is proposed which reduces the computational complexity greatly while achieving competitive prediction accuracies we also present nonlinear extensions of these lda algorithms based on kernel methods it is shown that a generalized eigenvalue problem can be formulated in the kernel based feature space and generalized lda algorithms are applied to solve the generalized eigenvalue problem resulting in nonlinear discriminant analysis performances of these linear and nonlinear discriminant analysis algorithms are compared extensively

testing equality of cell populations based on shape and geodesic distance image cytometry has emerged as a valuable in vitro screening tool and advances in automated microscopy have made it possible to readily analyze large cellular populations of image data the purpose of this paper is to illustrate the viability of using cell shape to test equality of cell populations based on image data shape space theory is reviewed from which differences between shapes can be quantified in terms of geodesic distance several multivariate nonparametric statistical hypothesis tests are adapted to test equality of cell populations it is illustrated that geodesic distance can be a better feature than cell spread area and roundness in distinguishing between cell populations tests based on geodesic distance are able to detect natural perturbations of cells whereas kolmogorov smirnov tests based on area and roundness are not

generalized discriminant analysis using a kernel approach we present a new method that we call generalized discriminant analysis gda to deal with nonlinear discriminant analysis using kernel function operator the underlying theory is close to the support vector machines svm insofar as the gda method provides a mapping of the input vectors into high dimensional feature space in the transformed space linear properties make it easy to extend and generalize the classical linear discriminant analysis lda to nonlinear discriminant analysis the formulation is expressed as an eigenvalue problem resolution using a different kernel one can cover a wide class of nonlinearities for both simulated data and alternate kernels we give classification results as well as the shape of the decision function the results are confirmed using real data to perform seed classification

multivariate two sample tests based on nearest neighbors abstract a new class of simple tests is proposed for the general multivariate two sample problem based on the possibly weighted proportion of all k nearest neighbor comparisons in which observations and their neighbors belong to the same sample large values of the test statistics give evidence against the hypothesis h of equality of the two underlying distributions asymptotic null distributions are explicitly determined and shown to involve certain nearest neighbor interaction probabilities simple infinite dimensional approximations are supplied the unweighted version yields a distribution free test that is consistent against all alternatives optimally weighted statistics are also obtained and asymptotic efficiencies are calculated each of the tests considered is easily adapted to a permutation procedure that conditions on the pooled sample power performance for finite sample sizes is assessed in simulations

inexact implementation using krylov subspace methods for large scale exponential discriminant analysis with applications to high dimensionality reduction problems abstract exponential discriminant analysis eda is a generalized discriminant analysis method based on matrix exponential it can essentially overcome the intrinsic difficulty of small sample size problem that exists in the classical linear discriminant analysis lda however for data with high dimensionality one has to solve a large matrix exponential eigenproblem in this method and the time complexity is dominated by the computation of exponential of large matrices in this paper we propose two inexact krylov subspace algorithms for solving the large matrix exponential eigenproblem efficiently the contribution of this work is threefold first we consider how to compute matrix exponential vector products efficiently which is the key step in the krylov subspace method second we compare the discriminant analysis criterion of eda and that of lda from a theoretical point of view third we establish the relationship between the accuracy of the approximate eigenvectors and the distance to nearest neighbor classifier and show why the matrix exponential eigenproblem can be solved approximately in practice numerical experiments on some real world databases show the superiority of our new algorithms over their original counterpart for face recognition

input space versus feature space in kernel based methods this paper collects some ideas targeted at advancing our understanding of the feature spaces associated with support vector sv kernel functions we first discuss the geometry of feature space in particular we review what is known about the shape of the image of input space under the feature space map and how this influences the capacity of sv methods following this we describe how the metric governing the intrinsic geometry of the mapped surface can be computed in terms of the kernel using the example of the class of inhomogeneous polynomial kernels which are often used in sv pattern recognition we then discuss the connection between feature space and input space by dealing with the question of how one can given some vector in feature space find a preimage exact or approximate in input space we describe algorithms to tackle this issue and show their utility in two applications of kernel methods first we use it to reduce the computational complexity of sv decision functions second we combine it with the kernel pca algorithm thereby constructing a nonlinear statistical denoising technique which is shown to perform well on real world data