worst case linear discriminant analysis as scalable semidefinite feasibility problems in this paper we propose an efficient semidefinite programming sdp approach to worst case linear discriminant analysis wlda compared with the traditional lda wlda considers the dimensionality reduction problem from the worst case viewpoint which is in general more robust for classification however the original problem of wlda is non convex and difficult to optimize in this paper we reformulate the optimization problem of wlda into a sequence of semidefinite feasibility problems to efficiently solve the semidefinite feasibility problems we design a new scalable optimization method with a quasi newton method and eigen decomposition being the core components the proposed method is orders of magnitude faster than standard interior point sdp solvers experiments on a variety of classification problems demonstrate that our approach achieves better performance than standard lda our method is also much faster and more scalable than standard interior point sdp solvers based wlda the computational complexity for an sdp with m constraints and matrices of size d by d is roughly reduced from mathcal o m 3 md 3 m 2 d 2 to mathcal o d 3 m d in our case

eigenfaces vs fisherfaces recognition using class specific linear projection we develop a face recognition algorithm which is insensitive to large variation in lighting direction and facial expression taking a pattern classification approach we consider each pixel in an image as a coordinate in a high dimensional space we take advantage of the observation that the images of a particular face under varying illumination but fixed pose lie in a 3d linear subspace of the high dimensional image space if the face is a lambertian surface without shadowing however since faces are not truly lambertian surfaces and do indeed produce self shadowing images will deviate from this linear subspace rather than explicitly modeling this deviation we linearly project the image into a subspace in a manner which discounts those regions of the face with large deviation our projection method is based on fisher s linear discriminant and produces well separated classes in a low dimensional subspace even under severe variation in lighting and facial expressions the eigenface technique another method based on linearly projecting the image space to a low dimensional subspace has similar computational requirements yet extensive experimental results demonstrate that the proposed fisherface method has error rates that are lower than those of the eigenface technique for tests on the harvard and yale face databases

discriminant subspace analysis a fukunaga koontz approach the fisher linear discriminant fld is commonly used in pattern recognition it finds a linear subspace that maximally separates class patterns according to the fisher criterion several methods of computing the fld have been proposed in the literature most of which require the calculation of the so called scatter matrices in this paper we bring a fresh perspective to fld via the fukunaga koontz transform fkt we do this by decomposing the whole data space into four subspaces with different discriminabilities as measured by eigenvalue ratios by connecting the eigenvalue ratio with the generalized eigenvalue we show where the fisher criterion is maximally satisfied we prove the relationship between fld and fkt analytically and propose a unified framework to understanding some existing work furthermore we extend our theory to the multiple discriminant analysis mda this is done by transforming the data into intraclass and extraclass spaces followed by maximizing the bhattacharyya distance based on our fkt analysis we identify the discriminant subspaces of mda fkt and propose an efficient algorithm which works even when the scatter matrices are singular or too large to be formed our method is general and may be applied to different pattern recognition problems we validate our method by experimenting on synthetic and real data

two dimensional supervised local similarity and diversity projection this paper presents a novel manifold learning method namely two dimensional supervised local similarity and diversity projection 2dslsdp for feature extraction the proposed method defines two weighted adjacency graphs namely similarity graph and diversity graph the affinity matrix of similarity graph is determined by the spatial relationship between vertices of this graph while affinity matrix of diversity graph is determined by the diversity information of vertices of its graph using these two graphs the proposed method constructs local similarity scatter and diversity scatter respectively a concise feature extraction criterion is then raised via minimizing the ratio of the local similarity scatter to local diversity scatter thus 2dslsdp can well preserve not only the adjacency similarity structure but also the diversity of data points which is important for the classification experiments on the ar and umist databases show the effectiveness of the proposed method

generalizing discriminant analysis using the generalized singular value decomposition discriminant analysis has been used for decades to extract features that preserve class separability it is commonly defined as an optimization problem involving covariance matrices that represent the scatter within and between clusters the requirement that one of these matrices be nonsingular limits its application to data sets with certain relative dimensions we examine a number of optimization criteria and extend their applicability by using the generalized singular value decomposition to circumvent the nonsingularity requirement the result is a generalization of discriminant analysis that can be applied even when the sample size is smaller than the dimension of the sample data we use classification results from the reduced representation to compare the effectiveness of this approach with some alternatives and conclude with a discussion of their relative merits

general tensor discriminant analysis and gabor features for gait recognition traditional image representations are not suited to conventional classification methods such as the linear discriminant analysis lda because of the undersample problem usp the dimensionality of the feature space is much higher than the number of training samples motivated by the successes of the two dimensional lda 2dlda for face recognition we develop a general tensor discriminant analysis gtda as a preprocessing step for lda the benefits of gtda compared with existing preprocessing methods such as the principal components analysis pca and 2dlda include the following 1 the usp is reduced in subsequent classification by for example lda 2 the discriminative information in the training tensors is preserved and 3 gtda provides stable recognition rates because the alternating projection optimization algorithm to obtain a solution of gtda converges whereas that of 2dlda does not we use human gait recognition to validate the proposed gtda the averaged gait images are utilized for gait representation given the popularity of gabor function based image decompositions for image understanding and object recognition we develop three different gabor function based image representations 1 gabord is the sum of gabor filter responses over directions 2 gabors is the sum of gabor filter responses over scales and 3 gaborsd is the sum of gabor filter responses over scales and directions the gabord gabors and gaborsd representations are applied to the problem of recognizing people from their averaged gait images a large number of experiments were carried out to evaluate the effectiveness recognition rate of gait recognition based on first obtaining a gabor gabord gabors or gaborsd image representation then using gdta to extract features and finally using lda for classification the proposed methods achieved good performance for gait recognition based on image sequences from the university of south florida usf humanid database experimental comparisons are made with nine state of the art classification methods in gait recognition

an analysis of strong discontinuities in multiplicative finite strain plasticity and their relation with the numerical simulation of strain localization in solids abstract this paper presents an analysis of strong discontinuities in inelastic solids at finite strains solutions exhibiting this type of discontinuities characterized by a discontinuous displacement field are shown to make physical and mathematical sense in a classical multiplicative plasticity continuum model if the softening modulus is reinterpreted as a singular distribution physically the strainsoftening is localized along the discontinuity conditions for the appearance of strong discontinuities in the geometrically nonlinear range are characterized as it is the response of the material during localization in addition these analytical results are exploited in the design of a new class of finite element methods the proposed methods fall within the class of enhanced strain methods and lead to solutions independent of the mesh size and insensitive to mesh alignment without requiring any regularization of the solutions by numerical parameters like a characteristic length

regularized discriminant analysis abstract linear and quadratic discriminant analysis are considered in the small sample high dimensional setting alternatives to the usual maximum likelihood plug in estimates for the covariance matrices are proposed these alternatives are characterized by two parameters the values of which are customized to individual situations by jointly minimizing a sample based estimate of future misclassification risk computationally fast implementations are presented and the efficacy of the approach is examined through simulation studies and application to data these studies indicate that in many circumstances dramatic gains in classification accuracy can be achieved

stable local dimensionality reduction approaches dimensionality reduction is a big challenge in many areas a large number of local approaches stemming from statistics or geometry have been developed however in practice these local approaches are often in lack of robustness since in contrast to maximum variance unfolding mvu which explicitly unfolds the manifold they merely characterize local geometry structure moreover the eigenproblems that they encounter are hard to solve we propose a unified framework that explicitly unfolds the manifold and reformulate local approaches as the semi definite programs instead of the above mentioned eigenproblems three well known algorithms locally linear embedding lle laplacian eigenmaps le and local tangent space alignment ltsa are reinterpreted and improved within this framework several experiments are presented to demonstrate the potential of our framework and the improvements of these local algorithms

on the discrete constitutive models induced by strong discontinuity kinematics and continuum constitutive equations on the basis of continuum constitutive models stress vs strain the introduction of strong discontinuity kinematics considering jumps in the displacement fields across a discontinuity interface induces projected discrete constitutive models traction displacement jumps in a consistent manner therefore this projection provides possible links between the classical continuum strain localization analysis and the non linear decohesive fracture mechanics techniques the strong discontinuity analysis shows that bandwidth based regularization of the hardening softening parameter is the crucial modification to be done on the continuum model to achieve such a projection and it also provides the strong discontinuity conditions that set restrictions on the stress state compatible with bifurcations in a strong discontinuity format the methodology is illustrated on the basis of two classical families of non linear constitutive models scalar continuum damage and elasto plasticity for which the corresponding discrete constitutive models and the strong discontinuity conditions are derived

graph embedding and extensions a general framework for dimensionality reduction a large family of algorithms supervised or unsupervised stemming from statistics or geometry theory has been designed to provide different solutions to the problem of dimensionality reduction despite the different motivations of these algorithms we present in this paper a general formulation known as graph embedding to unify them within a common framework in graph embedding each algorithm can be considered as the direct graph embedding or its linear kernel tensor extension of a specific intrinsic graph that describes certain desired statistical or geometric properties of a data set with constraints from scale normalization or a penalty graph that characterizes a statistical or geometric property that should be avoided furthermore the graph embedding framework can be used as a general platform for developing new dimensionality reduction algorithms by utilizing this framework as a tool we propose a new supervised dimensionality reduction algorithm called marginal fisher analysis in which the intrinsic graph characterizes the intraclass compactness and connects each data point with its neighboring points of the same class while the penalty graph connects the marginal points and characterizes the interclass separability we show that mfa effectively overcomes the limitations of the traditional linear discriminant analysis algorithm due to data distribution assumptions and available projection directions real face recognition experiments show the superiority of our proposed mfa in comparison to lda also for corresponding kernel and tensor extensions

adaptive maximum margin analysis for image recognition abstract most existing discriminant manifold learning methods aim to maximize the margin among nearby data which is determined in the high dimensional original space as such they do not necessarily best maximize the margin between different classes in the low dimensional space which is a critically important property for image classification to handle this problem we propose an adaptive maximum margin analysis amma for feature extraction amma aims to seek a projection matrix that best maximize the margin which is calculated in the low dimensional space it uses sparse representation to adaptively construct the intrinsic and penalty graphs finally an iterative algorithm is developed to solve the projection matrix extensive experimental results on several image databases illustrate the effectiveness of the proposed approach

deep adaptive feature embedding with local sample distributions for person re identification person re identification re id aims to match pedestrians observed by disjoint camera views it attracts increasing attention in computer vision due to its importance to surveillance systems to combat the major challenge of cross view visual variations deep embedding approaches are proposed by learning a compact feature space from images such that the euclidean distances correspond to their cross view similarity metric however the global euclidean distance cannot faithfully characterize the ideal similarity in a complex visual feature space because features of pedestrian images exhibit unknown distributions due to large variations in poses illumination and occlusion moreover intra personal training samples within a local range which are robust to guide deep embedding against uncontrolled variations cannot be captured by a global euclidean distance in this paper we study the problem of person re id by proposing a novel sampling to mine suitable positives i e intra class within a local range to improve the deep embedding in the context of large intra class variations our method is capable of learning a deep similarity metric adaptive to local sample structure by minimizing each sample s local distances while propagating through the relationship between samples to attain the whole intra class minimization to this end a novel objective function is proposed to jointly optimize similarity metric learning local positive mining and robust deep feature embedding this attains local discriminations by selecting local ranged positive samples and the learned features are robust to dramatic intra class variations experiments on benchmarks show state of the art results achieved by our method c 2017 elsevier ltd all rights reserved

feature extraction using constrained maximum variance mapping in this paper an efficient feature extraction method named as constrained maximum variance mapping cmvm is developed the proposed algorithm can be viewed as a linear approximation of multi manifolds learning based approach which takes the local geometry and manifold labels into account the cmvm and the original manifold learning based approaches have a point in common that the locality is preserved moreover the cmvm is globally maximizing the distances between different manifolds after the local scatters have been characterized the proposed method focuses on developing a linear transformation that can maximize the dissimilarities between all the manifolds under the constraint of locality preserving compared to most of the up to date manifold learning based methods this trick makes contribution to pattern classification from two aspects on the one hand the local structure in each manifold is still kept on the other hand the discriminant information between manifolds can be explored finally feret face database cmu pie face database and usps handwriting data are all taken to examine the effectiveness and efficiency of the proposed method experimental results validate that the proposed approach is superior to other feature extraction methods such as linear discriminant analysis lda locality preserving projection lpp unsupervised discriminant projection udp and maximum variance projection mvp

semisupervised generalized discriminant analysis generalized discriminant analysis gda is a commonly used method for dimensionality reduction in its general form it seeks a nonlinear projection that simultaneously maximizes the between class dissimilarity and minimizes the within class dissimilarity to increase class separability in real world applications where labeled data are scarce gda may not work very well however unlabeled data are often available in large quantities at very low cost in this paper we propose a novel gda algorithm which is abbreviated as semisupervised generalized discriminant analysis ssgda we utilize unlabeled data to maximize an optimality criterion of gda and formulate the problem as an optimization problem that is solved using the constrained concave convex procedure the optimization procedure leads to estimation of the class labels for the unlabeled data we propose a novel confidence measure and a method for selecting those unlabeled data points whose labels are estimated with high confidence the selected unlabeled data can then be used to augment the original labeled dataset for performing gda we also propose a variant of ssgda called m ssgda which adopts the manifold assumption to utilize the unlabeled data extensive experiments on many benchmark datasets demonstrate the effectiveness of our proposed methods

smoothing effects for schrodinger equations with electro magnetic potentials and applications to the maxwell schrodinger equations abstract we consider schrodinger equations in r 1 2 with electro magnetic potentials the potentials belong to h 1 and typically they are time independent or determined as solutions to inhomogeneous wave equations we prove kato type smoothing estimates for solutions we also apply this result to the maxwell schrodinger equations in the lorentz gauge and prove unique solvability of this system in the energy space

discriminative histograms of local dominant orientation d hldo for biometric image feature extraction this paper presents a simple and robust method namely discriminative histograms of local dominant orientation d hldo for biometric image feature extraction in d hldo the local dominant orientation map and the corresponding relative energy map are obtained by applying the singular value decomposition svd to the collected gradient vectors over a local patch the dominant orientation map and the relative energy map are then used to construct the concatenated histogram features local mean based nearest neighbor discriminant analysis lm nnda is finally employed to reduce the redundancy information and get the low dimensional and discriminative features the proposed method is applied to face finger knuckle print and palm biometrics and is examined using the ar cmu pie and frgcv2 0 face image databases the polyu palmprint database and the polyu finger knuckle print database experimental results demonstrate the effectiveness of the proposed d hldo method

fisher discrimination based low rank matrix recovery for face recognition in this paper we consider the issue of computing low rank lr recovery of matrices with sparse errors based on the success of low rank matrix recovery in statistical learning computer vision and signal processing a novel low rank matrix recovery algorithm with fisher discrimination regularization fdlr is proposed standard low rank matrix recovery algorithm decomposes the original matrix into a set of representative basis with a corresponding sparse error for modeling the raw data motivated by the fisher criterion the proposed fdlr executes low rank matrix recovery in a supervised manner i e taking the with class scatter and between class scatter into account when the whole label information are available the paper shows that the formulated model can be solved by the augmented lagrange multipliers and provides additional discriminating power over the standard low rank recovery models the representative bases learned by the proposed method are encouraged to be closer within the same class and as far as possible between different classes meanwhile the sparse error recovered by fdlr is not discarded as usual but treated as a feedback in the following classification tasks numerical simulations demonstrate that the proposed algorithm achieves the state of the art results

exploiting the relationships among several binary classifiers via data transformation the structural resemblance among several existing classifiers has motivated us to investigate their underlying relationships by exploring into the mapping solutions of these classifiers we found that they can be linked by simple feature data scaling in other words the key to these relationships lies upon how the replica of feature data are being scaled this finding leads us directly to an exploration of novel classifiers beyond existing settings based on an extensive empirical evaluation we show that the proposed formulation facilitates a tuning capability beyond existing settings for classifier generalization highlightsestablishment of relationship among several binary classifiers exploration of a random projection network for such classifier learning proposal of novel classifier which exploits the established relationship observe the performance of novel classifier on several performance measures

face recognition based on the uncorrelated discriminant transformation abstract the extraction of discriminant features is the most fundamental and important problem in face recognition this paper presents a method to extract optimal discriminant features for face images by using the uncorrelated discriminant transformation and kl expansion experiments on the orl database and the nust603 database have been performed experimental results show that the uncorrelated discriminant transformation is superior to the foley sammon discriminant transformation and the new method to extract uncorrelated discriminant features for face images is very effective an error rate of 2 5 is obtained with the experiments on the orl database an average error rate of 1 2 is obtained with the experiments on the nust603 database experiments show that by extracting uncorrelated discriminant features face recognition could be performed with higher accuracy on lower than 16 16 resolution mosaic images it is suggested that for the uncorrelated discriminant transformation the optimal face image resolution can be regarded as the resolution m n which makes the dimensionality n mn of the original image vector space be larger and closer to the number of known face classes

extracting the optimal dimensionality for local tensor discriminant analysis supervised dimensionality reduction with tensor representation has attracted great interest in recent years it has been successfully applied to problems with tensor data such as image and video recognition tasks however in the tensor based methods how to select the suitable dimensions is a very important problem since the number of possible dimension combinations exponentially increases with respect to the order of tensor manually selecting the suitable dimensions becomes an impossible task in the case of high order tensor in this paper we aim at solving this important problem and propose an algorithm to extract the optimal dimensionality for local tensor discriminant analysis experimental results on a toy example and real world data validate the effectiveness of the proposed method

a two stage linear discriminant analysis via qr decomposition linear discriminant analysis lda is a well known method for feature extraction and dimension reduction it has been used widely in many applications involving high dimensional data such as image and text classification an intrinsic limitation of classical lda is the so called singularity problems that is it fails when all scatter matrices are singular many lda extensions were proposed in the past to overcome the singularity problems among these extensions pca lda a two stage method received relatively more attention in pca lda the lda stage is preceded by an intermediate dimension reduction stage using principal component analysis pca most previous lda extensions are computationally expensive and not scalable due to the use of singular value decomposition or generalized singular value decomposition in this paper we propose a two stage lda method namely lda qr which aims to overcome the singularity problems of classical lda while achieving efficiency and scalability simultaneously the key difference between lda qr and pca lda lies in the first stage where lda qr applies qr decomposition to a small matrix involving the class centroids while pca lda applies pca to the total scatter matrix involving all training data points we further justify the proposed algorithm by showing the relationship among lda qr and previous lda methods extensive experiments on face images and text documents are presented to show the effectiveness of the proposed algorithm

an optimal orthonormal system for discriminant analysis abstract this paper proposes a new discriminant analysis with orthonormal coordinate axes of the feature space in general the number of coordinate axes of the feature space in the traditional discriminant analysis depends on the number of pattern classes therefore the discriminatory capability of the feature space is limited considerably the new discriminant analysis solves this problem completely in addition it is more powerful than the traditional one in so far as the discriminatory power and the mean error probability for coordinate axes are concerned this is also shown by a numerical example

geometric mean for subspace selection subspace selection approaches are powerful tools in pattern classification and data visualization one of the most important subspace approaches is the linear dimensionality reduction step in the fisher s linear discriminant analysis flda which has been successfully employed in many fields such as biometrics bioinformatics and multimedia information management however the linear dimensionality reduction step in flda has a critical drawback for a classification task with c classes if the dimension of the projected subspace is strictly lower than c 1 the projection to a subspace tends to merge those classes which are close together in the original feature space if separate classes are sampled from gaussian distributions all with identical covariance matrices then the linear dimensionality reduction step in flda maximizes the mean value of the kullback leibler kl divergences between different classes based on this viewpoint the geometric mean for subspace selection is studied in this paper three criteria are analyzed 1 maximization of the geometric mean of the kl divergences 2 maximization of the geometric mean of the normalized kl divergences and 3 the combination of 1 and 2 preliminary experimental results based on synthetic data uci machine learning repository and handwriting digits show that the third criterion is a potential discriminative subspace selection method which significantly reduces the class separation problem in comparing with the linear dimensionality reduction step in flda and its several representative extensions

dimensionality reduction of multimodal labeled data by local fisher discriminant analysis reducing the dimensionality of data without losing intrinsic information is an important preprocessing step in high dimensional data analysis fisher discriminant analysis fda is a traditional technique for supervised dimensionality reduction but it tends to give undesired results if samples in a class are multimodal an unsupervised dimensionality reduction method called locality preserving projection lpp can work well with multimodal data due to its locality preserving property however since lpp does not take the label information into account it is not necessarily useful in supervised learning scenarios in this paper we propose a new linear supervised dimensionality reduction method called local fisher discriminant analysis lfda which effectively combines the ideas of fda and lpp lfda has an analytic form of the embedding transformation and the solution can be easily computed just by solving a generalized eigenvalue problem we demonstrate the practical usefulness and high scalability of the lfda method in data visualization and classification tasks through extensive simulation studies we also show that lfda can be extended to non linear dimensionality reduction scenarios by applying the kernel trick

constitutive modeling of discontinuities by means of discrete and continuum approximations and damage models in this paper the mathematical modeling of discontinuities using the discrete approximation and the continuum approximation with weak discontinuities is presented first the kinematics of discontinuities is discussed then two constitutive models based on the continuum damage mechanics theory are developed the first model is an isotropic damage model and is used in the discrete approximation the second model is an anisotropic damage model and is used in the continuum approximation these models are characterized for weighing the mode of failure in the failure criterion an energy analysis is proposed to establish the equations that relate the parameters of both constitutive models the fulfillment of the involved equations guarantee that both models are energetically equivalent it is concluded that the proposed models are suitable to reproduce the constitutive behavior of discontinuities

a new lda based face recognition system which can solve the small sample size problem a new lda based face recognition system is presented in this paper linear discriminant analysis lda is one of the most popular linear projection techniques for feature extraction the major drawback of applying lda is that it may encounter the small sample size problem in this paper we propose a new lda based technique which can solve the small sample size problem we also prove that the most expressive vectors derived in the null space of the within class scatter matrix using principal component analysis pca are equal to the optimal discriminant vectors derived in the original space using lda the experimental results show that the new lda process improves the performance of a face recognition system signi cantly 2000 pattern recognition society published by elsevier science ltd all rights reserved

efficient and robust feature extraction by maximum margin criterion in pattern recognition feature extraction techniques are widely employed to reduce the dimensionality of data and to enhance the discriminatory information principal component analysis pca and linear discriminant analysis lda are the two most popular linear dimensionality reduction methods however pca is not very effective for the extraction of the most discriminant features and lda is not stable due to the small sample size problem in this paper we propose some new linear and nonlinear feature extractors based on maximum margin criterion mmc geometrically feature extractors based on mmc maximize the average margin between classes after dimensionality reduction it is shown that mmc can represent class separability better than pca as a connection to lda we may also derive lda from mmc by incorporating some constraints by using some other constraints we establish a new linear feature extractor that does not suffer from the small sample size problem which is known to cause serious stability problems for lda the kernelized nonlinear counterpart of this linear feature extractor is also established in the paper our extensive experiments demonstrate that the new feature extractors are effective stable and efficient

enhanced fisher discriminant criterion for image recognition many previous studies have shown that image recognition can be significantly improved by fisher linear discriminant analysis flda technique however flda ignores the variation of data points from the same class which characterizes the most important modes of variability of patterns and helps to improve the generalization capability of flda thus the performance of flda on testing data is not good enough in this paper we propose an enhanced fisher discriminant criterion efdc efdc explicitly considers the intra class variation and incorporates the intra class variation into the fisher discriminant criterion to build a robust and efficient dimensionality reduction function efdc obtains a subspace which best detects the discriminant structure and simultaneously preserves the modes of variability of patterns which will result in stable intraclass representation experimental results on four image database show a clear improvement over the results of flda based methods

tensor linear laplacian discrimination tlld for feature extraction discriminant feature extraction plays a central role in pattern recognition and classification in this paper we propose the tensor linear laplacian discrimination tlld algorithm for extracting discriminant features from tensor data tlld is an extension of linear discriminant analysis lda and linear laplacian discrimination lld in directions of both nonlinear subspace learning and tensor representation based on the contextual distance the weights for the within class scatters and the between class scatter can be determined to capture the principal structure of data clusters this makes tlld free from the metric of the sample space which may not be known moreover unlike lld the parameter tuning of tlld is very easy experimental results on face recognition texture classification and handwritten digit recognition show that tlld is effective in extracting discriminative features