pairwise sparsity preserving embedding for unsupervised subspace learning and classification two novel unsupervised dimensionality reduction techniques termed sparse distance preserving embedding sdpe and sparse proximity preserving embedding sppe are proposed for feature extraction and classification sdpe and sppe perform in the clean data space recovered by sparse representation and enhanced euclidean distances over noise removed data are employed to measure pairwise similarities of points in extracting informative features sdpe and sppe aim at preserving pairwise similarities between data points in addition to preserving the sparse characteristics this paper calculates the sparsest representation of all vectors jointly by a convex optimization the sparsest codes enable certain local information of data to be preserved and can endow sdpe and sppe a natural discriminating power adaptive neighborhood and robust characteristic against noise and errors in delivering low dimensional embeddings we also mathematically show sdpe and sppe can be effectively extended for discriminant learning in a supervised manner the validity of sdpe and sppe is examined by extensive simulations comparison with other related state of the art unsupervised algorithms show that promising results are delivered by our techniques

worst case linear discriminant analysis as scalable semidefinite feasibility problems in this paper we propose an efficient semidefinite programming sdp approach to worst case linear discriminant analysis wlda compared with the traditional lda wlda considers the dimensionality reduction problem from the worst case viewpoint which is in general more robust for classification however the original problem of wlda is non convex and difficult to optimize in this paper we reformulate the optimization problem of wlda into a sequence of semidefinite feasibility problems to efficiently solve the semidefinite feasibility problems we design a new scalable optimization method with a quasi newton method and eigen decomposition being the core components the proposed method is orders of magnitude faster than standard interior point sdp solvers experiments on a variety of classification problems demonstrate that our approach achieves better performance than standard lda our method is also much faster and more scalable than standard interior point sdp solvers based wlda the computational complexity for an sdp with m constraints and matrices of size d by d is roughly reduced from mathcal o m 3 md 3 m 2 d 2 to mathcal o d 3 m d in our case

modified principal component analysis an integration of multiple similarity subspace models we modify the conventional principal component analysis pca and propose a novel subspace learning framework modified pca mpca using multiple similarity measurements mpca computes three similarity matrices exploiting the similarity measurements 1 mutual information 2 angle information and 3 gaussian kernel similarity we employ the eigenvectors of similarity matrices to produce new subspaces referred to as similarity subspaces a new integrated similarity subspace is then generated using a novel feature selection approach this approach needs to construct a kind of vector set termed weak machine cell wmc which contains an appropriate number of the eigenvectors spanning the similarity subspaces combining the wrapper method and the forward selection scheme mpca selects a wmc at a time that has a powerful discriminative capability to classify samples mpca is very suitable for the application scenarios in which the number of the training samples is less than the data dimensionality mpca outperforms the other state of the art pca based methods in terms of both classification accuracy and clustering result in addition mpca can be applied to face image reconstruction mpca can use other types of similarity measurements extensive experiments on many popular real world data sets such as face databases show that mpca achieves desirable classification results as well as has a powerful capability to represent data

eigenfaces vs fisherfaces recognition using class specific linear projection we develop a face recognition algorithm which is insensitive to large variation in lighting direction and facial expression taking a pattern classification approach we consider each pixel in an image as a coordinate in a high dimensional space we take advantage of the observation that the images of a particular face under varying illumination but fixed pose lie in a 3d linear subspace of the high dimensional image space if the face is a lambertian surface without shadowing however since faces are not truly lambertian surfaces and do indeed produce self shadowing images will deviate from this linear subspace rather than explicitly modeling this deviation we linearly project the image into a subspace in a manner which discounts those regions of the face with large deviation our projection method is based on fisher s linear discriminant and produces well separated classes in a low dimensional subspace even under severe variation in lighting and facial expressions the eigenface technique another method based on linearly projecting the image space to a low dimensional subspace has similar computational requirements yet extensive experimental results demonstrate that the proposed fisherface method has error rates that are lower than those of the eigenface technique for tests on the harvard and yale face databases

compact and discriminative descriptor inference using multi cues feature descriptors around local interest points are widely used in human action recognition both for images and videos however each kind of descriptors describes the local characteristics around the reference point only from one cue to enhance the descriptive and discriminative ability from multiple cues this paper proposes a descriptor learning framework to optimize the descriptors at the source by learning a projection from multiple descriptors spaces to a new euclidean space in this space multiple cues and characteristics of different descriptors are fused and complemented for each other in order to make the new descriptor more discriminative we learn the multi cue projection by the minimization of the ratio of within class scatter to between class scatter and therefore the discriminative ability of the projected descriptor is enhanced in the experiment we evaluate our framework on the tasks of action recognition from still images and videos experimental results on two benchmark image and two benchmark video data sets demonstrate the effectiveness and better performance of our method

improved discriminate analysis for high dimensional data and its application to face recognition many pattern recognition applications involve the treatment of high dimensional data and the small sample size problem principal component analysis pca is a common used dimension reduction technique linear discriminate analysis lda is often employed for classification pca plus lda is a famous framework for discriminant analysis in high dimensional space and singular cases in this paper we examine the theory of this framework and find out that even if there is no small sample size problem the pca dimension reduction cannot guarantee the subsequent successful application of lda we thus develop an improved discriminate analysis method by introducing an inverse fisher criterion and adding a constrain in pca procedure so that the singularity phenomenon will not occur experiment results on face recognition suggest that this new approach works well and can be applied even when the number of training samples is one per class

stability of square root domains associated with elliptic systems of pdes on nonsmooth domains abstract we discuss stability of square root domains for uniformly elliptic partial differential operators l a a in l 2 with mixed boundary conditions on with respect to additive perturbations we consider open bounded and connected sets r n n n 1 that satisfy the interior corkscrew condition and prove stability of square root domains of the operator l a with respect to additive potential perturbations v l p l p n 2 special emphasis is put on the case of uniformly elliptic systems with mixed boundary conditions

constrained large margin local projection algorithms and extensions for multimodal dimensionality reduction a constrained large margin local projection cmlp technique for multimodal dimensionality reduction is proposed we elaborate the criterion of cmlp from a pairwise constrained marginal perspective four effective cmlp solution schemes are presented and the corresponding comparative analyses are given an equivalent weighted least squares formulation for cmlp is also detailed cmlp is originated from the criterion of locality preserving projections lpp but cmlp offers a number of attractive advantages over lpp to keep the intrinsic proximity relations of inter class and intra class similarity pairs the localized pairwise cannot link and must link constraints are applied to specify the types of those neighboring pairs by utilizing the cmlp criterion margins between inter and intra class clusters are significantly enlarged as a result multimodal distributions are effectively preserved to further optimize the cmlp criterion one feasible improvement strategy is described with kernel methods we present the kernelized extensions of our approaches mathematical comparisons and analyses between this work and the related works are also detailed extensive simulations including multivariate manifold visualization and classification on the benchmark ucl orl yale umist mit cbcl and usps datasets are conducted to verify the efficiency of our techniques the presented results reveal that our methods are highly competitive with and even outperform some widely used state of the art algorithms

discriminative sparsity preserving projections for image recognition previous works have demonstrated that image classification performance can be significantly improved by manifold learning however performance of manifold learning heavily depends on the manual selection of parameters resulting in bad adaptability in real world applications in this paper we propose a new dimensionality reduction method called discriminative sparsity preserving projections dspp different from the existing sparse subspace algorithms which manually construct a penalty adjacency graph dspp employs sparse representation model to adaptively build both intrinsic adjacency graph and penalty graph with weight matrix and then integrates global within class structure into the discriminant manifold learning objective function for dimensionality reduction extensive experimental results on four image databases demonstrate the effectiveness of the proposed approach we analyzed the vertex points of penalty graph using sparse representation we adaptively constructed both penalty adjacency graph and intrinsic graph our method simultaneously considered the global and local geometric structure of data

probabilistic visual learning for object representation we present an unsupervised technique for visual learning which is based on density estimation in high dimensional spaces using an eigenspace decomposition two types of density estimates are derived for modeling the training data a multivariate gaussian for unimodal distributions and a mixture of gaussians model for multimodal distributions those probability densities are then used to formulate a maximum likelihood estimation framework for visual search and target detection for automatic object recognition and coding our learning technique is applied to the probabilistic visual modeling detection recognition and coding of human faces and nonrigid objects such as hands

fast and orthogonal locality preserving projections for dimensionality reduction the locality preserving projections lpp algorithm is a recently developed linear dimensionality reduction algorithm that has been frequently used in face recognition and other applications however the projection matrix in lpp is not orthogonal thus creating difficulties for both reconstruction and other applications as the orthogonality property is desirable orthogonal lpp olpp has been proposed so that an orthogonal projection matrix can be obtained based on a step by step procedure however this makes the algorithm computationally more expensive therefore in this paper we propose a fast and orthogonal version of lpp called folpp which simultaneously minimizes the locality and maximizes the globality under the orthogonal constraint as a result the computation burden of the proposed algorithm can be effectively alleviated compared with the olpp algorithm experimental results on two face recognition data sets and two hyperspectral data sets are presented to demonstrate the effectiveness of the proposed algorithm

statistical resilience of random populations to random perturbations we consider populations represented by random collections of real valued points and explore their statistical resilience to random perturbations seeking populations whose statistics remain qualitatively unchanged by the action of arbitrary random perturbations of a certain type studying a general physical perturbation scheme we obtain an explicit characterization of statistically resilient populations show that these objects are fractal and comprehensively analyze their topological and statistical structures an application of statistical resilience attained is an alternative explanation of the ubiquity of power law statistics

learning prototype hyperplanes for face verification in the wild in this paper we propose a new scheme called prototype hyperplane learning phl for face verification in the wild using only weakly labeled training samples i e we only know whether each pair of samples are from the same class or different classes without knowing the class label of each sample by leveraging a large number of unlabeled samples in a generic data set our scheme represents each sample in the weakly labeled data set as a mid level feature with each entry as the corresponding decision value from the classification hyperplane referred to as the prototype hyperplane of one support vector machine svm model in which a sparse set of support vectors is selected from the unlabeled generic data set based on the learnt combination coefficients to learn the optimal prototype hyperplanes for the extraction of mid level features we propose a fisher s linear discriminant like fld like objective function by maximizing the discriminability on the weakly labeled data set with a constraint enforcing sparsity on the combination coefficients of each svm model which is solved by using an alternating optimization method then we use the recent work called side information based linear discriminant sild analysis for dimensionality reduction and a cosine similarity measure for final face verification comprehensive experiments on two data sets labeled faces in the wild lfw and youtube faces demonstrate the effectiveness of our scheme

algebraic feature extraction for image recognition based on an optimal discriminant criterion abstract a novel algebraic feature extraction method for image recognition is presented the main difference between the present method and any other existing algebraic feature extraction method is that the present method uses an optimal discriminant criterion to extract algebraic features for the given training image samples a set of optimal discriminant projection vectors is calculated according to a generalized fisher criterion function on the basis of the optimal discriminant projection vector the projective feature vectors of an image i e algebraic features are extracted by projecting the image onto all optimal discriminant projection vectors the method of calculating the set of optimal discriminant projection vectors is discussed a minimum distance classifier is also proposed for classifying projective feature vectors experimental results showed that the present method has good recognition performance an important conclusion about the present method is that the foley sammon optimal set of discriminant vectors is a special case of the set of optimal discriminant projection vectors

on the invariance of spatially inhomogeneous relaxation processes we consider a general relaxation model based on an ensemble of particles propagating randomly in a general space the particles trajectories are independent and identically distributed copies of an arbitrary random spatial curve and each particle has its own parameter triplet initiation epoch velocity and relaxation coefficient the relaxation rate of each particle is dynamic depending on its spatial location and quantified by an arbitrary spatial landscape function we study the relaxation rate of the entire ensemble and characterize the class of poissonian randomizations of the particles parameters that render the ensemble s relaxation rate invariant with respect to both the particles trajectories and the landscape function the invariant relaxation rates turn out to be governed by power laws and display a statistical behavior analogous to anomalous diffusion applications range from chemical reactions to target search and intelligence sifting

semi supervised dimension reduction using trace ratio criterion in this brief we address the trace ratio tr problem for semi supervised dimension reduction we first reformulate the objective function of the recent work semi supervised discriminant analysis sda in a tr form we also observe that in sda the low dimensional data representation f is constrained to be in the linear subspace spanned by the training data matrix x i e f xt w in order to relax this hard constraint we introduce a flexible regularizer f xt w 2 which models the regression residual into the reformulated objective function with such relaxation our method referred to as tr based flexible sda tr fsda can better cope with data sampled from a certain type of nonlinear manifold that is somewhat close to a linear subspace in order to address the non trivial optimization problem in tr fsda we further develop an iterative algorithm to simultaneously solve for the low dimensional data representation f and the projection matrix w moreover we theoretically prove that our iterative algorithm converges to the optimum based on the newton raphson method the experiments on two face databases one shape image database and one webpage database demonstrate that tr fsda outperforms the existing semi supervised dimension reduction methods

multi class fukunaga koontz discriminant analysis for enhanced face recognition linear subspace learning methods such as fisher s linear discriminant analysis lda unsupervised discriminant projection udp and locality preserving projections lpp have been widely used in face recognition applications as a tool to capture low dimensional discriminant information however when these methods are applied in the context of face recognition they often encounter the small sample size problem in order to overcome this problem a separate principal component analysis pca step is usually adopted to reduce the dimensionality of the data however such a step may discard dimensions that contain important discriminative information that can aid classification performance in this work we propose a new idea which we named multi class fukunaga koontz discriminant analysis fkda by incorporating the fukunaga koontz transform within the optimization for maximizing class separation criteria in lda udp and lpp in contrast to traditional lda udp and lpp our approach can work with very high dimensional data as input without requiring a separate dimensionality reduction step to make the scatter matrices full rank in addition the fkda formulation seeks optimal projection direction vectors that are orthogonal which the existing methods cannot guarantee and it has the capability of finding the exact solutions to the trace ratio objective in discriminant analysis problems while traditional methods can only deal with a relaxed and inexact ratio trace objective we have shown using six face database in the context of large scale unconstrained face recognition face recognition with occlusions and illumination invariant face recognition under closed set semi open set and open set recognition scenarios that our proposed fkda significantly outperforms traditional linear discriminant subspace learning methods as well as five other competing algorithms highlightssolve small sample size problem in lda udp lpp using fkt formulation can work with high dimensional data without inverting any scatter matrices finds optimal projection direction vectors that are orthogonal finds exact solutions to the objective in the form of trace ratio improvement in unconstrained face recognition scenarios

sparse and kernel opls feature extraction based on eigenvalue problem solving orthonormalized partial least squares opls is a popular multivariate analysis method to perform supervised feature extraction usually in machine learning papers opls projections are obtained by solving a generalized eigenvalue problem however in statistical papers the method is typically formulated in terms of a reduced rank regression problem leading to a formulation based on a standard eigenvalue decomposition a first contribution of this paper is to derive explicit expressions for matching the opls solutions derived under both approaches and discuss that the standard eigenvalue formulation is also normally more convenient for feature extraction in machine learning more importantly since optimization with respect to the projection vectors is carried out without constraints via a minimization problem inclusion of penalty terms that favor sparsity is straightforward in the paper we exploit this fact to propose modified versions of opls in particular relying on the 1 norm we propose a sparse version of linear opls as well as a non linear kernel opls with pattern selection we also incorporate a group lasso penalty to derive an opls method with true feature selection the discriminative power of the proposed methods is analyzed on a benchmark of classification problems furthermore we compare the degree of sparsity achieved by our methods and compare them with other state of the art methods for sparse feature extraction highlightswe establish the connection between opls and reduced rank regression problem evd formulation a novel sparse opls is proposed which enhances the solution obtained following the procrustres approach we propose a novel sparse kernel opls feature extractor for improved performance interpretability and efficiency

applications of the purely imaginary powers of operators in hilbert spaces in c r acad sci paris299 1984 173 176 we discussed purely imaginary powers aiy y of linear operators a in hilbert spaces here we utilize the results to consider the various problems generation of cosine families in hilbert spaces coincidence of the definition domains of the fractional powers of operators differentiability of the functions of the form a 0 0 1 where a is an operator valued function defined on an interval 0 t and so forth

spectral embedded clustering a framework for in sample and out of sample spectral clustering spectral clustering sc methods have been successfully applied to many real world applications the success of these sc methods is largely based on the manifold assumption namely that two nearby data points in the high density region of a low dimensional data manifold have the same cluster label however such an assumption might not always hold on high dimensional data when the data do not exhibit a clear low dimensional manifold structure e g high dimensional and sparse data the clustering performance of sc will be degraded and become even worse than k means clustering in this paper motivated by the observation that the true cluster assignment matrix for high dimensional data can be always embedded in a linear space spanned by the data we propose the spectral embedded clustering sec framework in which a linearity regularization is explicitly added into the objective function of sc methods more importantly the proposed sec framework can naturally deal with out of sample data we also present a new laplacian matrix constructed from a local regression of each pattern and incorporate it into our sec framework to capture both local and global discriminative information for clustering comprehensive experiments on eight real world high dimensional datasets demonstrate the effectiveness and advantages of our sec framework over existing sc methods and k means based clustering methods our sec framework significantly outperforms sc using the nystrom algorithm on unseen data

stable orthogonal local discriminant embedding for linear dimensionality reduction manifold learning is widely used in machine learning and pattern recognition however manifold learning only considers the similarity of samples belonging to the same class and ignores the within class variation of data which will impair the generalization and stableness of the algorithms for this purpose we construct an adjacency graph to model the intraclass variation that characterizes the most important properties such as diversity of patterns and then incorporate the diversity into the discriminant objective function for linear dimensionality reduction finally we introduce the orthogonal constraint for the basis vectors and propose an orthogonal algorithm called stable orthogonal local discriminate embedding experimental results on several standard image databases demonstrate the effectiveness of the proposed dimensionality reduction approach

multiclass classifiers based on dimension reduction with generalized lda linear discriminant analysis lda has been widely used for dimension reduction of data sets with multiple classes the lda has been recently extended to various generalized lda methods that are applicable regardless of the relative sizes between the data dimension and the number of data items in this paper we propose several multiclass classifiers based on generalized lda glda algorithms taking advantage of the dimension reducing transformation matrix without requiring additional training or parameter optimization a marginal linear discriminant classifier mldc a bayesian linear discriminant classifier bldc and a one dimensional bldc are introduced for multiclass classification our experimental results illustrate that these classifiers produce higher ten fold cross validation accuracy than knn and centroid based classifiers in the reduced dimensional space obtained from glda

on the optimal class representation in linear discriminant analysis linear discriminant analysis lda is a widely used technique for supervised feature extraction and dimensionality reduction lda determines an optimal discriminant space for linear data projection based on certain assumptions e g on using normal distributions for each class and employing class representation by the mean class vectors however there might be other vectors that can represent each class to increase class discrimination in this brief we propose an optimization scheme aiming at the optimal class representation in terms of fisher ratio maximization for lda based data projection compared with the standard lda approach the proposed optimization scheme increases class discrimination in the reduced dimensionality space and achieves higher classification rates in publicly available data sets

fractal poisson processes abstract the central limit theorem clt and extreme value theory evt study respectively the stochastic limit laws of sums and maxima of sequences of independent and identically distributed i i d random variables via an affine scaling scheme in this research we study the stochastic limit laws of populations of i i d random variables via nonlinear scaling schemes the stochastic population limits obtained are fractal poisson processes which are statistically self similar with respect to the scaling scheme applied and which are characterized by two elemental structures i a universal power law structure common to all limits and independent of the scaling scheme applied ii a specific structure contingent on the scaling scheme applied the sum projection and the maximum projection of the population limits obtained are generalizations of the classic clt and evt results extending them from affine to general nonlinear scaling schemes

symmetric subspace learning for image analysis subspace learning sl is one of the most useful tools for image analysis and recognition a large number of such techniques have been proposed utilizing a priori knowledge about the data in this paper new subspace learning techniques are presented that use symmetry constraints in their objective functions the rational behind this idea is to exploit the a priori knowledge that geometrical symmetry appears in several types of data such as images objects faces and so on experiments on artificial facial expression recognition face recognition and object categorization databases highlight the superiority and the robustness of the proposed techniques in comparison with standard sl techniques

compound rank k projections for bilinear analysis in many real world applications data are represented by matrices or high order tensors despite the promising performance the existing 2 d discriminant analysis algorithms employ a single projection model to exploit the discriminant information for projection making the model less flexible in this paper we propose a novel compound rank k projection crp algorithm for bilinear analysis the crp deals with matrices directly without transforming them into vectors and it therefore preserves the correlations within the matrix and decreases the computation complexity different from the existing 2 d discriminant analysis algorithms objective function values of crp increase monotonically in addition the crp utilizes multiple rank k projection models to enable a larger search space in which the optimal solution can be found in this way the discriminant ability is enhanced we have tested our approach on five data sets including uuim cvl pointing 04 usps and coil20 experimental results show that the performance of our proposed crp performs better than other algorithms in terms of classification accuracy

graph optimization for dimensionality reduction with sparsity constraints graph based dimensionality reduction dr methods play an increasingly important role in many machine learning and pattern recognition applications in this paper we propose a novel graph based learning scheme to conduct graph optimization for dimensionality reduction with sparsity constraints godrsc different from most of graph based dr methods where graphs are generally constructed in advance godrsc aims to simultaneously seek a graph and a projection matrix preserving such a graph in one unified framework resulting in an automatically updated graph moreover by applying an l 1 regularizer a sparse graph is achieved which models the locality structure of data and contains natural discriminating information finally extensive experiments on several publicly available uci and face databases verify the feasibility and effectiveness of the proposed method

universal self similarity of propagating populations this paper explores the universal self similarity of propagating populations the following general propagation model is considered particles are randomly emitted from the origin of a d dimensional euclidean space and propagate randomly and independently of each other in space all particles share a statistically common yet arbitrary motion pattern each particle has its own random propagation parameters emission epoch motion frequency and motion amplitude the universally self similar statistics of the particles displacements and first passage times fpts are analyzed statistics which are invariant with respect to the details of the displacement and fpt measurements and with respect to the particles underlying motion pattern analysis concludes that the universally self similar statistics are governed by poisson processes with power law intensities and by the fr echet and weibull extreme value laws

universal generation of statistical self similarity a randomized central limit theorem a universal mechanism for the generation of statistical self similarity i e fractality in the context of random processes is established we consider a generic system which superimposes independent stochastic signals producing a system output all signals share a common statistical signal pattern yet each signal has its own transmission parameters amplitude frequency and initiation epoch we characterize the class of parameter randomizations yielding statistically self similar outputs in a universal fashion i e for whatever signals fed into the system statistically self similar outputs with finite variance further display i anomalous diffusion behavior characterized by power law temporal variance growth and ii 1 f noise behavior characterized by power law power spectra the mechanism presented is a randomized central limit theorem for fractal statistics of random processes

adaptive maximum margin analysis for image recognition abstract most existing discriminant manifold learning methods aim to maximize the margin among nearby data which is determined in the high dimensional original space as such they do not necessarily best maximize the margin between different classes in the low dimensional space which is a critically important property for image classification to handle this problem we propose an adaptive maximum margin analysis amma for feature extraction amma aims to seek a projection matrix that best maximize the margin which is calculated in the low dimensional space it uses sparse representation to adaptively construct the intrinsic and penalty graphs finally an iterative algorithm is developed to solve the projection matrix extensive experimental results on several image databases illustrate the effectiveness of the proposed approach

graph embedding and extensions a general framework for dimensionality reduction a large family of algorithms supervised or unsupervised stemming from statistics or geometry theory has been designed to provide different solutions to the problem of dimensionality reduction despite the different motivations of these algorithms we present in this paper a general formulation known as graph embedding to unify them within a common framework in graph embedding each algorithm can be considered as the direct graph embedding or its linear kernel tensor extension of a specific intrinsic graph that describes certain desired statistical or geometric properties of a data set with constraints from scale normalization or a penalty graph that characterizes a statistical or geometric property that should be avoided furthermore the graph embedding framework can be used as a general platform for developing new dimensionality reduction algorithms by utilizing this framework as a tool we propose a new supervised dimensionality reduction algorithm called marginal fisher analysis in which the intrinsic graph characterizes the intraclass compactness and connects each data point with its neighboring points of the same class while the penalty graph connects the marginal points and characterizes the interclass separability we show that mfa effectively overcomes the limitations of the traditional linear discriminant analysis algorithm due to data distribution assumptions and available projection directions real face recognition experiments show the superiority of our proposed mfa in comparison to lda also for corresponding kernel and tensor extensions

fisher regularized support vector machine this paper proposes fisher regularized support vector machine fishersvm fishersvm is a graph based supervised learning method fishersvm has two regularization terms maximum margin and fisher regularizations fishersvm aims to maximize the margin and minimize the within class scatter support vector machine svm and fisher discriminant analysis fda are two commonly used methods in machine learning and pattern recognition a combined method of the linear svm and fda called svm lda linear discriminant analysis has been proposed only for the linear case this paper generalizes this combined method to the nonlinear case from the view of regularization a fisher regularization is defined and incorporated into svm to obtain a fisher regularized support vector machine fishersvm in fishersvm there are two regularization terms the maximum margin regularization and fisher regularization which allow fishersvm to maximize the classification margin and minimize the within class scatter roughly speaking fishersvm can approximatively fulfill the fisher criterion and obtain good statistical separability this paper also discusses the connections of fishersvm to graph based regularization methods and the mathematical programming method to kernel fisher discriminant analysis kfda it shows that fishersvm can be thought of as a graph based supervised learning method or a robust kfda experimental results on artificial and real world data show that fishersvm has a promising generalization performance

from ornstein uhlenbeck dynamics to long memory processes and fractional brownian motion this article establishes a natural physical path leading from regular ornstein uhlenbeck dynamics to anomalous long memory processes and thereafter to fractional brownian motion considering a system composed of n different parts each part conducting its own ornstein uhlenbeck dynamics and all parts being perturbed by a common external l evy noise we show that the collective system dynamics in the limit n ensuremath rightarrow ensuremath infty converges to a temporal moving average of the driving noise the limiting moving average process in turn can posses a long memory in which case when observed over large time scales further yields fractional brownian motion the temporal correlation structure of the limiting moving average process turns out to be determined by the structural statistical variability of the system s composing parts thus the emergence of a long memory is a consequence of the intrinsic quenched disorder present at the system s formation epoch rather than the consequence of the external annealed disorder carried in continuously by the driving noise

the solution of the kato problem for divergence form elliptic operators with gaussian heat kernel bounds which in turnhas applications to the perturbation theory for certain classes of hyperbolicequations see 16 we remark that 1 3 is equivalent to the oppositeinequality for the square root of the adjoint operator l in 13 14 kato conjectured that an abstract version of 1 3 might hold for regularly accretive operators see 14 18 for the details a counter example to this abstract conjecture was obtained by mcintosh 17 whothen reformulated the conjecture in the following form bearing in mind thatkato s interest in the problem had been motivated by the special case ofelliptic di erential operators conjecture 1 4 the estimate 1 3 holds for lde ned as in 1 2 for anyl

fractal probability laws we explore six classes of fractal probability laws defined on the positive half line weibull frechet levy hyper pareto hyper beta and hyper shot noise each of these classes admits a unique statistical power law structure and is uniquely associated with a certain operation of renormalization all six classes turn out to be one dimensional projections of underlying poisson processes which in turn are the unique fixed points of poissonian renormalizations the first three classes correspond to linear poissonian renormalizations and are intimately related to extreme value theory weibull frechet and to the central limit theorem levy the other three classes correspond to nonlinear poissonian renormalizations pareto s law commonly perceived as the universal fractal probability distribution is merely a special case of the hyper pareto class

benford s law a poisson perspective abstract benford s law is a counterintuitive statistical law asserting that the distribution of leading digits taken from a large ensemble of positive numerical values that range over many orders of scale is logarithmic rather than uniform as intuition suggests in this paper we explore benford s law from a poisson perspective considering ensembles of positive numerical values governed by poisson process statistics we show that this poisson setting naturally accommodates benford s law and i establish a poisson characterization and a poisson multidigit extension of benford s law ii study a system invariant leading digit distribution which generalizes benford s law and establish a poisson characterization and a poisson multidigit extension of this distribution iii explore the universal emergence of the system invariant leading digit distribution couple this universal emergence to the universal emergence of the weibull and frechet extreme value distributions and distinguish the special role of benford s law in this universal emergence iv study the continued fractions counterpart of the system invariant leading digit distribution and establish a poisson characterization of this distribution and v unveil the elemental connection between the system invariant leading digit distribution and its continued fractions counterpart this paper presents a panoramic poisson approach to benford s law to its system invariant generalization and to its continued fractions counterpart

optimal discriminant plane for a small number of samples and design method of classifier on the plane abstract in a previous work zi quan hong and jing yu yang minimum distance classifier on the optimal discriminant plane we suggested and derived a method for constructing classifier on the optimal discriminant plane using minimum distance criteria in this paper the problem of solving optimal discriminant plane for a small number of samples is discussed which is based on the above paper by the same authors in the case of a small number of samples generalized eigenequation ax bx established for a large number of samples usually has no solution because within class scatter matrix is singular to obtain the solution of the generalized eigenequation a new method is suggested in which the singular value perturbation is added to the within class scatter matrix such that the matrix becomes a nonsingular matrix therefore the generalized eigenequation can be solved using existing algorithms we proved that the generalized eigenequations on the optimal discriminant plane are stable in respect of eigenvalues and the generalized eigenvectors are indeed the optimal discriminant directions if the perturbation is subject to some certain conditions the experimental results have shown that our method works well and the minimum distance classifier on the optimal discriminant plane can be constructed with high performance even in the case of a small number of samples

towards robust discriminative projections learning via non greedy l2 1 norm minmax linear discriminant analysis lda is one of the most successful supervised dimensionality reduction methods and has been widely used in many real world applications however the l 2 norm is employed as the distance measure in objective of lda which is sensitive to outliers many previous works improve the robustness of lda by using l1 norm distance however the robustness against outliers is limited and the solver of l1 norm are mostly based on greedy search strategy which is time consuming and easy to get stuck in a local optimum in this paper we propose a novel robust lda measured by l 2 1 norm to learn robust discriminative projections since the proposed model needs to minimize and maximize minmax l 2 1 norm terms simultaneously it is challenging to solve as a result we first systematically derive an efficient iterative optimization algorithm to solve a general ratio minimization problem and rigorously prove its convergence more importantly an efficient non greedy iterative re weighted optimization algorithm is developed based on preceding approach for solving proposed l 2 1 norm minmax problem besides an optimal weighted mean mechanism is driven according to designed objective and solver which can be applied to other approach for robustness improvement extensive experimental results on several real world datasets show the effectiveness of proposed method

multiscale distance matrix for fast plant leaf recognition in this brief we propose a novel contour based shape descriptor called the multiscale distance matrix to capture the shape geometry while being invariant to translation rotation scaling and bilateral symmetry the descriptor is further combined with a dimensionality reduction to improve its discriminative power the proposed method avoids the time consuming pointwise matching encountered in most of the previously used shape recognition algorithms it is therefore fast and suitable for real time applications we applied the proposed method to the task of plan leaf recognition with experiments on two data sets the swedish leaf data set and the icl leaf data set the experimental results clearly demonstrate the effectiveness and efficiency of the proposed descriptor

nonnegative sparse locality preserving hashing it is a np hard problem to optimize the objective function of hash based similarity search algorithms such as spectral hashing and self taught hashing to make the problem solvable existing methods have relaxed the constraints on hash codes from binary values discrete to real values continuous then eigenvalue decomposition was employed to achieve the relaxed real solution the main problem is that the signs of the relaxed continuous solution are mixed such results may deviate severely from the true solution which has lead to significant semantic loss moreover eigenvalue decomposition confronts singularity problem when the dimension of the data is larger than the sample size to address these problems we propose a novel method named nonnegative sparse locality preserving hashing nslph nonnegative and sparse constraints are imposed for a more accurate solution which preserves semantic information well then we have applied nonnegative quadratic programming and multiplicative updating to solve the optimization problem which successfully avoids the singularity problem of the eigenvalue decomposition the extensive experiments presented in this paper demonstrate that the proposed approach outperforms the state of the art algorithms

trace ratio problem revisited dimensionality reduction is an important issue in many machine learning and pattern recognition applications and the trace ratio tr problem is an optimization problem involved in many dimensionality reduction algorithms conventionally the solution is approximated via generalized eigenvalue decomposition due to the difficulty of the original problem however prior works have indicated that it is more reasonable to solve it directly than via the conventional way in this brief we propose a theoretical overview of the global optimum solution to the tr problem via the equivalent trace difference problem eigenvalue perturbation theory is introduced to derive an efficient algorithm based on the newton raphson method theoretical issues on the convergence and efficiency of our algorithm compared with prior literature are proposed and are further supported by extensive empirical results

universal statistics and control of random transport processes a general model of random transport is considered independent agents are stochastically emitted into a space with general topology the agents motion in space is governed by a general random transport mechanism the agents are detected only when in a target zone which is an arbitrary subset of space we explore emission intensities which render the agents target zone statistics universal i e invariant with respect to the underlying spatial topology and transport mechanism the results established determine how to universally control the target zone statistics via the emission intensity and show that the resulting universal statistics are governed by power laws

optimal fisher discriminant analysis using the rank decomposition abstract the fisher optimal discriminant vector is a very efficient means in high dimensional pattern analysis in the case of a small number of samples the within class scatter matrix s w is singular therefore the calculation of the fisher optimal discriminant vector becomes very important in this paper the conception of the rank decomposition of matrices is first introduced and a new method for calculating the fisher optimal discriminant vector is presented which is particularly well suited to the case of a small number of samples in the sense that the scatter matrices are rank deficient the new method is compared with the pseudoinverse and perturbation methods in tian j opt soc am a 5 1670 1672 1988 and hong and yang pattern recognition 24 317 324 1991 an important conclusion is proved in the three methods our method is the best one for calculating the fisher optimal discriminant vectors in the cases of both a large and a small number of samples the experimental results have also shown that our method can give the highest recognition rate in the case of a small number of samples among the three methods

a hierarchical word merging algorithm with class separability measure in image recognition with the bag of features model a small sized visual codebook is usually preferred to obtain a low dimensional histogram representation and high computational efficiency such a visual codebook has to be discriminative enough to achieve excellent recognition performance to create a compact and discriminative codebook in this paper we propose to merge the visual words in a large sized initial codebook by maximally preserving class separability we first show that this results in a difficult optimization problem to deal with this situation we devise a suboptimal but very efficient hierarchical word merging algorithm which optimally merges two words at each level of the hierarchy by exploiting the characteristics of the class separability measure and designing a novel indexing structure the proposed algorithm can hierarchically merge 10 000 visual words down to two words in merely 90 seconds also to show the properties of the proposed algorithm and reveal its advantages we conduct detailed theoretical analysis to compare it with another hierarchical word merging algorithm that maximally preserves mutual information obtaining interesting findings experimental studies are conducted to verify the effectiveness of the proposed algorithm on multiple benchmark data sets as shown it can efficiently produce more compact and discriminative codebooks than the state of the art hierarchical word merging algorithms especially when the size of the codebook is significantly reduced

square roots of elliptic operators consider an elliptic sesquilinear form defined on v v by j u v ajk u xk t6v xj ak u xkv ju t6v xj auvdx where v is a closed subspace of h1 which contains c0 is a bounded lipschitz domain in rn ajk ak j a l and re ajk k j 0 for all cn with 1 let l be the operator with largest domain satisfying j u v lu v for all v then l i is a maximal accretive operator in l2 for a sufficiently large real number it is proved that l i 12 is a bounded operator from v to l2 provided mild regularity of the coefficients is assumed in addition it is shown that if the coefficients depend differentiably on a parameter t in an appropriate sense then the corresponding square root operators also depend differentiably on t the latter result is new even when the forms j are hermitian

the solution of the kato square root problem for second order elliptic operators on rn we prove the kato conjecture for elliptic operators on jfin more precisely we establish that the domain of the square root of a uniformly complex elliptic operator l div av with bounded measurable coefficients in ietn is the

a unified and universal explanation for levy laws and 1 f noises levy laws and 1 f noises are shown to emerge uniquely and universally from a general model of systems which superimpose the transmissions of many independent stochastic signals the signals are considered to follow statistically a common yet arbitrary generic signal pattern which may be either stationary or dissipative each signal is considered to have its own random transmission amplitude and frequency we characterize the amplitude frequency randomizations which render the system output s stationary law and power spectrum universal i e independent of the underlying generic signal pattern the classes of universal stationary laws and power spectra are shown to coincide respectively with the classes of levy laws and 1 f noises thus providing a unified and universal explanation for the ubiquity of these classes of anomalous statistics in various fields of science and engineering

a non greedy algorithm for l1 norm lda recently l1 norm based discriminant subspace learning has attracted much more attention in dimensionality reduction and machine learning however most existing approaches solve the column vectors of the optimal projection matrix one by one with greedy strategy thus the obtained optimal projection matrix does not necessarily best optimize the corresponding trace ratio objective function which is the essential criterion function for general supervised dimensionality reduction in this paper we propose a non greedy iterative algorithm to solve the trace ratio form of l1 norm based linear discriminant analysis we analyze the convergence of our proposed algorithm in detail extensive experiments on five popular image databases illustrate that our proposed algorithm can maximize the objective function value and is superior to most existing l1 lda algorithms

self weighted supervised discriminative feature selection in this brief a novel self weighted orthogonal linear discriminant analysis solda problem is proposed and a self weighted supervised discriminative feature selection ssd fs method is derived by introducing sparsity inducing regularization to the proposed solda problem by using the row sparse projection the proposed ssd fs method is superior to multiple sparse feature selection approaches which can overly suppress the nonzero rows such that the associated features are insufficient for selection more specifically the orthogonal constraint ensures the minimal number of selectable features for the proposed ssd fs method in addition the proposed feature selection method is able to harness the discriminant power such that the discriminative features are selected consequently the effectiveness of the proposed ssd fs method is validated theoretically and experimentally

regularized generalized eigen decomposition with applications to sparse supervised feature extraction and sparse discriminant analysis we propose a general technique for obtaining sparse solutions to generalized eigenvalue problems and call it regularized generalized eigen decomposition rged for decades fisher s discriminant criterion has been applied in supervised feature extraction and discriminant analysis and it is formulated as a generalized eigenvalue problem thus rged can be applied to effectively extract sparse features and calculate sparse discriminant directions for all variants of fisher discriminant criterion based models particularly rged can be applied to matrix based and even tensor based discriminant techniques for instance 2d linear discriminant analysis 2d lda furthermore an iterative algorithm based on the alternating direction method of multipliers is developed the algorithm approximately solves rged with monotonically decreasing convergence and at an acceptable speed for results of modest accuracy numerical experiments based on four data sets of different types of images show that rged has competitive classification performance with existing multidimensional and sparse techniques of discriminant analysis highlightswe propose a new technique called regularized generalized eigen decomposition rged rged solves generalized eigenvalue problems and obtains sparse solutions it is easy and straightforward applying rged to sparse discriminant analysis and feature extraction an algorithm is developed to solve it with monotonically decreasing convergence rged has competitive classification performance comparing with other methods

multiple instance discriminant analysis multiple instance discriminant analysis mida is proposed to cope with the feature extraction problem in multiple instance learning similar to midlabs mida is also derived from linear discriminant analysis lda and both algorithms can be treated as multiple instance extensions of lda different from midlabs which learns from the bag level mida is designed from the instance level mida consists of two versions i e binary class mida b mida and multi class mida m mida which are utilized to cope with binary class standard and multi class multiple instance learning tasks respectively the block coordinate ascent approach by which we seek positive prototypes the most positive instance in a positive bag is termed as the positive prototype of this bag and projection vectors alternatively and iteratively is proposed to optimize b mida and m mida to obtain lower dimensional transformation subspaces extensive experiments empirically demonstrate the effectiveness of b mida and m mida in extracting discriminative components and weakening class label ambiguities for instances in positive bags

a new lda based face recognition system which can solve the small sample size problem a new lda based face recognition system is presented in this paper linear discriminant analysis lda is one of the most popular linear projection techniques for feature extraction the major drawback of applying lda is that it may encounter the small sample size problem in this paper we propose a new lda based technique which can solve the small sample size problem we also prove that the most expressive vectors derived in the null space of the within class scatter matrix using principal component analysis pca are equal to the optimal discriminant vectors derived in the original space using lda the experimental results show that the new lda process improves the performance of a face recognition system signi cantly 2000 pattern recognition society published by elsevier science ltd all rights reserved

multi manifold discriminant isomap for visualization and classification isomap aims to recover the intrinsic geometric structure of manifold by preserving geodesic distances between all pairs of data points however it is an unsupervised dimensionality reduction method usually using class label information can increase the discriminating capability hence a new supervised isomap is proposed in this paper dubbed multi manifold discriminant isomap mmd isomap first data points are partitioned into different manifolds according to their class label information then mmd isomap aims at seeking an optimal nonlinear subspace to preserve the geometrical structure of each manifold according to the isomap criterion meanwhile to enhance the discriminating capability by maximizing the distances between data points of different manifolds finally the corresponding optimization problem is solved by using a majorization algorithm furthermore two new numerical metrics are designed to measure the performance of dimensionality reduction method in both visualization and classification experiments mmd isomap achieves improved performance over many state of the art methods preserve the geometrical structure and enhance the discriminating capability smacof algorithm is introduced to solve the optimization problem two metrics are designed to evaluate dimensionality reduction methods experiments show the effectiveness of the method in visualization and classification

regularized class specific subspace classifier in this paper we mainly focus on how to achieve the translated subspace representation for each class which could simultaneously indicate the distribution of the associated class and the differences from its complementary classes by virtue of the reconstruction problem the class specific subspace classifier cssc problem could be represented as a series of biobjective optimization problems which minimize and maximize the reconstruction errors of the related class and its complementary classes respectively besides the regularization term is specifically introduced to ensure the whole system s stability accordingly a regularized class specific subspace classifier rcssc method can be further proposed based on solving a general quadratic ratio problem the proposed rcssc method consistently converges to the global optimal subspace and translation under the variations of the regularization parameter furthermore the proposed rcssc method could be extended to the unregularized case which is known as unregularized cssc ucssc method via orthogonal decomposition technique as a result the effectiveness and the superiority of both proposed rcssc and ucssc methods can be verified analytically and experimentally

efficient and robust feature extraction by maximum margin criterion in pattern recognition feature extraction techniques are widely employed to reduce the dimensionality of data and to enhance the discriminatory information principal component analysis pca and linear discriminant analysis lda are the two most popular linear dimensionality reduction methods however pca is not very effective for the extraction of the most discriminant features and lda is not stable due to the small sample size problem in this paper we propose some new linear and nonlinear feature extractors based on maximum margin criterion mmc geometrically feature extractors based on mmc maximize the average margin between classes after dimensionality reduction it is shown that mmc can represent class separability better than pca as a connection to lda we may also derive lda from mmc by incorporating some constraints by using some other constraints we establish a new linear feature extractor that does not suffer from the small sample size problem which is known to cause serious stability problems for lda the kernelized nonlinear counterpart of this linear feature extractor is also established in the paper our extensive experiments demonstrate that the new feature extractors are effective stable and efficient

trace ratio criterion based generalized discriminative learning for semi supervised dimensionality reduction dealing with high dimensional data has always been a major problem in many pattern recognition and machine learning applications trace ratio criterion is a criterion that can be applicable to many dimensionality reduction methods as it directly reflects euclidean distance between data points of within or between classes in this paper we analyze the trace ratio problem and propose a new efficient algorithm to find the optimal solution based on the proposed algorithm we are able to derive an orthogonal constrained semi supervised learning framework the new algorithm incorporates unlabeled data into training procedure so that it is able to preserve the discriminative structure as well as geometrical structure embedded in the original dataset under such a framework many existing semi supervised dimensionality reduction methods such as sda lap lda ssdr ssmmc can be improved using our proposed framework which can also be used to formulate a corresponding kernel framework for handling nonlinear problems theoretical analysis indicates that there are certain relationships between linear and nonlinear methods finally extensive simulations on synthetic dataset and real world dataset are presented to show the effectiveness of our algorithms the results demonstrate that our proposed algorithm can achieve great superiority to other state of art algorithms

regularized max min linear discriminant analysis several dimensionality reduction methods based on the max min idea have been proposed in recent years and can obtain good classification performance in this paper inspired by the idea we develop max min linear discriminant analysis mmlda which maximizes the minimum ratio of each two class scatter measure to the within class scatter measure however the method ignores equal emphasis on the distances between class centers and there may be room to improve the classification performance we then propose regularized max min linear discriminant analysis rmmlda which introduces the shannon entropy and the corresponding distance difference regularization terms based on mmlda the changing trends of distances between class centers can be precisely controlled in optimization and the separation between classes can be emphasized approximately equally as a result rmmlda may obtain better classification performance experiments on synthetic data sets and three publicly available data sets demonstrate its effectiveness highlightswe propose max min linear discriminant analysis we present regularized max min linear discriminant analysis using shannon entropy the improved form can obtain better classification results for high dimensional data

multiple kernel sparse representations for supervised and unsupervised learning in complex visual recognition tasks it is typical to adopt multiple descriptors which describe different aspects of the images for obtaining an improved recognition performance descriptors that have diverse forms can be fused into a unified feature space in a principled manner using kernel methods sparse models that generalize well to the test data can be learned in the unified kernel space and appropriate constraints can be incorporated for application in supervised and unsupervised learning in this paper we propose to perform sparse coding and dictionary learning in the multiple kernel space where the weights of the ensemble kernel are tuned based on graph embedding principles such that class discrimination is maximized in our proposed algorithm dictionaries are inferred using multiple levels of 1d subspace clustering in the kernel space and the sparse codes are obtained using a simple levelwise pursuit scheme empirical results for object recognition and image clustering show that our algorithm outperforms existing sparse coding based approaches and compares favorably to other state of the art methods

tensor rank preserving discriminant analysis for facial recognition facial recognition one of the basic topics in computer vision and pattern recognition has received substantial attention in recent years however for those traditional facial recognition algorithms the facial images are reshaped to a long vector thereby losing part of the original spatial constraints of each pixel in this paper a new tensor based feature extraction algorithm termed tensor rank preserving discriminant analysis trpda for facial image recognition is proposed the proposed method involves two stages in the first stage the low dimensional tensor subspace of the original input tensor samples was obtained in the second stage discriminative locality alignment was utilized to obtain the ultimate vector feature representation for subsequent facial recognition on the one hand the proposed trpda algorithm fully utilizes the natural structure of the input samples and it applies an optimization criterion that can directly handle the tensor spectral analysis problem thereby decreasing the computation cost compared those traditional tensor based feature selection algorithms on the other hand the proposed trpda algorithm extracts feature by finding a tensor subspace that preserves most of the rank order information of the intra class input samples experiments on the three facial databases are performed here to determine the effectiveness of the proposed trpda algorithm

noise cascades and levy correlations we explore a general model of stochastic noise cascades which can be illustrated by the example of rain dropping down on the earth and then seeping through layers of ground pouring down layer by layer the rain represents an input noise that is assumed to be spatially uncorrelated and each ground layer represents a stochastic convolution filter as the input noise percolates through the layered filters spatial correlations which are initially nonexistent build up we study this build up of correlations and focus on the following question are there universally emergent forms of spatial correlations the answer is proved affirmative and is shown to be uniquely characterized by power spectra that coincide with the fourier transform of the spherically symmetric l vy distribution we term these universally emergent spatial correlations l vy correlations

face recognition using lda based algorithms low dimensional feature representation with enhanced discriminatory power is of paramount importance to face recognition fr systems most of traditional linear discriminant analysis lda based methods suffer from the disadvantage that their optimality criteria are not directly related to the classification ability of the obtained feature representation moreover their classification accuracy is affected by the small sample size sss problem which is often encountered in fr tasks in this paper we propose a new algorithm that deals with both of the shortcomings in an efficient and cost effective manner the proposed method is compared in terms of classification accuracy to other commonly used fr methods on two face databases results indicate that the performance of the proposed method is overall superior to those of traditional fr approaches such as the eigenfaces fisherfaces and d lda methods

kernel quadratic discriminant analysis for small sample size problem it is generally believed that quadratic discriminant analysis qda can better fit the data in practical pattern recognition applications compared to linear discriminant analysis lda method this is due to the fact that qda relaxes the assumption made by lda based methods that the covariance matrix for each class is identical however it still assumes that the class conditional distribution is gaussian which is usually not the case in many real world applications in this paper a novel kernel based qda method is proposed to further relax the gaussian assumption by using the kernel machine technique the proposed method solves the complex pattern recognition problem by combining the qda solution and the kernel machine technique and at the same time tackles the so called small sample size problem through a regularized estimation of the covariance matrix extensive experimental results indicate that the proposed method is a more sophisticated solution outperforming many traditional kernel based learning algorithms

inexact implementation using krylov subspace methods for large scale exponential discriminant analysis with applications to high dimensionality reduction problems abstract exponential discriminant analysis eda is a generalized discriminant analysis method based on matrix exponential it can essentially overcome the intrinsic difficulty of small sample size problem that exists in the classical linear discriminant analysis lda however for data with high dimensionality one has to solve a large matrix exponential eigenproblem in this method and the time complexity is dominated by the computation of exponential of large matrices in this paper we propose two inexact krylov subspace algorithms for solving the large matrix exponential eigenproblem efficiently the contribution of this work is threefold first we consider how to compute matrix exponential vector products efficiently which is the key step in the krylov subspace method second we compare the discriminant analysis criterion of eda and that of lda from a theoretical point of view third we establish the relationship between the accuracy of the approximate eigenvectors and the distance to nearest neighbor classifier and show why the matrix exponential eigenproblem can be solved approximately in practice numerical experiments on some real world databases show the superiority of our new algorithms over their original counterpart for face recognition

diversity of poissonian populations populations represented by collections of points scattered randomly on the real line are ubiquitous in science and engineering the statistical modeling of such populations leads naturally to poissonian populations poisson processes on the real line with a distinguished maximal point poissonian populations are infinite objects underlying key issues in statistical physics probability theory and random fractals due to their infiniteness measuring the diversity of poissonian populations depends on the lower bound cut off applied this research characterizes the classes of poissonian populations whose diversities are invariant with respect to the cut off level applied and establishes an elemental connection between these classes and extreme value theory the measures of diversity considered are variance and dispersion simpson s index and inverse participation ratio shannon s entropy and renyi s entropy and gini s index

single site localization via maximum discrimination multipath fingerprinting a novel approach to single site localization based on maximum discrimination multipath fingerprinting is presented in contrast to the existing approach which extracts each fingerprint only from the data of that location the new approach uses also the data of all the other locations in the database and leverages it to extract a fingerprint that is as different as possible from the other fingerprints in the database the performance of this approach validated with both simulated and real data is superior to the existing approach demonstrating single site localization accuracy of 1 m in typical indoor environments the new approach has also a lower computational complexity

adaptive data embedding framework for multiclass classification the objective of this paper is the design of an engine for the automatic generation of supervised manifold embedding models it proposes a modular and adaptive data embedding framework for classification referred to as defc which realizes in different stages including initial data preprocessing relation feature generation and embedding computation for the computation of embeddings the concepts of friend closeness and enemy dispersion are introduced to better control at local level the relative positions of the intraclass and interclass data samples these are shown to be general cases of the global information setup utilized in the fisher criterion and are employed for the construction of different optimization templates to drive the defc model generation for model identification we use a simple but effective bilevel evolutionary optimization which searches for the optimal model and its best model parameters the effectiveness of defc is demonstrated with experiments using noisy synthetic datasets possessing nonlinear distributions and real world datasets from different application fields

formulating robust linear regression estimation as a one class lda criterion discriminative hat matrix linear discriminant analysis such as fisher s criterion is a statistical learning tool traditionally devoted to separating a training dataset into two or even several classes by the way of linear decision boundaries in this paper we show that this tool can formalize the robust linear regression problem as a robust estimator will do more precisely we develop a one class fischer s criterion in which the maximization provides both the regression parameters and the separation of the data in two classes typical data and atypical data or outliers this new criterion is built on the statistical properties of the subspace decomposition of the hat matrix from this angle we improve the discriminative properties of the hat matrix which is traditionally used as outlier diagnostic measure in linear regression naturally we call this new approach discriminative hat matrix the proposed algorithm is fully nonsupervised and needs only the initialization of one parameter synthetic and real datasets are used to study the performance both in terms of regression and classification of the proposed approach we also illustrate its potential application to image recognition and fundamental matrix estimation in computer vision

enhanced fisher discriminant criterion for image recognition many previous studies have shown that image recognition can be significantly improved by fisher linear discriminant analysis flda technique however flda ignores the variation of data points from the same class which characterizes the most important modes of variability of patterns and helps to improve the generalization capability of flda thus the performance of flda on testing data is not good enough in this paper we propose an enhanced fisher discriminant criterion efdc efdc explicitly considers the intra class variation and incorporates the intra class variation into the fisher discriminant criterion to build a robust and efficient dimensionality reduction function efdc obtains a subspace which best detects the discriminant structure and simultaneously preserves the modes of variability of patterns which will result in stable intraclass representation experimental results on four image database show a clear improvement over the results of flda based methods

discriminative graph embedding for label propagation in many applications the available information is encoded in graph structures this is a common problem in biological networks social networks web communities and document citations we investigate the problem of classifying nodes labels on a similarity graph given only a graph structure on the nodes conventional machine learning methods usually require data to reside in some euclidean spaces or to have a kernel representation applying these methods to nodes on graphs would require embedding the graphs into these spaces by embedding and then learning the nodes on graphs most methods are either flexible with different learning objectives or efficient enough for large scale applications we propose a method to embed a graph into a feature space for a discriminative purpose our idea is to include label information into the embedding process making the space representation tailored to the task we design embedding objective functions that the following learning formulations become spectral transforms we then reformulate these spectral transforms into multiple kernel learning problems our method while being tailored to the discriminative tasks is efficient and can scale to massive data sets we show the need of discriminative embedding on some simulations applying to biological network problems our method is shown to outperform baselines

randomized central limit theorems a unified theory the central limit theorems clts characterize the macroscopic statistical behavior of large ensembles of independent and identically distributed random variables the clts assert that the universal probability laws governing ensembles aggregate statistics are either gaussian or levy and that the universal probability laws governing ensembles extreme statistics are frechet weibull or gumbel the scaling schemes underlying the clts are deterministic scaling all ensemble components by a common deterministic scale however there are random environment settings in which the underlying scaling schemes are stochastic scaling the ensemble components by different random scales examples of such settings include holtsmark s law for gravitational fields and the stretched exponential law for relaxation times in this paper we establish a unified theory of randomized central limit theorems rclts in which the deterministic clt scaling schemes are replaced with stochastic scaling schemes and present randomized counterparts to the classic clts the rclt scaling schemes are shown to be governed by poisson processes with power law statistics and the rclts are shown to universally yield the levy frechet and weibull probability laws

the kato square root problem for mixed boundary conditions we consider the negative laplacian subject to mixed boundary conditions on a bounded domain we prove under very general geometric assumptions that slightly above the critical exponent 12 its fractional power domains still coincide with suitable sobolev spaces of optimal regularity in combination with a reduction theorem recently obtained by the authors this solves the kato square root problem for elliptic second order operators and systems in divergence form under the same geometric assumptions thereby we answer a question posed by j l lions in 1962 30

a generalized optimal set of discriminant vectors abstract a generalized optimal set of discriminant vectors for linear feature extraction is presented first the criteria of selecting the generalized optimal discriminant vectors are introduced and then a unified solving method is derived to solve the vectors of the generalized optimal set in both cases of a large number of samples and a small number of samples the experimental results show that the present method is superior to the foley sammon method foley and sammon ieee trans comput 24 281 289 1975 the positive pseudoinverse method tian et al opt engng 25 7 834 839 1986 the perturbation method hong and yang pattern recognition 24 317 324 1991 and the matrix rank decomposition method cheng et al pattern recognition 25 101 111 1992 in terms of correct classification rate