scaled rotation regularization abstract a new regularization method a scaled rotation is proposed and compared with the standard linear regularized discriminant analysis a sense of the method consists in the singular value decomposition s tdt of a sample covariance matrix s and a use of the following representation of an inverse of the covariance matrix s 1 t d i 1 t for certain data structures the scaled rotation helps to reduce the generalization error in small learning set and high dimensionality cases efficacy of the scaled rotation increases if one transforms the data by y d i 1 2 t x and uses an optimally stopped single layer perceptron classifier afterwards

improved discriminate analysis for high dimensional data and its application to face recognition many pattern recognition applications involve the treatment of high dimensional data and the small sample size problem principal component analysis pca is a common used dimension reduction technique linear discriminate analysis lda is often employed for classification pca plus lda is a famous framework for discriminant analysis in high dimensional space and singular cases in this paper we examine the theory of this framework and find out that even if there is no small sample size problem the pca dimension reduction cannot guarantee the subsequent successful application of lda we thus develop an improved discriminate analysis method by introducing an inverse fisher criterion and adding a constrain in pca procedure so that the singularity phenomenon will not occur experiment results on face recognition suggest that this new approach works well and can be applied even when the number of training samples is one per class

ridge regression biased estimation for nonorthogonal problems in multiple regression it is shown that parameter estimates based on minimum residual sum of squares have a high probability of being unsatisfactory if not incorrect if the prediction vectors are not orthogonal proposed is an estimation procedure based on adding small positive quantities to the diagonal of x x introduced is the ridge trace a method for showing in two dimensions the effects of nonorthogonality it is then shown how to augment x x to obtain biased estimates with smaller mean square error

estimation of the mean of a multivariate normal distribution estimation of the means of independent normal random variables is considered using sum of squared errors as loss an unbiased estimate of risk is obtained for an arbitrary estimate and certain special classes of estimates are then discussed the results are applied to smoothing by use of moving averages and to trimmed analogs of the james stein estimate a suggestion is made for calculating approximate confidence sets for the mean vector centered at an arbitrary estimate

chaotic phase similarities and recurrences in a damped driven duffing oscillator we report strong evidence of remarkably close periodic repetitions of the structuring of the parameter space of a damped driven duffing oscillator as the amplitude of the drive increases families of period adding cascades and some intricate networks of periodic oscillations embedded in chaotic phases are also found to recur closely as the driving force grows such surprising regularities suggest that some hitherto unknown renormalization mechanism may be operating in higher codimension controlling the alternation of chaos and order in parameter space of certain flows

towards a generalized singular value decomposition we suggest a form for and give a constructive derivation of the generalized singular value decomposition of any two matrices having the same number of columns we outline its desirable characteristics and compare it to an earlier suggestion by van loan siam j numer anal 13 1976 pp 76 83 the present form largely follows from the work of van loan but is slightly more general and computationally more amenable than that in the paper cited we also prove a useful extension of a theorem of stewart siam rev 19 1977 pp 634 662 on unitary decompositions of submatrices of a unitary matrix

the discrimination subspace model abstract for a p variate normal random vector measured in two populations we propose a method of discrimination under the constraint that all differences between the two populations occur in a subspace of dimension q p this method of classification is based on the discrimination subspace model denoted by dsm q and is intermediate between linear and quadratic discrimination it combines the ideas of dimension reduction and constraints on the parameter space thus substantially reducing the number of parameters to be estimated the maximum likelihood estimators of the model are presented and the performance of dsm q versus quadratic and linear discrimination is assessed via simulation it is generally shown that discrimination based on dsm q consistently yields noticeably lower expected actual error rates relative to the traditional methods the method is illustrated with a real data example and is compared to linear and quadratic discrimination using a leave one out method the example confirms t

spectral density of sparse sample covariance matrices applying the replica method of statistical mechanics we evaluate the eigenvalue density of the large random matrix sample covariance matrix of the form j a t a where a is an m n real sparse random matrix the difference from a dense random matrix is the most significant in the tail region of the spectrum we compare the results of several approximation schemes focusing on the behaviour in the tail region

inferring hidden states in langevin dynamics on large networks average case performance we present average performance results for dynamical inference problems in large networks where a set of nodes is hidden while the time trajectories of the others are observed examples of this scenario can occur in signal transduction and gene regulation networks we focus on the linear stochastic dynamics of continuous variables interacting via random gaussian couplings of generic symmetry we analyze the inference error given by the variance of the posterior distribution over hidden paths in the thermodynamic limit and as a function of the system parameters and the ratio ensuremath alpha between the number of hidden and observed nodes by applying kalman filter recursions we find that the posterior dynamics is governed by an effective drift that incorporates the effect of the observations we present two approaches for characterizing the posterior variance that allow us to tackle respectively equilibrium and nonequilibrium dynamics the first appeals to random matrix theory and reveals average spectral properties of the inference error and typical posterior relaxation times the second is based on dynamical functionals and yields the inference error as the solution of an algebraic equation

sparse orthogonal linear discriminant analysis in this paper sparse orthogonal linear discriminant analysis olda is studied the main contributions of the present work include the following i all minimum frobenius norm dimension solutions of the optimization problem used for establishing olda are characterized explicitly and ii this explicit characterization leads to two numerical algorithms for computing a sparse linear transformation for olda the first is based on the gradient flow approach while the second is a sequential linear bregman method we experiment with real world datasets to illustrate that the sequential linear bregman method is much better than the gradient flow approach the sequential linear bregman method always achieves comparable classification accuracy with the normal olda satisfactory sparsity and orthogonality and acceptable cpu times

lower current large deviations for zero range processes on a ring we study lower large deviations for the current of totally asymmetric zero range processes on a ring with concave current density relation we use an approach by jensen and varadhan which has previously been applied to exclusion processes to realize current fluctuations by travelling wave density profiles corresponding to non entropic weak solutions of the hyperbolic scaling limit of the process we further establish a dynamic transition where large deviations of the current below a certain value are no longer typically attained by non entropic weak solutions but by condensed profiles where a non zero fraction of all the particles accumulates on a single fixed lattice site this leads to a general characterization of the rate function which is illustrated by providing detailed results for four generic examples of jump rates including constant rates decreasing rates unbounded sublinear rates and asymptotically linear rates our results on the dynamic transition are supported by numerical simulations using a cloning algorithm

what is the effective impact of the explosive orbital growth in discrete time one dimensional polynomial dynamical systems abstract we study the distribution of periodic orbits in one dimensional two parameter maps specifically we report an exact expression to quantify the growth of the number of periodic orbits for discrete time dynamical systems governed by polynomial equations of motion of arbitrary degree in addition we compute high resolution phase diagrams for quartic and for both normal forms of cubic dynamics and show that their stability phases emerge all distributed in a similar way preserving a characteristic invariant ordering such coincidences are remarkable since our exact expression shows the total number of orbits of these systems to differ dramatically by more than several millions even for quite low periods all this seems to indicate that surprisingly the total number and the distribution of stable phases is not significantly affected by the specific nature of the nonlinearity present in the equations of motion

regularized discriminant analysis abstract linear and quadratic discriminant analysis are considered in the small sample high dimensional setting alternatives to the usual maximum likelihood plug in estimates for the covariance matrices are proposed these alternatives are characterized by two parameters the values of which are customized to individual situations by jointly minimizing a sample based estimate of future misclassification risk computationally fast implementations are presented and the efficacy of the approach is examined through simulation studies and application to data these studies indicate that in many circumstances dramatic gains in classification accuracy can be achieved

information loss of the mahalanobis distance in high dimensions application to feature selection when an infinite training set is used the mahalanobis distance between a pattern measurement vector of dimensionality d and the center of the class it belongs to is distributed as a chi2 with d degrees of freedom however the distribution of mahalanobis distance becomes either fisher or beta depending on whether cross validation or resubstitution is used for parameter estimation in finite training sets the total variation between chi2 and fisher as well as between chi2 and beta allows us to measure the information loss in high dimensions the information loss is exploited then to set a lower limit for the correct classification rate achieved by the bayes classifier that is used in subset feature selection

joint estimation of inverse covariance matrices lying in an unknown subspace we consider the problem of joint estimation of inverse covariance matrices lying in an unknown subspace of the linear space of symmetric matrices we perform the estimation using groups of measurements with different covariances assuming the inverse covariances span a low dimensional subspace our aim is to determine this subspace and to exploit this knowledge in order to improve the estimation we develop a novel optimization algorithm discovering and exploiting the underlying low dimensional subspace we provide a computationally efficient algorithm and derive a tight upper performance bound numerical simulations on synthetic and real world data are presented to illustrate the performance benefits of the algorithm

slyrb measures natural invariant measures for chaotic systems in many applications it is useful to consider not only the set that constitutes an attractor but also if it exists the asymptotic distribution of a typical trajectory converging to the attractor indeed in the physics literature such a distribution is often assumed to exist when it exists it is called a natural invariant measure the results by lasota and yorke and by sinai ruelle and bowen represent two approaches both of which establish the existence of an invariant measure the goal of this paper is to relate the lasota yorke measure for chaotic attractors in one dimensional maps and the sinai ruelle bowen measure for chaotic attractors in higher dimensional dynamical systems we introduce the notion of slyrb measure we pronounce the term slyrb as a single word slurb the srb concept of measure can be motivated by asking how a trajectory from a typical initial point is distributed asymptotically similarly the slyrb concept of measure can be motivated by asking what the average distribution is for trajectories of a large collection of initial points in some region not necessarily restricted to a single basin the latter is analogous to ask where all the rain drops from a rain storm go and the former asks about where a single rain drop goes perhaps winding up distributed throughout a particular lake

model sparsity and brain pattern interpretation of classification models in neuroimaging interest is increasing in applying discriminative multivariate analysis techniques to the analysis of functional neuroimaging data model interpretation is of great importance in the neuroimaging context and is conventionally based on a brain map derived from the classification model in this study we focus on the relative influence of model regularization parameter choices on both the model generalization the reliability of the spatial patterns extracted from the classification model and the ability of the resulting model to identify relevant brain networks defining the underlying neural encoding of the experiment for a support vector machine logistic regression and fisher s discriminant analysis we demonstrate that selection of model regularization parameters has a strong but consistent impact on the generalizability and both the reproducibility and interpretable sparsity of the models for both 2 and 1 regularization importantly we illustrate a trade off between model spatial reproducibility and prediction accuracy we show that known parts of brain networks can be overlooked in pursuing maximization of classification accuracy alone with either 2 and or 1 regularization this supports the view that the quality of spatial patterns extracted from models cannot be assessed purely by focusing on prediction accuracy our results instead suggest that model regularization parameters must be carefully selected so that the model and its visualization enhance our ability to interpret the brain

the dynamics of the henon map the purpose of this paper is to develop machinery which is suitable for a study of the long time behavior of systems such as the henon map with expansion combined with strong contraction our study is modeled on the treatment of the one dimensional system x 1 ax 2 and we study the perturbation of a from the value a 2 and b small a surprising number of complications arise when b 0 the computer simulations give some indication of the complicated structure of the attractor but it can safely be stated that the real difficulties do not show up the central aim of the paper is to introduce what we consider the relevant concepts and techniques needed to study mappings such as the henon one

dissecting shrimps results for some one dimensional physical models this paper describes how certain shrimp like clusters of stability organize themselves in the parameter space of dynamical systems clusters are composed of an infinite affine similar repetition of a basic elementary cell containing two primary noble points a head and a tail defining an axis of approximate symmetry knowledge of the axis and the skewness of the k periodic main cell of a k 2 cluster is enough to define the orientation of the whole cluster in space peculiarly simple directions along which shrimp like clusters align are formed by the locus of doubly degenerate saddle zero multipliers corresponding to the main shrimp head in addition we report a family of models having the boundaries of all isoperiodic domains of stability totally degenerate and describe different aspects of their mathematical arrangement and some of their consequences for example that shrimps are diffeomorphic copies of shrimps

hydrodynamic limit of mean zero condensing zero range processes with sub critical initial profiles condensing zero range processes are interacting particle systems with zero range interaction exhibiting phase separation at densities above a finite critical density we prove the hydrodynamic limit of mean zero condensing zero range processes with bounded local jump rate for sub critical initial profiles i e for initial profiles that are everywhere strictly below the critical density the proof is based on h t yau s relative entropy method and is made possible by a generalisation of the one block estimate

person re identification by dual regularized kiss metric learning person re identification aims to match the images of pedestrians across different camera views from different locations this is a challenging intelligent video surveillance problem that remains an active area of research due to the need for performance improvement person re identification involves two main steps feature representation and metric learning although the keep it simple and straightforward kiss metric learning method for discriminative distance metric learning has been shown to be effective for the person re identification the estimation of the inverse of a covariance matrix is unstable and indeed may not exist when the training set is small resulting in poor performance here we present dual regularized kiss dr kiss metric learning by regularizing the two covariance matrices dr kiss improves on kiss by reducing overestimation of large eigenvalues of the two estimated covariance matrices and in doing so guarantees that the covariance matrix is irreversible furthermore we provide theoretical analyses for supporting the motivations specifically we first prove why the regularization is necessary then we prove that the proposed method is robust for generalization we conduct extensive experiments on three challenging person re identification datasets viper grid and cuhk 01 and show that dr kiss achieves new state of the art performance

large deviations of the current for driven periodic diffusions we study the large deviations of the time integrated current for a driven diffusion on the circle often used as a model of nonequilibrium systems we obtain the large deviation functions describing the current fluctuations using a fourier bloch decomposition of the so called tilted generator and we also construct from this decomposition the effective biased auxiliary or driven markov process describing the diffusion as current fluctuations are observed in time this effective process provides a clear physical explanation of the various fluctuation regimes observed it is used here to obtain an upper bound on the current large deviation function which we compare to a recently derived entropic bound and to study the low noise limit of large deviations

absolutely continuous invariant measures for one parameter families of one dimensional maps given a one parameter familyf x of maps of the interval 0 1 we consider the set of parameter values for whichf has an invariant measure absolutely continuous with respect to lebesgue measure we show that this set has positive measure for two classes of maps i f x f x where 0 4 andf x is a functionc3 near the quadratic mapx 1 x and ii f x f x mod 1 wheref isc3 f 0 f 1 0 andf has a unique nondegenerate critical point in 0 1

estimates of optimal storage conditions in neural network memories based on random matrix theory the authors formulate a method for estimating the critical conditions for storage of sets of data in neural network memory this variational method is based on random matrix theory and calculating the average spectrum of a matrix whose elements are given by overlaps of the stored patterns several generic cases of random overlap matrices are considered they investigate the cases of simply uncorrelated random patterns and spatially and semantically correlated ones they obtain bounds of the critical curve in the control parameters space which determine the stability of the stored data sets

a study of regularized gaussian classifier in high dimension small sample set case based on mdl principle with application to spectrum recognition in classifying high dimensional patterns such as stellar spectra by a gaussian classifier the covariance matrix estimated with a small number sample set becomes unstable leading to degraded classification accuracy in this paper we investigate the covariance matrix estimation problem for small number samples with high dimension setting based on minimum description length mdl principle a new covariance matrix estimator is developed and a formula for fast estimation of regularization parameters is derived experiments on spectrum pattern recognition are conducted to investigate the classification accuracy with the developed covariance matrix estimator higher classification accuracy results are obtained and demonstrated in our new approach

sparse canonical correlation analysis new formulation and algorithm in this paper we study canonical correlation analysis cca which is a powerful tool in multivariate data analysis for finding the correlation between two sets of multidimensional variables the main contributions of the paper are 1 to reveal the equivalent relationship between a recursive formula and a trace formula for the multiple cca problem 2 to obtain the explicit characterization for all solutions of the multiple cca problem even when the corresponding covariance matrices are singular 3 to develop a new sparse cca algorithm and 4 to establish the equivalent relationship between the uncorrelated linear discriminant analysis and the cca problem we test several simulated and real world datasets in gene classification and cross language document retrieval to demonstrate the effectiveness of the proposed algorithm the performance of the proposed method is competitive with the state of the art sparse cca algorithms

accuracy of pseudo inverse covariance learning a random matrix theory analysis for many learning problems estimates of the inverse population covariance are required and often obtained by inverting the sample covariance matrix increasingly for modern scientific data sets the number of sample points is less than the number of features and so the sample covariance is not invertible in such circumstances the moore penrose pseudo inverse sample covariance matrix constructed from the eigenvectors corresponding to nonzero sample covariance eigenvalues is often used as an approximation to the inverse population covariance matrix the reconstruction error of the pseudo inverse sample covariance matrix in estimating the true inverse covariance can be quantified via the frobenius norm of the difference between the two the reconstruction error is dominated by the smallest nonzero sample covariance eigenvalues and diverges as the sample size becomes comparable to the number of features for high dimensional data we use random matrix theory techniques and results to study the reconstruction error for a wide class of population covariance matrices we also show how bagging and random subspace methods can result in a reduction in the reconstruction error and can be combined to improve the accuracy of classifiers that utilize the pseudo inverse sample covariance matrix we test our analysis on both simulated and benchmark data sets

cross validation the jackknife and the bootstrap excess error estimation in forward logistic regression abstract given a prediction rule based on a set of patients what is the probability of incorrectly predicting the outcome of a new patient call this probability the true error an optimistic estimate is the apparent error or the proportion of incorrect predictions on the original set of patients and it is the goal of this article to study estimates of the excess error or the difference between the true and apparent errors i consider three estimates of the excess error cross validation the jackknife and the bootstrap using simulations and real data the three estimates for a specific prediction rule are compared when the prediction rule is allowed to be complicated overfitting becomes a real danger and excess error estimation becomes important the prediction rule chosen here is moderately complicated involving a variable selection procedure based on forward logistic regression

subclass graph embedding and a marginal fisher analysis paradigm subspace learning techniques have been extensively used for dimensionality reduction dr in many pattern classification problem domains recently methods like subclass discriminant analysis sda and clustering based discriminant analysis cda which use subclass information for the discrimination between the data classes have attracted much attention in parallel important work has been accomplished on graph embedding ge which is a general framework unifying several subspace learning techniques in this paper ge has been extended in order to integrate subclass discriminant information resulting to the novel subclass graph embedding sge framework which is the main contribution of our work it is proven that sge encapsulates a diversity of both supervised and unsupervised unimodal methods like locality preserving projections lpp principal component analysis pca and linear discriminant analysis lda the theoretical link of sda and cda methods with sge is also established along these lines it is shown that sge comprises a generalization of the typical ge framework including subclass dr methods moreover it allows for an easy utilization of kernels for confronting non linearly separable data employing sge in this paper a novel dr algorithm which uses subclass discriminant information called subclass marginal fisher analysis smfa has been proposed through a series of experiments on various real world datasets it is shown that smfa outperforms in most of the cases the state of the art demonstrating the efficacy and power of sge as a platform to develop new methods highlightsgraph embedding is extended in order to integrate subclass informationthe novel subclass graph embedding framework is proposed the kernelized version of the new framework is presentedsubclass graph embedding encapsulates various subspace learning methods a novel subclass marginal fisher analysis method is proposed

joint deconvolution and classification with applications to passive acoustic underwater multipath this paper addresses the problem of classifying signals that have been corrupted by noise and unknown linear time invariant lti filtering such as multipath given labeled uncorrupted training signals a maximum a posteriori approach to the deconvolution and classification is considered which produces estimates of the desired signal the unknown channel and the class label for cases in which only a class label is needed the classification accuracy can be improved by not committing to an estimate of the channel or signal a variant of the quadratic discriminant analysis qda classifier is proposed that probabilistically accounts for the unknown lti filtering and which avoids deconvolution the proposed qda classifier can work either directly on the signal or on features whose transformation by lti filtering can be analyzed as an example a classifier for subband power features is derived results on simulated data and real bowhead whale vocalizations show that jointly considering deconvolution with classification can dramatically improve classification performance over traditional methods over a range of signal to noise ratios

first order tree type dependence between variables and classification performance structuralization of the covariance matrix reduces the number of parameters to be estimated from the training data and does not affect an increase in the generalization error asymptotically as both the number of dimensions and training sample size grow a method to benefit from approximately correct assumptions about the first order tree dependence between components of the feature vector is proposed we use a structured estimate of the covariance matrix to decorrelate and scale the data and to train a single layer perceptron in the transformed feature space we show that training the perceptron can reduce negative effects of inexact a priori information experiments performed with 13 artificial and 10 real world data sets show that the first order tree type dependence model is the most preferable one out of two dozen of the covariance matrix structures investigated

estimating the error rate of a prediction rule improvement on cross validation abstract we construct a prediction rule on the basis of some data and then wish to estimate the error rate of this rule in classifying future observations cross validation provides a nearly unbiased estimate using only the original data cross validation turns out to be related closely to the bootstrap estimate of the error rate this article has two purposes to understand better the theoretical basis of the prediction problem and to investigate some related estimators which seem to offer considerably improved estimation in small samples

functional linear discriminant analysis for irregularly sampled curves we introduce a technique for extending the classical method of linear discriminant analysis lda to data sets where the predictor variables are curves or functions this procedure which we call functional linear discriminant analysis flda is particularly useful when only fragments of the curves are observed all the techniques associated with lda can be extended for use with flda in particular flda can be used to produce classifications on new test curves give an estimate of the discriminant function between classes and provide a one or two dimensional pictorial representation of a set of curves we also extend this procedure to provide generalizations of quadratic and regularized discriminant analysis

towards a theory of early visual processing we propose a theory of the early processing in the mammalian visual pathway the theory is formulated in the language of information theory and hypothesizes that the goal of this processing is to recode in order to reduce a generalized redundancy subject to a constraint that specifies the amount of average information preserved in the limit of no noise this theory becomes equivalent to barlow s redundancy reduction hypothesis but it leads to very different computational strategies when noise is present a tractable approach for finding the optimal encoding is to solve the problem in successive stages where at each stage the optimization is performed within a restricted class of transfer functions we explicitly find the solution for the class of encodings to which the parvocellular retinal processing belongs namely linear and nondivergent transformations the solution shows agreement with the experimentally observed transfer functions at all levels of signal to noise

compound windows of the henon map abstract for the two parameter second order henon map the shapes and locations of the periodic windows continua of parameter values for which solutions x 0 x 1 can be stably periodic embedded in larger regions where chaotic solutions or solutions of other periods prevail are found by a random searching procedure and displayed graphically many windows have a typical shape consisting of a central body from which four narrow antennae extend such windows to be called compound windows are often arranged in bands to be called window streets that are made up largely of small detected but poorly resolved compound windows for each fundamental subwindow the portion of a window where a fundamental period prevails a stability measure u is introduced where the solution is stable u 1 curves of constant u are found by numerical integration along one line in parameter space the henon map reduces to the one parameter first order logistic map and two antennae from each compound window intersect this line the curves where u 1 and u 1 that bound either antenna are close together within these intersections but as either curve with u 1 leaves the line it diverges from the curve where u 1 crosses the other curve where u 1 and nears the other curve where u 1 forming another antenna the region bounded by the numerically determined curves coincides with the subwindow as found by random searching a fourth degree equation for an idealized curve of constant u is established points in parameter space producing periodic solutions where x 0 x m 0 for given values of m are found to lie on cantor sets of curves that closely fit the window streets points producing solutions where x 0 x m 0 and satisfying a third condition approximating the condition that x n be bounded as n lie on curves to be called street curves of order m that approximate individual members of the cantor set and individual window streets compound windows of period m m tend to occur near the intersections of street curves of orders m and m some exceptions to what appear to be fairly general results are noted the exceptions render it difficult to establish general theorems

from mandelbrot like sets to arnold tongues a transition from mandelbrot like sets to arnold tongues is characterized via a coupling of two non identical quadratic maps proposed by us a two dimensional parameter space considering the parameters of the individual quadratic maps was used to demonstrate numerically the event the location of the parameter sets where naimark sacker bifurcations occur which is exactly the place where arnold tongues of arbitrary periods are born was computed analytically

diffusions conditioned on occupation measures a markov process fluctuating away from its typical behavior can be represented in the long time limit by another markov process called the effective or driven process having the same stationary states as the original process conditioned on the fluctuation observed we construct here this driven process for diffusions spending an atypical fraction of their evolution in some region of state space corresponding mathematically to stochastic differential equations conditioned on occupation measures as an illustration we consider the langevin equation conditioned on staying for a fraction of time in different intervals of the real line including the positive half line which leads to a generalization of the brownian meander problem other applications related to quasi stationary distributions metastable states noisy chemical reactions queues and random walks are discussed

estimation of a covariance matrix under stein s loss on decrit la technique generale de stein pour ameliorer l estimateur sans biais invariant le meilleur et l estimateur minimax de la matrice de covariance normale

efficient integration of sufficient dimension reduction and prediction in discriminant analysis abstractsufficient dimension reduction sdr methods are popular model free tools for preprocessing and data visualization in regression problems where the number of variables is large unfortunate

generalized consistent error estimator of linear discriminant analysis a classifier is epistemologically vacuous without an accurate estimate of its true error rate in situations where the number of sample points is of the same order of magnitude as the dimension of observations serious issues arise with respect to the performance of error estimators in this paper we place the problem of synthesizing an error rate estimator of a common linear classifier in an asymptotic setting in which the number of sample points is kept comparable in magnitude to the dimension of observations double asymptotic we construct a generalized consistent estimator of the true error rate for linear discriminant analysis in the multivariate gaussian model under the assumption of a common covariance matrix in other words the estimator converges to true error rate in the double asymptotic sense we employ simulations using both synthetic and real data to compare the performance of the new estimator to the classical consistent estimator of the true error plug in estimator as well as other well known estimators we observe that the constructed estimator can outperform other estimators of the true error in many situations in terms of bias and root mean square rms error

robust singular value decompositions a new approach to projection pursuit abstract robust location and covariance estimators are developed via general m estimation for covariance matrix eigenvectors and eigenvalues the solution to this gm estimation problem is obtained by transforming it into a series of robust regression problems based on a new algorithm for the singular value decomposition it is shown here that the singular value decomposition can be represented as an iteration of two steps a least squares regression fit of the data matrix followed by a rotation to the regression hyperplanes an algorithm to obtain the solution to this gm estimation problem is presented along with results of a monte carlo study and examples of its application in addition it is shown how the output of this algorithm can be used to numerically search for multivariate outliers which is especially useful in exploratory data analysis with high dimensional data and large sample sizes where standard graphical techniques are difficult to implement because the algorithm computes robust estima

novel fisher discriminant classifiers at the present several applications need to classify high dimensional points belonging to highly unbalanced classes unfortunately when the training set cardinality is small compared to the data dimensionality small sample size problem the classification performance of several well known classifiers strongly decreases similarly the classification accuracy of several discriminative methods decreases when non linearly separable and unbalanced classes are treated in this paper we firstly survey state of the art methods that employ improved versions of linear discriminant analysis lda to deal with the above mentioned problems secondly we propose a family of classifiers based on the fisher subspace estimation which efficiently deal with the small sample size problem non linearly separable classes and unbalanced classes the promising results obtained by the proposed techniques on benchmark datasets and the comparison with state of the art predictors show the efficacy of the proposed techniques

efficient and robust feature extraction by maximum margin criterion in pattern recognition feature extraction techniques are widely employed to reduce the dimensionality of data and to enhance the discriminatory information principal component analysis pca and linear discriminant analysis lda are the two most popular linear dimensionality reduction methods however pca is not very effective for the extraction of the most discriminant features and lda is not stable due to the small sample size problem in this paper we propose some new linear and nonlinear feature extractors based on maximum margin criterion mmc geometrically feature extractors based on mmc maximize the average margin between classes after dimensionality reduction it is shown that mmc can represent class separability better than pca as a connection to lda we may also derive lda from mmc by incorporating some constraints by using some other constraints we establish a new linear feature extractor that does not suffer from the small sample size problem which is known to cause serious stability problems for lda the kernelized nonlinear counterpart of this linear feature extractor is also established in the paper our extensive experiments demonstrate that the new feature extractors are effective stable and efficient

bilinear probabilistic principal component analysis probabilistic principal component analysis ppca is a popular linear latent variable model for performing dimension reduction on 1 d data in a probabilistic manner however when used on 2 d data such as images ppca suffers from the curse of dimensionality due to the subsequently large number of model parameters to overcome this problem we propose in this paper a novel probabilistic model on 2 d data called bilinear ppca bppca this allows the establishment of a closer tie between bppca and its nonprobabilistic counterpart moreover two efficient parameter estimation algorithms for fitting bppca are also developed experiments on a number of 2 d synthetic and real world data sets show that bppca is more accurate than existing probabilistic and nonprobabilistic dimension reduction methods

an optimization criterion for generalized discriminant analysis on undersampled problems an optimization criterion is presented for discriminant analysis the criterion extends the optimization criteria of the classical linear discriminant analysis lda through the use of the pseudoinverse when the scatter matrices are singular it is applicable regardless of the relative sizes of the data dimension and sample size overcoming a limitation of classical lda the optimization problem can be solved analytically by applying the generalized singular value decomposition gsvd technique the pseudoinverse has been suggested and used for undersampled problems in the past where the data dimension exceeds the number of data points the criterion proposed in this paper provides a theoretical justification for this procedure an approximation algorithm for the gsvd based approach is also presented it reduces the computational complexity by finding subclusters of each cluster and uses their centroids to capture the structure of each cluster this reduced problem yields much smaller matrices to which the gsvd can be applied efficiently experiments on text data with up to 7 000 dimensions show that the approximation algorithm produces results that are close to those produced by the exact algorithm

distribution of eigenvalues of ensembles of asymmetrically diluted hopfield matrices using grassmann variables and an analogy with two dimensional electrostatics we obtain the average eigenvalue distribution of ensembles of asymmetrically diluted hopfield matrices in the limit we found that in the limit of strong dilution the distribution is uniform in a circle in the complex plane

global structure of periodicity hubs in lyapunov phase diagrams of dissipative flows infinite cascades of periodicity hubs were predicted and very recently observed experimentally to organize stable oscillations of some dissipative flows here we describe the global mechanism underlying the genesis and organization of networks of periodicity hubs in control parameter space of a simple prototypical flow namely a r ossler s oscillator we show that spirals associated with periodicity hubs emerge and accumulate at the folding of certain fractal like sheaves of shilnikov homoclinic bifurcations of a common saddle focus equilibrium the specific organization of hub networks is found to depend strongly on the interaction between the homoclinic orbits and the global structure of the underlying attractor

thermodynamic limit for the invariant measures in supercritical zero range processes we prove a strong form of the equivalence of ensembles for the invariant measures of zero range processes conditioned to a supercritical density of particles it is known that in this case there is a single site that accomodates a macroscopically large number of the particles in the system we show that in the thermodynamic limit the rest of the sites have joint distribution equal to the grand canonical measure at the critical density this improves the result of groskinsky schutz and spohn where convergence is obtained for the finite dimensional marginals we obtain as corollaries limit theorems for the order statistics of the components and for the fluctuations of the bulk

appearance models based on kernel canonical correlation analysis this paper introduces a new approach to constructing appearance models based on kernel canonical correlation analysis kernel cca kernel cca is a non linear extension of cca where a non linear transformation of the input data is performed implicitly using kernel methods although in this respect it is similar to other generalized linear methods kernel cca is especially well suited for relating two sets of measurements the benefits of our method compared to standard feature extraction methods based on pca will be illustrated experimentally for the task of estimating an object s pose from raw brightness images

an exploration of the henon quadratic map abstract the two dimensional mapping introduced by henon in 1976 which yields a strange attractor for the parameter values a 1 4 b 0 3 has been investigated over the extended region 2 a 6 1 b 1 analytical solutions have been obtained for the onset of real stable cycles of periods 1 through 6 the associated bifurcations to stable cycles of twice the period 2 to 12 have also been determined all the resulting boundary curves are displayed in the a b parameter plane finally an interesting three dimensional generalization of the henon map is announced

dynamics of batch training in a perceptron early stopping and weight decay are studied in a linear perceptron using a new simplified approach to the dynamics in the thermodynamical limit the approach is directly deduced from the gradient descent weight update it allows an exact description of the dynamics of the batch training process the results are compared with a recent study of early stopping and weight decay based on the equilibrium statistical mechanics approach it is shown that the equilibrium results for early stopping are good approximations and are exact for weight decay furthermore in the dynamical approach it is possible to determine the necessary number of training steps to fulfil certain termination conditions it can be shown that asymptotically i e if the number of examples is large only two batch steps are required to reach the optimal convergence if the learning rate is optimally chosen

eigenvalues of covariance matrices application to neural network learning the learing time of a simple neural network model is obtained through an analytic computation of the eigenvalue spectrum for the hessian matrix which describes the second order properties of the objective function in the space of coupling coefficients the results are generic for symmetric matrices obtained by summing outer products of random vectors the form of the eigenvalue distribution suggests new techniques for accelerating the learning process and provides a theoretical justification for the choice of centered versus biased state variables

kernel quadratic discriminant analysis for small sample size problem it is generally believed that quadratic discriminant analysis qda can better fit the data in practical pattern recognition applications compared to linear discriminant analysis lda method this is due to the fact that qda relaxes the assumption made by lda based methods that the covariance matrix for each class is identical however it still assumes that the class conditional distribution is gaussian which is usually not the case in many real world applications in this paper a novel kernel based qda method is proposed to further relax the gaussian assumption by using the kernel machine technique the proposed method solves the complex pattern recognition problem by combining the qda solution and the kernel machine technique and at the same time tackles the so called small sample size problem through a regularized estimation of the covariance matrix extensive experimental results indicate that the proposed method is a more sophisticated solution outperforming many traditional kernel based learning algorithms

inexact implementation using krylov subspace methods for large scale exponential discriminant analysis with applications to high dimensionality reduction problems abstract exponential discriminant analysis eda is a generalized discriminant analysis method based on matrix exponential it can essentially overcome the intrinsic difficulty of small sample size problem that exists in the classical linear discriminant analysis lda however for data with high dimensionality one has to solve a large matrix exponential eigenproblem in this method and the time complexity is dominated by the computation of exponential of large matrices in this paper we propose two inexact krylov subspace algorithms for solving the large matrix exponential eigenproblem efficiently the contribution of this work is threefold first we consider how to compute matrix exponential vector products efficiently which is the key step in the krylov subspace method second we compare the discriminant analysis criterion of eda and that of lda from a theoretical point of view third we establish the relationship between the accuracy of the approximate eigenvectors and the distance to nearest neighbor classifier and show why the matrix exponential eigenproblem can be solved approximately in practice numerical experiments on some real world databases show the superiority of our new algorithms over their original counterpart for face recognition

rapid and brief communications modified linear discriminant analysis in this paper a modified fisher linear discriminant analysis flda is proposed and aims to not only overcome the rank limitation of flda that is at most only finding a discriminant vector for 2 class problem based on fisher discriminant criterion but also relax singularity of the within class scatter matrix and finally improves classification performance of flda experiments on nine publicly available datasets show that the proposed method has better or comparable performance on all the datasets than flda

current large deviations for asymmetric exclusion processes with open boundaries we study the large deviation functional of the current for the weakly asymmetric simple exclusion process in contact with two reservoirs we compare this functional in the large drift limit to the one of the totally asymmetric simple exclusion process in particular to the jensen varadhan functional conjectures for generalizing the jensen varadhan functional to open systems are also stated

penalized discriminant analysis fisher s linear discriminant analysis lda is a popular data analytic tool for studying the relationship between a set of predictors and a categorical response in this paper we describe a penalized version of lda it is designed for situations in which there are many highly correlated predictors such as those obtained by discretizing a function or the grey scale values of the pixels in a series of images in cases such as these it is natural efficient and sometimes essential to impose a spatial smoothness constraint on the coefficients both for improved prediction performance and interpretability we cast the classification problem into a regression framework via optimal scoring using this our proposal facilitates the use of any penalized regression technique in the classification setting the technique is illustrated with examples in speech recognition and handwritten character recognition

enhanced fisher discriminant criterion for image recognition many previous studies have shown that image recognition can be significantly improved by fisher linear discriminant analysis flda technique however flda ignores the variation of data points from the same class which characterizes the most important modes of variability of patterns and helps to improve the generalization capability of flda thus the performance of flda on testing data is not good enough in this paper we propose an enhanced fisher discriminant criterion efdc efdc explicitly considers the intra class variation and incorporates the intra class variation into the fisher discriminant criterion to build a robust and efficient dimensionality reduction function efdc obtains a subspace which best detects the discriminant structure and simultaneously preserves the modes of variability of patterns which will result in stable intraclass representation experimental results on four image database show a clear improvement over the results of flda based methods

condensation in stochastic particle systems with stationary product measures we study stochastic particle systems with stationary product measures that exhibit a condensation transition due to particle interactions or spatial inhomogeneities we review previous work on the stationary behaviour and put it in the context of the equivalence of ensembles providing a general characterization of the condensation transition for homogeneous and inhomogeneous systems in the thermodynamic limit this leads to strengthened results on weak convergence for subcritical systems and establishes the equivalence of ensembles for spatially inhomogeneous systems under very general conditions extending previous results which focused on attractive and finite systems we use relative entropy techniques which provide simple proofs making use of general versions of local limit theorems for independent random variables